Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications.Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

    Every piece of the web application stack, from HTTP requests to browser-side scripts, comes with important yet subtle security consequences. To keep users safe, it is essential for developers to confidently navigate this landscape.In The Tangled Web, Michal Zalewski, one of the world's top browser security experts, offers a compelling narrative that explains exactly how browsers work and why they're fundamentally insecure. Rather than dispense simplistic advice on vulnerabilities, Zalewski examines the entire browser security model, revealing weak points and providing crucial information for shoring up web application security. You'll learn how to:Perform common but surprisingly complex tasks such as URL parsing and HTML sanitization Use modern security features like Strict Transport Security, Content Security Policy, and Cross-Origin Resource Sharing Leverage many variants of the same-origin policy to safely compartmentalize complex web applications and protect user credentials in case of XSS bugs Build mashups and embed gadgets without getting stung by the tricky frame navigation policy Embed or host user-supplied content without running into the trap of content sniffing For quick reference, "Security Engineering Cheat Sheets" at the end of each chapter offer ready solutions to problems you're most likely to encounter. With coverage extending as far as planned HTML5 features, The Tangled Web will help you create secure web applications that stand the test of time.

    And it’s still short, profusely illustrated…and best of all–fun to read.If you’ve read it before, you’ll rediscover what made Don’t Make Me Think so essential to Web designers and developers around the world. If you’ve never read it, you’ll see why so many people have said it should be required reading for anyone working on Web sites.

    Restructured into six parts, this new edition covers a wide spectrum of software processes from initial requirements solicitation through design and development.

    It's used worldwide by companies who need to produce reliable, efficient, and scalable applications. Invest in learning Erlang now.Moore's Law is the observation that the amount you can do on a single chip doubles every two years. But Moore's Law is taking a detour. Rather than producing faster and faster processors, companies such as Intel and AMD are producing multi-core devices: single chips containing two, four, or more processors. If your programs aren't concurrent, they'll only run on a single processor at a time. Your users will think that your code is slow.Erlang is a programming language designed for building highly parallel, distributed, fault-tolerant systems. It has been used commercially for many years to build massive fault-tolerated systems that run for years with minimal failures.Erlang programs run seamlessly on multi-core computers: this means your Erlang program should run a lot faster on a 4 core processor than on a single core processor, all without you having to change a line of code.Erlang combines ideas from the world of functional programming with techniques for building fault-tolerant systems to make a powerful language for building the massively parallel, networked applications of the future.This book presents Erlang and functional programming in the familiar Pragmatic style. And it's written by Joe Armstrong, one of the creators of Erlang.It includes example code you'll be able to build upon. In addition, the book contains the full source code for two interesting applications:A SHOUTcast server which you can use to stream music to every computer in your house, and a full-text indexing and search engine that can index gigabytes of data. Learn how to write programs that run on dozens or even hundreds of local and remote processors. See how to write robust applications that run even in the face of network and hardware failure, using the Erlang programming language.

    Tracy Kidder got a preview of this world in the late 1970s when he observed the engineers of Data General design and build a new 32-bit minicomputer in just one year. His thoughtful, prescient book, The Soul of a New Machine, tells stories of 35-year-old "veteran" engineers hiring recent college graduates and encouraging them to work harder and faster on complex and difficult projects, exploiting the youngsters' ignorance of normal scheduling processes while engendering a new kind of work ethic.These days, we are used to the "total commitment" philosophy of managing technical creation, but Kidder was surprised and even a little alarmed at the obsessions and compulsions he found. From in-house political struggles to workers being permitted to tease management to marathon 24-hour work sessions, The Soul of a New Machine explores concepts that already seem familiar, even old-hat, less than 20 years later. Kidder plainly admires his subjects; while he admits to hopeless confusion about their work, he finds their dedication heroic. The reader wonders, though, what will become of it all, now and in the future. —Rob Lightner

    They aren't conscious of all the choices they make, like how they format their source, the names they use for variables, or the kinds of loops they use. They're focused entirely on problems they're solving, solutions they're creating, and algorithms they're implementing. So they write code in the way that seems natural, that happens intuitively, and that feels good.But if you're serious about your profession, intuition isn't enough. Perl Best Practices author Damian Conway explains that rules, conventions, standards, and practices not only help programmers communicate and coordinate with one another, they also provide a reliable framework for thinking about problems, and a common language for expressing solutions. This is especially critical in Perl, because the language is designed to offer many ways to accomplish the same task, and consequently it supports many incompatible dialects.With a good dose of Aussie humor, Dr. Conway (familiar to many in the Perl community) offers 256 guidelines on the art of coding to help you write better Perl code--in fact, the best Perl code you possibly can. The guidelines cover code layout, naming conventions, choice of data and control structures, program decomposition, interface design and implementation, modularity, object orientation, error handling, testing, and debugging.They're designed to work together to produce code that is clear, robust, efficient, maintainable, and concise, but Dr. Conway doesn't pretend that this is the one true universal and unequivocal set of best practices. Instead, Perl Best Practices offers coherent and widely applicable suggestions based on real-world experience of how code is actually written, rather than on someone's ivory-tower theories on how software ought to be created.Most of all, Perl Best Practices offers guidelines that actually work, and that many developers around the world are already using. Much like Perl itself, these guidelines are about helping you to get your job done, without getting in the way.Praise for Perl Best Practices from Perl community members:"As a manager of a large Perl project, I'd ensure that every member of my team has a copy of Perl Best Practices on their desk, and use it as the basis for an in-house style guide." -- Randal Schwartz"There are no more excuses for writing bad Perl programs. All levels of Perl programmer will be more productive after reading this book." -- Peter Scott"Perl Best Practices will be the next big important book in the evolution of Perl. The ideas and practices Damian lays down will help bring Perl out from under the embarrassing heading of "scripting languages". Many of us have known Perl is a real programming language, worthy of all the tasks normally delegated to Java and C++. With Perl Best Practices, Damian shows specifically how and why, so everyone else can see, too." -- Andy Lester"Damian's done what many thought impossible: show how to build large, maintainable Perl applications, while still letting Perl be the powerful, expressive language that programmers have loved for years." -- Bill Odom"Finally, a means to bring lasting order to the process and product of real Perl development teams." -- Andrew Sundstrom"Perl Best Practices provides a valuable education in how to write robust, maintainable P

    It combines the nice features of a scripting language with the powerful features of a production environment—features like persistent data structures and clean multithreading that you'll need for industrial-strength application development.The Joy of Clojure goes beyond just syntax to show you how to write fluent and idiomatic Clojure code. You'll learn a functional approach to programming and will master Lisp techniques that make Clojure so elegant and efficient. The book gives you easy access to hard soft ware areas like concurrency, interoperability, and performance. And it shows you how great it can be to think about problems the Clojure way. Purchase of the print book comes with an offer of a free PDF, ePub, and Kindle eBook from Manning. Also available is all code from the book. What's InsideThe what and why of ClojureHow to work with macrosHow to do elegant application designFunctional programming idiomsWritten for programmers coming to Clojure from another programming background—no prior experience with Clojure or Lisp is required.

    because the language seems like a good way to accomplish programming tasks that don't require the low-level bit handling power of C.-- Richard Bejtlich, TaoSecurityPython is a new and innovative scripting language. It is set to replace Perl as the programming language of choice for shell scripters, and for serious application developers who want a feature-rich, yet simple language to deploy their products.Dive Into Python is a hands-on guide to the Python language. Each chapter starts with a real, complete code sample, proceeds to pick it apart and explain the pieces, and then puts it all back together in a summary at the end.This is the perfect resource for you if you like to jump into languages fast and get going right away. If you're just starting to learn Python, first pick up a copy of Magnus Lie Hetland's Practical Python.

    Many examples are taken from real-world projects. The book focuses on high-level design as well as the gritty details of the Python syntax. The provided exercises inspire the reader to think about his or her own code, rather than providing solved problems. If you're new to Object Oriented Programming techniques, or if you have basic Python skills and wish to learn in depth how and when to correctly apply Object Oriented Programming in Python, this is the book for you. If you are an object-oriented programmer for other languages, you too will find this book a useful introduction to Python, as it uses terminology you are already familiar with. Python 2 programmers seeking a leg up in the new world of Python 3 will also find the book beneficial, and you need not necessarily know Python 2.

    Switching between these with the mouse takes up valuable time and can break your concentration. By using tmux, you can improve your productivity and regain your focus. This book will show you how.You’ll learn how to manage multiple terminal sessions within tmux using only your keyboard. You’ll see how to manage and run programs side-by-side in panes, and you’ll learn how to create the perfect development environment with custom scripts so that when you’re ready to work, your programs are waiting for you. Then you’ll discover how to manipulate text with tmux’s copy and paste buffers. Once you’ve got the basics down, you’ll discover how easy it is to use tmux to collaborate remotely with others. Finally, you’ll explore more advanced usage as you manage multiple tmux sessions, add custom scripts into the tmux status line, and integrate tmux with your system.Whether you’re an application developer or a system administrator, you’ll find many useful tricks and techniques to help you take control of your terminal.

    Booch illustrates essential concepts, explains the method, and shows successful applications in a variety of fields. Booch also gives pragmatic advice on a host of issues, including classification, implementation strategies, and cost-effective project management. A two-time winner of Software Development's coveted Jolt Cola Product Excellence Award!

    With this long-awaited revision, the authors continue to present the theory in a concise and straightforward manner, now with an eye out for the practical applications. They have revised this book to make it more accessible to today's students, including the addition of more material on writing proofs, more figures and pictures to convey ideas, side-boxes to highlight other interesting material, and a less formal writing style. Exercises at the end of each chapter, including some new, easier exercises, help readers confirm and enhance their understanding of the material. *NEW! Completely rewritten to be less formal, providing more accessibility to todays students. *NEW! Increased usage of figures and pictures to help convey ideas. *NEW! More detail and intuition provided for definitions and proofs. *NEW! Provides special side-boxes to present supplemental material that may be of interest to readers. *NEW! Includes more exercises, including many at a lower level. *NEW! Presents program-like notation for PDAs and Turing machines. *NEW! Increas

    These topics lie at the heart of many exciting areas of contemporary science and engineering - communication, signal processing, data mining, machine learning, pattern recognition, computational neuroscience, bioinformatics, and cryptography. This textbook introduces theory in tandem with applications. Information theory is taught alongside practical communication systems, such as arithmetic coding for data compression and sparse-graph codes for error-correction. A toolbox of inference techniques, including message-passing algorithms, Monte Carlo methods, and variational approximations, are developed alongside applications of these tools to clustering, convolutional codes, independent component analysis, and neural networks. The final part of the book describes the state of the art in error-correcting codes, including low-density parity-check codes, turbo codes, and digital fountain codes -- the twenty-first century standards for satellite communications, disk drives, and data broadcast. Richly illustrated, filled with worked examples and over 400 exercises, some with detailed solutions, David MacKay's groundbreaking book is ideal for self-learning and for undergraduate or graduate courses. Interludes on crosswords, evolution, and sex provide entertainment along the way. In sum, this is a textbook on information, communication, and coding for a new generation of students, and an unparalleled entry point into these subjects for professionals in areas as diverse as computational biology, financial engineering, and machine learning.

    As expected, .NET 4 brings a wealth of new APIs to the base class libraries and several new syntactical constructs to C#. For example, it is now possible to create C# methods that support optional and named arguments. As well, you are able to resolve types completely at runtime using the new dynamic keyword. As far as APIs are concerned, this edition of Pro C# 2010 and the .NET 4 Platform will cover a number of new details, including:• the Dynamic Language Runtime (DLR)• the Task Parallel Library (TPL, including PLINQ)• the ADO.NET Entity Framework (as well as LINQ to EF)• expanded coverage of the Windows Presentation Foundation (WPF) API• improved support for COM interopIf you're checking out this book for the first time, understand that it targets experienced software professionals and students of computer science—so please don't expect three chapters devoted to "variable scope" here! The mission of this text is to provide you with a rock-solid foundation in the C# programming language (including a full discussion of OOP) and the critical aspects of the .NET 4 platform.Once you digest the information presented in the book you hold in your hands, you'll be in a perfect position to apply this knowledge to your specific programming assignments, and you'll be well-equipped to explore the .NET universe on your own terms.Andrew TroelsenMicrosoft MVP, Visual Developer—Visual C#; Microsoft Certified Trainer (MCT)