Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications.Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

    And this may sound weirdly obvious, but every single one of those pieces of software was written by a programmer. Programmers are thus among the most quietly influential people on the planet. As we live in a world made of software, they're the architects. The decisions they make guide our behavior. When they make something newly easy to do, we do a lot more of it. If they make it hard or impossible to do something, we do less of it.If we want to understand how today's world works, we ought to understand something about coders. Who exactly are the people that are building today's world? What makes them tick? What type of personality is drawn to writing software? And perhaps most interestingly -- what does it do to them?One of the first pieces of coding a newbie learns is the program to make the computer say "Hello, world!" Like that piece of code, Clive Thompson's book is a delightful place to begin to understand this vocation, which is both a profession and a way of life, and which essentially didn't exist little more than a generation ago, but now is considered just about the only safe bet we can make about what the future holds. Thompson takes us close to some of the great coders of our time, and unpacks the surprising history of the field, beginning with the first great coders, who were women. Ironically, if we're going to traffic in stereotypes, women are arguably "naturally" better at coding than men, but they were written out of the history, and shoved out of the seats, for reasons that are illuminating. Now programming is indeed, if not a pure brotopia, at least an awfully homogenous community, which attracts people from a very narrow band of backgrounds and personality types. As Thompson learns, the consequences of that are significant - not least being a fetish for disruption at scale that doesn't leave much time for pondering larger moral issues of collateral damage. At the same time, coding is a marvelous new art form that has improved the world in innumerable ways, and Thompson reckons deeply, as no one before him has, with what great coding in fact looks like, who creates it, and where they come from. To get as close to his subject has he can, he picks up the thread of his own long-abandoned coding practice, and tries his mightiest to up his game, with some surprising results.More and more, any serious engagement with the world demands an engagement with code and its consequences, and to understand code, we must understand coders. In that regard, Clive Thompson's Hello, World! is a marvelous and delightful master class.

    You know you need concurrent applications, but also know these are almost impossible to get right. Meet Elixir, a functional, concurrent language built on the rock-solid Erlang VM. Elixir’s pragmatic syntax and built-in support for metaprogramming will make you productive and keep you interested for the long haul. This book is the introduction to Elixir for experienced programmers.Maybe you need something that’s closer to Ruby, but with a battle-proven environment that’s unrivaled for massive scalability, concurrency, distribution, and fault tolerance. Maybe the time is right for the Next Big Thing. Maybe it’s Elixir.

    This is necessary because, in the words of Michael Polanyi, "we can know more than we can tell." Our design choices are not the result of an ineluctable chain of logic; they come from a deeper place, one which is visceral and inarticulate.Polanyi calls this "tacit knowledge", a thing which we only understand as part of something else. When we speak, we do not focus on making sounds, we focus on our words. We understand the muscular act of speech, but would struggle to explain it.To write software, we must learn where to draw boundaries. Good software is built through effective indirection. We seem to have decided that this skill can only be learned through practice; it cannot be taught, except by example. Our decisions may improve with time, but not our ability to explain them. It's true that the study of these questions cannot yield a closed-form solution for judging software design. We can make our software simple, but we cannot do the same to its problem domain, its users, or the physical world. Our tacit knowledge of this environment will always inform our designs.This doesn't mean that we can simply ignore our design process. Polanyi tells us that tacit knowledge only suffices until we fail, and the software industry is awash with failure. Our designs may never be provably correct, but we can give voice to the intuition that shaped them. Our process may always be visceral, but it doesn't have to be inarticulate.And so this book does not offer knowledge, it offers clarity. It is aimed at readers who know Clojure, but struggle to articulate the rationale of their designs to themselves and others. Readers who use other languages, but have a passing familiarity with Clojure, may also find this book useful.

    But can you also build web sites that are usable by machines? That's where the future lies, and that's what RESTful Web Services shows you how to do. The World Wide Web is the most popular distributed application in history, and Web services and mashups have turned it into a powerful distributed computing platform. But today's web service technologies have lost sight of the simplicity that made the Web successful. They don't work like the Web, and they're missing out on its advantages. This book puts the "Web" back into web services. It shows how you can connect to the programmable web with the technologies you already use every day. The key is REST, the architectural style that drives the Web. This book:Emphasizes the power of basic Web technologies -- the HTTP application protocol, the URI naming standard, and the XML markup language Introduces the Resource-Oriented Architecture (ROA), a common-sense set of rules for designing RESTful web services Shows how a RESTful design is simpler, more versatile, and more scalable than a design based on Remote Procedure Calls (RPC) Includes real-world examples of RESTful web services, like Amazon's Simple Storage Service and the Atom Publishing Protocol Discusses web service clients for popular programming languages Shows how to implement RESTful services in three popular frameworks -- Ruby on Rails, Restlet (for Java), and Django (for Python) Focuses on practical issues: how to design and implement RESTful web services and clients This is the first book that applies the REST design philosophy to real web services. It sets down the best practices you need to make your design a success, and the techniques you need to turn your design into working code. You can harness the power of the Web for programmable applications: you just have to work with the Web instead of against it. This book shows you how.

    You’ll learn to spot which tests are performing, which need refactoring, and which need to be deleted entirely! Upgrade your testing suite with new testing styles, good patterns, and reliable automated testing.

    Designed for readers who don't need the academic formality, it's a fast and easy computer science guide. It teaches essential concepts for people who want to program computers effectively. First, it introduces discrete mathematics, then it exposes the most common algorithms and data structures. It also shows the principles that make computers and programming languages work.

    You'll learn how to create responsive, data-driven websites with PHP, MySQL, and JavaScript, regardless of whether you already know how to program. Discover how the powerful combination of PHP and MySQL provides an easy way to build modern websites complete with dynamic data and user interaction. You'll also learn how to add JavaScript to create rich Internet applications and websites.Learning PHP, MySQL, and JavaScript explains each technology separately, shows you how to combine them, and introduces valuable web programming concepts, including objects, XHTML, cookies, and session management. You'll practice what you've learned with review questions in each chapter, and find a sample social networking platform built with the elements introduced in this book. This book will help you:-Understand PHP essentials and the basics of object-oriented programming-Master MySQL, from database structure to complex queries-Create web pages with PHP and MySQL by integrating forms and other HTML features-Learn about JavaScript, from functions and event handling to accessing the Document Object Model-Use libraries and packages, including the Smarty web template system, PEAR program repository, and the Yahoo! User Interface Library -Make Ajax calls and turn your website into a highly dynamic environment-Upload and manipulate files and images, validate user input, and secure your applications

    The concepts and techniques described are the actual ones used by real game studios like Electronic Arts and Naughty Dog. The examples are often grounded in specific technologies, but the discussion extends way beyond any particular engine or API. The references and citations make it a great jumping off point for those who wish to dig deeper into any particular aspect of the game development process.Intended as the text for a college level series in game programming, this book can also be used by amateur software engineers, hobbyists, self-taught game programmers, and existing members of the game industry. Junior game engineers can use it to solidify their understanding of game technology and engine architecture. Even senior engineers who specialize in one particular field of game development can benefit from the bigger picture presented in these pages.

    With the sophisticated algorithms in this book, you can write smart programs to access interesting datasets from other web sites, collect data from users of your own applications, and analyze and understand the data once you've found it.Programming Collective Intelligence takes you into the world of machine learning and statistics, and explains how to draw conclusions about user experience, marketing, personal tastes, and human behavior in general -- all from information that you and others collect every day. Each algorithm is described clearly and concisely with code that can immediately be used on your web site, blog, Wiki, or specialized application. This book explains:Collaborative filtering techniques that enable online retailers to recommend products or media Methods of clustering to detect groups of similar items in a large dataset Search engine features -- crawlers, indexers, query engines, and the PageRank algorithm Optimization algorithms that search millions of possible solutions to a problem and choose the best one Bayesian filtering, used in spam filters for classifying documents based on word types and other features Using decision trees not only to make predictions, but to model the way decisions are made Predicting numerical values rather than classifications to build price models Support vector machines to match people in online dating sites Non-negative matrix factorization to find the independent features in a dataset Evolving intelligence for problem solving -- how a computer develops its skill by improving its own code the more it plays a game Each chapter includes exercises for extending the algorithms to make them more powerful. Go beyond simple database-backed applications and put the wealth of Internet data to work for you. "Bravo! I cannot think of a better way for a developer to first learn these algorithms and methods, nor can I think of a better way for me (an old AI dog) to reinvigorate my knowledge of the details."-- Dan Russell, Google "Toby's book does a great job of breaking down the complex subject matter of machine-learning algorithms into practical, easy-to-understand examples that can be directly applied to analysis of social interaction across the Web today. If I had this book two years ago, it would have saved precious time going down some fruitless paths."-- Tim Wolters, CTO, Collective Intellect

    Revised and updated for the Elixir 1.7, Elixir in Action, Second Edition teaches you how to apply Elixir to practical problems associated with scalability, fault tolerance, and high availability. Along the way, you'll develop an appreciation for, and considerable skill in, a functional and concurrent style of programming.

    Readers will learn to understand object-oriented design with inheritance or composition, object aggregation and association, and the difference between interfaces and implementations. Readers will also become more efficient and better thinkers in terms of object-oriented development." This revised edition focuses on interoperability across various technologies, primarily using XML as the communication mechanism. A more detailed focus is placed on how business objects operate over networks, including client/server architectures and web services.