Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications.Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

    It guides you step by step from simple tests to tests that are maintainable, readable, and trustworthy. It covers advanced subjects like mocks, stubs, and frameworks such as Typemock Isolator and Rhino Mocks. And you'll learn about advanced test patterns and organization, working with legacy code and even untestable code. The book discusses tools you need when testing databases and other technologies. It's written for .NET developers but others will also benefit from this book.Purchase of the print book comes with an offer of a free PDF, ePub, and Kindle eBook from Manning. Also available is all code from the book.Table of ContentsThe basics of unit testingA first unit testUsing stubs to break dependenciesInteraction testing using mock objectsIsolation (mock object) frameworksTest hierarchies and organizationThe pillars of good testsIntegrating unit testing into the organizationWorking with legacy code

    Readers will progress from building a basic physics engine to creating intelligent moving objects and complex systems, setting the foundation for further experiments in generative design. Subjects covered include forces, trigonometry, fractals, cellular automata, self-organization, and genetic algorithms. The book's examples are written in Processing, an open-source language and development environment built on top of the Java programming language. On the book's website (http://www.natureofcode.com), the examples run in the browser via Processing's JavaScript mode.

    It's available on almost every OS--if you master the techniques in this book, you'll never need another text editor. Practical Vim shows you 120 vim recipes so you can quickly learn the editor's core functionality and tackle your trickiest editing and writing tasks. Vim, like its classic ancestor vi, is a serious tool for programmers, web developers, and sysadmins. No other text editor comes close to Vim for speed and efficiency; it runs on almost every system imaginable and supports most coding and markup languages. Learn how to edit text the "Vim way:" complete a series of repetitive changes with The Dot Formula, using one keystroke to strike the target, followed by one keystroke to execute the change. Automate complex tasks by recording your keystrokes as a macro. Run the same command on a selection of lines, or a set of files. Discover the "very magic" switch, which makes Vim's regular expression syntax more like Perl's. Build complex patterns by iterating on your search history. Search inside multiple files, then run Vim's substitute command on the result set for a project-wide search and replace. All without installing a single plugin! You'll learn how to navigate text documents as fast as the eye moves--with only a few keystrokes. Jump from a method call to its definition with a single command. Use Vim's jumplist, so that you can always follow the breadcrumb trail back to the file you were working on before. Discover a multilingual spell-checker that does what it's told.Practical Vim will show you new ways to work with Vim more efficiently, whether you're a beginner or an intermediate Vim user. All this, without having to touch the mouse.What You Need: Vim version 7

    After splitting a monolithic application or building a microservice ecosystem from scratch, many engineers are left wondering what s next. In this practical book, author Susan Fowler presents a set of microservice standards in depth, drawing from her experience standardizing over a thousand microservices at Uber. You ll learn how to design microservices that are stable, reliable, scalable, fault tolerant, performant, monitored, documented, and prepared for any catastrophe.Explore production-readiness standards, including:Stability and Reliability: develop, deploy, introduce, and deprecate microservices; protect against dependency failuresScalability and Performance: learn essential components for achieving greater microservice efficiencyFault Tolerance and Catastrophe Preparedness: ensure availability by actively pushing microservices to fail in real timeMonitoring: learn how to monitor, log, and display key metrics; establish alerting and on-call proceduresDocumentation and Understanding: mitigate tradeoffs that come with microservice adoption, including organizational sprawl and technical debt"

    Windows Presentation Foundation (WPF) is the recommended technology for creating Windows user interfaces, giving you the power to create richer and more compelling applications than you dreamed possible. Whether you want to develop traditional user interfaces or integrate 3D graphics, audio/video, animation, dynamic skinning, multi-touch, rich document support, speech recognition, or more, WPF enables you to do so in a seamless, resolution-independent manner. WPF 4 Unleashed is the authoritative book that covers it all, in a practical and approachable fashion, authored by WPF guru and Microsoft developer Adam Nathan. Covers everything you need to know about Extensible Application Markup Language (XAML) Examines the WPF feature areas in incredible depth: controls, layout, resources, data binding, styling, graphics, animation, and more Highlights the latest features, such as multi-touch, text rendering improvements, XAML language enhancements, new controls, the Visual State Manager, easing functions, and much more Delves into topics that aren't covered by most books: 3D, speech, audio/video, documents, effects Shows how to create popular UI elements, such as Galleries, ScreenTips, and more Demonstrates how to create sophisticated UI mechanisms, such as Visual Studio-like collapsible/dockable panes Explains how to create first-class custom controls for WPF Demonstrates how to create hybrid WPF software that leverages Windows Forms, DirectX, ActiveX, or other non-WPF technologies Explains how to exploit new Windows 7 features, such as Jump Lists and taskbar customizations

    And now, with the release of the 5.0 version of Java, O'Reilly has given the book that defined the "in a Nutshell" category another impressive tune-up.In this latest revision, readers will find "Java in a Nutshell," 5th Edition, does more than just cover the extensive changes implicit in 5.0, the newest version of Java. It's undergone a complete makeover--in scope, size, and type of coverage--in order to more closely meet the needs of the modern Java programmer.To wit, "Java in a Nutshell," 5th Edition now places less emphasis on coming to Java from C and C++, and adds more discussion on tools and frameworks. It also offers new code examples to illustrate the working of APIs, and, of course, extensive coverage of Java 5.0. But faithful readers take comfort: it still hasn't lost any of its core elements that made it such a classic to begin with.This handy reference gets right to the heart of the program with an accelerated introduction to the Javaprogramming language and its key APIs--ideal for developers wishing to start writing code right away. And, as was the case in previous editions, " Java in a Nutshell," 5th Edition is once again chock-full of poignant tips, techniques, examples, and practical advice. For as longas Java has existed, "Java in a Nutshell" has helped developers maximize the capabilities of the program's newest versions. And this latest edition is no different.

    Over the past five years, authors Dustin Boswell and Trevor Foucher have analyzed hundreds of examples of "bad code" (much of it their own) to determine why they’re bad and how they could be improved. Their conclusion? You need to write code that minimizes the time it would take someone else to understand it—even if that someone else is you.This book focuses on basic principles and practical techniques you can apply every time you write code. Using easy-to-digest code examples from different languages, each chapter dives into a different aspect of coding, and demonstrates how you can make your code easy to understand.Simplify naming, commenting, and formatting with tips that apply to every line of codeRefine your program’s loops, logic, and variables to reduce complexity and confusionAttack problems at the function level, such as reorganizing blocks of code to do one task at a timeWrite effective test code that is thorough and concise—as well as readable"Being aware of how the code you create affects those who look at it later is an important part of developing software. The authors did a great job in taking you through the different aspects of this challenge, explaining the details with instructive examples." —Michael Hunger, passionate Software Developer

    This book contains the fundamental laws of software development, the primary pieces of understanding that make the difference between a mid-level/junior programmer and the high-level senior software engineer. The book exists to help all programmers understand the process of writing software, on a very fundamental level that can be applied to any programming language or project, from here into eternity. Code Simplicity is also written in such a way that even non-technical managers of software teams can gain an understanding of what the “right way” and the “wrong way” is (and why they are right and wrong) when it comes to software design. The focus of the book is primarily on “software design,” the process of creating a plan for a software project and making technical decisions about the pattern and structure of a system.

    The content assumes no previous programming experience, other than knowing how to create a basic web page in HTML & CSS. You'll learn how to achieve techniques seen on many popular websites (such as adding animation, tabbed panels, content sliders, form validation, interactive galleries, and sorting data)..Introduces core programming concepts in JavaScript and jQueryUses clear descriptions, inspiring examples, and easy-to-follow diagramsTeaches you how to create scripts from scratch, and understand the thousands of JavaScripts, JavaScript APIs, and jQuery plugins that are available on the webDemonstrates the latest practices in progressive enhancement, cross-browser compatibility, and when you may be better off using CSS3If you're looking to create more enriching web experiences and express your creativity through code, then this is the book for you.This book is also available as part of a set in hardcover - Web Design with HTML, CSS, JavaScript and jQuery, 9781119038634 - and in softcover - Web Design with HTML, CSS, JavaScript and jQuery, 9781118907443.

     While very complex, the Internet operates on a few relatively simple concepts that anyone can understand. Networks and networked applications are embedded in our lives. Understanding how these technologies work is invaluable.  This book was written for everyone - no technical knowledge is required!While this book is not specifically about the Network+ or CCNA certifications, it as a way to give students interested in these certifications a starting point.

    What they don't realize—and what Daniel Hillis's short book brilliantly demonstrates—is that computers' seemingly complex operations can be broken down into a few simple parts that perform the same simple procedures over and over again.Computer wizard Hillis offers an easy-to-follow explanation of how data is processed that makes the operations of a computer seem as straightforward as those of a bicycle. Avoiding technobabble or discussions of advanced hardware, the lucid explanations and colorful anecdotes in The Pattern on the Stone go straight to the heart of what computers really do.Hillis proceeds from an outline of basic logic to clear descriptions of programming languages, algorithms, and memory. He then takes readers in simple steps up to the most exciting developments in computing today—quantum computing, parallel computing, neural networks, and self-organizing systems.Written clearly and succinctly by one of the world's leading computer scientists, The Pattern on the Stone is an indispensable guide to understanding the workings of that most ubiquitous and important of machines: the computer.

    You'll learn how to use Haskell in a variety of practical ways, from short scripts to large and demanding applications. Real World Haskell takes you through the basics of functional programming at a brisk pace, and then helps you increase your understanding of Haskell in real-world issues like I/O, performance, dealing with data, concurrency, and more as you move through each chapter. With this book, you will:Understand the differences between procedural and functional programming Learn the features of Haskell, and how to use it to develop useful programs Interact with filesystems, databases, and network services Write solid code with automated tests, code coverage, and error handling Harness the power of multicore systems via concurrent and parallel programming You'll find plenty of hands-on exercises, along with examples of real Haskell programs that you can modify, compile, and run. Whether or not you've used a functional language before, if you want to understand why Haskell is coming into its own as a practical language in so many major organizations, Real World Haskell is the best place to start.

    It is the technology behind photo tagging systems at Facebook and Google, self-driving cars, speech recognition systems on your smartphone, and much more.In particular, Deep learning excels at solving machine perception problems: understanding the content of image data, video data, or sound data. Here's a simple example: say you have a large collection of images, and that you want tags associated with each image, for example, "dog," "cat," etc. Deep learning can allow you to create a system that understands how to map such tags to images, learning only from examples. This system can then be applied to new images, automating the task of photo tagging. A deep learning model only has to be fed examples of a task to start generating useful results on new data.

    We are living in an age where data comes in abundance, and thanks to the self-learning algorithms from the field of machine learning, we can turn this data into knowledge. Automated speech recognition on our smart phones, web search engines, e-mail spam filters, the recommendation systems of our favorite movie streaming services – machine learning makes it all possible.Thanks to the many powerful open-source libraries that have been developed in recent years, machine learning is now right at our fingertips. Python provides the perfect environment to build machine learning systems productively.This book will teach you the fundamentals of machine learning and how to utilize these in real-world applications using Python. Step-by-step, you will expand your skill set with the best practices for transforming raw data into useful information, developing learning algorithms efficiently, and evaluating results.You will discover the different problem categories that machine learning can solve and explore how to classify objects, predict continuous outcomes with regression analysis, and find hidden structures in data via clustering. You will build your own machine learning system for sentiment analysis and finally, learn how to embed your model into a web app to share with the world