Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications.Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

    It is the definitive reference guide, now in a second edition. Although the first edition was written in 1978, it continues to be a worldwide best-seller. This second edition brings the classic original up to date to include the ANSI standard. From the Preface: We have tried to retain the brevity of the first edition. C is not a big language, and it is not well served by a big book. We have improved the exposition of critical features, such as pointers, that are central to C programming. We have refined the original examples, and have added new examples in several chapters. For instance, the treatment of complicated declarations is augmented by programs that convert declarations into words and vice versa. As before, all examples have been tested directly from the text, which is in machine-readable form. As we said in the first preface to the first edition, C "wears well as one's experience with it grows." With a decade more experience, we still feel that way. We hope that this book will help you to learn C and use it well.

    With this practical guide, you'll learn how PHP has become a full-featured, mature language with object-orientation, namespaces, and a growing collection of reusable component libraries.Author Josh Lockhart--creator of PHP The Right Way, a popular initiative to encourage PHP best practices--reveals these new language features in action. You'll learn best practices for application architecture and planning, databases, security, testing, debugging, and deployment. If you have a basic understanding of PHP and want to bolster your skills, this is your book.Learn modern PHP features, such as namespaces, traits, generators, and closuresDiscover how to find, use, and create PHP componentsFollow best practices for application security, working with databases, errors and exceptions, and moreLearn tools and techniques for deploying, tuning, testing, and profiling your PHP applicationsExplore Facebook's HVVM and Hack language implementations--and how they affect modern PHPBuild a local development environment that closely matches your production server

    And through CODE, we see how this ingenuity and our very human compulsion to communicate have driven the technological innovations of the past two centuries. Using everyday objects and familiar language systems such as Braille and Morse code, author Charles Petzold weaves an illuminating narrative for anyone who’s ever wondered about the secret inner life of computers and other smart machines. It’s a cleverly illustrated and eminently comprehensible story—and along the way, you’ll discover you’ve gained a real context for understanding today’s world of PCs, digital media, and the Internet. No matter what your level of technical savvy, CODE will charm you—and perhaps even awaken the technophile within.

    Plenty of books describe what agile development is or why it helps software projects succeed, but very few combine information for developers, managers, testers, and customers into a single package that they can apply directly. This book provides no-nonsense advice on agile planning, development, delivery, and management taken from the authors' many years of experience with Extreme Programming (XP). You get a gestalt view of the agile development process, including comprehensive guidance for non-technical readers and hands-on technical practices for developers and testers. The Art of Agile Development gives you clear answers to questions such as:How can we adopt agile development? Do we really need to pair program? What metrics should we report? What if I can't get my customer to participate? How much documentation should we write? When do we design and architect? As a non-developer, how should I work with my agile team? Where is my product roadmap? How does QA fit in? The book teaches you how to adopt XP practices, describes each practice in detail, then discusses principles that will allow you to modify XP and create your own agile method. In particular, this book tackles the difficult aspects of agile development: the need for cooperation and trust among team members. Whether you're currently part of an agile team, working with an agile team, or interested in agile development, this book provides the practical tips you need to start practicing agile development. As your experience grows, the book will grow with you, providing exercises and information that will teach you first to understand the rules of agile development, break them, and ultimately abandon rules altogether as you master the art of agile development. "Jim Shore and Shane Warden expertly explain the practices and benefits of Extreme Programming. They offer advice from their real-world experiences in leading teams. They answer questions about the practices and show contraindications - ways that a practice may be mis-applied. They offer alternatives you can try if there are impediments to applying a practice, such as the lack of an on-site customer. --Ken Pugh, Author of Jolt Award Winner, Prefactoring "I will leave a copy of this book with every team I visit." --Brian Marick, Exampler Consulting

    If you're anexperienced developer, you can start creating robust mobile Androidapps right away with this professional guide to Android 4application development. Written by one of Google's lead Androiddeveloper advocates, this practical book walks you through a seriesof hands-on projects that illustrate the features of the AndroidSDK. That includes all the new APIs introduced in Android 3 and 4, including building for tablets, using the Action Bar, Wi-Fi Direct, NFC Beam, and more.Shows experienced developers how to create mobile applicationsfor Android smartphones and tabletsRevised and expanded to cover all the Android SDK releasesincluding Android 4.0 (Ice Cream Sandwich), including all updatedAPIs, and the latest changes to the Android platform.Explains new and enhanced features such as drag and drop, fragments, the action bar, enhanced multitouch support, newenvironmental sensor support, major improvements to the animationframework, and a range of new communications techniques includingNFC and Wi-Fi direct.Provides practical guidance on publishing and marketing yourapplications, best practices for user experience, and moreThis book helps you learn to master the design, lifecycle, andUI of an Android app through practical exercises, which you canthen use as a basis for developing your own Android apps.

    This book surveys the most important computer algorithms currently in use and provides a full treatment of data structures and algorithms for sorting, searching, graph processing, and string processing -- including fifty algorithms every programmer should know. In this edition, new Java implementations are written in an accessible modular programming style, where all of the code is exposed to the reader and ready to use.The algorithms in this book represent a body of knowledge developed over the last 50 years that has become indispensable, not just for professional programmers and computer science students but for any student with interests in science, mathematics, and engineering, not to mention students who use computation in the liberal arts.The companion web site, algs4.cs.princeton.edu contains An online synopsis Full Java implementations Test data Exercises and answers Dynamic visualizations Lecture slides Programming assignments with checklists Links to related material The MOOC related to this book is accessible via the "Online Course" link at algs4.cs.princeton.edu. The course offers more than 100 video lecture segments that are integrated with the text, extensive online assessments, and the large-scale discussion forums that have proven so valuable. Offered each fall and spring, this course regularly attracts tens of thousands of registrants.Robert Sedgewick and Kevin Wayne are developing a modern approach to disseminating knowledge that fully embraces technology, enabling people all around the world to discover new ways of learning and teaching. By integrating their textbook, online content, and MOOC, all at the state of the art, they have built a unique resource that greatly expands the breadth and depth of the educational experience.

    By applying the design patterns outlined in this book, you’ll learn how to write flexible and resilient code that’s easier—not harder—to work with as your code base grows.JavaScript has become one of the most widely used—and essential—programming languages for the Web, on both the client-side and server-side. In the real world, JavaScript applications are fragile, and when you change them things often break. Author Eric Elliott shows you how to add features without creating bugs or negatively impacting the rest of your code during the course of building a large JavaScript application.Examine the anatomy of a modern JavaScript applicationLearn best practices for code organization, modularity, and reuseApply Model-View-Controller architectures to client-side web developmentDelve into client-side (browser) and server-side (Node) approachesUse Node to design and program RESTful APIsLearn the processes teams use to build, test, deploy, and scale large JavaScript applicationsExpand your application’s reach through platform targets and internationalization

    This book draws on material Michael created for his renowned Object Mentor seminars, techniques Michael has used in mentoring to help hundreds of developers, technical managers, and testers bring their legacy systems under control. The topics covered include: Understanding the mechanics of software change, adding features, fixing bugs, improving design, optimizing performance Getting legacy code into a test harness Writing tests that protect you against introducing new problems Techniques that can be used with any language or platform, with examples in Java, C++, C, and C# Accurately identifying where code changes need to be made Coping with legacy systems that aren't object-oriented Handling applications that don't seem to have any structureThis book also includes a catalog of twenty-four dependency-breaking techniques that help you work with program elements in isolation and make safer changes.

    It covers topics ranging from personal responsibility and career development to architectural techniques for keeping your code flexible and easy to adapt and reuse. Read this book, and you'll learn how toFight software rot; Avoid the trap of duplicating knowledge; Write flexible, dynamic, and adaptable code; Avoid programming by coincidence; Bullet-proof your code with contracts, assertions, and exceptions; Capture real requirements; Test ruthlessly and effectively; Delight your users; Build teams of pragmatic programmers; and Make your developments more precise with automation. Written as a series of self-contained sections and filled with entertaining anecdotes, thoughtful examples, and interesting analogies, The Pragmatic Programmer illustrates the best practices and major pitfalls of many different aspects of software development. Whether you're a new coder, an experienced programmer, or a manager responsible for software projects, use these lessons daily, and you'll quickly see improvements in personal productivity, accuracy, and job satisfaction. You'll learn skills and develop habits and attitudes that form the foundation for long-term success in your career. You'll become a Pragmatic Programmer.

    The book is broken into two parts, the first deals with security-related reverse engineering and the second explores the more practical aspects of reverse engineering. In addition, the author explains how to reverse engineer a third-party software library to improve interfacing and how to reverse engineer a competitor's software to build a better product. * The first popular book to show how software reverse engineering can help defend against security threats, speed up development, and unlock the secrets of competitive products * Helps developers plug security holes by demonstrating how hackers exploit reverse engineering techniques to crack copy-protection schemes and identify software targets for viruses and other malware * Offers a primer on advanced reverse-engineering, delving into disassembly-code-level reverse engineering-and explaining how to decipher assembly language

    Now he's sharing his collection of antipatterns--the most common errors he's identified in those thousands of requests for help. Most developers aren't SQL experts, and most of the SQL that gets used is inefficient, hard to maintain, and sometimes just plain wrong. This book shows you all the common mistakes, and then leads you through the best fixes. What's more, it shows you what's behind these fixes, so you'll learn a lot about relational databases along the way. Each chapter in this book helps you identify, explain, and correct a unique and dangerous antipattern. The four parts of the book group the anti​patterns in terms of logical database design, physical database design, queries, and application development. The chances are good that your application's database layer already contains problems such as Index Shotgun, Keyless Entry, Fear of the Unknown, and Spaghetti Query. This book will help you and your team find them. Even better, it will also show you how to fix them, and how to avoid these and other problems in the future. SQL Antipatterns gives you a rare glimpse into an SQL expert's playbook. Now you can stamp out these common database errors once and for all. Whatever platform or programming language you use, whether you're a junior programmer or a Ph.D., SQL Antipatterns will show you how to design and build databases, how to write better database queries, and how to integrate SQL programming with your application like an expert. You'll also learn the best and most current technology for full-text search, how to design code that is resistant to SQL injection attacks, and other techniques for success.

    "Programming Entity Framework, 1st Edition" offers experienced developers a thorough introduction to Microsoft's core framework for modeling and interacting with data in .NET applications. This hands-on tour provides a deep understanding of Entity Framework's architecture and APIs, and explains how to use the framework in a variety of applications built with Visual Studio 2008 and .NET 3.5.From the Entity Data Model (EDM) and Object Services to EntityClient and the Metadata Workspace, this highly acclaimed first edition covers it all.Understand the core concepts you need to make the best use of the Entity Framework (EF) in your applicationsLearn to query your data, using either LINQ to Entities or Entity SQLCreate Windows Forms, WPF, and ASP.NET applicationsBuild ASMX web services and WCF servicesUse Object Services to work directly with your entity objectsDelve into model customization, relationship management, change tracking, data concurrency, and moreOne important note: while many of the lessons from this book will continue to be valuable as you move to .NET 4, the thoroughly revised second edition of "Programming Entity Framework" (August 2010) specifically targets Visual Studio 2010 and .NET 4 -- where there have been many advancements and additions to the framework.

    Fueled by significant benefits and an impressive portfolio of real-world applications already in production, Rails is destined to continue making significant inroads in coming years.Each new Rails application showing up on the web adds yet more to the collective wisdom of the Rails development community. Yesterday's best practices yield to today's latest and greatest techniques, as the state of the art is continually refined in kitchens all across the Internet. Indeed, these are times of great progress.At the same time, it's easy to get left behind in the wake of progress. Advanced Rails Recipes keeps you on the cutting edge of Rails development and, more importantly, continues to turn this fast-paced framework to your advantage.Advanced Rails Recipes is filled with pragmatic recipes you'll use on every Rails project. And by taking the code in these recipes and slipping it into your application you'll not only deliver your application quicker, you'll do so with the confidence that it's done right.The book includes contributions from Aaron Batalion, Adam Keys, Adam Wiggins, Andre Lewis, Andrew Kappen, Benjamin Curtis, Ben Smith, Chris Bernard, Chris Haupt, Chris Wanstrath, Cody Fauser, Dan Benjamin, Dan Manges, Daniel Fischer, David Bock, David Chelimsky, David Heinemeier Hansson, Erik Hatcher, Ezra Zygmuntowicz, Geoffrey Grosenbach, Giles Bowkett, Greg Hansen, Gregg Pollack, Hemant Kumar, Hugh Bien, Jamie Orchard-Hays, Jamis Buck, Jared Haworth, Jarkko Laine, Jason LaPier, Jay Fields, John Dewey, Jonathan Dahl, Josep Blanquer, Josh Stephenson, Josh Susser, Kevin Clark, Luke Francl, Mark Bates, Marty Haught, Matthew Bass, Michael Slater, Mike Clark, Mike Hagedorn, Mike Mangino, Mike Naberezny, Mike Subelsky, Nathaniel Talbott, PJ Hyett, Patrick Reagan, Peter Marklund, Pierre-Alexandre Meyer, Rick Olson, Ryan Bates, Scott Barron, Tony Primerano, Val Aleksenko, and Warren Konkel.

    Led by three renowned internals experts, this classic guide is fully updated for Windows 7 and Windows Server 2008 R2—and now presents its coverage in two volumes.As always, you get critical insider perspectives on how Windows operates. And through hands-on experiments, you’ll experience its internal behavior firsthand—knowledge you can apply to improve application design, debugging, system performance, and support.In Part 1, you will:Understand how core system and management mechanisms work—including the object manager, synchronization, Wow64, Hyper-V, and the registryExamine the data structures and activities behind processes, threads, and jobsGo inside the Windows security model to see how it manages access, auditing, and authorizationExplore the Windows networking stack from top to bottom—including APIs, BranchCache, protocol and NDIS drivers, and layered servicesDig into internals hands-on using the kernel debugger, performance monitor, and other tools