Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications.Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

    As a modern application developer you need to understand the emerging field of data management, both RDBMS and NoSQL. Seven Databases in Seven Weeks takes you on a tour of some of the hottest open source databases today. In the tradition of Bruce A. Tate's Seven Languages in Seven Weeks, this book goes beyond your basic tutorial to explore the essential concepts at the core each technology. Redis, Neo4J, CouchDB, MongoDB, HBase, Riak and Postgres. With each database, you'll tackle a real-world data problem that highlights the concepts and features that make it shine. You'll explore the five data models employed by these databases-relational, key/value, columnar, document and graph-and which kinds of problems are best suited to each. You'll learn how MongoDB and CouchDB are strikingly different, and discover the Dynamo heritage at the heart of Riak. Make your applications faster with Redis and more connected with Neo4J. Use MapReduce to solve Big Data problems. Build clusters of servers using scalable services like Amazon's Elastic Compute Cloud (EC2). Discover the CAP theorem and its implications for your distributed data. Understand the tradeoffs between consistency and availability, and when you can use them to your advantage. Use multiple databases in concert to create a platform that's more than the sum of its parts, or find one that meets all your needs at once.Seven Databases in Seven Weeks will take you on a deep dive into each of the databases, their strengths and weaknesses, and how to choose the ones that fit your needs.What You Need: To get the most of of this book you'll have to follow along, and that means you'll need a *nix shell (Mac OSX or Linux preferred, Windows users will need Cygwin), and Java 6 (or greater) and Ruby 1.8.7 (or greater). Each chapter will list the downloads required for that database.

    You'll find new chapters covering all the important new features that make .NET 4.5 the most comprehensive release yet, including:.NET APIs for Windows 8 style UI appsNew asynchronous task-based model for async operationsHow HTML5 support is being wrapped into C# web applicationsNew programming interfaces for HTTP applications, including improved IPv6 supportExpanded WPF, WCF and WF libraries giving C# more power than ever beforeThis comes on top of award winning coverage of core C# features, both old and new, that have made the previous editions of this book so popular (you'll find everything from generics to pLINQ covered here).The mission of this text is to provide you with a rock-solid foundation in the C# programming language and the core aspects of the .NET platform (assemblies, remoting, Windows Forms, Web Forms, ADO.NET, XML web services, etc.). Once you digest the information presented in these 25 chapters, you'll be in a perfect position to apply this knowledge to your specific programming assignments, and you'll be well equipped to explore the .NET universe on your own terms.

    Led by three renowned internals experts, this classic guide is fully updated for Windows 7 and Windows Server 2008 R2—and now presents its coverage in two volumes.As always, you get critical insider perspectives on how Windows operates. And through hands-on experiments, you’ll experience its internal behavior firsthand—knowledge you can apply to improve application design, debugging, system performance, and support.In Part 1, you will:Understand how core system and management mechanisms work—including the object manager, synchronization, Wow64, Hyper-V, and the registryExamine the data structures and activities behind processes, threads, and jobsGo inside the Windows security model to see how it manages access, auditing, and authorizationExplore the Windows networking stack from top to bottom—including APIs, BranchCache, protocol and NDIS drivers, and layered servicesDig into internals hands-on using the kernel debugger, performance monitor, and other tools

    Generics and the greatly expanded collection libraries have tremendously increased the power of Java 5 and Java 6. But they have also confused many developers who haven't known how to take advantage of these new features.Java Generics and Collections covers everything from the most basic uses of generics to the strangest corner cases. It teaches you everything you need to know about the collections libraries, so you'll always know which collection is appropriate for any given task, and how to use it.Topics covered include:• Fundamentals of generics: type parameters and generic methods• Other new features: boxing and unboxing, foreach loops, varargs• Subtyping and wildcards• Evolution not revolution: generic libraries with legacy clients and generic clients with legacy libraries• Generics and reflection• Design patterns for generics• Sets, Queues, Lists, Maps, and their implementations• Concurrent programming and thread safety with collections• Performance implications of different collectionsGenerics and the new collection libraries they inspired take Java to a new level. If you want to take your software development practice to a new level, this book is essential reading.Philip Wadler is Professor of Theoretical Computer Science at the University of Edinburgh, where his research focuses on the design of programming languages. He is a co-designer of GJ, work that became the basis for generics in Sun's Java 5.0.Maurice Naftalin is Technical Director at Morningside Light Ltd., a software consultancy in the United Kingdom. He has most recently served as an architect and mentor at NSB Retail Systems plc, and as the leader of the client development team of a major UK government social service system."A brilliant exposition of generics. By far the best book on the topic, it provides a crystal clear tutorial that starts with the basics and ends leaving the reader with a deep understanding of both the use and design of generics." Gilad Bracha, Java Generics Lead, Sun Microsystems

    From extended integer types and compound literals to Boolean support and variable-length arrays, this book helps you learn to create practical and real-world applications with C programming. It contains review questions and programming exercises.

    In a self-contained manner it proceeds from mathematical and theoretical considerations to actual practical computer routines. With over 100 new routines bringing the total to well over 300, plus upgraded versions of the original routines, the new edition remains the most practical, comprehensive handbook of scientific computing available today.

    Collected from the series of blog posts starting at: https://bartoszmilewski.com/2014/10/2...Hardcover available at: http://www.blurb.com/b/9008339-catego...

    Built for your brain, this book covers C# 3.0 and Visual Studio 2008, and teaches everything from language fundamentals to advanced topics including garbage collection, extension methods, and double-buffered animation. You'll also master C#'s hottest and newest syntax, LINQ, for querying SQL databases, .NET collections, and XML documents. By the time you're through, you'll be a proficient C# programmer, designing and coding large-scale applications. Every few chapters you will come across a lab that lets you apply what you've learned up to that point. Each lab is designed to simulate a professional programming task, increasing in complexity until-at last-you build a working Invaders game, complete with shooting ships, aliens descending while firing, and an animated death sequence for unlucky starfighters. This remarkably engaging book will have you going from zero to 60 with C# in no time flat.

    Thoroughly updated to include Java 6, the Third Edition of Horstmann's bestselling text helps you absorb computing concepts and programming principles, develop strong problem-solving skills, and become a better programmer, all while exploring the elements of Java that are needed to write real-life programs. A top-notch introductory text for beginners, Big Java, Third Edition is also a thorough reference for students and professionals alike to Java technologies, Internet programming, database access, and many other areas of computer science.Features of the Third Edition: The 'Objects Gradual' approach leads you into object-oriented thinking step-by-step, from using classes, implementing simple methods, all the way to designing your own object-oriented programs. A strong emphasis on test-driven development encourages you to consider outcomes as you write programming code so you design better, more usable programs Helpful "Testing Track" introduces techniques and tools step by step, ensuring that you master one before moving on to the next New teaching and learning tools in WileyPLUS--including a unique assignment checker that enables you to test your programming problems online before you submit them for a grade Graphics topics are developed gradually throughout the text, conveniently highlighted in separate color-coded sections Updated coverage is fully compatible with Java 5 and includes a discussion of the latest Java 6 features

    With the sophisticated algorithms in this book, you can write smart programs to access interesting datasets from other web sites, collect data from users of your own applications, and analyze and understand the data once you've found it.Programming Collective Intelligence takes you into the world of machine learning and statistics, and explains how to draw conclusions about user experience, marketing, personal tastes, and human behavior in general -- all from information that you and others collect every day. Each algorithm is described clearly and concisely with code that can immediately be used on your web site, blog, Wiki, or specialized application. This book explains:Collaborative filtering techniques that enable online retailers to recommend products or media Methods of clustering to detect groups of similar items in a large dataset Search engine features -- crawlers, indexers, query engines, and the PageRank algorithm Optimization algorithms that search millions of possible solutions to a problem and choose the best one Bayesian filtering, used in spam filters for classifying documents based on word types and other features Using decision trees not only to make predictions, but to model the way decisions are made Predicting numerical values rather than classifications to build price models Support vector machines to match people in online dating sites Non-negative matrix factorization to find the independent features in a dataset Evolving intelligence for problem solving -- how a computer develops its skill by improving its own code the more it plays a game Each chapter includes exercises for extending the algorithms to make them more powerful. Go beyond simple database-backed applications and put the wealth of Internet data to work for you. "Bravo! I cannot think of a better way for a developer to first learn these algorithms and methods, nor can I think of a better way for me (an old AI dog) to reinvigorate my knowledge of the details."-- Dan Russell, Google "Toby's book does a great job of breaking down the complex subject matter of machine-learning algorithms into practical, easy-to-understand examples that can be directly applied to analysis of social interaction across the Web today. If I had this book two years ago, it would have saved precious time going down some fruitless paths."-- Tim Wolters, CTO, Collective Intellect

    The book covers lambdas, streams, and functional-style programming. With Java 8's functional features you can now write more concise code in less time, and also automatically benefit from multicore architectures. It's time to dig in!

    The book is a complete revision of the commands section of Sobell's Practical Guide to Linux - a proven best-seller. The book is Linux distribution and release agnostic. It will appeal to users of ALL Linux distributions. Superior examples make this book the the best option on the market! System administrators, software developers, quality assurance engineers and others working on a Linux system need to work from the command line in order to be effective. Linux is famous for its huge number of command line utility programs, and the programs themselves are famous for their large numbers of options, switches, and configuration files. But the truth is that users will only use a limited (but still significant) number of these utilities on a recurring basis, and then only with a subset of the most important and useful options, switches and configuration files. This book cuts through all the noise and shows them which utilities are most useful, and which options most important. And it contains examples, lot's and lot's of examples. programmability. Utilities are designed, by default, to work wtih other utilities within shell programs as a way of automating system tasks. This book contains a superb introduction to Linux shell programming. And since shell programmers need to write their programs in text editors, this book covers the two most popular ones: vi and emacs.

    Understanding Computation explains theoretical computer science in a context you’ll recognize, helping you appreciate why these ideas matter and how they can inform your day-to-day programming.Rather than use mathematical notation or an unfamiliar academic programming language like Haskell or Lisp, this book uses Ruby in a reductionist manner to present formal semantics, automata theory, and functional programming with the lambda calculus. It’s ideal for programmers versed in modern languages, with little or no formal training in computer science.* Understand fundamental computing concepts, such as Turing completeness in languages* Discover how programs use dynamic semantics to communicate ideas to machines* Explore what a computer can do when reduced to its bare essentials* Learn how universal Turing machines led to today’s general-purpose computers* Perform complex calculations, using simple languages and cellular automata* Determine which programming language features are essential for computation* Examine how halting and self-referencing make some computing problems unsolvable* Analyze programs by using abstract interpretation and type systems

    This one's a keeper." --Robert C. Martin "If you want to be an expert in the state of the art in TDD, you need to understand the ideas in this book."--Michael Feathers Test-Driven Development (TDD) is now an established technique for delivering better software faster. TDD is based on a simple idea: Write tests for your code before you write the code itself. However, this simple idea takes skill and judgment to do well. Now there's a practical guide to TDD that takes you beyond the basic concepts. Drawing on a decade of experience building real-world systems, two TDD pioneers show how to let tests guide your development and "grow" software that is coherent, reliable, and maintainable. Steve Freeman and Nat Pryce describe the processes they use, the design principles they strive to achieve, and some of the tools that help them get the job done. Through an extended worked example, you'll learn how TDD works at multiple levels, using tests to drive the features and the object-oriented structure of the code, and using Mock Objects to discover and then describe relationships between objects. Along the way, the book systematically addresses challenges that development teams encounter with TDD--from integrating TDD into your processes to testing your most difficult features. Coverage includes - Implementing TDD effectively: getting started, and maintaining your momentum throughout the project - Creating cleaner, more expressive, more sustainable code - Using tests to stay relentlessly focused on sustaining quality - Understanding how TDD, Mock Objects, and Object-Oriented Design come together in the context of a real software development project - Using Mock Objects to guide object-oriented designs - Succeeding where TDD is difficult: managing complex test data, and testing persistence and concurrency