Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications.Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

    

    But they've often been hidden in plain sight, their inventions and contributions touching our lives in ways we don't even realize.Author Claire L. Evans finally gives these unsung female heroes their due with her social history of the Broad Band, the women who made the internet what it is today. Learn from Ada Lovelace, the daughter of Lord Byron, who wove numbers into the first program for a mechanical computer in 1842. Seek inspiration from Grace Hopper, the tenacious mathematician who democratized computing by leading the charge for machine-independent programming languages after World War II. Meet Elizabeth "Jake" Feinler, the one-woman Google who kept the earliest version of the Internet online, and Stacy Horn, who ran one of the first-ever social networks on a shoestring out of her New York City apartment in the 1980s. Evans shows us how these women built and colored the technologies we can't imagine life without.Join the ranks of the pioneers who defied social convention and the longest odds to become database poets, information-wranglers, hypertext dreamers, and glass ceiling-shattering dot com-era entrepreneurs.

    It’s time we stand face-to-digital-face with the true powers and limitations of the algorithms that already automate important decisions in healthcare, transportation, crime, and commerce. Hello World is indispensable preparation for the moral quandaries of a world run by code, and with the unfailingly entertaining Hannah Fry as our guide, we’ll be discussing these issues long after the last page is turned.

    How can you work with it efficiently? Recently updated for Spark 1.3, this book introduces Apache Spark, the open source cluster computing system that makes data analytics fast to write and fast to run. With Spark, you can tackle big datasets quickly through simple APIs in Python, Java, and Scala. This edition includes new information on Spark SQL, Spark Streaming, setup, and Maven coordinates. Written by the developers of Spark, this book will have data scientists and engineers up and running in no time. You’ll learn how to express parallel jobs with just a few lines of code, and cover applications from simple batch jobs to stream processing and machine learning. Quickly dive into Spark capabilities such as distributed datasets, in-memory caching, and the interactive shell Leverage Spark’s powerful built-in libraries, including Spark SQL, Spark Streaming, and MLlib Use one programming paradigm instead of mixing and matching tools like Hive, Hadoop, Mahout, and Storm Learn how to deploy interactive, batch, and streaming applications Connect to data sources including HDFS, Hive, JSON, and S3 Master advanced topics like data partitioning and shared variables

    Along the way, we encounter black holes, turtles, snakes, dragons, axe-sharpening, and yak-shaving—and take a guided tour through the theories and methods, both brilliant and misguided, that litter the history of software development, from the famous ‘mythical man-month’ to Extreme Programming. Not just for technophiles but for anyone captivated by the drama of invention, Dreaming in Code offers a window into both the information age and the workings of the human mind.

    How can we catch schools that cheat on standardized tests? How does Netflix know which movies you’ll like? What is causing the rising incidence of autism? As best-selling author Charles Wheelan shows us in Naked Statistics, the right data and a few well-chosen statistical tools can help us answer these questions and more.For those who slept through Stats 101, this book is a lifesaver. Wheelan strips away the arcane and technical details and focuses on the underlying intuition that drives statistical analysis. He clarifies key concepts such as inference, correlation, and regression analysis, reveals how biased or careless parties can manipulate or misrepresent data, and shows us how brilliant and creative researchers are exploiting the valuable data from natural experiments to tackle thorny questions.And in Wheelan’s trademark style, there’s not a dull page in sight. You’ll encounter clever Schlitz Beer marketers leveraging basic probability, an International Sausage Festival illuminating the tenets of the central limit theorem, and a head-scratching choice from the famous game show Let’s Make a Deal—and you’ll come away with insights each time. With the wit, accessibility, and sheer fun that turned Naked Economics into a bestseller, Wheelan defies the odds yet again by bringing another essential, formerly unglamorous discipline to life.

    In 1997, she wroteClose to the Machine, the now classic and still definitive account of life as a coder at the birth of what would be a sweeping technological, cultural, and financial revolution.The intervening twenty years has seen, among other things, the rise of the Internet, the ubiquity of once unimaginably powerful computers, and the thorough transformation of our economy and society—as Ullman’s clique of socially awkward West Coast geeks became our new elite, elevated for and insulated by a technical mastery that few could achieve.In Life in Code, Ullman presents a series of essays that unlock and explain—and don’t necessarily celebrate—how we got to now, as only she can, with a fluency and expertise that’s unusual in someone with her humanistic worldview, and with the sharp insight and brilliant prose that are uniquely her own. Life in Code is an essential text toward our understanding of the last twenty years—and the next twenty.

    Increasingly, the decisions that affect our lives--where we go to school, whether we can get a job or a loan, how much we pay for health insurance--are being made not by humans, but by machines. In theory, this should lead to greater fairness: Everyone is judged according to the same rules.But as mathematician and data scientist Cathy O'Neil reveals, the mathematical models being used today are unregulated and uncontestable, even when they're wrong. Most troubling, they reinforce discrimination--propping up the lucky, punishing the downtrodden, and undermining our democracy in the process.

    Master the fundamentals of C++ and object-oriented programming Understand how C++11 features help you write compact and efficient code using concepts such as lambda expressions, move constructors, and assignment operators Learn the Standard Template Library, including containers and algorithms used in most real-world C++ applications Test your knowledge and expertise using exercises at the end of every lesson Learn on your own time, at your own pace: No previous programming experience required Learn C++11, object-oriented programming, and analysis Write fast and powerful C++ programs, compile the source code with a gcc compiler, and create executable files Use the Standard Template Library's (STL) algorithms and containers to write feature-rich yet stable C++ applications Develop sophisticated programming techniques using lambda expressions, smart pointers, and move constructors Learn to expand your program's power with inheritance and polymorphism Master the features of C++ by learning from programming experts Learn C++11 features that allow you to program compact and high-performance C++ applications TABLE OF CONTENTSPART I: THE BASICS LESSON 1: Getting Started with C++11 LESSON 2: The Anatomy of a C++ Program LESSON 3: Using Variables, Declaring Constants LESSON 4: Managing Arrays and Strings LESSON 5: Working with Expressions, Statements, and Operators LESSON 6: Controlling Program Flow LESSON 7: Organizing Code with Functions LESSON 8: Pointers and References Explained PART II: FUNDAMENTALS OF OBJECT-ORIENTED C++ PROGRAMMING LESSON 9: Classes and Objects LESSON 10: Implementing Inheritance LESSON 11: Polymorphism LESSON 12: Operator Types and Operator Overloading LESSON 13: Casting Operators LESSON 14: An Introduction to Macros and Templates PART III: LEARNING THE STANDARD TEMPLATE LIBRARY (STL) LESSON 15: An Introduction to the Standard Template LibraryLESSON 16: The STL String ClassLESSON 17: STL Dynamic Array ClassesLESSON 18: STL list and forward_listLESSON 19: STL Set ClassesLESSON 20: STL Map ClassesPART IV: MORE STL LESSON 21: Understanding Function ObjectsLESSON 22: C++11 Lambda ExpressionsLESSON 23: STL AlgorithmsLESSON 24: Adaptive Containers: Stack and QueueLESSON 25: Working with Bit Flags Using STLPART V: ADVANCED C++ CONCEPTS LESSON 26: Understanding Smart PointersLESSON 27: Using Streams for Input and OutputLESSON 28: Exception HandlingLESSON 29: Going Forward APPENDIXES A: Working with Numbers: Binary and Hexadecimal B: C++ Keywords C: Operator Precedence D: Answers E: ASCII Codes

    In Turing’s Cathedral, George Dyson focuses on a small group of men and women, led by John von Neumann at the Institute for Advanced Study in Princeton, New Jersey, who built one of the first computers to realize Alan Turing’s vision of a Universal Machine. Their work would break the distinction between numbers that mean things and numbers that do things—and our universe would never be the same. Using five kilobytes of memory (the amount allocated to displaying the cursor on a computer desktop of today), they achieved unprecedented success in both weather prediction and nuclear weapons design, while tackling, in their spare time, problems ranging from the evolution of viruses to the evolution of stars. Dyson’s account, both historic and prophetic, sheds important new light on how the digital universe exploded in the aftermath of World War II. The proliferation of both codes and machines was paralleled by two historic developments: the decoding of self-replicating sequences in biology and the invention of the hydrogen bomb. It’s no coincidence that the most destructive and the most constructive of human inventions appeared at exactly the same time.  How did code take over the world? In retracing how Alan Turing’s one-dimensional model became John von Neumann’s two-dimensional implementation, Turing’s Cathedral offers a series of provocative suggestions as to where the digital universe, now fully three-dimensional, may be heading next.

    Like its sister textbook, Discovering Statistics Using R is written in an irreverent style and follows the same ground-breaking structure and pedagogical approach. The core material is enhanced by a cast of characters to help the reader on their way, hundreds of examples, self-assessment tests to consolidate knowledge, and additional website material for those wanting to learn more.

    

    You can use it to write traditional, imperative, object-oriented code. But you can also leverage its higher level of abstraction to take full advantage of modern, multicore systems. Programming Scala will show you how to use this powerful functional programming language to create highly scalable, highly concurrent applications on the Java Platform. The increasing popularity and availability of multicore processors is creating a whole new set of challenges--although you can enjoy true concurrency, you're now faced with higher contention and synchronization issues. Deploying an existing application on a multicore processor may bring out previously hidden concurrency issues. Java's multi-threading facility by itself isn't enough---it's a very low level abstraction. Instead, you need a paradigm that provides a higher level of abstraction to deal with concurrency. It's time to embrace Functional Programming. Scala is a hybrid Object-Oriented/Functional Programming language on the JVM. Using Scala, you can create traditional imperative programs, intermix them with Java code, and at the same time take advantage of higher levels of abstraction. You can use features that lead to concise, highly expressive code that remove the pain of dealing with concurrency.Programming Scala will show you the fundamentals of functional programming using Scala. Very quickly, you'll learn how this statically typed language can give you dynamic capabilities to create concise, scalable, highly capable concurrent code. Pragmatic programmers always use the right tool for the job. For concurrent programming on the Java VM, Scala is the tool, and Programming Scala by award-winning author Venkat Subramaniam is your guide.

    The book will teach you about:* Neural networks, a beautiful biologically-inspired programming paradigm which enables a computer to learn from observational data* Deep learning, a powerful set of techniques for learning in neural networksNeural networks and deep learning currently provide the best solutions to many problems in image recognition, speech recognition, and natural language processing. This book will teach you the core concepts behind neural networks and deep learning.