Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications.Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

    Then he wrote a groundbreaking operating system and distributed it via the Internet -- for free. Today Torvalds is an international folk hero. And his creation LINUX is used by over 12 million people as well as by companies such as IBM.Now, in a narrative that zips along with the speed of e-mail, Torvalds gives a history of his renegade software while candidly revealing the quirky mind of a genius. The result is an engrossing portrayal of a man with a revolutionary vision, who challenges our values and may change our world.

    Norbert Wiener's classic is one in that small company. Founder of the science of cybernetics—the study of the relationship between computers and the human nervous system—Wiener was widely misunderstood as one who advocated the automation of human life. As this book reveals, his vision was much more complex and interesting. He hoped that machines would release people from relentless and repetitive drudgery in order to achieve more creative pursuits. At the same time he realized the danger of dehumanizing and displacement. His book examines the implications of cybernetics for education, law, language, science, technology, as he anticipates the enormous impact—in effect, a third industrial revolution—that the computer has had on our lives.

    Historically and etymologically richer than its predecessor, it supplies additional background on existing entries and clarifies the murky origins of several important jargon terms (overturning a few long-standing folk etymologies) while still retaining its high giggle value.Sample definitionhacker n. [originally, someone who makes furniture with an axe] 1. A person who enjoys exploring the details of programmable systems and how to stretch their capabilities, as opposed to most users, who prefer to learn only the minimum necessary. 2. One who programs enthusiastically (even obsessively) or who enjoys programming rather than just theorizing about programming. 3. A person capable of appreciating {hack value}. 4. A person who is good at programming quickly. 5. An expert at a particular program, or one who frequently does work using it or on it; as in `a UNIX hacker'. (Definitions 1 through 5 are correlated, and people who fit them congregate.) 6. An expert or enthusiast of any kind. One might be an astronomy hacker, for example. 7. One who enjoys the intellectual challenge of creatively overcoming or circumventing limitations. 8. [deprecated] A malicious meddler who tries to discover sensitive information by poking around. Hence `password hacker', `network hacker'. The correct term is {cracker}.The term 'hacker' also tends to connote membership in the global community defined by the net (see {network, the} and {Internet address}). It also implies that the person described is seen to subscribe to some version of the hacker ethic (see {hacker ethic, the}).It is better to be described as a hacker by others than to describe oneself that way. Hackers consider themselves something of an elite (a meritocracy based on ability), though one to which new members are gladly welcome. There is thus a certain ego satisfaction to be had in identifying yourself as a hacker (but if you claim to be one and are not, you'll quickly be labeled {bogus}). See also {wannabee}.

    Though this tech is still in its infancy, to those on the inside, it is the future. VR will change how we work, how we experience entertainment, how we feel pleasure and other emotions, how we see ourselves, and most importantly, how we relate to each other in the real world. And we will never be the same.Peter Rubin, senior culture editor for Wired and the industry’s go-to authority on the subject, calls it an "intimacy engine." While once we needed another person to feel the sensations of closeness, trust, vulnerability, confidence, and titillation, VR will give us the ability to induce these sensations by ourselves for the first time in human history. This metamorphosis, Rubin argues, is going to have a powerful impact on relationships that will ripple throughout our society and our individual lives.A journey into this uncertain future and a glimpse at the cultural implications and promises of a new reality, Future Presence explores a host of complex questions about what makes us human, what connects us, and what is real. Offering a glimpse into the mind-blowing things happening in universities, labs, and tech companies around the world, Rubin leads readers on an entertaining tour of the weirdest, wildest corners of this fascinating new universe. Describing this book as "half travelogue and half crystal ball", Rubin will:Introduce readers to the creators and consumers of VR technologyShow readers what an experience is like inside the current VR devicesExplain how this technology will upend everything we know about human connection in the futureAt once the incredible, inevitable story of virtual reality’s rise and a look towards the future of our fantasies, Future Presence is a deeply personal examination of what connects us, and an analysis of what relationships, empathy, and sex could look like—sooner than we think.

    Whether you're a budding hobbyist or an engineer, you'll benefit from the perfectly paced lessons that walk you through useful, artistic, and educational exercises that gradually get more advanced. In addition to specific projects, the book shares best practices in programming and design that you can apply to your own projects. Code snippets and schematics will serve as a useful reference for future projects even after you've mastered all the topics in the book.Includes a number of projects that utilize different capabilities of the Arduino, while interfacing with external hardware Features chapters that build upon each other, tying in concepts from previous chapters to illustrate new ones Includes aspects that are accompanied by video tutorials and other multimedia content Covers electrical engineering and programming concepts, interfacing with the world through analog and digital sensors, communicating with a computer and other devices, and internet connectivity Explains how to combine smaller topics into more complex projects Shares downloadable materials and source code for everything covered in the book Projects compatible with many official Arduino boards including Arduino Uno; Arduino Leonardo; Arduino Mega 2560; Arduino Due; Arduino Nano; Arduino Mega ADK; LilyPad Arduino and may work with Arduino-compatible boards such as Freeduino and new third party certified boards such as the Intel Galileo Exploring Arduino takes you on an adventure and provides you with exclusive access to materials not found anywhere else!

    Tracy Kidder got a preview of this world in the late 1970s when he observed the engineers of Data General design and build a new 32-bit minicomputer in just one year. His thoughtful, prescient book, The Soul of a New Machine, tells stories of 35-year-old "veteran" engineers hiring recent college graduates and encouraging them to work harder and faster on complex and difficult projects, exploiting the youngsters' ignorance of normal scheduling processes while engendering a new kind of work ethic.These days, we are used to the "total commitment" philosophy of managing technical creation, but Kidder was surprised and even a little alarmed at the obsessions and compulsions he found. From in-house political struggles to workers being permitted to tease management to marathon 24-hour work sessions, The Soul of a New Machine explores concepts that already seem familiar, even old-hat, less than 20 years later. Kidder plainly admires his subjects; while he admits to hopeless confusion about their work, he finds their dedication heroic. The reader wonders, though, what will become of it all, now and in the future. —Rob Lightner

    But for all the heat of claim and counter-claim, the argument is essentially beside the point: it’s here; it’s everywhere. The real question is, do we direct technology, or do we let ourselves be directed by it and those who have mastered it? “Choose the former,” writes Rushkoff, “and you gain access to the control panel of civilization. Choose the latter, and it could be the last real choice you get to make.” In ten chapters, composed of ten “commands” accompanied by original illustrations from comic artist Leland Purvis, Rushkoff provides cyberenthusiasts and technophobes alike with the guidelines to navigate this new universe.In this spirited, accessible poetics of new media, Rushkoff picks up where Marshall McLuhan left off, helping readers come to recognize programming as the new literacy of the digital age––and as a template through which to see beyond social conventions and power structures that have vexed us for centuries. This is a friendly little book with a big and actionable message. World-renowned media theorist and counterculture figure Douglas Rushkoff is the originator of ideas such as “viral media,” “social currency” and “screenagers.” He has been at the forefront of digital society from its beginning, correctly predicting the rise of the net, the dotcom boom and bust, as well as the current financial crisis. He is a familiar voice on NPR, face on PBS, and writer in publications from Discover Magazine to the New York Times.“Douglas Rushkoff is one of the great thinkers––and writers––of our time.” —Timothy Leary“Rushkoff is damn smart. As someone who understood the digital revolution faster and better than almost anyone, he shows how the internet is a social transformer that should change the way your business culture operates." —Walter Isaacson

    Whether you're building the next killer app, investing in a startup, or simply curious about the technology, this practical book is essential reading.Bitcoin, the first successful decentralized digital currency, is still in its infancy and it's already spawned a multi-billion dollar global economy. This economy is open to anyone with the knowledge and passion to participate. Mastering Bitcoin provides you with the knowledge you need (passion not included).This book includes:A broad introduction to bitcoin--ideal for non-technical users, investors, and business executivesAn explanation of the technical foundations of bitcoin and cryptographic currencies for developers, engineers, and software and systems architectsDetails of the bitcoin decentralized network, peer-to-peer architecture, transaction lifecycle, and security principlesOffshoots of the bitcoin and blockchain inventions, including alternative chains, currencies, and applicationsUser stories, analogies, examples, and code snippets illustrating key technical concepts

    Completely updated to comprehensively reflect the most recent changes to the ISTQB Foundation Syllabus, the book adopts a practical, hands-on approach, covering the fundamental topics that every system and software tester should know. The authors are themselves developers of the ISTQB syllabus and are highly respected international authorities, teachers and authors within the field of software testing.

    Soon, few of us will own our own automobiles and instead will get around in driverless electric vehicles that we summon with the touch of an app. We will be liberated from driving, prevent over 90% of car crashes, provide freedom of mobility to the elderly and disabled, and decrease our dependence on fossil fuels. Autonomy is the story of the maverick engineers and computer nerds who are creating the revolution. Longtime advisor to the Google Self-Driving Car team and former GM research and development chief Lawrence D. Burns provides the perfectly-timed history of how we arrived at this point, in a character-driven and heavily reported account of the unlikely thinkers who accomplished what billion-dollar automakers never dared.Beginning with the way 9/11 spurred the U.S. government to set a million-dollar prize for a series of off-road robot races in the Mojave Desert up to the early 2016 stampede to develop driverless technology, Autonomy is a page-turner that represents a chronicle of the past, diagnosis of the present, and prediction of the future—the ultimate guide to understanding the driverless car and navigating the revolution it sparks.

    This book is both an example-driven programmer's guide and a keep-on-your-desk reference, with new chapters that explain everything you need to know to get the most out of JavaScript, including:Scripted HTTP and Ajax XML processing Client-side graphics using the canvas tag Namespaces in JavaScript--essential when writing complex programs Classes, closures, persistence, Flash, and JavaScript embedded in Java applicationsPart I explains the core JavaScript language in detail. If you are new to JavaScript, it will teach you the language. If you are already a JavaScript programmer, Part I will sharpen your skills and deepen your understanding of the language.Part II explains the scripting environment provided by web browsers, with a focus on DOM scripting with unobtrusive JavaScript. The broad and deep coverage of client-side JavaScript is illustrated with many sophisticated examples that demonstrate how to:Generate a table of contents for an HTML document Display DHTML animations Automate form validation Draw dynamic pie charts Make HTML elements draggable Define keyboard shortcuts for web applications Create Ajax-enabled tool tips Use XPath and XSLT on XML documents loaded with Ajax And much morePart III is a complete reference for core JavaScript. It documents every class, object, constructor, method, function, property, and constant defined by JavaScript 1.5 and ECMAScript Version 3.Part IV is a reference for client-side JavaScript, covering legacy web browser APIs, the standard Level 2 DOM API, and emerging standards such as the XMLHttpRequest object and the canvas tag.More than 300,000 JavaScript programmers around the world have made this their indispensable reference book for building JavaScript applications."A must-have reference for expert JavaScript programmers...well-organized and detailed."-- Brendan Eich, creator of JavaScript

    The doctor-patient relationship--the heart of medicine--is broken: doctors are too distracted and overwhelmed to truly connect with their patients, and medical errors and misdiagnoses abound. In Deep Medicine, leading physician Eric Topol reveals how artificial intelligence can help. AI has the potential to transform everything doctors do, from notetaking and medical scans to diagnosis and treatment, greatly cutting down the cost of medicine and reducing human mortality. By freeing physicians from the tasks that interfere with human connection, AI will create space for the real healing that takes place between a doctor who can listen and a patient who needs to be heard.Innovative, provocative, and hopeful, Deep Medicine shows us how the awesome power of AI can make medicine better, for all the humans involved.

    In today’s economy, we have far too much dismay along with our amazement, and technology bears some of the blame. In this combination of memoir, business strategy guide, and call to action, Tim O'Reilly, Silicon Valley’s leading intellectual and the founder of O’Reilly Media, explores the upside and the potential downsides of today's WTF? technologies. What is the future when an increasing number of jobs can be performed by intelligent machines instead of people, or done only by people in partnership with those machines? What happens to our consumer based societies—to workers and to the companies that depend on their purchasing power? Is income inequality and unemployment an inevitable consequence of technological advancement, or are there paths to a better future? What will happen to business when technology-enabled networks and marketplaces are better at deploying talent than traditional companies? How should companies organize themselves to take advantage of these new tools? What’s the future of education when on-demand learning outperforms traditional institutions? How can individuals continue to adapt and retrain? Will the fundamental social safety nets of the developed world survive the transition, and if not, what will replace them? O'Reilly is "the man who can really can make a whole industry happen," according to Eric Schmidt, Executive Chairman of Alphabet (Google.) His genius over the past four decades has been to identify and to help shape our response to emerging technologies with world shaking potential—the World Wide Web, Open Source Software, Web 2.0, Open Government data, the Maker Movement, Big Data, and now AI. O’Reilly shares the techniques he's used at O’Reilly Media  to make sense of and predict past innovation waves and applies those same techniques to provide a framework for thinking about how today’s world-spanning platforms and networks, on-demand services, and artificial intelligence are changing the nature of business, education, government, financial markets, and the economy as a whole. He provides tools for understanding how all the parts of modern digital businesses work together to create marketplace advantage and customer value, and why ultimately, they cannot succeed unless their ecosystem succeeds along with them.The core of the book's call to action is an exhortation to businesses to DO MORE with technology rather than just using it to cut costs and enrich their shareholders. Robots are going to take our jobs, they say. O'Reilly replies, “Only if that’s what we ask them to do! Technology is the solution to human problems, and we won’t run out of work till we run out of problems." Entrepreneurs need to set their sights on how they can use big data, sensors, and AI to create amazing human experiences and the economy of the future, making us all richer in the same way the tools of the first industrial revolution did. Yes, technology can eliminate labor and make things cheaper, but at its best, we use it to do things that were previously unimaginable! What is our poverty of imagination? What are the entrepreneurial leaps that will allow us to use the technology of today to build a better future, not just a more efficient one? Whether technology brings the WTF? of wonder or the WTF? of dismay isn't inevitable. It's up to us!

    What's more, its potential is nearly limitless--every hour the sun beams down more energy than the world uses in a year. But in Taming the Sun, energy expert Varun Sivaram warns that the world is not yet equipped to harness erratic sunshine to meet most of its energy needs. And if solar's current surge peters out, prospects for replacing fossil fuels and averting catastrophic climate change will dim.Innovation can brighten those prospects, Sivaram explains, drawing on firsthand experience and original research spanning science, business, and government. Financial innovation is already enticing deep-pocketed investors to fund solar projects around the world, from the sunniest deserts to the poorest villages. Technological innovation could replace today's solar panels with coatings as cheap as paint and employ artificial photosynthesis to store intermittent sunshine as convenient fuels. And systemic innovation could add flexibility to the world's power grids and other energy systems so they can dependably channel the sun's unreliable energy.Unleashing all this innovation will require visionary public policy: funding researchers developing next-generation solar technologies, refashioning energy systems and economic markets, and putting together a diverse clean energy portfolio. Although solar can't power the planet by itself, it can be the centerpiece of a global clean energy revolution.A Council on Foreign Relations Book