Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications.Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

    Tracing the rise, fall, and alarming resurrection of the digital mafia behind the two largest spam pharmacies and countless viruses, phishing, and spyware attacks he delivers the first definitive narrative of the global spam problem and its threat to consumers everywhere.Blending cutting-edge research, investigative reporting, and firsthand interviews, this terrifying true story reveals how we unwittingly invite these digital thieves into our lives every day. From unassuming computer programmers right next door to digital mobsters like "Cosma" who unleashed a massive malware attack that has stolen thousands of Americans' logins and passwords, Krebs uncovers the shocking lengths to which these people will go to profit from our data and our wallets.Not only are hundreds of thousands of Americans exposing themselves to fraud and dangerously toxic products from rogue online pharmacies, but even those who never open junk messages are at risk. As Krebs notes, spammers can—and do—hack into accounts through these emails, harvest personal information like usernames and passwords, and sell them on the digital black market. The fallout from this global epidemic doesn't just cost consumers and companies billions, it costs lives too.Fast-paced and utterly gripping, Spam Nation ultimately proposes concrete solutions for protecting ourselves online and stemming this tidal wave of cybercrime, before it's too late."Krebs's talent for exposing the weaknesses in online security has earned him respect in the IT business and loathing among cybercriminals. His track record of scoops has helped him become the rare blogger who supports himself on the strength of his reputation for hard-nosed reporting." Bloomberg Businessweek

    However, these activities can be viewed as two facets of the same field, and together they have undergone substantial development over the past ten years. In particular, Bayesian methods have grown from a specialist niche to become mainstream, while graphical models have emerged as a general framework for describing and applying probabilistic models. Also, the practical applicability of Bayesian methods has been greatly enhanced through the development of a range of approximate inference algorithms such as variational Bayes and expectation propagation. Similarly, new models based on kernels have had a significant impact on both algorithms and applications. This new textbook reflects these recent developments while providing a comprehensive introduction to the fields of pattern recognition and machine learning. It is aimed at advanced undergraduates or first-year PhD students, as well as researchers and practitioners, and assumes no previous knowledge of pattern recognition or machine learning concepts. Knowledge of multivariate calculus and basic linear algebra is required, and some familiarity with probabilities would be helpful though not essential as the book includes a self-contained introduction to basic probability theory.

    You can think of Python as your tool to solve problems that are far beyond the capability of a spreadsheet. It is an easy-to-use and easy-to learn programming language that is freely available on Windows, Macintosh, and Linux computers. There are free downloadable copies of this book in various electronic formats and a self-paced free online course where you can explore the course materials. All the supporting materials for the book are available under open and remixable licenses. This book is designed to teach people to program even if they have no prior experience.

    Dobbs Jolt Award Finalist since Bruce Schneier's Secrets and Lies and Applied Cryptography!Adam Shostack is responsible for security development lifecycle threat modeling at Microsoft and is one of a handful of threat modeling experts in the world. Now, he is sharing his considerable expertise into this unique book. With pages of specific actionable advice, he details how to build better security into the design of systems, software, or services from the outset. You'll explore various threat modeling approaches, find out how to test your designs against threats, and learn effective ways to address threats that have been validated at Microsoft and other top companies.Systems security managers, you'll find tools and a framework for structured thinking about what can go wrong. Software developers, you'll appreciate the jargon-free and accessible introduction to this essential skill. Security professionals, you'll learn to discern changing threats and discover the easiest ways to adopt a structured approach to threat modeling.Provides a unique how-to for security and software developers who need to design secure products and systems and test their designs Explains how to threat model and explores various threat modeling approaches, such as asset-centric, attacker-centric and software-centric Provides effective approaches and techniques that have been proven at Microsoft and elsewhere Offers actionable how-to advice not tied to any specific software, operating system, or programming language Authored by a Microsoft professional who is one of the most prominent threat modeling experts in the world As more software is delivered on the Internet or operates on Internet-connected devices, the design of secure software is absolutely critical. Make sure you're ready with Threat Modeling: Designing for Security.

    But that’s old hat for today’s researchers, who are creating technologies that will have a much deeper impact on the quality of people’s lives over the next quarter century.   In this exhilarating tour of the Media Lab's inner sanctums, we'll meet the professors and their students - the Sorcerers and their Apprentices - and witness first hand the creative magic behind inventions such as:  * Nexi, a mobile humanoid robot with such sophisticated social skills she can serve as a helpful and understanding companion for the sick and elderly. * CityCar, a foldable, stackable, electric vehicle of the future that will redefine personal transportation in cities and revolutionize urban life. * Sixth Sense, a compact wearable device that transforms any surface – wall, tabletop or even your hand - into a touch screen computer.* PowerFoot, a lifelike robotic prosthesis that enables amputees to walk as naturally as if it were a real biological limb.  Through inspiring stories of people who are using Media Lab innovations to confront personal challenges - like a man with cerebral palsy who is unable to hum a tune or pick up an instrument yet is using an ingenious music composition system to unleash his “inner Mozart”, and a woman with a rare life-threatening condition who co-invented a revolutionary web service that enables patients to participate in the search for their own cures - we’ll see how the Media Lab is empowering us all with the tools to take control of our health, wealth, and happiness.   Along the way, Moss reveals the highly unorthodox approach to creativity and invention that makes all this possible, explaining how the Media Lab cultivates an open and boundary-less environment where researchers from a broad array of disciplines – from musicians to neuroscientists to visual artists to computer engineers - have the freedom to follow their passions and take bold risks unthinkable elsewhere.  The Sorcerers and Their Apprentices can serve as a blueprint for how to fix our broken innovation ecosystem and bring about the kind of radical change required to meet the challenges of the 21st century.  It is a must-read for anyone striving to be more innovative as an individual, as a businessperson, or as a member of society. Also includes 16 pages of color photos highlighting some of the lab's most visually stunning inventions - and the people who make them possible.

    When we use consumer products from Google, Microsoft, Facebook, Apple, or Baidu, we are often interacting with a deep learning system. In this volume in the MIT Press Essential Knowledge series, computer scientist John Kelleher offers an accessible and concise but comprehensive introduction to the fundamental technology at the heart of the artificial intelligence revolution.Kelleher explains that deep learning enables data-driven decisions by identifying and extracting patterns from large datasets; its ability to learn from complex data makes deep learning ideally suited to take advantage of the rapid growth in big data and computational power. Kelleher also explains some of the basic concepts in deep learning, presents a history of advances in the field, and discusses the current state of the art. He describes the most important deep learning architectures, including autoencoders, recurrent neural networks, and long short-term networks, as well as such recent developments as Generative Adversarial Networks and capsule networks. He also provides a comprehensive (and comprehensible) introduction to the two fundamental algorithms in deep learning: gradient descent and backpropagation. Finally, Kelleher considers the future of deep learning—major trends, possible developments, and significant challenges.

    Underlying this trend is a single idea: the belief that our existence is understandable through computation, and more data is enough to help us build a better world.   In actual fact, we are lost in a sea of information, increasingly divided by fundamentalism, simplistic narratives, conspiracy theories, and post-factual politics. Meanwhile, those in power use our lack of understanding to further their own interests. Despite the accessibility of information, we’re living in a new Dark Age.   From rogue financial systems to shopping algorithms, from artificial intelligence to state secrecy, we no longer understand how our world is governed or presented to us. The media is filled with unverifiable speculation, much of it generated by anonymous software, while companies dominate their employees through surveillance and the threat of automation.   In his brilliant new work, leading artist and writer James Bridle excavates the limits of technology and how it aids our understanding of the world. Surveying the history of art, technology, and information systems, he explores the dark clouds that gather over our dreams of the digital sublime.

    Our world is unsettled and rapidly changing, and we face existential risks over the next century. Various outcomes--good and bad--are possible. Yet our approach to the future is characterized by short-term thinking, polarizing debates, alarmist rhetoric, and pessimism. In this short, exhilarating book, renowned scientist and bestselling author Martin Rees argues that humanity's prospects depend on our taking a very different approach to planning for tomorrow.The future of humanity is bound to the future of science and hinges on how successfully we harness technological advances to address our challenges. If we are to use science to solve our problems while avoiding its dystopian risks, we must think rationally, globally, collectively, and optimistically about the long term. Advances in biotechnology, cybertechnology, robotics, and artificial intelligence--if pursued and applied wisely--could empower us to boost the developing and developed world and overcome the threats humanity faces on Earth, from climate change to nuclear war. At the same time, further advances in space science will allow humans to explore the solar system and beyond with robots and AI. But there is no "Plan B" for Earth--no viable alternative within reach if we do not care for our home planet.Rich with fascinating insights into cutting-edge science and technology, this accessible book will captivate anyone who wants to understand the critical issues that will define the future of humanity on Earth and beyond.

    Season 1, comprising 20 essays totaling approximately 30,000 words, is written by Venkatesh Rao and illustrated by Grace Witherell. This inaugural season is an in-depth exploration of Marc Andreessen’s observation that “software is eating the world.”

    Featuring core writings by seminal thinkers in the speculative possibilities of the posthuman condition, essays address key philosophical arguments for and against human enhancement, explore the inevitability of life extension, and consider possible solutions to the growing issues of social and ethical implications and concerns. Edited by the internationally acclaimed founders of the philosophy and social movement of transhumanism, The Transhumanist Reader is an indispensable guide to our current state of knowledge of the quest to expand the frontiers of human nature.

    The Computer Revolution may be even more life-changing than the Industrial Revolution. We can do things with computers that could never be done before, and computers can do things for us that could never be done before.But our love of computers should not cloud our thinking about their limitations.We are told that computers are smarter than humans and that data mining can identify previously unknown truths, or make discoveries that will revolutionize our lives. Our lives may well be changed, but not necessarily for the better. Computers are very good at discovering patterns, but are uselessin judging whether the unearthed patterns are sensible because computers do not think the way humans think.We fear that super-intelligent machines will decide to protect themselves by enslaving or eliminating humans. But the real danger is not that computers are smarter than us, but that we think computers are smarter than us and, so, trust computers to make important decisions for us.The AI Delusion explains why we should not be intimidated into thinking that computers are infallible, that data-mining is knowledge discovery, and that black boxes should be trusted.

    The results of Artificial Intelligence have been invaluable to biologists, psychologists, and linguists in helping to understand the processes of memory, learning, and language from a fresh angle. As a concept, Artificial Intelligence has fuelled and sharpened the philosophical debates concerning the nature of the mind, intelligence, and the uniqueness of human beings. Margaret A. Boden reviews the philosophical and technological challenges raised by Artificial Intelligence, considering whether programs could ever be really intelligent, creative or even conscious, and shows how the pursuit of Artificial Intelligence has helped us to appreciate how human and animal minds are possible.

    Brian Arthur puts forth the first complete theory of the origins and evolution of technology, in a major work that achieves for the invention of new technologies what Darwin’s theory achieved for the emergence of new species. Brian Arthur is a pioneer of complexity theory and the discoverer of the highly influential "theory of increasing returns," which took Silicon Valley by storm, famously explaining why some high-tech companies achieve breakaway success. Now, in this long-awaited and ground-breaking book, he solves the great outstanding puzzle of technology—where do transformative new technologies come from?—putting forth the first full theory of how new technologies emerge and offering a definitive answer to the mystery of why some cultures—Silicon Valley, Cambridge, England in the 1920s—are so extraordinarily inventive. He has discovered that rather than springing from insight moments of individual genius, new technologies arise in a process akin to evolution. Technology evolves by creating itself out of itself, much as a coral reef builds itself from activities of small organisms. Drawing on a wealth of examples, from the most ancient to cutting-edge inventions of today, Arthur takes readers on a delightful intellectual journey, bringing to life the wonders of this process of technological evolution. The Nature of Technology is the work of one of our greatest thinkers at the top of his game, composing a classic for our times that is sure to generate wide acclaim.

    Taking the Three Laws of Robotics literally, Prime Intellect makes every human immortal and provides instantly for every stated human desire. Caroline finds no meaning in this life of purposeless ease, and forgets her emptiness only in moments of violent and profane exhibitionism. At turns shocking and humorous, Prime Intellect looks unflinchingly at extremes of human behavior that might emerge when all limits are removed.