Book picks similar to
Facing the Intelligence Explosion by Luke Muehlhauser


non-fiction
rationality
artificial-intelligence
technology

Smarter Than Us: The Rise of Machine Intelligence


Stuart Armstrong - 2014
    The power of an artificial intelligence (AI) comes from its intelligence, not physical strength and laser guns. Humans steer the future not because we're the strongest or the fastest but because we're the smartest. When machines become smarter than humans, we'll be handing them the steering wheel. What promises—and perils—will these powerful machines present? Stuart Armstrong’s new book navigates these questions with clarity and wit.Can we instruct AIs to steer the future as we desire? What goals should we program into them? It turns out this question is difficult to answer! Philosophers have tried for thousands of years to define an ideal world, but there remains no consensus. The prospect of goal-driven, smarter-than-human AI gives moral philosophy a new urgency. The future could be filled with joy, art, compassion, and beings living worthwhile and wonderful lives—but only if we’re able to precisely define what a "good" world is, and skilled enough to describe it perfectly to a computer program.AIs, like computers, will do what we say—which is not necessarily what we mean. Such precision requires encoding the entire system of human values for an AI: explaining them to a mind that is alien to us, defining every ambiguous term, clarifying every edge case. Moreover, our values are fragile: in some cases, if we mis-define a single piece of the puzzle—say, consciousness—we end up with roughly 0% of the value we intended to reap, instead of 99% of the value.Though an understanding of the problem is only beginning to spread, researchers from fields ranging from philosophy to computer science to economics are working together to conceive and test solutions. Are we up to the challenge?A mathematician by training, Armstrong is a Research Fellow at the Future of Humanity Institute (FHI) at Oxford University. His research focuses on formal decision theory, the risks and possibilities of AI, the long term potential for intelligent life (and the difficulties of predicting this), and anthropic (self-locating) probability. Armstrong wrote Smarter Than Us at the request of the Machine Intelligence Research Institute, a non-profit organization studying the theoretical underpinnings of artificial superintelligence.

Superintelligence: Paths, Dangers, Strategies


Nick Bostrom - 2014
    The human brain has some capabilities that the brains of other animals lack. It is to these distinctive capabilities that our species owes its dominant position. If machine brains surpassed human brains in general intelligence, then this new superintelligence could become extremely powerful--possibly beyond our control. As the fate of the gorillas now depends more on humans than on the species itself, so would the fate of humankind depend on the actions of the machine superintelligence.But we have one advantage: we get to make the first move. Will it be possible to construct a seed Artificial Intelligence, to engineer initial conditions so as to make an intelligence explosion survivable? How could one achieve a controlled detonation?

The Singularity is Near: When Humans Transcend Biology


Ray Kurzweil - 2005
    In his classic The Age of Spiritual Machines, he argued that computers would soon rival the full range of human intelligence at its best. Now he examines the next step in this inexorable evolutionary process: the union of human and machine, in which the knowledge and skills embedded in our brains will be combined with the vastly greater capacity, speed, and knowledge-sharing ability of our creations.

To Be a Machine : Adventures Among Cyborgs, Utopians, Hackers, and the Futurists Solving the Modest Problem of Death


Mark O'Connell - 2017
    It has found adherents in Silicon Valley billionaires Ray Kurzweil and Peter Diamandis. Google has entered the picture, establishing a bio-tech subsidiary aimed at solving the problem of aging.In To Be a Machine, journalist Mark O'Connell takes a headlong dive into this burgeoning movement. He travels to the laboratories, conferences, and basements of today's foremost transhumanists, where he's presented with the staggering possibilities and moral quandaries of new technologies like mind uploading, artificial superintelligence, cryonics, and device implants.A contributor to Slate, The New Yorker, and The New York Times Magazine, O'Connell serves as a sharp and lively guide to the outer limits of technology in the twenty first century. In investigating what it means to be a machine, he offers a surprising, singular meditation on what it means to be human."

The Precipice: Existential Risk and the Future of Humanity


Toby Ord - 2020
    

Life 3.0: Being Human in the Age of Artificial Intelligence


Max Tegmark - 2017
    It doesn't shy away from the full range of viewpoints or from the most controversial issues--from superintelligence to meaning, consciousness and the ultimate physical limits on life in the cosmos.

Human Compatible: Artificial Intelligence and the Problem of Control


Stuart Russell - 2019
    Conflict between humans and machines is seen as inevitable and its outcome all too predictable.In this groundbreaking book, distinguished AI researcher Stuart Russell argues that this scenario can be avoided, but only if we rethink AI from the ground up. Russell begins by exploring the idea of intelligence in humans and in machines. He describes the near-term benefits we can expect, from intelligent personal assistants to vastly accelerated scientific research, and outlines the AI breakthroughs that still have to happen before we reach superhuman AI. He also spells out the ways humans are already finding to misuse AI, from lethal autonomous weapons to viral sabotage.If the predicted breakthroughs occur and superhuman AI emerges, we will have created entities far more powerful than ourselves. How can we ensure they never, ever, have power over us? Russell suggests that we can rebuild AI on a new foundation, according to which machines are designed to be inherently uncertain about the human preferences they are required to satisfy. Such machines would be humble, altruistic, and committed to pursue our objectives, not theirs. This new foundation would allow us to create machines that are provably deferential and provably beneficial.In a 2014 editorial co-authored with Stephen Hawking, Russell wrote, "Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last." Solving the problem of control over AI is not just possible; it is the key that unlocks a future of unlimited promise.

The Alignment Problem: Machine Learning and Human Values


Brian Christian - 2020
    Today’s "machine-learning" systems, trained by data, are so effective that we’ve invited them to see and hear for us?and to make decisions on our behalf. But alarm bells are ringing. Recent years have seen an eruption of concern as the field of machine learning advances. When the systems we attempt to teach will not, in the end, do what we want or what we expect, ethical and potentially existential risks emerge. Researchers call this the alignment problem.Systems cull résumés until, years later, we discover that they have inherent gender biases. Algorithms decide bail and parole?and appear to assess Black and White defendants differently. We can no longer assume that our mortgage application, or even our medical tests, will be seen by human eyes. And as autonomous vehicles share our streets, we are increasingly putting our lives in their hands.The mathematical and computational models driving these changes range in complexity from something that can fit on a spreadsheet to a complex system that might credibly be called “artificial intelligence.” They are steadily replacing both human judgment and explicitly programmed software.In best-selling author Brian Christian’s riveting account, we meet the alignment problem’s “first-responders,” and learn their ambitious plan to solve it before our hands are completely off the wheel. In a masterful blend of history and on-the ground reporting, Christian traces the explosive growth in the field of machine learning and surveys its current, sprawling frontier. Readers encounter a discipline finding its legs amid exhilarating and sometimes terrifying progress. Whether they—and we—succeed or fail in solving the alignment problem will be a defining human story.The Alignment Problem offers an unflinching reckoning with humanity’s biases and blind spots, our own unstated assumptions and often contradictory goals. A dazzlingly interdisciplinary work, it takes a hard look not only at our technology but at our culture—and finds a story by turns harrowing and hopeful.

Our Final Invention: Artificial Intelligence and the End of the Human Era


James Barrat - 2013
    Corporations & government agencies around the world are pouring billions into achieving AI’s Holy Grail—human-level intelligence. Once AI has attained it, scientists argue, it will have survival drives much like our own. We may be forced to compete with a rival more cunning, more powerful & more alien than we can imagine. Thru profiles of tech visionaries, industry watchdogs & groundbreaking AI systems, James Barrat's Our Final Invention explores the perils of the heedless pursuit of advanced AI. Until now, human intelligence has had no rival. Can we coexist with beings whose intelligence dwarfs our own? Will they allow us to?

Who Owns the Future?


Jaron Lanier - 2013
    Who Owns the Future? is his visionary reckoning with the most urgent economic and social trend of our age: the poisonous concentration of money and power in our digital networks.Lanier has predicted how technology will transform our humanity for decades, and his insight has never been more urgently needed. He shows how Siren Servers, which exploit big data and the free sharing of information, led our economy into recession, imperiled personal privacy, and hollowed out the middle class. The networks that define our world—including social media, financial institutions, and intelligence agencies—now threaten to destroy it.But there is an alternative. In this provocative, poetic, and deeply humane book, Lanier charts a path toward a brighter future: an information economy that rewards ordinary people for what they do and share on the web.

AI Superpowers: China, Silicon Valley, and the New World Order


Kai-Fu Lee - 2018
    Kai-Fu Lee—one of the world’s most respected experts on AI and China—reveals that China has suddenly caught up to the US at an astonishingly rapid and unexpected pace.In AI Superpowers, Kai-Fu Lee argues powerfully that because of these unprecedented developments in AI, dramatic changes will be happening much sooner than many of us expected. Indeed, as the US-Sino AI competition begins to heat up, Lee urges the US and China to both accept and to embrace the great responsibilities that come with significant technological power.Most experts already say that AI will have a devastating impact on blue-collar jobs. But Lee predicts that Chinese and American AI will have a strong impact on white-collar jobs as well. Is universal basic income the solution? In Lee’s opinion, probably not.  But he provides a clear description of which jobs will be affected and how soon, which jobs can be enhanced with AI, and most importantly, how we can provide solutions to some of the most profound changes in human history that are coming soon.

The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies


Erik Brynjolfsson - 2014
    Digital technologies—with hardware, software, and networks at their core—will in the near future diagnose diseases more accurately than doctors can, apply enormous data sets to transform retailing, and accomplish many tasks once considered uniquely human.In The Second Machine Age MIT’s Erik Brynjolfsson and Andrew McAfee—two thinkers at the forefront of their field—reveal the forces driving the reinvention of our lives and our economy. As the full impact of digital technologies is felt, we will realize immense bounty in the form of dazzling personal technology, advanced infrastructure, and near-boundless access to the cultural items that enrich our lives.Amid this bounty will also be wrenching change. Professions of all kinds—from lawyers to truck drivers—will be forever upended. Companies will be forced to transform or die. Recent economic indicators reflect this shift: fewer people are working, and wages are falling even as productivity and profits soar.Drawing on years of research and up-to-the-minute trends, Brynjolfsson and McAfee identify the best strategies for survival and offer a new path to prosperity. These include revamping education so that it prepares people for the next economy instead of the last one, designing new collaborations that pair brute processing power with human ingenuity, and embracing policies that make sense in a radically transformed landscape.A fundamentally optimistic book, The Second Machine Age alters how we think about issues of technological, societal, and economic progress.

The Age of Em: Work, Love and Life When Robots Rule the Earth


Robin Hanson - 2016
    Train an em to do some job and copy it a million times: an army of workers is at your disposal. When they can be made cheaply, within perhaps a century, ems will displace humans in most jobs. In this new economic era, the world economy may double in size every few weeks. Some say we can't know the future, especially following such a disruptive new technology, but Professor Robin Hanson sets out to prove them wrong. Applying decades of expertise in physics, computer science, and economics, he uses standard theories to paint a detailed picture of a world dominated by ems. While human lives don't change greatly in the em era, em lives are as different from ours as our lives are from those of our farmer and forager ancestors. Ems make us question common assumptions of moral progress, because they reject many of the values we hold dear. Read about em mind speeds, body sizes, job training and career paths, energy use and cooling infrastructure, virtual reality, aging and retirement, death and immortality, security, wealth inequality, religion, teleportation, identity, cities, politics, law, war, status, friendship and love. This book shows you just how strange your descendants may be, though ems are no stranger than we would appear to our ancestors. To most ems, it seems good to be an em.

Rise of the Robots: Technology and the Threat of a Jobless Future


Martin Ford - 2015
    In Rise of the Robots, Silicon Valley entrepreneur Martin Ford argues that this is absolutely not the case. As technology continues to accelerate and machines begin taking care of themselves, fewer people will be necessary. Artificial intelligence is already well on its way to making “good jobs” obsolete: many paralegals, journalists, office workers, and even computer programmers are poised to be replaced by robots and smart software. As progress continues, blue and white collar jobs alike will evaporate, squeezing working- and middle-class families ever further. At the same time, households are under assault from exploding costs, especially from the two major industries—education and health care—that, so far, have not been transformed by information technology. The result could well be massive unemployment and inequality as well as the implosion of the consumer economy itself.In Rise of the Robots, Ford details what machine intelligence and robotics can accomplish, and implores employers, scholars, and policy makers alike to face the implications. The past solutions to technological disruption, especially more training and education, aren't going to work, and we must decide, now, whether the future will see broad-based prosperity or catastrophic levels of inequality and economic insecurity. Rise of the Robots is essential reading for anyone who wants to understand what accelerating technology means for their own economic prospects—not to mention those of their children—as well as for society as a whole.

The AI Does Not Hate You: Superintelligence, Rationality and the Race to Save the World


Tom Chivers - 2019
    But it's also more importantly about a community of people who are trying to think rationally about intelligence, and the places that these thoughts are taking them, and what insight they can and can't give us about the future of the human race over the next few years. It explains why these people are worried, why they might be right, and why they might be wrong. It is a book about the cutting edge of our thinking on intelligence and rationality right now by the people who stay up all night worrying about it.Along the way, we discover why we probably don't need to worry about a future AI resurrecting a perfect copy of our minds and torturing us for not inventing it sooner, but we perhaps should be concerned about paperclips destroying life as we know it; how Mickey Mouse can teach us an important lesson about how to program AI; and how a more rational approach to life could be what saves us all.