Confluence Book Note: The Coming Wave by Mustafa Suleyman
Suleyman makes the case that exponential change is coming -- and that we're nowhere near ready for it.
We launched Confluence as a way to share what’s capturing our attention at the intersection of AI and communication. Usually that comes in the form of insights and reflection applied to news, company announcements, published studies, or our own experiences or experiments. Occasionally, however, a book comes along that catches our attention and demands a post of its own. Mustafa Suleyman’s The Coming Wave is one of those books.
Overview
Mustafa Suleyman is the co-founder of DeepMind and CEO of Inflection AI. He’s one of the most accomplished builder-entrepreneurs and one of the deepest thinkers in AI today. The Coming Wave1 is his attempt to raise awareness of—and spur action against—what he characterizes as “the 21st Century’s greatest dilemma”:
The coming wave is defined by two core technologies: artificial intelligence and synthetic biology. Together they will usher in a new dawn for humanity, creating wealth and surplus unlike anything ever seen. And yet their rapid proliferation also threatens to empower a diverse array of bad actors to unleash disruption, instability, and even catastrophe on an unimaginable scale. This wave creates an immense challenge that will define the twenty-first century: our future both depends on these technologies and is imperiled by them.
The first half of the book focuses on the proliferation of technology from a deep historical perspective before providing an overview of the coming wave of technologies and what they might mean for society. The second half of the book then looks at possible implications on the nation state and geopolitics before proposing the initial outline of a framework for containing these technologies.
In our view, the entire book is worth reading. The questions in the second half of the book are important, if daunting. But for our purposes, we’ll focus primarily on the key insights and takeaways from the book’s first half — that is, on the technology itself and on the more micro-level impacts, rather than on broader geopolitical and existential questions.
Notes
The history of technology is a history of proliferation. We should expect the coming wave of technologies to proliferate faster than any previous wave of technologies.
“Almost every foundational technology ever invented, from pickaxes to plows, pottery to photography, phones to planes, and everything in between, follows a single, seemingly immutable law: it gets cheaper and easier to use, and ultimately it proliferates far and wide.”
“Technology has a clear, inevitable trajectory: mass diffusion in great rolling waves … Costs continue to fall. Capabilities rise. Experiment, repeat, use. Grow, improve, adapt. This is the inescapable evolutionary nature of technology.”
As the technology itself improves, proliferation turns into turbo-proliferation: “Since the early 1970s the number of transistors per chip has increased ten-million-fold. Their power has increased by ten orders of magnitude — a seventeen-billion-fold improvement. Fairchild Semiconductor sold one hundred transistors for $150 each in 1958. Transistors are now produced in the tens of trillions per second, at billionths of a dollar per transistor: the fastest, most extensive proliferation in history.”
General-purpose technologies are rare, transformative, and unpredictable in their impact. AI and synthetic biology are the general-purpose technologies that define the coming wave.
“So, what is a wave? Put simply, a wave is a set of technologies coming together around the same time, powered by one or several new general-purpose technologies with profound societal implications. By ‘general-purpose technologies,’ I mean those that enable seismic advances in what human beings can do. Society unfolds in concert with those leaps. We see it over and over; a new piece of technology, like the internal combustion engine, proliferates and transforms everything around it.”
“General-purpose technologies are accelerants. Invention sparks invention. Waves lay the ground for further scientific and technological experimentation, nudging open the doors of possibility.”
“The irony of general-purpose technologies is that, before long, they become invisible and we take them for granted. Language, agriculture, writing — each was a general-purpose technology at the center of an early wave … One major study pegged the number of general-purpose technologies that have emerged over the entire span of human history at just twenty-four … There aren’t many of them, but they matter.”
“The coming wave of technology is built primarily on two general-purpose technologies capable of operating at the grandest and most granular levels alike: artificial intelligence and synthetic biology. For the first time core components of our technological ecosystem directly address two fundamental properties of our world: intelligence and life. In other words, technology is undergoing a phase transition. No longer simply a tool, it’s going to engineer life and rival—and surpass—our own intelligence.”
We should expect dramatic improvements in AI capabilities in the near to medium term. Current unsolved problems should not be confused with lasting limitations.
“We are only beginning to scratch at the profound impact large language models are about to have. If DQN and AlphaGo were the early sign of something lapping at the shore, ChatGPT and LLMs are the first signs of the wave beginning to crash around us. In 1996, thirty-six million people used the internet; this year it will be well over five billion. That’s the kind of trajectory we should expect for these tools, only much faster. Over the next few years, I believe, AI will become as ubiquitous as the internet itself: just as available, and yet even more consequential.”
“Significant challenges with real-world applications linger, including material questions of bias and fairness, reproducibility, security vulnerabilities, and legal liability. Urgent ethical gaps and unsolved safety questions cannot be ignored. Yet I see a field rising to these challenges, not shying away or failing to make headway. I see obstacles but also a track record of overcoming them. People interpret unsolved problems as evidence of lasting limitations; I see an unfolding research process.”
“The future of AI is, at least in one sense, fairly easy to predict. Over the next five years, vast resources will continue to be invested. Some of the smartest people on the planet are working on these problems. Orders of magnitude more computation will train the top models. All of this will lead to more dramatic leaps forward, including breakthroughs toward AI that can imagine, reason, plan, and exhibit common sense. It won’t be long before AI can transfer what it ‘knows’ from one domain to another, seamlessly, as humans do. What are now only tentative signs of self-reflection and self-improvement will leap forward. These ACI systems will be plugged into the internet, capable of interfacing with everything humans do, but on a platform of deep knowledge and ability. It will be not just language that they’ve mastered but a bewildering array of tasks, too.”
As capabilities improve, we’ll need a new Turing Test — and a new classification for capable AIs that are not yet Artificial General Intelligence (AGI) or superintelligence.
Today’s large language models are very close to passing the original Turing Test2. One could argue that many of them are capable of passing the Turing test already. In any case, “intelligence is about so much more than just language (or indeed any facet of intelligence taken in isolation). One particularly important dimension is in the ability to take actions. We don’t just care about what a machine can say; we also care about what it can do … What we would really like to know is, can I give an AI an ambiguous, open-ended, complex goal that requires interpretation, judgment, creativity, decision-making, and acting across multiple domains, over an extended time period, and then see the AI accomplish that goal?”
“Rather than getting too distracted by questions of consciousness, then, we should refocus the entire debate around near-term capabilities and how they will evolve in the coming years … While [current models] are already having an enormous impact, they will be dwarfed by what happens as we progress through the next few doublings and as AIs complete complex, multistep end-to-end tasks on their own.”
“I think of this as ‘artificial capable intelligence’ (ACI), the point at which AI can achieve complex goals and tasks with minimal oversight. AI and AGI3 are both parts of the everyday discussion, but we need a concept encapsulating a middle layer in which the Modern Turing Test is achieved but before systems display runaway ‘superintelligence.’”
“Conscious superintelligence? Who knows. But highly capable learning systems, ACIs, that can pass some version of the Modern Turing Test? Make no mistake: they are on their way, are already here in embryonic form. There will be thousands of these models, and they will be used by the majority of the world’s population. It will take us to a point where everyone can have an ACI in their pocket that can help or even directly accomplish a vast array of conceivable goals: planning and running your vacation, designing and building more efficient solar panels, helping win an election.”
Four features of the coming wave of technologies will make them particularly hard to contain: Asymmetry, Hyper-Evolution, Omni-Use, and Autonomy.
Asymmetry: A Colossal Transfer of Power — “This new wave of technology has unlocked powerful capabilities that are cheap, easy to access and use, targeted, and scalable … A single AI program can write as much text as all of humanity. A single two-gigabyte image-generation model running on your laptop can compress all the pictures on the open web into a tool that generates images with extraordinary creativity and precision. A single pathogenic experiment could spark a pandemic, a tiny molecular event with global ramifications. One viable quantum computer could render the world’s entire encryption infrastructure redundant. Prospects for asymmetric impact are growing all around, and also in the positive sense—single systems can deliver huge benefits as well.”
Hyper-Evolution: Endless Acceleration — “The next forty years will see both the world of atoms rendered into bits at new levels of complexity and fidelity and, crucially, the world of bits rendered back into tangible atoms with a speed and ease unthinkable until recently … Put simply, innovation in the ‘real world’ could start moving at a digital pace, in near real-time, with reduced friction and fewer dependencies.”
Omni-Use: More is More — “A more appropriate term for the technologies of the coming wave is ‘omni-use,’ a concept that grasps at the sheer levels of generality, the extreme versatility on display. Omni-use technologies like steam or electricity have wider societal effects and spillovers than narrower technologies. If AI is indeed the new electricity, then like electricity it will be an on-demand utility that permeates and powers almost every aspect of daily life, society, the economy: a general-purpose technology embedded everywhere.”
Autonomy: Will Humans Be in the Loop? — “For all of history technology has been ‘just’ a tool, but what if the tool comes to life? … Autonomous systems are able to interact with their surroundings and take actions without the immediate approval of humans. For centuries the idea hat technology is somehow running out of control, a self-directed and self-propelling force beyond the realms of human agency, remained a fiction. Not anymore.”
The notes above come primarily from Parts I (Homo Technologicus) and II (The Next Wave) of the book. Parts III (States of Failure) and IV (Through the Wave) are essential to Suleyman's arguments about the societal impacts of this technology and his call to action toward a containment strategy. We chose not to cover those chapters not because they're not important, but because they are not as relevant to the focus of Confluence. For anyone interested in going further on those topics, we do recommend reading the book in its entirety.
Quick Quotes
“Inventions cannot be uninvented or blocked indefinitely, knowledge unlearned or stopped from spreading.”
“The seeming inevitability of waves comes not from the absence of resistance but from demand overwhelming it.”
“There’s a recurrent problem with making sense of progress in AI. We quickly adapt, even to breakthroughs that astound us initially, and within no time they seem routine, even mundane.”
“The boom never lasts, but the raw speculative drive produces lasting change, a new technological substrate.”
“Technology is ultimately political because technology is a form of power. And perhaps the single overriding characteristic of the coming wave is that it will democratize access to power.”
“Openness is the default, imitations are endemic, cost curves relentlessly go down, and barriers to access crumble.”
“Social media created a few giants and a million tribes.”
“AI is both valuable and dangerous precisely because it’s an extension of our best and worst selves.”
“Technology is not a niche; it is a hyper-object dominating human existence.”
“The most urgent task is not to ride or vainly stop the wave but to sculpt it.”
“We are going to live in an epoch where the majority of our daily interactions are not with other people but with AIs.”
AI Disclosure: We used generative AI in creating imagery for this post. We also used it as a proofreader.
Suleyman, Mustafa. (2023). The Coming Wave: Technology, Power, and the Twenty-first Century's Greatest Dilemma. Crown.
Here is GPT-4’s three-sentence description of the Turing Test: The Turing Test is a measure of a machine's ability to exhibit human-like intelligence, specifically the ability to engage in natural language conversation. Proposed by Alan Turing in 1950, the test involves a human evaluator who converses with an unseen interlocutor, which could be either a human or a machine, and then judges which one they are conversing with. If the evaluator cannot reliably distinguish between the machine and human responses, the machine is said to have passed the Turing Test.
Here is GPT-4’s three-sentence description of Artificial General Intelligence (AGI): Artificial General Intelligence (AGI) refers to machines that possess the ability to understand, learn, and apply knowledge across a broad range of tasks, much like a human being. Unlike narrow or specialized AI, which is designed for specific tasks such as image recognition or language translation, AGI has the cognitive flexibility to adapt and solve novel problems. The development of AGI is considered a long-term goal in the field of artificial intelligence, and it has yet to be achieved.