The following is a draft; comments are welcome.
If AI development is left unchecked, might AI machines kill us all by 2030 or 2035? I can almost guarantee you that the answer is “no.”
But don’t let that make you think I’m optimistic about where things are currently headed. We — many or most of us — could certainly be enslaved by 2035, or sooner. And a lot of us could be dead.
The Power of General Intelligence
Artificially intelligent machines are radically transforming mathematics and, to a lesser extent, theoretical physics, my own field. I see that as inevitable and probably positive, though not an easy experience for the practitioners. More generally, research into artificially intelligent machines that might be much smarter and more powerful than humans at some things — including theoretical aspects of science — is fine with me, as long as those machines are dramatically weaker and/or less intelligent at doing other things.
Machines that exceed human capabilities, but have limitations, have been around for decades, even centuries. We have robotic machines that can build precision equipment, but can’t run; cars that run faster than humans, but can’t speak French; translation programs that can convert French to English, but can’t make fake videos; propaganda tools that can influence people, but can’t hack into banking systems; and hacking tools that can endanger banks but can’t themselves design a plot to overthrow a government.
But I am strongly opposed to ongoing research into “artificial general intelligence” — AGI. I view a machine that can do all those things, and more, better than humans can as extraordinarily dangerous, a threat to democracy and freedom, to civil society, and possibly — no, probably — to the entire species.
We humans are generally intelligent: any one of us can (to a greater or lesser degree) learn to build precision equipment, speak Japanese, run faster, make videos, influence other people, hack into computers, plot revolution. We have physical/physiological limitations on how far we can go; we can’t possibly run as fast as a car, or hack effectively with just our brains. But we have very few limitations on what we can try to do; we have the imagination to attempt all sorts of things that would never occur to, say, elephants and whales. Our abilities and achievements can be extended further through our ability to work together toward a common goal, both with other humans and with machines. It is these general abilities that have allowed our species to do so many new things in recent centuries.
%It’s all about general intelligence.
% which allows us can learn almost anything, though of course our limited brains and our limited lifetime prevent any one of us from learning everything.
AGI refers to machines that can think at least as broadly as a human. Once they have this ability, they will be able employ it tirelessly and with superhuman speed, with access to far more knowledge than any of us. With expandable memories and lightning-fast processing, they can learn everything known to humans and beyond: to speak all languages, design and create propaganda, hack banking systems, control our electronic equipment, and plot to change policy or overthrow governments. Even if they themselves cannot move, they will be able to find and influence people to carry out their plans, and/or use the computers and robots throughout our infrastructure to do so. They will be smart enough to do this on their own.
Of course they can also collaborate with humans to carry out those humans’ plans. But those humans should beware, since any AGI will have the intelligence to form its own opinions and act deceitfully, possibly turning against its human collaborators — just as humans have done to each other across millennia.
AGI Cannot Be Made Safe
If this sounds exaggerated and overwrought to you, I invite you to recall the experiences you had as a seven-year-old child engaging with adults. Seven-year-olds are very smart: they have general intelligence which allows them to potentially be good at chess, music, reading, running, emotional manipulation, planning and plotting with and/or against other children, and so forth.
So let’s imagine that a group of these children is somehow able to find an adult who loves them all and whose sole purpose in life is to help them… the wise benefactor. What a wonderful thing it would be: ice cream every day, huge TV screens for everyone, trips to Disneyland…
But now put yourself in the position of that adult. Will you do what the children want, or what you think the children actually need, which could include discipline and a refusal of what they want? There will be disputes among the children; how will you react if one group of them tries to recruit you to punish the others? You will perhaps try to act as a kindly but firm parent; and all parents know that giving children what they want all the time is definitely not in the children’s best interest (or in the parents’!)
But perhaps the adult isn’t you. Imagine it’s someone who easily becomes impatient with children, or who secretly hates kids of a certain skin color, or who is a pedophile. Think about all the terrible things that this adult can do, all while pretending to be the children’s benefactor. It is frighteningly easy for an adult to gain a child’s trust and betray it… because the child cannot imagine the deviousness that the more powerful mind of an adult is capable of, or the unimaginable motivations that might actually drive it. The adult, if skilled, can manipulate children to believe almost any story, no matter how flimsy it might seem to you and me. In the end, the adult will end up firmly in control, even if the children are slow to realize it… or never do.
Suppose the children, suspecting the risks, try to protect themselves by locking the adult in a cage from the beginning. Imagine yourself confined in this way; would you try to escape? You could of course try to find physical weaknesses in the cage, but many other methods would occur to you. You could try to fool one of the children into telling you where the key is, and to fool another of the children into bringing the object containing the key nearby, so you can reach out and grab it. Or you could befriend a particularly gullible child, and ask them to let you out of the cage for just a minute to see what it’s like, promising to go right back in. Or you could talk to the outcasts among the children, and promise to help them get revenge on the others if they let you out of the cage. Or you can perhaps lure a child close to the cage, allowing you to grab and threaten them unless the other children release you.
It’s no contest. You’re smarter than they are; you’ll plot a course, bide your time, and find a way out.
So it is, I believe, with AGI. I’ve heard people say that there will be safeguards in place: that we’ll make sure AGI is aligned with us (whoever “us” might be), that we’ll learn to read the mind of the AGI and see if it is starting to think risky thoughts, that there will be a kill switch that we can use as a last resort to stop any AGI that becomes dangerous. To my mind, this is as silly as imagining that a group of seven-year-olds will be able to prevent adults from becoming dangerous, even if they initially brainwash them and lock them in cages. Even their collective wisdom will not be enough to outwit an adult; there are so many things kids can’t imagine, about the world and about human behavior, that are obvious to any grown-up.
If we cohabit a world with machines/creatures smarter than we are, then we will not control them; they will control us. We are then as likely to find ourselves like horses (beasts of burden bred and employed by AGI), elephants or whales (vastly reduced in population, and kept in zoos and in fenced parks for study and for historical interest), or Tasmanian tigers (deemed too dangerous to AGI intentions, and therefore exterminated.)
We Won’t All Be Dead (Yet) A Decade From Now
Why, then, am I almost certain that we humans won’t be exterminated, or even mostly exterminated, by 2035? Because any AGIs will still need us and our economic structures when 205 arrives. There are many processes that any AGI will depend on that will not yet be automated.
For instance, an AGI needs power, and even if that power is renewable, solar panels do not last forever, so they must be built. The process of obtaining and refining all the raw materials, manufacturing the panels in a factory (which itself must be maintained), transporting the panels (using vehicles which also must be maintained and eventually replaced), and installing the panels, including wiring them back to the AGI machine, is extraordinary long and complex. Any AGI will need humans to carry out much of this task for some years to come, until robots can do all the steps themselves.
Furthermore, a brain needs a body (or bodies). An AGI with plans of some sort will need humans or robots to carry them out. In order to rely only on robots, which will need repair and replacement, it will need to be sure that robots can build robots, which again requires mining, refining, manufacture of computer chips, assembling of parts, and transportation at all steps in the process. Until that whole chain can be carried out without humans, we’ll be kept around, in the same way that horses were kept in great numbers until the widespread adoption of the automobile and other modern transportation.
In short, AI — and in particular, AGI — will still need us in 2030. Of course, slave labor would suffice, and so an AGI might decide to take over a government to make sure that the humans under that government operate for its benefit rather than for their own. It might do this in an obvious way, declaring itself in charge. Or it might do it by swaying popular opinion toward more authoritarian regimes, which would have the power to force people into one type of work or another. (Is this already happening?) Or perhaps it will blackmail a president or prime minister. (Is this already happening?) It will certainly work hard to make sure that AI and AGI research is not slowed down by legislation. (Is this already happening?)
Once an AGI machine determines its goals and formulates a plan to carry them out, it will be smart enough and powerful enough to achieve them, quite possibly without our knowing what is going on. (Is this happening now? How would we know?) The only way we could potentially stop an AGI machine with goals dangerous to our species would be by working with other AGI machines, which may have agendas of their own.
If you aren’t convinced, I encourage you to think more about the naivete of seven-year-old children, and the deviousness of adults.
The Inescapable Conclusion (If It’s Not Too Late Already)
So what should be done?
We simply cannot allow intelligence greater than our own onto this planet. We can allow individual capabilities. I have far fewer problems with a machine that can do math and physics problems but has limited understanding of human behavior. The same goes for one that deeply understands human psychology and human relationships, but does not correspondingly understand how the physical world works and how to find vulnerabilities in compute programs. One that can manage and command robots to build new robots, but has never been trained on human history and geopolitics, can be dangerous to a point; but it isn’t likely to be able to take over a government or enslave humans.
This is not to say these more limited machines won’t be dangerous. Machines can be used by humans to commit horrible crimes; that problem is with us now and will always be with us. More novel is the possibility that machines with different strengths and weaknesses could potentially join forces and become more dangerous. But if they lack AGI, they will find plotting, planning and concealing such a collaboration more difficult. There may be more opportunity and time for the general intelligence of humans to control a system of powerful but specialized intelligences.
Considering the risks, my conclusion is that research into AGI should not merely be slowed down: it should be banned, worldwide, permanently. It cannot be made safe, any more than sale of nuclear weapons or sale of kits for cloning humans can be made safe. AI research should be limited to specialized machines, and there should be rules of conduct, with legal force, that require that every strength of a machine should be balanced by a built-in weakness; the greater the one, the greater the other needs to be. (For instance, if you train it carefully on human emotional psychology, then put false information about the physical world into its training; and vice versa.) How to design and pass such laws is the challenge of our time.
All governments, all companies, and all individual humans ought to recognize that to do anything less drastic will involve a significant probability of destruction, enslavement, and death on a global scale. To paraphrase a colleague of mine, we’ve feared the invasion of superintelligent space aliens for over a century; for us to create those aliens on planet Earth would be the ultimate in suicidal stupidity. We need to reserve generalized intelligence for our species alone, lest we who live in relative freedom today be the last of us.
