The extinction-level dangers of technological progress

How we prevent technological or scientific black balls in the future

Welcome to Designing the Future,

the newsletter about the future, emerging trends, and how to leverage them today.

In this newsletter, we share:

• an introduction to black ball technologies

• the Vulnerable World Hypothesis

• prevention we can do today

---

About Technological or Scientific Black Balls

Be aware, the current issue of Designing the Future is dark. We’re going to talk about technological or scientific black balls — world-ending stuff. This issue is based on the “The Vulnerable World Hypothesis” created by Nick Bostrom at the Future of Humanity Institute at the University of Oxford. All credits go to him.

Ever thought about newly developed technologies or scientific discoveries in the future, and how they could cause harm to the world? So much harm that it could end the world as we know it?

If so, then you’ve thought about a “Technological or Scientific Black Ball”.

Human discoveries throughout history can be compared to pulling balls out of a giant jar, with each ball representing ideas, discoveries, or inventions. Most of these balls have been white (beneficial to the human race), but there have been balls around in several shades of gray (moderately harmful or mixed blessings). In general, however, when looking at the overall picture, human discoveries have so far brought a lot of positive developments for our human species.

A ball we haven’t picked from our jar yet is a black ball: a technology or scientific finding that by default destroys the civilization that invents it.

---

The Vulnerable World Hypothesis

Resulting from the concept of technological and scientific black balls is the vulnerable world hypothesis. The vulnerable world hypothesis states that a new technology or scientific discovery can be made that results in civilization almost certainly getting destroyed unless rigorous and historically unprecedented preventive policies and global governance are implemented.

“If technological development continues then a set of capabilities will at some point be attained that make the devastation of civilization extremely likely, unless civilization sufficiently exits the semi-anarchic default condition.”

- The Vulnerable World Hypothesis by Nick Bostrom

Good to mention is what we mean by the devastation of civilization. Writer Nick Bostrom takes a wider definition than what most people assume. With the devastation of civilization, we mean that the devastation itself is catastrophic enough so that the reasonable objections to prevent the devastation far outweigh the drastic developments needed to prevent it.

So, what are the types of vulnerabilities resulting from black balls?

• Type-1 (‘easy nukes’)

We develop a technology or make a discovery that is so destructive and so easy to use that, given the way that the world works today, it makes civilizational devastation extremely likely. For example, a situation where the average joe can make a nuclear weapon in his kitchen sink.

• Type- 2a (‘safe first strike’)

There is a technology that strongly incentivizes powerful actors to use their powers to cause mass destruction. For example, a key reason countries have nuclear weapons at the moment is so other countries fear using their weapons in your country. However, it is possible that this fear of mutual destruction has become irrelevant because of a newly developed technology. This makes the use of weapons of mass destruction more likely because of, for example, possible economic gains.

• Type-2b (‘worse global warming’)

There is some level of technology in which a great many actors face incentives to take some slightly damaging action such that the combined effect of those actions is civilizational devastation. For example, a new technology is developed that worsens global warming on a much faster scale, but the incentive to continue using this technology is also very high.

• Type-0 (‘surprising strangelets’)

There is some technology that carries a hidden risk such that the default outcome when it is discovered is the devastation of our civilization. This type of vulnerability is not about intentional use, but mainly about a hidden risk that is discovered while, for example, testing a technology. An example includes the research done at CERN in Switzerland. A few years ago, there were rumors about how the new particle accelerator of CERN would cause black holes that would consume the earth. Fortunately, this didn’t happen, but it is a classic example of a Type-0 vulnerability.

---

Preventing a black ball

Until now, humanity has not grabbed a black ball from the jar. The main reason is not that we have been particularly careful or wise in our technology policy. Un-inventing Innovations has not been one of our strengths. So far luck has been on our side.

There are several reasons why it’s hard to prevent a black ball with new policy:

1. Limited capacity for preventive policing. There are insufficient means of real-time surveillance and interception to prevent every small group of people in the world to carry out illegal activities.

2. Limited capacity for global governance. Taking climate change as an example, there is no reliable way yet that we can quickly solve global problems through global coordination. Especially when national security interests are involved.

3. A wide range of motives. Many actors in the world act mainly on self-interest, fearing nothing to gain more money, power, or status. Some of them would even act in ways that destroy civilizations, even when there is a high cost to themselves.

What we can do

However, we are not lost yet. There are several ways we can achieve stability and prevent black balls from happening.

1. Restrict technological development - slow down the development of harmful technologies and accelerate the development of technologies that reduce existential risks.

2. Ensure there is no large population with a wide distribution of motives - this would only work for certain types of vulnerabilities and would provide partial protection.

3. Establish extremely effective preventive policy - for example, prevent dangerous information from spreading, establish surveillance mechanisms, or restrict access to materials needed to build the technology.

4. Establish effective global governance - for example by developing a strong capacity among countries to solve the most serious global problems together.

---

Hope you found our writing interesting and will learn from every new piece of writing we publish. Not subscribed yet?

Until next time,

Koen Vegter

---

Please note that this article is not about stating that the WHDHD hypothesis is true. It is merely used here to surface important considerations and possibilities regarding humanity’s macro-strategic considerations.

Bostrom, N. (2019). The vulnerable world hypothesis. Global Policy, 10(4), 455-476.

This article is based on the excellent work of Nick Bostrom in his paper “The Vulnerable World Hypothesis”, written in 2019. It is a simplification of the extensive research of Nick Bostrom, which you can read in the whole paper here.