Big Risks: Pandemics, Nuclear War, Advanced AI

In short: Most of the forces in this section unfold over decades. A few risks are different: they are unlikely in any given year, but could do enormous damage very quickly. A severe pandemic, a nuclear war, or advanced AI that escapes human control could set humanity back for generations. They get little attention and little money compared with the harm they could cause, which makes reducing them some of the most important work there is.

The Question

The other forces are already reshaping society. This article looks at the ones that might not happen at all, but would change everything if they did: which risks could cause a global catastrophe, and what can be done about them?

Why Rare Risks Deserve Attention

We tend to ignore what has not happened yet. But a risk that is small each year adds up over a century. And when the harm would be huge, even a small chance deserves serious effort. That is how we already treat fire, flooding and car crashes: we buy insurance, build defences and set rules, even though most of us never need them.

Pandemics

COVID-19 killed around 7 million people according to official counts, and many more according to estimates of excess deaths, and disrupted life for almost everyone on Earth. Yet it was far from the worst possible pandemic. A virus that spread as easily but was much more deadly, whether natural or made in a lab, could be far worse. As biotechnology becomes cheaper, the ability to create dangerous pathogens spreads too.

What lowers the risk: early warning systems that spot new diseases fast, the ability to make vaccines within months, better ventilation and air cleaning, stockpiles of protective equipment, and strict rules on the most dangerous lab research.

Nuclear War

There are still about 12,000 nuclear warheads in the world, most held by Russia and the United States. A large nuclear war would kill many millions directly. Smoke from burning cities could also cool the planet and ruin harvests for years, threatening billions with hunger. Several near misses during the Cold War came down to luck and the judgement of single individuals.

What lowers the risk: arms control treaties, direct hotlines between rivals, fewer weapons on hair-trigger alert, and keeping more countries from building them.

Advanced AI

AI systems are becoming more capable very fast. See Impact of Artificial Intelligence. Many leading AI researchers warn that systems more capable than humans could become hard to control, or could be misused to design weapons, attack critical systems or concentrate power in very few hands. In 2023, hundreds of AI scientists and industry leaders signed a statement that reducing the risk of extinction from AI should be a global priority, alongside pandemics and nuclear war.

What lowers the risk: research into making AI systems safe and understandable, independent testing before powerful systems are released, international agreements, and public oversight of the most powerful models.

Examples

• Little money for big risks: the philosopher Toby Ord pointed out that the international convention banning biological weapons has had a budget smaller than that of an average McDonald’s restaurant.
• Cooperation is possible: at the height of the Cold War, the United States and the Soviet Union still signed treaties limiting nuclear tests and weapons. Rivals can agree when the danger is shared.

The Human Side

Thinking about catastrophes is uncomfortable, and it is natural to look away. But these risks are not fate. Each of them can be reduced by human choices, and many people are already working on them. Paying attention without panicking is part of taking the future seriously. See Fear management in the Personal Guide.

Open Questions

• How much should societies spend to reduce risks that may never happen?
• Who should decide on research that could be both very useful and very dangerous?
• Can international agreements keep up with fast-moving technologies like AI and biotechnology?

Where Next

• Previous: Life After Automation
• Next: the solutions, starting with Plant-Based Consumption
• Related: Vision & Roadmap

Sources

• Toby Ord, The Precipice: Existential Risk and the Future of Humanity (2020)
• World Health Organization, COVID-19 dashboard and excess mortality estimates
• Stockholm International Peace Research Institute (SIPRI), Yearbook 2025, world nuclear forces
• Xia et al., “Global food insecurity and famine from reduced crop, marine fishery and livestock production due to climate disruption from nuclear war soot injection”, Nature Food (2022)
• Center for AI Safety, “Statement on AI Risk” (2023)
• International AI Safety Report (2025), chaired by Yoshua Bengio