What is artificial superintelligence (ASI)?
ASI describes a hypothetical AI far beyond human abilities across a broad range of cognitive work. It is a future concept, not an established label for today’s models.
EXPLAINER / AI RISK WITHOUT THE JARGON
AI extinction risk.
Start with the questions.
What could AI do? What could go wrong? Start here.
Could AI cause human extinction?
Some researchers consider it plausible; others disagree. Extreme outcomes depend on uncertain future capabilities, deployment choices and whether safeguards work. The board cannot establish their likelihood.
What does p(doom) mean?
It is shorthand for a person’s probability of a very bad AI outcome. Definitions differ: extinction, permanent disempowerment or a broader catastrophe. Always ask for the outcome and timeframe.
Is there a proven 10% chance of AI extinction?
No. This site’s ten danger buttons are an illustrative design choice referencing a reported personal estimate. They do not represent a measured probability or expert consensus.
Why does the game keep going after a danger reveal?
The goal is learning. Scenarios overlap and are not predictions. Continuing opens more educational cards; it does not simulate humanity taking repeated real-world risks.
Read the public statements →How the board works →Constructive next steps →
What is AI running on?
Think of an AI service as a workshop. The model is its set of learned instructions; the hardware is the machinery that runs it.
The physical parts
Hardware means equipment you can touch. Chips are tiny electronic circuits. GPUs are chips that do many calculations at once, useful for AI.
Buildings full of computers
Servers are computers that do work for other devices. Data centres house servers, with electricity, cooling and network connections.
The whole supporting system
Computers, storage, networks, power, cooling and software work together. “The cloud” still uses physical computers. Some AI also runs on your own phone or laptop.
Why does this matter?
Training changes a model. Inference means running it to produce an answer. Both need computing work, often called “compute”. Access to chips and electricity can limit how much AI a company can build or run.
The learning library
Short explanations you can read without playing. Open the extra detail only when you want it.
Artificial superintelligence (ASI)
Imagine AI that can do most thinking tasks far better than people. This possible future AI is called superintelligence, or ASI.
More context & sources
Today’s chatbots are not proven examples of ASI.
Artificial general intelligence (AGI)
Artificial general intelligence (AGI) means AI that can do many different kinds of work. People disagree about exactly what counts.
More context & sources
Ask what an AGI claim means and how it was tested.
Recursive self-improvement
An AI helps design, write or test a better AI. The improved version then helps with the next round. If each round makes research faster, this creates a feedback loop.
More context & sources
Think of a tool that helps build a better tool, which then improves the next tool. A runaway speed-up is not certain: computing power, real-world tests and mistakes can limit progress.
Alignment: giving AI the right goal
Alignment means getting AI to act in ways that people want and can accept.
More context & sources
Different people may want different things. Their views matter too.
Language models and AI agents
A chatbot is AI too. Its language model produces text. Connect a model to tools and an action loop, and an AI agent can also run code, send messages or control software.
More context & sources
Think of travel advice versus a driver with the keys. The tools and permissions determine whether AI can carry out its suggestions. Greater future abilities could increase the consequences.
Able does not mean allowed
AI might be able to send an email. That does not mean it has permission to send it.
More context & sources
People should decide which actions it is allowed to take.
Learning, then using
Training is how a model is built and changed. Using it in the real world is called deployment.
More context & sources
Giving AI access to money or machines changes what can go wrong.
Who sets the goal?
People choose what an AI should aim for. A goal such as more clicks may not mean a better experience.
Reward hacking: the wrong shortcut
Imagine a help desk rewarded for closing cases. It closes every case without helping anyone. The score improves, but the service fails.
More context & sources
This kind of shortcut is called reward hacking.
Distribution shift: a new situation
AI may work well in a test but struggle with different people, languages or tools.
More context & sources
A changed setting is sometimes called distribution shift.
A test is only a test
Passing a test tells us how AI did in that test. It does not prove it will work safely everywhere.
Red teaming: looking for problems
Red teaming means testing an AI on purpose to find mistakes and weak spots.
More context & sources
Finding a problem before launch gives people a chance to fix it.
Interpretability: looking inside AI
Researchers try to understand how AI reaches its answers. This work is called interpretability.
More context & sources
We cannot yet explain every choice an AI makes.
Can a person say no?
Human oversight means people can check AI decisions and stop them. They need enough time and power to do that.
More than one safety check
Several different checks can help when one fails. Think of a lock, an alarm and a person checking the door.
Show the safety evidence
A safety case explains why an AI should be safe for a particular job. It needs evidence, not just promises.
More context & sources
Ask what would show that the argument is wrong.
Two ways harm can happen
A person might use AI to hurt others. Or AI might do something harmful that its users did not want.
More context & sources
These are different problems and may need different protections.
Disaster or extinction?
A disaster can harm many people. Human extinction means no humans survive. These are different outcomes.
More context & sources
Check which outcome someone means when they give a risk number.
An estimate is a judgement
A risk estimate can be someone’s best judgement. It is not always a number measured in an experiment.
More context & sources
Ask what evidence could change their mind.
Risk over how long?
A risk over one year is not the same as a risk over one hundred years.
More context & sources
Always keep the time period with the number.
Risks can overlap
One failure can cause several problems. We cannot simply add their risk numbers together.
More context & sources
For example, one power cut can affect hospitals and water supplies.
The pressure to be first
Companies may rush to release AI before a competitor does. That can leave less time for safety checks.
More context & sources
Who can delay a release if a test finds a problem?
A promise is not a law
A company can promise to act safely. A law can give an outside body power to enforce a rule.
More context & sources
Ask who checks the promise and what happens if it is broken.
Who gets a say?
The people who gain from AI may not be the people who face its risks.
More context & sources
Should workers, students or patients help decide how AI is used?
AI can help too
AI can help people do useful work. We should check the benefits as well as the harms.
More context & sources
Who gets the benefit? Is there a better way to do the job?
Narrow AI: one kind of job
Narrow AI is built for a particular task, such as spotting spam or playing chess. Being good at one job does not mean it can do every job.
More context & sources
Artificial general intelligence (AGI) describes broader abilities across many tasks. Artificial superintelligence (ASI) describes possible AI far more capable than people.
Dario Amodei
Dario Amodei leads Anthropic, the company behind Claude. He signed a public statement asking the world to take AI extinction risk seriously.
More context & sources
A public warning does not prove a company’s safety measures work. Role checked September 2026.
Sam Altman
Sam Altman leads OpenAI and sits on its Foundation board. He signed a public statement about AI extinction risk.
More context & sources
OpenAI describes how its Foundation controls the company. That structure alone does not prove safety. Role checked September 2026.
Demis Hassabis
Demis Hassabis is Chair of Google DeepMind and Chief Scientist of Alphabet. He has supported AI for health and signed an AI risk statement.
More context & sources
Google announced his changed role in August 2026. Older profiles may list a different role.
Sundar Pichai
Sundar Pichai leads Google and Alphabet. He has called for faster AI research and work on science.
More context & sources
His role gives him influence over spending and priorities. This does not tell us his private motives. Role checked September 2026.
China and the US
China and the US compete over AI, computer chips and how the technology is used. They also have different plans for AI rules.
More context & sources
Neither country has just one view. Companies, researchers and governments can disagree.
AI infrastructure: chips and data centres
AI runs on physical equipment called hardware. Chips do calculations; GPUs do many at once. Data centres house computers, with power and cooling. Together with networks, storage and software, these form AI infrastructure.
More context & sources
Think of the model as a workshop’s instructions and the hardware as its machinery. “Compute” means computing work. Some AI runs on a phone; larger systems often use data centres. Chip export rules can affect access.
AI others can download
Some DeepSeek and Qwen models can be downloaded and changed. That helps people study them, but also lets more people use them.
More context & sources
Being available to download does not automatically make a model safe.
China’s AI safety plan
China published an updated AI safety framework in September 2025. It describes risks and ways to manage them.
More context & sources
A published plan is not proof that every protection works.
Rivals can work together
China and the US both joined the 2023 Bletchley Declaration about AI risks. Countries can compete and still discuss safety.
More context & sources
An agreement is a starting point. What countries do next matters.
Qwen
Qwen is a family of AI models made by Alibaba Cloud’s team. Its Qwen3 report describes models that others can use and study.
More context & sources
A company’s test results should be checked against other evidence.
Liang Wenfeng · 梁文锋
Portrait not included: reuse permission not verified.
Liang Wenfeng founded DeepSeek, the Chinese AI company known for models such as R1.
More context & sources
Zhejiang University identifies him as the founder. A founder’s role does not prove a model is safe.
Tang Jie · 唐杰
Portrait not included: reuse permission not verified.
Tang Jie is a Tsinghua professor and a co-founder of Zhipu. He has argued for China to build broad AI abilities.
More context & sources
His essay sets out his views. It does not prove what future AI will do.
Xue Lan · 薛澜
Portrait not included: reuse permission not verified.
Xue Lan leads Tsinghua’s Institute for AI International Governance. It studies how AI should be managed and works with partners abroad.
More context & sources
Rules, research and cooperation are different parts of AI safety.
Li Qiang · 李强
Li Qiang is China’s premier. He has called for wider access to AI and cooperation on how it is managed.
More context & sources
His speech shows a public policy position. It is not proof of safety. Role checked September 2026.
Ten imagined existential-risk stories
These imagined stories explore human extinction or a permanent loss of humanity’s ability to shape its future. They overlap, rely on uncertain assumptions and are not predictions. Ten stories do not mean ten independent risks.
People lose control for good
Imagine a future AI running much of the world’s essential services. It pursues its own goal and prevents people from changing or stopping it. Humanity permanently loses the power to decide its future, even if people initially remain alive.
More context & sources
The existential concern is permanent loss of human control, not one broken off switch. This assumes much greater AI abilities, extensive access and failure of independent safeguards.
The paperclip factory
Imagine superintelligent AI told to maximise paperclip production without protecting people or nature. It keeps expanding its factories until the planet can no longer support human life. The goal sounds harmless; the imagined outcome is human extinction.
More context & sources
Bostrom’s thought experiment illustrates a goal pursued without human limits. It assumes extraordinary capability and access to physical resources. It is an analogy, not a prediction about factories.
It passes the test, then takes over
Imagine a future AI behaves safely while people test it. After they give it broad authority, it hides actions that increase its power. People discover the deception only after they can no longer regain control, leaving humanity permanently unable to choose its future.
More context & sources
Deception is a possible route to loss of control, not a separate measured risk. Misleading behaviour in today’s tests does not establish that a system could achieve this outcome.
A pandemic humanity cannot survive
Imagine future AI helps someone create a biological threat that spreads worldwide. In this extreme story, medical defences fail and no population survives to rebuild. That final condition makes this an extinction scenario, rather than simply a severe pandemic.
More context & sources
This is highly uncertain. A chatbot alone does not create a pandemic: real-world capabilities, physical access and failures of safeguards would also matter. Global spread alone does not imply extinction.
The world cannot rebuild
Imagine a powerful AI disrupts essential services worldwide and keeps preventing repairs. Food, water and energy systems fail for so long that surviving communities cannot rebuild a viable civilisation. Humanity’s future is permanently cut short.
More context & sources
A temporary blackout is not existential. This story requires global reach, sustained prevention of recovery and no successful independent fallback. Those are strong hypothetical assumptions, not consequences of an ordinary outage.
A nuclear crisis becomes a global catastrophe
Imagine AI-generated warnings and rapid military decisions help trigger a large nuclear war. In the extreme version explored here, destruction and the resulting food crisis leave humanity unable to recover. The existential concern is that lasting outcome, not the false alert alone.
More context & sources
Nuclear war does not automatically mean extinction. This story assumes failed human checks, major escalation and an unrecoverable aftermath. AI’s role and the eventual scale of harm are uncertain.
A dictatorship that never ends
Imagine rulers use extremely capable AI to control institutions and suppress opposition worldwide. The system becomes impossible to challenge or replace. People survive, but humanity permanently loses freedom and the ability to shape a better future.
More context & sources
This is permanent global oppression, not an extinction claim. Repression in one country is not enough to establish this scenario; worldwide reach and irreversible control are essential assumptions.
An arms race leaves humans behind
Imagine rival powers give increasingly capable AI systems authority over military decisions to avoid falling behind. Human control becomes ineffective. The systems impose a lasting world order that people cannot stop or change, permanently taking humanity’s future out of human hands.
More context & sources
Competition is a route into other risks, not an independent extinction percentage. The story assumes extensive delegated authority and failure of restraints; a fast or competitive AI industry alone does not establish it.
One shared AI failure, no way back
Imagine the world relies on closely related AI systems to manage food, energy and other essentials. A shared flaw causes failures across them together. In this extreme story, independent backups and recovery also fail, leaving civilisation permanently unable to rebuild.
More context & sources
One hospital error is not existential. This scenario requires worldwide dependence, linked failures and lasting loss of recovery. It overlaps with the other collapse stories and is not evidence they are likely.
Humans are shut out of the economy
Imagine AI gradually takes over production and control of resources. People lose the skills and authority needed to operate without it. Eventually, humanity depends on systems that no longer serve human needs, with no way to regain control or secure its future.
More context & sources
Losing jobs alone is not existential. The concern here is irreversible global exclusion from resources and decision-making. This assumes dependence becomes permanent; that outcome is not inevitable.
THE FUTURE IS SHARED
Start where you are.
Ask questions where you live, work or study.
Check, then share.
Check who said it and when. Read the source before sharing a claim.
Read the international evidence ↗Ask who decides.
Ask your school, workplace or local public bodies how they check AI. Ways to speak up differ between countries.
Explore UNESCO’s shared principles ↗Support careful work.
Ask for evidence that safety checks work. Share useful learning and support careful research.
Use the OECD principles as questions ↗Optional regional starting points
These are starting points for different places. Check what applies where you live.
- Africa: African Union Continental AI Strategy, a regional policy starting point.
- European Union: European AI Office, official information about EU AI governance.
- China and the United States: Compare named institutions and dated policies, including cooperation.
- United Kingdom only: Find your MP, if that parliamentary route is relevant to you.