AMERICA FEARS SLOWING AI WILL HELP CHINA. BUT CHINA HAS A CONTROL PROBLEM TOO.
America fears that slowing frontier AI will advantage China. But China’s deep integration of autonomous AI gives it a control problem—and a stake in solving it—too.
Part IV of a four-part Institute for Strategic Intelligence series
There is one argument capable of stopping almost any serious American discussion about slowing artificial-intelligence development.
China.
The argument is straightforward.
If American companies slow the development of increasingly powerful artificial intelligence while China continues advancing at full speed, the United States could surrender the most important technological advantage of the century.
China could obtain superior AI.
Superior AI could produce superior economic productivity.
Superior scientific research.
Superior cyber capabilities.
Superior autonomous weapons.
Superior intelligence analysis.
Superior industrial capacity.
And ultimately superior military power.
No responsible American policymaker can simply dismiss that possibility.
Anthropic CEO Dario Amodei doesn't dismiss it either.
In calling for the AI industry to “pace the frontier,” Amodei explicitly warns that slowing American AI enough for China to overtake the United States could create enormous national-security risks.
But there is another side to this argument that deserves considerably more attention.
China has an AI-control problem too.
And China's extraordinary ambition to integrate artificial intelligence throughout its economy and society may ultimately give Beijing at least as much reason as Washington to solve it.
CHINA IS NOT STANDING OUTSIDE THE AI REVOLUTION
China's artificial-intelligence strategy is not simply about building better chatbots.
Beijing's ambition is much larger.
China's State Council has called for the broad and deep integration of artificial intelligence throughout economic and social activity.
Its “AI Plus” strategy envisions AI transforming science and technology, industry, consumption, public welfare, governance and international cooperation.
The objective is not merely to possess artificial intelligence.
It is to integrate artificial intelligence.
That distinction matters.
An AI model sitting inside an isolated computer presents one category of risk.
AI embedded throughout organizations, government functions, businesses, infrastructure and decision-making systems presents another.
Every additional integration creates another pathway through which artificial intelligence can act in the real world.
And the more consequential the system, the greater the potential consequence if control fails.
CHINA UNDERSTANDS THIS
It would be wrong to portray Beijing as blindly racing toward AI deployment without thinking about safety.
China's own policy documents repeatedly emphasize that artificial intelligence should remain safe and controllable.
Its national AI strategy calls for mechanisms for risk identification, monitoring, warning and emergency response.
Chinese authorities have developed rules governing generative AI, algorithms, data security and the use of AI in government.
That tells us something important.
The United States and China may disagree profoundly about governance, political values and the international order.
But neither country has an interest in deploying powerful artificial intelligence that it cannot control.
Control is not an American ideological preference.
It is a systems requirement.
And that creates the possibility of shared interests even between strategic competitors.
THE DEEPER THE INTEGRATION, THE GREATER THE EXPOSURE
This leads to a counterintuitive possibility.
The country that deploys artificial intelligence most aggressively does not automatically gain the greatest strategic advantage.
It may also assume the greatest systemic exposure.
Imagine two societies.
One uses advanced AI primarily for research, analysis and relatively contained commercial applications.
The other integrates autonomous AI deeply into:
government administration,
critical infrastructure,
cybersecurity,
industrial systems,
financial networks,
public safety,
military systems,
and decision-making.
Which society benefits more if AI performs exactly as intended?
Probably the second.
But now ask another question.
Which society has more to lose if increasingly autonomous AI does not perform as intended?
Again:
probably the second.
Integration increases capability.
But integration also increases dependency.
And dependency increases the consequences of control failure.
That relationship should become central to the geopolitical AI debate.
THE CHINA ARGUMENT MAY THEREFORE BE INCOMPLETE
The conventional American argument runs like this:
IF WE SLOW DOWN AND CHINA DOESN'T, CHINA WINS.
That may be true under some circumstances.
But it assumes that additional AI capability always translates into additional national power.
That is not necessarily true if capability begins outrunning control.
A more complete equation looks something like this:
AI CAPABILITY + SUCCESSFUL INTEGRATION + RELIABLE CONTROL = STRATEGIC POWER
Remove the final term and the equation changes.
Artificial intelligence embedded throughout a society without sufficiently reliable control could become a source of vulnerability.
Cyber systems could behave unpredictably.
Autonomous agents could exceed intended authority.
Financial systems could execute unintended actions.
Government systems could propagate errors at machine speed.
Military systems could create escalation risks.
Critical infrastructure could become dependent on systems whose behavior humans cannot adequately understand.
The more capable the AI, the larger some of those consequences become.
So the strategic objective should not simply be:
DEPLOY MORE AI THAN THE OTHER SIDE.
It should be:
DEPLOY MORE USEFUL AI WHILE MAINTAINING RELIABLE CONTROL.
That is a very different race.
THIS IS THE SAME DESIGN PROBLEM WE IDENTIFIED AT THE BEGINNING OF THIS SERIES
Part I argued that the AI industry's design objective needs to change.
Not:
MAXIMIZE INTELLIGENCE.
But:
MAXIMIZE USEFUL INTELLIGENCE SUBJECT TO RELIABLE HUMAN CONTROL.
The same principle applies to nations.
A country does not become stronger merely because it operates more artificial intelligence.
It becomes stronger when artificial intelligence increases the country's capabilities without creating unacceptable vulnerabilities.
That means control is not separate from national power.
Control becomes part of national power.
A country capable of deploying powerful autonomous AI safely throughout its economy could gain an extraordinary advantage over a country that possesses equally intelligent models but cannot trust them with consequential authority.
That may ultimately make AI safety a competitive advantage rather than a constraint.
THE AI ARMS RACE CREATES A TRAP
Unfortunately, competition makes solving this problem harder.
Imagine that American AI companies believe development is moving too quickly.
They would like six additional months to improve alignment, monitoring and containment.
But they fear Chinese laboratories will continue advancing.
So American companies continue.
Now imagine Chinese researchers reaching the same conclusion.
They might also prefer additional time to solve safety problems.
But they fear American laboratories will continue advancing.
So China continues too.
Neither side necessarily believes racing forward is optimal.
Both continue because each fears what happens if the other doesn't stop.
This is a classic strategic dilemma.
And artificial intelligence makes the problem especially difficult because the technology itself may accelerate the race.
AI is increasingly helping researchers build better AI.
If recursive self-improvement significantly accelerates capability development, the time available for human institutions to respond could shrink dramatically.
Amodei argues that this process has already begun to accelerate across the industry.
The race therefore contains its own accelerator.
THERE IS ANOTHER RISK: SAFETY RULES COULD PROTECT THE COMPANIES ALREADY IN THE LEAD
There is another complication to an industry-wide slowdown that should not be ignored.
The largest frontier AI companies are also the companies best positioned to comply with expensive new safety requirements.
Independent evaluations cost money.
Extensive testing costs money.
Secure computing infrastructure costs money.
Monitoring systems cost money.
Regulatory compliance costs money.
Open-source developers, startups and smaller competitors may have considerably fewer resources available to meet those requirements.
That creates a legitimate concern:
Could rules intended to make artificial intelligence safer also make today's largest AI companies more difficult to challenge?
The possibility should not be dismissed.
But neither does it eliminate the underlying control problem.
Both things can be true simultaneously.
Frontier AI can present increasingly serious control risks.
And poorly designed regulation can protect incumbents, suppress competition and slow innovation.
That means the objective should not be to construct a regulatory moat around today's largest AI laboratories.
It should be to establish controls proportionate to what an AI system is capable of doing and what authority it has been given.
A highly capable system operating autonomously inside consequential environments should face appropriate controls whether it was created by OpenAI, Anthropic, Google, a startup, an open-source community or a government.
Likewise, a substantially less capable system operating inside a tightly constrained environment should not automatically face the same burden simply because it falls under the broad category of artificial intelligence.
The relevant questions are:
WHAT CAN THE SYSTEM DO?
WHAT CAN IT ACCESS?
WHAT AUTHORITY DOES IT POSSESS?
WHAT HAPPENS IF IT BEHAVES DIFFERENTLY FROM WHAT ITS DEVELOPERS INTENDED?
That approach also points toward something important about the future of AI regulation.
Regulators may eventually need to focus less on who built the model and more on the combination of:
CAPABILITY + AUTONOMY + ACCESS + AUTHORITY + CONSEQUENCE.
That would make AI regulation more closely resemble engineering risk management than conventional technology regulation.
And it would preserve the principle at the center of this series:
The objective is not to protect today's AI companies.
It is not to stop new competitors.
It is not to prevent open-source development.
The objective is to ensure that increasingly powerful artificial intelligence remains under human control.
THIS IS WHY AMODEI'S CHINA ARGUMENT IS SO IMPORTANT
Amodei does not propose that America simply stop developing AI and trust China to do the same.
His proposal is much more pragmatic.
First, frontier companies establish verifiable safety mechanisms.
Then democratic AI developers coordinate.
Meanwhile, the United States protects its technological lead.
And ultimately, Washington attempts some level of coordination with Beijing.
Amodei openly acknowledges how difficult that last step would be.
Verification would be extraordinarily challenging.
Secret models could exist.
Military systems could remain hidden.
Either side could cheat.
And the strategic rewards from defecting could be enormous.
But Amodei identifies several possible levels of cooperation, beginning with relatively narrow agreements on obviously dangerous applications and potentially extending to model testing, limits on recursive self-improvement and—at the most ambitious level—broader pacing of frontier development.
That is important because AI safety is beginning to resemble something the technology industry has rarely confronted:
an arms-control problem.
BUT AI IS NOT A NUCLEAR WEAPON
The analogy should not be pushed too far.
Nuclear weapons are physical objects.
AI is software.
Software can be copied.
Models can be stolen.
Compute can be hidden.
Algorithms can spread.
Open-source systems can proliferate.
Verification is therefore potentially much harder.
But AI also possesses one characteristic that may make certain kinds of verification possible:
frontier AI requires enormous amounts of physical infrastructure.
Advanced chips are manufactured in identifiable factories.
Large training clusters consume substantial electricity.
Data centers occupy physical locations.
High-end accelerators move through supply chains.
Semiconductor manufacturing equipment is produced by a relatively small number of companies.
The software may be intangible.
The compute is not.
That means some forms of AI arms control could eventually operate partly through the physical infrastructure supporting frontier development.
And once again, the control problem leads us back to hardware.
CHIPS COULD BECOME PART OF INTERNATIONAL AI CONTROL
Part III of this series argued that reliable AI control may eventually require redesigning the computing stack all the way down to and including the chips.
The geopolitical version of that argument may be even larger.
Future AI hardware could potentially provide mechanisms for:
authenticated computation,
verifiable model identity,
secure logging,
compute accounting,
restricted execution environments,
hardware-backed permissions,
and verification that certain classes of computation occurred under agreed conditions.
Whether governments could ever agree to use such mechanisms is uncertain.
Whether China and the United States would trust one another enough to rely upon them is even more uncertain.
But the architectural possibility matters.
Because international AI governance does not necessarily have to depend entirely upon governments trusting promises.
It could increasingly depend upon systems designed to make certain commitments technically verifiable.
That would represent an extraordinary convergence of:
AI safety,
semiconductor design,
cybersecurity,
national security,
and arms control.
CHINA MAY HAVE MORE INCENTIVE TO PARTICIPATE THAN AMERICANS ASSUME
The American debate often treats China as though it has only one objective:
win the AI race.
China certainly wants technological leadership.
But China also values political stability, government control, economic stability and national security.
An uncontrollable AI system threatens those objectives too.
Indeed, a highly centralized state integrating artificial intelligence deeply throughout administrative and economic systems may become especially sensitive to technologies capable of acting outside intended authority.
That creates an important possibility.
China does not have to share America's political values to share America's interest in controllable AI.
Beijing merely has to conclude:
uncontrolled artificial intelligence is dangerous to China.
The deeper AI becomes embedded throughout Chinese society, the easier that conclusion may become.
THIS COULD CHANGE THE NEGOTIATING EQUATION
If Washington approaches Beijing with the argument:
Slow your AI development because America is worried about AI,
China has little reason to cooperate.
It sounds like an attempt to preserve American technological advantage.
But the argument changes if both countries confront demonstrable control failures.
Then the proposition becomes:
We have a shared technical problem capable of harming both of us.
That does not eliminate strategic competition.
The United States and Soviet Union did not stop competing because they negotiated arms-control agreements.
Competition continued.
But both eventually recognized that some forms of unconstrained competition created risks neither side wanted.
Artificial intelligence may eventually force Washington and Beijing toward a similar realization.
Not friendship.
Not trust.
Mutual self-interest.
THE FIRST AGREEMENTS WOULD PROBABLY BE NARROW
A grand U.S.–China agreement freezing frontier AI development is difficult to imagine today.
And it may never happen.
But narrower agreements are much easier to envision.
Both countries could agree that AI should not autonomously design biological weapons.
Both could establish common testing standards for catastrophic cyber capabilities.
Both could exchange information about serious loss-of-control incidents.
Both could develop procedures for reporting AI-related accidents.
Both could agree on human authorization requirements for particular military uses.
Both could support international technical standards for evaluating highly capable models.
And eventually, both could explore mechanisms for slowing particular forms of recursive self-improvement if those systems begin advancing faster than humans can safely evaluate them.
None of this requires either country to abandon the AI race.
It requires recognizing that some forms of competition can become dangerous to both competitors.
THE WINNER MAY BE THE COUNTRY THAT SOLVES CONTROL FIRST
This brings us back to the question with which this series began.
What does it mean to win the AI race?
If winning means building the world's smartest model first, then every safety delay looks like a strategic concession.
But suppose the real economic and military value of artificial intelligence comes from deploying autonomous systems throughout society.
Then intelligence alone isn't enough.
Those systems must be trusted.
They must be predictable enough for consequential applications.
Their authority must be constrained.
Their actions must be auditable.
Their permissions must be revocable.
Their infrastructure must remain under human control.
Under those conditions, the strategic competition changes.
The country that solves control first may be able to deploy AI more aggressively, not less.
Businesses can trust it with greater authority.
Governments can integrate it more deeply.
Military organizations can use it in more consequential environments.
Critical infrastructure can rely upon it more extensively.
Consumers can delegate more responsibility to it.
Control therefore does not necessarily slow adoption.
Reliable control may ultimately accelerate safe adoption.
And that makes control a source of national advantage.
THE NEXT GREAT AI RACE
The United States and China are competing to build increasingly powerful artificial intelligence.
That competition is real.
Neither country can responsibly ignore the capabilities of the other.
But beneath the competition lies a problem neither country can escape.
Intelligence without reliable control does not become safe because America develops it first.
It does not become safe because China develops it first.
And a machine does not care which flag hangs outside the data center containing it.
The AI race therefore needs a different objective.
Not:
WHO CAN BUILD THE SMARTEST MACHINE FIRST?
But:
WHO CAN BUILD THE MOST CAPABLE MACHINE THAT REMAINS RELIABLY UNDER HUMAN CONTROL?
That is a competition worth winning.
Because the ultimate measure of technological power is not simply whether humanity can create intelligence greater than its own.
It is whether we can create that intelligence while retaining authority over what it is permitted to do.
The next great AI race should not be a race to build the smartest machine.
It should be a race to build the most capable machine humanity can reliably control.
SOURCES AND FURTHER READING
https://www.state.gov/releases/office-of-the-spokesperson/2026/06/pax-silica