An apparent artificial intelligence-generated error led the United States military to be at-risk of conflict with China. CNN has reported that U.S. forces were preparing to intercept and search a Chinese vessel in the Middle East after an intelligence report indicated it was transporting equipment associated with a nuclear weapons development program. Aircraft were reportedly airborne prior when officials reviewing the intelligence found the claim to be incorrect. A source characterized the intelligence as “entirely false” and stated “it almost started a war.”
This event demonstrates some of the potential dangers associated with utilizing generative AI for critical military decision-making. Generative AI systems can generate convincing but fictional information, commonly referred to as hallucinations.
When these fictitious claims appear within formal intelligence documents, particularly in an authoritative format, they may pose significant danger.
As such, while human review did identify the mistake prior to U.S. forces boarding the Chinese vessel, this near miss illustrates why AI-generated intelligence should be subject to independent verification by humans prior to military actions being initiated.
The uncomfortable question is what happens the next time?
Similar events have occurred prior to this, which is frightening
There are several historical examples dating back to the Cold War era which demonstrate how false information, technological malfunctions or misinterpretation can create an illusion that a nuclear strike is occurring. For example, in 1980, a defective $0.46 microchip caused U.S. warning systems to indicate that a non-existent Soviet missile attack was underway. As a result, military personnel remained on alert until additional sensors confirmed that no missiles were present.
In 1983, Soviet satellites incorrectly identified U.S. launched nuclear missiles approaching Russia. However, officer Stanislav Petrov questioned the warning and appropriately decided it was likely an anomaly.
The problem continued after the Cold War. In 1995, Russian radar initially believed a Norwegian-American scientific rocket was a U.S. submarine-launched missile and consequently activated President Boris Yeltsin’s nuclear command system. Before determining there was no actual attack, officials resolved the issue.
It’s quite evident that technology can generate convincing false alarms, and therefore human verification becomes a crucial last line of defense against potentially catastrophic escalations.
