Military Surgeon Warns AI Weapons Could Kill Civilians in 1 'Kill Box'

2 articles · Updated · The Guardian · Aug 18

Dr Darren Mann said AI is already embedded in weapons systems, making loss of control an immediate battlefield risk that can kill civilians today rather than pose only a future superintelligence threat.
A drone ordered to clear one “kill box,” he argued, cannot reliably distinguish a lawful target from a child inside it, creating a direct risk of indiscriminate killing.
Mann said the safeguard should be a deterministic, auditable system grounded in the Geneva conventions, with its judgments returned to a human rather than left to another opaque AI.
He also warned autonomy is marketed as war without a nation’s own dead, but easier-to-start “bloodless” wars still produce casualties once ground must be taken and held.
The broader test, he wrote, is whether militaries can make AI obey the laws of war; machines that kill without distinction and answer to no one leave no one safe.