The First AI War
What Palantir’s COO Shyam Sankar called the “first AI war” opened, on its first day, with a strike on a school in Minab, Iran, coordinated with his company’s products — killing scores of children and staff in an attack which the UN has now denounced as a possible war crime. Now, according to reporting by Bloomberg, the Pentagon has admitted that over-reliance on Project Maven, an AI decision support system (DSS) built with Palantir, was partly to blame for the strike on the elementary school. Like hundreds of other civilians who lost their lives in the first week of the war, the children of Minab stood in the way of targets generated within hours by Project Maven. Making the situation worse, the Pentagon office charged with overseeing civilian harm was slashed by 90% (from 40 full-time employees to just 9 full-time employees) weeks before the strike, opening the door for critical oversight and widespread civilian harm.
For how much time we spend discussing the possibility of AI killing us all, we spend a surprisingly small portion of it discussing how AI is actually used, today, to kill people. Recent Pentagon admissions regarding the February bombing of the elementary school in Minab, as well as conversations stirring up around the film NAZA by Israeli journalists Yuval Abraham and Rachel Szor — which reveals the large numbers of civilians knowingly killed with autonomous targeting weapons by the IDF in the aftermath of October 7 — are finally starting to wake people up to man-made horrors of AI warfare. These stories are bringing the spotlight to an “AI risk” that is not some distant horizon, but which has already been realized: the ruthless killing of civilians using autonomous targeting systems like Project Maven.
For how much time we spend discussing the possibility of AI killing us all, we spend a surprisingly small portion of it discussing how AI is actually used, today, to kill people.
The Pentagon’s obsession with information superiority, speed and lethality (terms I first learned to use in the context of war while marketing software products at Palantir), combined with the rapid adoption of largely untested tools, has opened the door for war crimes like the Minab bombing — as well as a severe lack of accountability. But it could get much worse. Sources recently told CNN that, weeks after the Minab strike, the US Military almost boarded a Chinese vessel in the Middle East due to a faulty, AI-generated report which misidentified the materials it was carrying as nuclear components. The report was put together by an analyst at Special Operations Command — which uses Palantir to integrate AI models — and, according to the source, “almost started a war.”
Unfortunately, the media is not concerned with AI that kills, could kill, or is killing — even if it has already enabled genocide or put us at risk of global war. On the contrary, it is caught in the spell of marketing claims made by executives or press releases about existential risk or rogue agents attacking businesses and government websites. The dominant narratives that are setting policy at the federal level, and commanding the attention of the media, are not concerned with the people most at risk by the introduction of AI to the infrastructure of warfare, immigration enforcement, and surveillance. Instead, those who are profiting from this very infrastructure are the only ones with a seat at the table — both at the endless media circuit of panels, op-eds, and interviews which feed their ideas to the public; and literally at the White House. Recently, Dario Amodei (Anthropic), Mark Zuckerberg (Meta), Jensen Huang (Nvidia), Alex Karp (Palantir), Greg Brockman (OpenAI), Elon Musk (X), Sundar Pichai (Google) and Jeff Bezos (Amazon), among others, met with the President to sign the most vaguely worded agreement possible to “self-police” each other’s companies.
Taken together, these men — because they are all men, which is a major part of the problem — have found a way to command the media’s attention (and buy access to policymaking circles) in a way that obscures far more important contributions to the study of AI risk and ethics, principally by women. Their ability to set the narrative is currently enabling crimes without accountability, and if we are not able to steer away from their fantasies towards a realistic assessment of AI decision support systems and their risks, the consequences could actually be catastrophic for humanity.
How AI Kills People (Today)
When I worked at Palantir for a year and a half after the COVID-19 pandemic, it was my job to illustrate and explain the architecture of AI tools used in war. It was also my job to brand Palantir programs like FedStart, which, after I left, would allow startups like Anthropic to access the government contracts (which DOGE conveniently opened up as it fired thousands of government employees.) Pentagon projects like Project Maven, built with Palantir, are now taking the proliferation of AI technology across the armed forces to unprecedented levels. Today, Project Maven is making AI central to targeting decisions across every branch of the military and introducing new, private generative AI models and agents — such as Anthropic’s Claude and xAI’s Grok — to the military’s lethal decision matrix, known as the ‘AI kill chain.’
But how is AI actually used to target and kill people? Project Maven fuses enormous amounts of data — collected via drone, satellite, cell-tower interception, social media surveillance, and other, more classified types of data collection — with geographic and AI models to build “pattern of life” studies on people. This is done by creating “data mosaics” that link disparate information about targets to the places, events, and people in their lives so they can be more easily found, and possibly, killed. These ISTAR (Intelligence, Surveillance, Target Acquisition and Reconnaissance) tools can generate thousands of targets quickly based on parameters that are set by human operators. Palantir’s tools then allow LLMs like Claude to analyze this data, including with the use of agents, to support decisions at every step of the targeting “kill chain”.
This technology has been a central part of the wars in Palestine, Lebanon, Ukraine, Iran, and even the illegal drone strikes on vessels in the Pacific — which targeted innocent fishermen from my birth country, Ecuador, in international waters. ISTAR tools are also behind the Palantir tools ICE uses to capture undocumented immigrants in the United States.
With my background, I came to see Palantir’s AI products as infrastructure that is supplanting human thinking and judgment at some of the most important institutions of government, and replacing it with privately owned tools which can be designed to circumvent democratic decision-making. While these platforms make government agencies dependent on external tools and talent, they give the designers of these platforms increasing influence over how resources are allocated, and potentially — as is the case with the use of Palantir by the DHS — in determining who is an enemy of the state. It is these AI decision-support systems (DSS) which increasingly decide who is deserving of healthcare and humanitarian aid as well as who is a terrorist or a target in war. Worse yet, questioning these systems can now mark you as a “foreign agent” and target for surveillance yourself.
But without understanding how AI systems work, we will not be able to challenge them when they are implemented in our schools, workplaces, neighborhoods, cities and states. This is the reason why I spent the past two years working with organizers, teachers, and constituents in Colorado to fight back against the proliferation of AI surveillance technologies and in support of regulation for AI — but the catastrophic risks are already here. Katrina Mason recently reported that, after bombing the Iranian school, the Pentagon rushed to make product changes to Project Maven. But it’s impossible to have any faith in these systems (or their software updates) when they could be put to use imminently to plan, coordinate, and execute threats such as those Trump put forth a few weeks ago at the United Nations: to wipe out the entire civilization of Iran.
We have placed our focus so far out on AI’s existential problems — often in the service of marketing and regulatory control — that we have lost sight of a much more urgent question: how a software platform like Project Maven, developed by a private contractor, could be used, imminently, to carry out a new genocide several orders of magnitude larger than the one the world witnessed in Gaza — with little to no public resistance or accountability.
A Better Perspective
How do we steer the conversation on AI risks towards those most harmed today? One answer is to stop listening to what a small group of male AI billionaires have to say. While tech-bros — many calling themselves “effective altruists” or “accelerationists” — have been busy arguing hypotheticals on Twitter, Substack, and happy hours in San Francisco, a new generation of writers and academics around the world have been observing the real ways in which AI companies are profoundly harming people — and bringing much-needed nuance to the current conversation on AI risks and governance by algorithm.
In 2011, Laura A. Dickinson published Outsourcing War and Peace, a book about the privatization of military and foreign-aid functions, studying what happens to democratic accountability when public power is exercised through private companies. By the middle of the decade, researchers were tracking a parallel problem in the growing authority of algorithms wielded by those companies: Cathy O’Neil’s Weapons of Math Destruction showed how opaque mathematical systems could reproduce and intensify existing inequalities, while Timnit Gebru’s “The TESCREAL Bundle” (with Emile P. Torres) traced the eugenicist roots underpinning the ongoing quest for algorithmic super-intelligence.
More recently, scholars including Meredith Whittaker, Heidy Khlaaf, Sarah Myers West, Jessica Dorsey, and Sophia Goodfriend have pushed research on algorithms and state power into the military domain. They are asking what happens when commercially developed AI systems are built into the structures that surveil, target, and kill human beings. In their 2024 paper Mind the Gap, Khlaaf, Myers West, and Whittaker argue that debates over hypothetical and catastrophic AI risks have obscured the conversation on the real ISTAR AI tools that are killing people today. Dorsey’s recently published paper, The Warification of International Humanitarian Law and the Artifice of Artificial Intelligence in Decision-Support Systems, introduces the term “warification” to describe how nation-states use AI platforms to interpret international humanitarian law “in a way that allows for the maximum permissible incidental civilian harm in pursuit of military advantage.”
Meanwhile, Sophia Goodfriend’s work has helped trace technologies of AI surveillance and targeting from their testing grounds in Palestine to their use by ICE in the United States today.
Working in Europe today, scholars such as Elke Schwarz and Carissa Véliz are interrogating the narratives that make AI technologies appear neutral, inevitable, or more rational than human judgment. Schwarz has challenged the language of “precision” and “ethical killing,” and described the “sacralization” of AI technology in order to drive investor interest in new technologies. Véliz, meanwhile, has focused on the political power of data and prediction, arguing that more data collection does not guarantee more safety or productivity, but instead, concentrates power in the hands of those who use prediction as a method to bend behavior and reinforce power.
This is a small sample of the many women who are moving the debate away from speculative fears about what AI might become, toward a more useful, and urgent question: who owns, designs, and governs the AI systems through which consequential (lethal and possibly criminal) decisions are being made today? Who is most at risk? And who is responsible for harms? Although they might not have the hundreds of millions of dollars which companies like Palantir and OpenAI have for marketing and lobbying for deregulation, the reality they are describing is becoming impossible to ignore, and the consequences harder to escape.