All revisions

Revision #1

System

about 10 hours ago

The AI Bioterrorism Debate: How Real Is the Threat Behind the Alarm?

In early 2024, RAND Corporation published what remains the most cited empirical test of AI-assisted bioterrorism planning. A year later, Anthropic reported its own Claude models provided a 2.53× "uplift" in bioweapons acquisition planning. The divergence between those two findings captures a broader disagreement among biosecurity researchers, intelligence officials, and AI companies about whether large language models have materially changed the risk landscape for biological attacks—or whether the alarm is outrunning the evidence.

The Red-Team Evidence

RAND's 2024 Experiment

RAND assembled 12 cells of three participants each, giving them 80 hours over seven weeks to develop plans for one of four biological attack scenarios. Some cells had access to large language models; others relied solely on the internet [1]. The result: "no statistically significant difference in the viability of plans generated with or without the assistance of the current generation of large language models" [2]. The study concluded that LLM outputs "generally mirrored information readily available on the internet" [2].

OpenAI's GPT-4 Assessment

OpenAI tested 100 participants—50 biology PhDs with wet-lab experience and 50 students with at least one university biology course—randomly assigned to internet-only or internet-plus-GPT-4 conditions [3]. GPT-4 produced a mean accuracy uplift of 0.88 points on a 10-point scale for experts, a difference the researchers characterized as "at most a mild uplift" that was "not large enough to be conclusive" [3][4].

Anthropic's Claude Trials

Anthropic's more recent evaluation told a different story. In a controlled trial where participants had up to two days to draft bioweapons acquisition plans, those with access to Claude 4 models "received much higher scores and developed plans with substantially fewer critical failures compared to the internet-only control group" [5]. The measured uplift of 2.53× was high enough that Anthropic activated its AI Safety Level 3 (ASL-3) protections, meaning the company could not rule out that its model crossed a meaningful biosecurity threshold [6].

A subsequent evaluation of Claude Opus 4.5 found it was "meaningfully more helpful to participants than previous models," producing higher scores and fewer critical errors—though the generated protocols still contained critical errors that rendered them non-viable in practice [6].

What AI Can and Cannot Do

The Knowledge Gap AI Closes

An MIT study demonstrated that non-experts using chatbot tools could identify pandemic-capable pathogens, reverse genetics methods, and pathogen acquisition strategies within a single hour [7]. Anthropic's CEO Dario Amodei warned in 2024 that within "two to three years" AI tools could "greatly widen the range of actors with the technical capability to conduct a large-scale biological attack" [7].

The Physical Bottleneck That Remains

Even skeptics of AI safety alarm acknowledge the knowledge dimension is shifting. But the consensus among multiple research groups is that experimental validation—the actual laboratory work—remains the binding constraint [8].

The process requires biosafety cabinets, fermenters, centrifuges, and reliable pathogen or precursor acquisition channels. A plausible home wet-lab can be established for $16,000–$20,000, and CRISPR gene-editing kits are available online for approximately $85 [5][7]. A 2026 discovery of an illegal home wet-lab in Las Vegas demonstrated that financial and logistical barriers have been crossed in at least one documented case [5].

However, the production of a bioweapon still "requires experimental validation, which is resource-intensive and therefore remains an outstanding bottleneck" [5]. Current AI tools "are not capable of the de novo design of a self-replicating organism such as a virus," and building and testing still demands "an extensive footprint" [5].

A preliminary wet-lab uplift trial by Anthropic in 2024 found that participants with Claude access did not perform measurably better at hands-on laboratory tasks, suggesting that tacit knowledge—the physical intuition gained through years of bench work—may still function as a meaningful barrier, though the researchers cautioned this finding was preliminary [5].

How AI Companies Evaluate the Risk

AI Lab Biorisk Evaluations by Company (2024-2025)
Source: Epoch AI
Data as of Sep 1, 2025CSV

The rigor of biosecurity evaluations varies dramatically across AI labs. According to an analysis by Epoch AI, Anthropic conducted roughly 10 distinct biorisk evaluations for Claude Opus 4, including uplift trials and expert red-teaming. OpenAI reported four biorisk evaluations for its o3/o4-mini models. Google DeepMind conducted four to five evaluations but relied heavily on multiple-choice formats. Meta disclosed virtually no specifics about its CBRNE evaluations for Llama 4 [9].

Critical problems undermine these assessments. Public biosecurity benchmarks like ProtocolQA and WMDP-Bio have "rapidly saturated," making them unreliable indicators of real-world risk [9]. None of the evaluations from OpenAI, Google DeepMind, or Meta specified "clear eval-specific risk thresholds" [9]. Non-public evaluations often omit rubrics, score ranges, and elicitation techniques, making independent verification impossible.

Anthropic's own cross-model testing found that DeepSeek's model had "absolutely no blocks whatsoever" against generating information relevant to bioweapons production [9]—illustrating the uneven landscape of safeguards across the industry.

Both the U.S. Center for AI Standards and Innovation (CAISI) and the UK AI Security Institute have signed agreements with OpenAI and Anthropic for pre-release access to frontier models [10].

Government Assessments

Intelligence Community Posture

The 2025 Annual Threat Assessment from the Office of the Director of National Intelligence addresses AI-enabled biological threats, though much of the specific intelligence assessment remains classified [11]. DARPA's Biological Technologies Office continues to solicit research proposals integrating AI and machine learning for threat detection and countermeasure development, with a new Broad Agency Announcement issued in October 2025 accepting submissions through September 2026 [11].

Federal Response Architecture

No single federal agency has publicly declared AI-enabled bioterrorism an "active operational threat" as opposed to an emerging concern. The FBI's Weapons of Mass Destruction Directorate, DARPA, and the Department of Homeland Security's Countering Weapons of Mass Destruction Office all maintain awareness programs, but public statements treat the risk as evolving rather than imminent.

The CSIS assessment published in 2025 recommended strengthening gene-synthesis screening, imposing identity verification on cloud lab platforms, and establishing audit trails to "prevent or at least detect misuse" [12].

The Case That Risk Is Overstated

Several lines of argument challenge the dominant alarm narrative:

The empirical record is thin. The two largest controlled studies—RAND's 2024 red team and OpenAI's 100-participant trial—both found no statistically significant uplift from current LLMs [1][2][3]. Anthropic's 2.53× finding, while concerning, involved removing all safety guardrails for testing purposes and still produced plans with critical viability errors [5][6].

The internet already provided this information. RAND's study explicitly concluded that LLM outputs mirrored what was already available through conventional internet searches [2]. Critics argue that the relevant comparison is not "AI vs. no information" but "AI vs. a motivated searcher with Google and access to academic literature"—a comparison in which the marginal AI contribution shrinks.

Institutional incentives align with threat inflation. Some analysts note that biosecurity researchers benefit professionally from elevated threat perceptions through increased funding and policy influence, while AI companies may use biological risk as justification for regulatory frameworks that entrench incumbents against open-source competitors. The Centre for International Governance Innovation published an analysis asking "What's the real risk?" and concluded that "further analysis is needed" on whether current capabilities combined with available tools present material danger [13].

Physical barriers dominate. As multiple studies note, knowledge is not the binding constraint. The number of non-state actors with access to biosafety cabinets, fermenters, and reliable pathogen acquisition networks remains extremely small, and the operational security challenges of weaponizing a biological agent without self-exposure remain formidable [5][8].

The Scholarly Response

Research Publications on "AI biosecurity bioweapons"
Source: OpenAlex
Data as of Jan 1, 2026CSV

Academic attention to the AI-biosecurity intersection has grown sharply, with publications rising from 9 papers in 2022 to 62 in 2024 before settling at roughly 60 in 2025 [14]. A 2025 RAND Delphi study convened biology and AI experts who assessed that "in 2025 and the near term, AI is and will likely continue to be an assistive tool rather than an independent driver of biological design" [15]. The expert panel identified biological trade-offs—such as the inverse relationship between transmissibility and environmental stability—as constraints that AI cannot currently circumvent.

However, the same panel expressed "very uncertain" assessments about how rapidly capabilities would evolve after 2027, suggesting the window for regulatory intervention may be narrowing [15].

Which AI Capabilities Concern Experts Most

The Johns Hopkins Center for Health Security convened representatives from the National Security Council, Department of Energy, and leading AI companies in November 2023 to assess threats at the AI-biology intersection [16]. Their recommendations addressed biological data governance, open-source model risks, and export controls.

A 2022 experiment at a pharmaceutical research firm demonstrated that drug-design AI could be "tweaked to design highly toxic chemicals" rather than therapeutics, highlighting dual-use vulnerabilities in specialized biological AI tools beyond general-purpose LLMs [7].

The categories of greatest concern span multiple AI types: protein-folding tools (like AlphaFold) that predict molecular structures, large language models that provide procedural guidance, and automated cloud laboratory platforms that could execute experiments remotely. The combination—where an LLM guides experimental design and a cloud lab executes it—represents what researchers consider the most dangerous convergence [12][5].

The Legal and Regulatory Landscape

Existing Frameworks

The Biological Weapons Convention (BWC), in force since 1975, prohibits the development and stockpiling of biological weapons but lacks a verification mechanism. The U.S. Select Agent Program regulates possession of dangerous pathogens. Neither framework was designed to address AI-generated biological knowledge [17].

Emerging Policy

At the September 2025 UN General Assembly, President Trump announced U.S. intentions to lead "an AI verification system" for the BWC [17]. The BWC's seventh Working Group session in December 2025 produced draft recommendations for an Open-Ended Working Group on Compliance and Verification, with four BWC meetings scheduled in 2026 [17].

The EU plans to introduce a Biotech Act in Q3 2026 (July–September) that would address dual-use AI capabilities in biotechnology [18]. In the United States, the incoming administration in January 2025 revoked the 2023 AI Executive Order, removing reporting requirements for models exceeding certain computational thresholds, though the U.S. AI Safety Institute remains operational [18].

The Office of Science and Technology Policy recommended in May 2024 that computational models capable of designing novel biological agents be subject to oversight, but this recommendation has not been codified into binding regulation [18].

The Governance Gap

Current regulation treats traditional biological materials (pathogens, toxins, equipment) and digital information (AI models, training data, generated protocols) under entirely separate legal frameworks. No existing statute comprehensively governs the intersection—a gap that multiple research organizations have identified as the central policy failure [12][17][18].

What Comes Next

The RAND Delphi panel identified a potential inflection point after 2027 when AI capabilities in biological design may shift from assistive to autonomous [15]. Policymakers face a classic precautionary dilemma: act now based on uncertain projections and risk over-regulation, or wait for clearer evidence and risk being too late.

The empirical picture as of mid-2026 is that AI provides measurable but contested uplift in the planning stages of biological threats, that physical and tacit-knowledge barriers remain the primary constraint on actual weapon production, and that the regulatory architecture has not kept pace with either the technology or the research documenting its dual-use potential.

Sources (18)

  1. [1]
    The Operational Risks of AI in Large-Scale Biological Attacks: A Red-Team Approachrand.org

    RAND red-team study methodology: 12 cells of three participants given 80 hours over seven weeks to develop biological attack plans with and without LLM access.

  2. [2]
    The Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Studyrand.org

    Found no statistically significant difference in the viability of biological attack plans generated with or without LLM assistance.

  3. [3]
    Building an early warning system for LLM-aided biological threat creationopenai.com

    OpenAI study with 100 participants found GPT-4 provides at most a mild uplift (0.88/10 for experts) in biological threat creation accuracy.

  4. [4]
    GPT-4 gives 'a mild uplift' to creating a biochemical weapontheregister.com

    Coverage of OpenAI's finding that GPT-4 provided marginal, non-conclusive assistance in bioweapons-related tasks.

  5. [5]
    Biorisk - Anthropic Red Teamingred.anthropic.com

    Anthropic's biorisk evaluation found 2.53× uplift with Claude 4 models. Participants developed plans with fewer critical failures. Notes physical barriers remain key bottleneck.

  6. [6]
    Do the biorisk evaluations of AI labs actually measure the risk of developing bioweapons?epoch.ai

    Analysis finding most public biosecurity benchmarks have saturated, labs lack clear risk thresholds, and evaluations provide insufficient evidence to confirm or deny bioweapons uplift risk.

  7. [7]
    Biosecurity in the Age of AI: What's the Risk?belfercenter.org

    Harvard Belfer Center analysis noting LLMs have not generated directly actionable bioweapon instructions, while MIT study showed non-experts could identify pathogens and strategies within one hour.

  8. [8]
    AI-Enabled Biological Design and the Risks of Synthetic Biologyncbi.nlm.nih.gov

    NCBI analysis noting AI-enabled tools do not remove the need for physical building and testing, which requires extensive infrastructure footprint.

  9. [9]
    Do the biorisk evaluations of AI labs actually measure the risk of developing bioweapons?epochai.substack.com

    Detailed analysis of evaluation methodologies across Anthropic (~10 evals), OpenAI (4), Google DeepMind (4-5), and Meta (minimal disclosure).

  10. [10]
    Google DeepMind, Microsoft and xAI Sign Agreements for US National Security AI Testingtechnobezz.com

    CAISI and UK AI Security Institute secured pre-release testing agreements with major AI developers for biosecurity and national security evaluations.

  11. [11]
    DARPA Biological Technologies Officedarpa.mil

    DARPA BTO integrates AI/ML into biological threat detection and countermeasure development, with new BAA issued October 2025.

  12. [12]
    Opportunities to Strengthen U.S. Biosecurity from AI-Enabled Bioterrorism: What Policymakers Should Knowcsis.org

    CSIS policy analysis recommending gene-synthesis screening, identity verification for cloud labs, and audit trails to prevent AI-enabled bioterrorism.

  13. [13]
    AI Is Reviving Fears Around Bioterrorism. What's the Real Risk?cigionline.org

    Centre for International Governance Innovation analysis questioning whether AI bioterrorism fears are proportionate to demonstrated capabilities.

  14. [14]
    OpenAlex: AI Biosecurity Bioweapons Publication Trendopenalex.org

    Academic publications on AI and biosecurity grew from 9 papers in 2022 to 62 in 2024, with 211 total papers tracked.

  15. [15]
    Understanding the Theoretical Limits of AI-Enabled Pathogen Design: Insights from a Delphi Studyrand.org

    RAND Delphi study finding AI remains assistive rather than autonomous in biological design through 2027, with expert uncertainty about subsequent capabilities.

  16. [16]
    Johns Hopkins Center for Health Securitycenterforhealthsecurity.org

    Convened November 2023 meeting with NSC, DOE, and major AI companies to assess AI-biosecurity threats and develop policy recommendations.

  17. [17]
    What will be the impact of AI on the bioweapons treaty?thebulletin.org

    Analysis of AI's impact on the Biological Weapons Convention, including U.S. proposal for AI verification system announced at 2025 UN General Assembly.

  18. [18]
    Governing the Unseen: AI, Dual-Use Biology, and the Illusion of Controlmoderndiplomacy.eu

    Analysis of EU Biotech Act planned for Q3 2026 and the revocation of U.S. AI Executive Order reporting requirements in January 2025.