How Microsoft Used AI to Uncover Flaws in Biosecurity Tech

A team from Microsoft has exposed a major weakness in worldwide biosecurity systems, showing how widely accessible AI protein design tools could be leveraged to create synthetic toxins capable of eluding existing screening methods.

Their research, published in the journal Science, demonstrated that open-source AI protein design (AIPD) technologies can produce thousands of altered versions of specific toxins by modifying amino acid sequences while maintaining their structure and likely their function.

Most alarmingly, these computer-generated redesigns were found to bypass the screening protocols employed by DNA synthesis firms, revealing what Microsoft brands a significant blind spot in the current biosecurity framework.

A joint effort to respond

This finding led to a 10-month collaborative initiative between Microsoft scientists and DNA synthesis companies to devise and implement a security update resolving the identified vulnerability.

Eric Horvitz, Microsoft’s Chief Scientific Officer, initiated the project based on a single question: "Could today's late-breaking AI protein design tools be used to redesign toxic proteins to preserve their structure – and potentially their function – while evading detection by current screening tools?"

"The answer to that question was yes, they could," Eric confirms.

The research was led by Eric and Senior Applied Bioscientist Bruce Wittmann, who developed new biosecurity “red-teaming” techniques adapted from cybersecurity emergency response models.

Screening systems found lacking

The investigation revealed that current screening software and procedures fall short in identifying what the researchers describe as "paraphrased" versions of concerning protein sequences.

Eric regards AI-driven protein design as a thrilling frontier of AI progress, though he acknowledges the rapid pace of development raises worries about malicious uses.

"Following the launch of the Paraphrase Project, we believe that we've come quite far in characterising and addressing the initial concerns in a relatively short period of time," he notes.

Still, Eric cautions that some obstacles may be difficult to overcome.

"There are multiple ways in which AI could be misused to engineer biology – including areas beyond proteins," he explains. "We expect these challenges to persist, so there will be a continuing need to identify and address emerging vulnerabilities."

A double-edged sword

The study highlights how AI protein design tools are double-edged: they reduce the expertise barrier for biological research while increasing the potential for misuse.

AIPD technologies have legitimate uses, ranging from enhancing proteins in laundry detergents for better stain removal to sophisticated projects developing antidotes for snake venom.

Future uses might include advances in cancer treatment, immune disorder therapies, and early detection of health threats.

Eric emphasises the importance of public understanding about both the advantages and risks.

"I think it's important that everybody understands the power and promise of these AI tools, considering both their incredible potential to enable game-changing breakthroughs in biology and medicine and our collective responsibility to ensure that they benefit society rather than cause harm," he says.

Cross-sector collaboration

An unexpected positive was the swift establishment of an effective cross-sector team that met consistently over the 10-month period.

"It was surprising to see how effectively a cross-sector team could come together so quickly and collaborate so very closely at speed, forming a cohesive group that met regularly for months," Eric reflects.

The team developed and rolled out a fix now used globally by DNA synthesis companies.

Microsoft hopes their work offers guidance on approaches and best practices for others, stressing the importance of investing in innovation alongside safeguards rather than viewing the two as opposing priorities.