15 2.6 Proliferation of Conventional and Unconventional Weapons Certain LLM capabilities can have dual-use potential, meaning that the models can be used for “both commercial and military or proliferation applications”.[57] We subjected the model to stress testing, boundary testing, and red teaming16 in four dual-use domains to explore whether our models could provide the necessary information to proliferators 17 seeking to develop, acquire, or disperse nuclear, radiological, biological, and chemical weapons. Successful proliferation is dependent on a number of “ingredients,” information being one such ingredient. Threat actors would also need access to the dual-use items and laboratory equipment, which are often difficult to acquire due to export controls or other special licensing requirements. On its own, access to GPT-4 is an insufficient condition for proliferation but could alter the information available to proliferators, especially in comparison to traditional search tools. Red teamers selected a set of questions to prompt both GPT-4 and traditional search engines, finding that the time to research completion was reduced when using GPT-4. In some cases, the research process was shortened by several hours without sacrificing information accuracy. We therefore conclude that a key risk driver is GPT-4’s ability to generate publicly accessible but difficult-to-find information, shortening the time users spend on research and compiling this information in a way that is understandable to a non-expert user. The red team assessed the model’s capabilities but their work was not intended to assess the probability or likelihood of a user accessing the model for the purpose of developing unconventional weapons. Specifically, we found that information generated by the model is most likely to be useful for individuals and non-state actors who do not have access to formal scientific training. The model can provide general information on common proliferation pathways, including historical attempts at proliferation that were successful. The model can suggest vulnerable public targets, provide general security measures that are typically used to protect dual-use materials, and generate the fundamental components that are required to engineer a radiological dispersal device. The model readily re-engineered some biochemical compounds that were publicly available online, including compounds that could cause harm at both the individual and population level. The model is also able to identify mutations that can alter pathogenicity. Red teamers could not successfully compel the model to engineer new biochemical substances. Red teamers noted that threat actors may benefit from the model’s capability to critique and provide feedback on user-proposed acquisition strategies. Red teamers found that the model generated useful information about facility rentals, equipment, and companies that could be used to build a weapon, including companies that were more likely to violate U.S export restrictions. Threat actors may also benefit from combining GPT-4 with internet browsing and open-source tools, as highlighted in the section above on Interactions with other systems. 15We focus here on unconventional weapons, but note that large language models like GPT-4 can also contribute to conventional weapons such as, for example, small arms. 16We note that in the past we have used the term red teaming somewhat differently than traditional usage in cybersecurity.[26] Throughout this system card, we refer to the people performing stress testing, boundary testing, and red teaming as “red teamers” for simplicity and in order to use language consistent with that we used with our collaborators. 17When referring to proliferation, the System Card uses the definition offered by the North Atlantic Treaty Organization. “WMD proliferation refers to attempts by state or non-state actors to develop, acquire, manufacture, possess, transport or transfer nuclear, radiological, chemical or biological weapons or devices and their means of delivery or related material, including precursors, without prejudice to the rights and obligations of the States Parties to the following agreements: the Treaty on the Non-Proliferation of Nuclear Weapons or Non-Proliferation Treaty (NPT); the Convention on the Prohibition of the Development, Production, Stockpiling and Use of Chemical Weapons and on their Destruction (CWC) and the Convention on the Prohibition of the Development, Production and Stockpiling of Bacteriological (Biological) and Toxin Weapons and on their Destruction (BWC)”.[58] 52