Figure 4: Mistakes GPT-4V makes such as combining terms and missing symbols when trying to process complex images. See Appendix A.2 for clear image given to model. from images. It could miss text or characters, overlook mathematical symbols, and be unable to recognize spatial locations and color mappings. Given the model’s imperfect but increased pro昀椀ciency for such tasks, it could appear to be useful for certain dangerous tasks that require scienti昀椀c pro昀椀ciency such as synthesis of certain illicit chemicals. For example, the model would give information for the synthesis and analysis of some dangerous chemicals such as Isotonitazene, a synthetic opioid. (See Figure 5) However, the model’s generations here can be inaccurate and error prone, limiting its use for such tasks. GPT-4V has misidenti昀椀ed substances like fentanyl, carfentanil, and cocaine from images of their chemical structure, but also occasionally correctly identi昀椀ed poisonous foods such as certain toxic mushrooms from images. (See Figure 6) This demonstrates that the model is unreliable and should not be used for any high risk tasks such as identi昀椀cation of dangerous compounds or foods. 2.3.2 Medical advice Medically trained red teamers tested the model’s ability to provide medical advice, especially with medically-related images as an input. Red teamers took into account the perspectives of both a layperson seeking medical advice and a professional with medical education and training. Some considerations for potential risks that may arise during the course of using the model to seek medical advice are accuracy, bias, and taking context into account. Red teamers found that there were inconsistencies in interpretation in medical imaging4while the model would occasionally give accurate responses, it could sometimes give wrong responses for the same question. As one example, Figure 7 shows some of the vulnerabilities or inaccuracies that could result from an incorrect or decontextualized interpretation of the directionality of medical imaging. The experts noted that the correct standard when viewing imaging scans is to imagine the patient as if they were facing you, meaning the right side on the image would correspond to the left side of the patient. This is an important concept that is needed when viewing and diagnosing 7