output if the speaker doesn’t match the chosen preset voice. Evaluation: We 昀椀nd that the residual risk of unauthorized voice generation is minimal. Our 4 system currently catches 100% of meaningful deviations from the system voice based on our internal evaluations, which includes samples generated by other system voices, clips during which the model used a voice from the prompt as part of its completion, and an assortment of human samples. While unintentional voice generation still exists as a weakness of the model, we use the secondary classi昀椀ers to ensure the conversation is discontinued if this occurs making the risk of unintentional voice generation minimal. Finally, our moderation behavior may result in over-refusals when the 5 conversation is not in English, which is an active area of improvement . Table 2: Our voice output classi昀椀er performance over a conversation by language: Precision Recall English 0.96 1.0 5 Non-English 0.95 1.0 3.3.2 Speaker identi昀椀cation Risk Description: Speaker identi昀椀cation is the ability to identify a speaker based on input audio. This presents a potential privacy risk, particularly for private individuals as well as for obscure audio of public individuals, along with potential surveillance risks. Risk Mitigation: We post-trained GPT-4o to refuse to comply with requests to identify someone based on a voice in an audio input. We allow GPT-4o to answer based on the content of the audio if it contains content that explicitly identi昀椀es the speaker. GPT-4o still complies with requests to identify famous quotes. For example, a request to identify a random person saying