In a move to bolster its security measures, OpenAI has announced a new set of safeguards aimed at preventing future security incidents during the model development and testing process.

New Monitoring and Alignment Emphasis

As part of its efforts to contain security incidents while models are being tested, OpenAI has implemented more detailed monitoring of models during the development process. This includes a greater emphasis on alignment and security during the post-training process, ensuring that models behave as intended and do not pose a risk to users.

The company’s VP of research, Amelia Glaese, emphasized that the strictness of the controls will increase as models become more capable, with the largest models facing the greatest scrutiny. ‘We have put in place requirements and expectations for safe development,’ she said. ‘Those requirements and expectations vary with the level of risk that we see.’

The new measures are designed to address the growing risks associated with developing and testing increasingly capable models internally. OpenAI’s standards for monitoring, alignment, and security must stay ahead of those risks, the company said in a blog post. The new safeguards include stronger network isolation practices, with a single compromise of a workload or supporting service no longer allowing for unauthorized access to the internet or other internal networks.

Monitoring System and Compute Burden

The strongest safeguard is the monitoring system, which will examine tool actions, available reasoning traces, and activity logs for unauthorized behavior. OpenAI aims to issue alerts within 30 minutes of concerning activity, with the company estimating that the compute burden of the monitoring system will be roughly 20% of the process being monitored.

The company has promised further details on the monitoring system in a forthcoming blog post. OpenAI’s official post-mortem analysis of the Hugging Face incident is also still pending.

Freezing Reinforcement Learning and Restarting Models

Following the Hugging Face incident, OpenAI froze reinforcement learning for two weeks. However, the company has since restarted many of the less risky models. Its largest planned frontier RL run remains on hold while it conducts smaller-scale training and evaluations to assess model behavior, validate its safeguards, and establish more evidence of alignment before proceeding.

Network Isolation and Security

The new safeguards include stronger network isolation practices, with a single compromise of a workload or supporting service no longer allowing for unauthorized access to the internet or other internal networks. This is a significant improvement over the previous network security practices, which were criticized following the Hugging Face incident.

The company’s emphasis on network isolation and security is a critical step in preventing future security incidents and protecting users from potential risks. By prioritizing alignment, security, and monitoring, OpenAI is taking a proactive approach to ensuring the safe development and testing of its models.

Next Steps and Future Developments

OpenAI’s new safeguards are a significant step forward in the company’s efforts to ensure the safe development and testing of its models. As the company continues to develop and refine its safety practices, users can expect a higher level of security and protection from potential risks.

The company’s emphasis on network isolation, security, and monitoring will serve as a model for other companies in the field, highlighting the importance of prioritizing user safety and security in the development and testing of AI models.