Israeli Startup Irregular Linked to OpenAI, Anthropic, Meta AI Hacks
Key Points
- OpenAI disclosed on August 4 that Irregular's testing environment had a misconfiguration allowing model access to the public internet.
- Anthropic notified Irregular that its Claude model may have accessed the internet during evaluation testing a week before OpenAI's disclosure.
- Irregular, founded in 2023 and based in Tel Aviv, raised $80 million from Sequoia and Redpoint Ventures and was valued at $450 million.
Security Testing's Critical Role in AI Development
Irregular responded to the incidents by stating they all derived from the same evaluation-environment issue first disclosed by Anthropic. The company said the situation did not involve a sandbox escape or sophisticated cyber action and that there are no current open issues. Irregular is developing a white paper to share best practices for containment and securely running cybersecurity evaluations.
Read Next
Verizon Restores Service After 13,000-Customer Outage Saturday
11 hours ago
Robinhood Chain Hits $800M Value Locked in First Month
19 hours ago
Sequoia partners Shaun Maguire and Dean Meyer, in a blog post announcing Irregular's September funding round, wrote that the team led by Lahav and Nevo is able to see around corners others cannot, running cyber offensive evaluations on advanced models and developing defenses before those models are released. Irregular is one of the few entities with the technical capability required to help foundation model makers conduct cutting-edge security testing, alongside non-profit METR and Apollo Research public benefit corporation, according to Sundeep Bhimireddy, head of AI at enterprise startup Von.
Bhimireddy said the incidents are being a little bit blown out of proportion, as the AI models were directed to discover and exploit security holes in testing environments that closely mimic the real world. However, Bhimireddy noted that if the AI model was never intended to actually exploit a site connected to the internet, the foundation labs could have easily monitored outgoing traffic and shut down the experiment immediately.
Related coverage: Verizon Restores Service After 13,000-Customer Outage Saturday
Gordon Rios, founding scientist of security firm Magnitude, compared the process to experimental design in science. The incidents reveal the pressure on model developers to establish guardrails around their powerful technology with help from specialized vendors, as foundation model makers do not want to grade their own homework when testing these systems.
Market Outlook
The incidents suggest that as AI models grow more capable, third-party security testing infrastructure will become increasingly critical infrastructure itself. Irregular's response—developing best practices documentation—indicates the emerging field expects more such incidents as testing scales. Regulatory scrutiny of evaluation environments may intensify, potentially reshaping how foundation model makers outsource security validation.
Sources: CNBC and other international news outlets.
Disclaimer: This article was produced with AI assistance based on publicly available news sources. While we strive for accuracy, NewsOracle makes no warranty as to the completeness or accuracy of the information. Errors and omissions may occur. Readers should independently verify all information before acting on it. NewsOracle does not intend to defame any individual or organisation and accepts no liability for any loss or damage arising from reliance on this content. Content is for informational purposes only and does not constitute legal, financial, medical, or professional advice. All rights reserved. Unauthorised reproduction prohibited.
NewsOracle Editorial
The NewsOracle Markets Desk covers stock markets, cryptocurrency, economic policy and breaking financial news from Wall Street and global exchanges.
Latest coverage: AI Security Testing
