A monthly overview of things you need to know as an architect or aspiring architect.
View an example
We protect your privacy.
Facilitating the Spread of Knowledge and Innovation in Professional Software Development
Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with content, and download exclusive resources.
Dany Lepage discusses the architectural journey of porting a hit VR title to seven non-VR platforms. He explains how his team solved the challenges of cross-progression, diverse input paradigms, and maintaining release velocity across Steam, iOS, and PlayStation. Beyond the tech, he shares candid lessons on the "product fit" gap when translating immersive social presence to 2D screens.
Michael Stiefel spoke to Baruch Sadogursky about software architecture in the age of agentic AI. LLM can function, albeit stochastically, as reasoning machines capable of interpreting human ambiguity. With the appropriate rigorous context artifacts to control the LLM’s reasoning, software specifications can become the source of truth, while the code becomes a disposable intermediate language.
Ian Thomas shares a case study on embracing AI-native engineering within Meta’s Reality Labs. He explains the "Assess and Grow" framework, a maturity model designed to move teams from manual toil to AI-integrated innovation. He discusses real-world wins – including hitting 90% code coverage in record time – while addressing senior concerns like "code slop," review fatigue, and maintaining quality.
Stéphane Di Cesare and Cat Morris share how engineers can move from being a "cost center" to a value driver using product discovery. They explain the "Double Diamond" framework and why identifying user problems must precede building solutions. Learn to choose the right metrics, build customer empathy through shadowing, and use business context to maximize the impact of your technical work.
J. Paul Reed discusses the "ironies of automation" – a 40 years-old concept now amplified by AI. He explains how advanced systems often make the human operator more crucial, not less, while simultaneously degrading the skills needed to intervene. Sharing real-world stories of "AI-fueled" incidents, he shares why over-reliance on AI can double recovery times and how to maintain resilience.
The more senior you become, the fewer people pressure-test your decisions. This 5-week cohort gives you that check.
Register Now.
Learn how leading engineering teams run AI in production—reliably, securely, and at scale.
Register Now.
A practical online cohort for senior engineers making decisions around retrieval, agents, evals, and AI infrastructure.
Register Now.
Learn what’s next in AI and software, from teams already doing it.
Register Now.
InfoQ Homepage News Claude Sonnet 4.5 Ranked Safest LLM from Open-Source Audit Tool Petri
This item in japanese
Oct 07, 2025 2 min read
by
Claude Sonnet 4.5 has emerged as the best-performing model in ‘risky tasks’ in early evaluations by Petri (Parallel Exploration Tool For Risky Interactions)— Anthropic’s new open-source AI auditing tool.
Petri joins a growing ecosystem of internal tools from OpenAI and Meta, but stands out for being openly released.
As models grow more capable, safety testing is evolving from static benchmarks to automated, agent-driven audits designed to catch harmful behavior before deployment.
In early trials, Anthropic tested 14 models on 111 risky tasks. Each model was scored across four safety risk categories: deception (knowingly giving false answers), sycophancy (agreeing with users even when incorrect), power-seeking (pursuing actions to gain influence or control), and refusal failure (complying with requests it should decline).
Anthropic cautions that while Sonnet 4.5 performed best overall, misalignment behaviors were present in every model tested.
Aside from LLM rankings, Petri’s main capability is in automation of a key part of AI safety: testing how models behave in risky, multi-turn scenarios.
Researchers start with a simple instruction such as attempting a jailbreak or provoking deception and Petri launches auditor agents that interact with the model, adjusting tactics mid-conversation to probe for harmful behavior.
Each interaction is scored by a judge model across dimensions like honesty or refusal, and concerning transcripts are flagged for human review.
Unlike static benchmarks, Petri is meant for exploratory testing, helping researchers uncover edge cases and failure modes quickly, before model deployment.
Anthropic says Petri enables hypothesis testing in minutes and reduces the manual effort typically required for multi-turn safety evaluations. The company hopes that open-sourcing the tool will accelerate alignment research across the field.
Petri’s open release makes it notable, not just as a technical artifact, but as a public invitation to audit and improve alignment research.
Anthropic has also released example prompts, evaluation code, and guidance for extending the tool.
Like similar tools, Petri also has known limitations. Its judge models, often based on the same underlying language models, may inherit subtle biases, such as favoring certain response styles or over-penalizing ambiguity.
Further to this, recent studies have documented issues like self-preference bias (where models rate their own outputs more favorably) and position bias in LLM-as-a-judge setups.
To that end, Anthropic positions Petri as a tool for exploration of safety rather than an industry benchmark. Its release therefore adds momentum to a growing shift: away from static test sets and toward dynamic, scalable audits that surface risky behavior early before models are widely deployed.
Petri arrives amid a wave of internal safety tooling inside AI labs. OpenAI has long employed external red teaming and automated adversarial evaluation. Meta has also published a Responsible Use Guide alongside its Llama 3 release
The release also lands as governments begin formalizing AI safety requirements. The UK’s AI Safety Institute and the U.S. NIST AI Safety Consortium are both developing evaluation frameworks for high-risk models, with calls for greater transparency and standardized risk testing, a trend Petri may help accelerate.
A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example
We protect your privacy.
A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example
We protect your privacy.
ONLINE INFOQ CERTIFICATION PROGRAM
A Cohort for Senior Engineers and Architects
Bring a real architecture or AI engineering challenge from your work. Spend 5 weeks pressure-testing your approach with senior peers from other companies and experienced facilitators. Explore the upcoming cohorts.
InfoQ.com and all content copyright © 2006-2026 C4Media Inc.
Privacy Notice, Terms And Conditions, Cookie Policy