Anthropic partners with IT company Accenture to evaluate frontier AI models
Anthropic is partnering with an Ireland-based information technology company to start evaluating its frontier artificial intelligence models.
TL;DR
- Anthropic has partnered with Accenture, an Ireland-based IT company, for independent evaluation of its frontier AI models.
- This partnership introduces the concept of 'embedded evaluation,' where evaluators have near-employee access within AI companies.
- Anthropic and Accenture are each investing at least $1 billion over five years to develop this evaluation practice.
- Embedded evaluators can verify safety practices, report incidents, and assess how AI companies operate and adhere to safety commitments.
- Anthropic CEO Dario Amodei introduced embedded evaluation as the first stage in a three-stage plan to 'pace the frontier.'
- The initiative is also looking to collaborate with other nonprofit evaluators like METR.
- OpenAI CEO Sam Altman supports the concept of embedded evaluation.
- Concerns about AI safety have been amplified recently due to an Anthropic researcher's resignation citing existential fears.