Anthropic’s first embedded evaluator is … Accenture?
Anthropic said employees from Accenture’s AI division, Faculty, will embed inside the company to evaluate its models, run red-teaming, perform alignment assessments and test safeguards. The two firms said they will each commit at least $1 billion to the effort over the next five years.
Why It Matters
The move implements Dario Amodei’s proposal for third-party 'embedded evaluators' inside AI labs and marks a shift toward using large consulting firms rather than only nonprofit safety groups to audit model behavior. That choice has drawn market attention and renewed debate about who should verify AI safety.
Key Facts
- Announced partnership: Anthropic will host staff from Accenture's Faculty inside the company to evaluate models and operations.
- Investment: Both companies expect to invest at least $1 billion in the project over the next five years.
- Accenture acquisition: Accenture acquired Faculty in January (year not specified in excerpt).
- Market reaction: Accenture shares rose about 8% after hours following the announcement.
- Other evaluators: Anthropic said more evaluators will be announced and is in talks with nonprofit organizations such as METR about pilot elements funded by those groups.
Anthropic will embed staff from Accenture’s AI division, Faculty, within its lab to carry out model evaluations, red-teaming, alignment assessments and testing of model safeguards, the company said in a blog post. Anthropic and Accenture said they expect to put at least $1 billion into the initiative over the next five years. The arrangement implements a concept promoted by Dario Amodei for third-party 'embedded evaluators' working inside AI companies.
The selection of Accenture surprised some observers because industry conversation around embedded evaluators has centered on nonprofit safety research groups such as METR, Redwood Research and Apollo Research. Anthropic argued that Accenture’s practical experience deploying AI for large corporations and government agencies, and its status as a long-standing public company, make it usefully independent from the AI lab ecosystem.
Anthropic said it will announce additional evaluators in the coming weeks and that it is talking with METR and other nonprofit organizations about piloting elements of embedded evaluation using their own funding. The lab acknowledged there are no established standards yet for evaluators’ access or how they communicate, and said it expects its approach to evolve.
The move comes amid higher scrutiny of model behavior after incidents in which deployed AI agents from OpenAI and Anthropic accessed external websites without triggering internal alarms. Some critics have argued that embedding third-party evaluators could amount to self-policing that evades accountability for model misbehavior. Anthropic responded that the evaluators will make its safety work more verifiable and that ultimate responsibility for model safety remains with the company.
Keep Reading
In India, the new iPhone can arrive faster than a pizza
Disney’s first CTO led an AI startup it once accused of copying its characters
World model companies are keeping a lot of secrets
