Anthropic gives outside evaluators deeper access: why AI safety needs more than a company promise

Announcement date: October 15, 2026

On October 15, 2026, Anthropic said it had begun giving a team from METR, a nonprofit that evaluates advanced AI, access similar to that of employees. The aim is to let outside evaluators examine risks with more information than a public chatbot window provides.

Anthropic says METR will publish its risk-assessment results. The company also described work with SecureBio and Accenture, along with early discussions with other organisations. The announcement concerns the testing process. It is not a new certificate saying every Claude answer is safe or correct.

What does an AI evaluator do?

An evaluator designs tests to find out how a system behaves. Some tests look at useful abilities, such as completing a task. Others look for failures, such as following a harmful instruction, hiding a mistake or taking an action outside the user’s request.

Imagine buying a bicycle. A shop display can show that the wheels turn. A more useful inspection checks the brakes, loose parts and what happens under pressure. AI evaluation has a similar purpose: look beyond an impressive demonstration and investigate where things can go wrong.

Why outside access matters

A company developing a model knows a great deal about it, but also has an interest in a successful launch. An outside team can ask different questions and challenge assumptions. That value depends on the access, independence and ability to report findings that the team actually has.

Anthropic’s explanation is that some safety information is commercially sensitive or could help people misuse a model if published openly. Its proposed approach is to give trusted evaluators deeper access while limiting public disclosure of that sensitive material.

The useful follow-up is to read the evaluator’s eventual findings, rather than treating the announcement of a partnership as the final result.

How beginners can read a safety claim

Look for four things. Who performed the test? Which version of the model was tested? What behaviour did the test measure? What limits did the researchers describe?

A result on one benchmark is evidence about that test. It is not evidence that the assistant will never make a mistake in your spreadsheet, research summary or email. Different tasks can expose different weaknesses.

It also helps to separate a capability from a safeguard. Capability describes what a model can do. A safeguard is a measure intended to limit misuse or harmful behaviour. A strong score on one does not automatically prove the other is strong.

What this changes for everyday use

You do not need to become an AI safety researcher before using a chatbot. You can still keep decisions proportionate to the task. Drafting a fictional invitation needs a different level of checking from sending a message to a customer or changing important records.

When an assistant can use tools, define the job and the point at which a person should review the result. A clear boundary is useful even when the model has passed many evaluations.

QueryFun’s take

Deeper independent testing is worth following because it can make AI claims easier to question. The next useful evidence will be the published assessments and their limits. Until then, treat this as a development in accountability, not a reason to stop checking your own results.

For another common reliability problem, read our beginner explanation of AI hallucinations.

Original source: Anthropic’s update on independent evaluators. Company statements are attributed to their source. Practical examples and commentary are QueryFun’s explanation, not an independent product test.