Independent Evaluation of AI Systems

Independent third-party evaluation is the assessment of an AI system by an entity that is organizationally separate from its developer or deployer and can exercise impartial judgement. For a high-risk AI system, it provides credible evidence on whether the system satisfies defined safety requirements and is fit for its intended purpose, rather than relying solely on the provider's claims.

Translating safety requirements into tests

The evaluator first identifies the system's intended use, affected people and operating environment because AI performance and harm are context-dependent. Applicable legal, technical, contractual or organizational requirements are converted into documented criteria, test scenarios and acceptance thresholds.

• The evaluation plan should cover reasonably foreseeable failures, misuse pathways and the severity and likelihood of resulting harm.
• The evaluator needs sufficient access to the relevant model version, data, documentation and deployment arrangements.

Testing the system and its controls

Independent evaluation combines technical testing with examination of governance processes and documentation. Evidence may be compared with predetermined requirements to identify compliance, residual risk and necessary safeguards.

• Testing can examine validity, reliability, safety, security, resilience, privacy and harmful bias under normal, unusual and adversarial conditions.
• Red-teaming can probe misuse, unexpected behaviour and failures under hostile or unusual inputs.
• An audit may review data governance, risk controls, human oversight, documentation and incident-management arrangements.
• Evaluation can occur before deployment and through post-deployment monitoring, since systems and their environments may change.

Findings, decisions and limitations

The evaluator reports the tested version, use case, methods, assumptions, limitations and unmet requirements. Findings can inform deployment, safeguards, remediation, procurement or withdrawal, but responsibility remains with the developer and deployer.

• Credibility requires evaluator competence, transparent methods and management of conflicts of interest.
• A one-time assessment is only a snapshot and may miss emerging, rare or real-world harms.
• Benchmarks may be gamed or may not represent actual deployment conditions.
• Independent evaluation supplies evidence for governance; it does not automatically amount to certification, regulatory approval or proof that a system is risk-free.