AI Model Monitoring

Post-deployment monitoring is the continuous observation of an artificial intelligence system after it begins operating in real settings. It compares actual behaviour with expected performance and safety requirements so that emerging failures, misuse and harmful impacts can be identified across the system's life cycle.

What monitoring observes

Monitoring combines technical measurements with evidence from users and affected persons. The relevant indicators depend on the system's purpose and risk profile.

• Inputs and outputs are checked for data drift, unusual patterns, declining accuracy and departures from validated operating conditions.
• Results can be disaggregated across relevant groups to detect recurring errors, exclusion or discriminatory outcomes.
• System logs, access records and anomaly alerts can reveal adversarial inputs, unauthorised use, data leakage or other security incidents.
• Complaints, appeals, human overrides and reports of near misses reveal harms that aggregate performance metrics may conceal.

How signals become risk findings

Observed behaviour is compared with documented baselines, test results, legal obligations and predefined risk thresholds. Repeated deviations, sudden changes or severe individual incidents trigger investigation rather than being treated automatically as proof of failure.

• Investigators trace an alert to possible causes such as changed data, model updates, faulty interfaces, automation bias or operation outside the intended context.
• Periodic testing and independent review help determine whether a pattern is a genuine systemic risk rather than random variation.
• Maintaining model versions, data provenance and decision logs supports auditability and assignment of responsibility.

Corrective action and safeguards

Monitoring is effective only when findings lead to proportionate action through an established incident-response process.

• Responses may include human review, recalibration, retraining, access restrictions, rollback, user notification or temporary withdrawal of the system.
• Serious incidents should be documented and escalated to responsible management and, where required, competent authorities.
• Monitoring itself must protect privacy through purpose limitation, secure logs and proportionate data collection.
• Because monitoring may miss novel or poorly measured harms, it complements rather than replaces pre-deployment testing, human oversight and impact assessment.

How UPSC asks this