ETSI CABCA AI System Quality Audit

Overview#

Continuous Auditing-Based Conformity Assessment turns AI quality compliance from an annual scramble into a standing capability that always has audit-ready evidence on hand. This module implements the CABCA method defined in ETSI TS 104 008, running automated assessment cycles that score an AI system against all nine ETSI TR 103 910 quality criteria and produce machine-readable conformity records on completion.

Organisations deploying AI in regulated sectors such as law enforcement, emergency services, healthcare, and financial crime face continuous pressure to demonstrate quality conformity under the EU AI Act harmonised standards track. Manual point-in-time audits cannot keep pace with models that are retrained, redeployed, and exposed to drifting data every week. This capability replaces those one-off reviews with continuous automated evidence collection, scoring, and tamper-evident record keeping, so compliance teams always hold defensible proof of system quality without manual effort.

Because assessments fire automatically on model deployment, drift detection, security incidents, and accuracy drops, conformity gaps surface as operational events rather than as findings discovered later by an external assessor. Every result is cryptographically anchored, scoped to a single tenant, and presented through the compliance dashboard for live conformity status and historical trend analysis.

Key Features#

Full CABCA Lifecycle Automation: Implements Continuous Auditing-Based Conformity Assessment per ETSI TS 104 008, orchestrating each assessment cycle from start through evidence collection, scoring, status determination, and conformity record generation without manual intervention.
Nine Quality Criteria Coverage: Every comprehensive cycle assesses all nine ETSI TR 103 910 criteria: Model Relevance (QC-1), Correctness (QC-2), Robustness (QC-3), Bias Avoidance (QC-4), Information Security (QC-5), Exploitation Safeguards (QC-6), Vulnerability Security (QC-7), Explainability (QC-8), and Data Quality (QC-9).
Scheduled Assessment Cadence: Weekly quick scans focus on correctness, robustness, and information security; monthly cycles run the full criteria set; quarterly cycles add a documentation completeness check and emit a conformity record for the period.
Event-Triggered Assessments: Cycles fire automatically when a new model version is deployed, when drift is detected, when a security incident is raised, or when accuracy drops below threshold, each running the subset of criteria most relevant to that event so gaps are caught before they become regulatory findings.
Tamper-Evident Evidence Chain: Each evidence item, whether a metric snapshot, test result, review record, or configuration capture, is hashed with SHA-256 and assembled into a Merkle tree. The resulting root is stored with the assessment cycle, giving auditors a cryptographic anchor that proves the evidence set has not been altered.
Machine-Readable Conformity Records: On completion the service produces a conformity record keyed against ETSI TR 103 910, TR 104 119, or TS 104 008, recording a full, partial, or non-conformant level together with validity windows and a certificate hash, ready for export to regulators and oversight bodies.
Historical Conformity Trending: Conformity records persist over time, building a defensible history of how an AI system's quality posture has changed and supporting trend analysis that demonstrates sustained conformity rather than a single snapshot.
Per-Tenant Isolation: All assessment cycles, evidence items, and conformity records are scoped to the owning organisation, so multi-tenant deployments keep each organisation's audit data fully separated.

Use Cases#

Law Enforcement and Public Safety: Forces operating analytical or biometric AI demonstrate ongoing TR 103 910 quality conformity to oversight bodies, with deployment-triggered assessments confirming that each model update still meets correctness and robustness expectations before it goes live.
Emergency Services: Control rooms running triage and dispatch support models gain continuous proof of model relevance and data quality, with drift-triggered cycles flagging degradation in the data feeding life-critical decisions.
Healthcare: Providers deploying clinical decision support or diagnostic AI maintain audit-ready evidence of correctness, bias avoidance, and explainability conformity, aligning quality governance with regulatory expectations for high-risk systems.
Financial Crime: Fraud and anti-money-laundering teams show that scoring models remain accurate and unbiased over time, with accuracy-threshold triggers automatically opening an assessment when performance slips.
Regulated AI Deployments Generally: Any organisation operating AI under the EU AI Act harmonised standards track replaces periodic manual audits with continuous automated conformity assessment, cutting preparation effort while improving the credibility of the evidence presented.

Integration#

The capability exposes assessment cycles, evidence items, and conformity records through a GraphQL schema that powers the compliance dashboard, giving teams live conformity status, criterion-level scores, and the Merkle root for each cycle. Conformity records are emitted in a machine-readable structure that can be exported to external governance, risk, and compliance platforms or attached to regulator-facing reports.

Assessment cycles can be invoked on demand or driven automatically by platform events. Model deployment, drift detection, security incident, and accuracy-threshold signals from elsewhere in the platform start the matching event-triggered cycle, while scheduled cadences run unattended. Evidence is drawn from existing sources including model performance metrics, automated test results, and documentation completeness checks, so customers plug their AI lifecycle events and metrics into one place and receive continuous conformity assessment in return. Authentication follows the platform standard of OAuth2 with JWT bearer tokens, and all data access is tenant-scoped.

Open Standards#

ETSI TS 104 008: Continuous Auditing-Based Conformity Assessment (CABCA) for AI systems. Defines the assessment cycle, evidence, conformity record, and trigger model this capability implements end to end.
ETSI TR 103 910: AI system quality criteria, providing the nine criteria QC-1 through QC-9 that each assessment cycle scores against.
ETSI TR 104 119: AI documentation completeness requirements, evaluated as part of the quarterly comprehensive assessment and recorded as a conformity reference.
SHA-256 (FIPS 180-4): Secure hash applied to every evidence item to produce the content hashes that anchor the tamper-evident chain.
Merkle Tree (hash tree): Binary hash tree construction that aggregates evidence hashes into a single tamper-evident root stored with each assessment cycle.
OAuth2 and JSON Web Token (RFC 6749 and RFC 7519): Token-based authentication and authorisation governing access to assessment and conformity data.

Security & Compliance#

Every piece of evidence collected during an assessment is hashed with SHA-256 and bound into a Merkle tree, so any later alteration of the evidence set breaks the stored root and is immediately detectable. Conformity records carry a certificate hash derived from that root, giving each record an independently checkable link back to the evidence that produced it.

All assessment cycles, evidence items, and conformity records are persisted with strict per-organisation isolation, ensuring that no tenant can see or influence another tenant's audit data. Conformity records retain validity windows and a full history, supporting both current-state reporting and the longitudinal evidence regulators increasingly expect. Assessment activity is written to the platform audit trail, so the act of auditing is itself auditable.

Last Reviewed: 2026-05-26 Last Updated: 2026-05-26