Review Queue Domain

Overview#

An AI entity-matching pass links two records as the same person with 73% confidence, just below the auto-merge threshold. The system drops that proposed match into the review queue, tagged High priority, routed to an analyst with experience in identity resolution. She approves the merge with a confidence score and a brief justification. Elsewhere in the same queue, a case disposition that requires supervisor sign-off sits as a Critical item with a four-hour time-to-live. If it is not reviewed by then, it escalates automatically. The Review Queue domain manages this entire human-in-the-loop layer: intelligent assignment, escalation paths, confidence band classification, and performance analytics so that quality stays consistent even as volume grows.

Key Features#

• Priority-Based Queue Management: Items are organised by priority levels (Critical, High, Medium, Low) with configurable time-to-live settings that ensure urgent items receive prompt attention and stale items are automatically expired.

• Intelligent Assignment: Distribute review items to available reviewers with workload balancing and specialisation matching, ensuring fair distribution and routing complex items to qualified reviewers.

• Decision Workflow: Reviewers can approve, reject, escalate, or defer items with required justification, confidence scoring, and evidence linking for a complete audit trail of every decision.

• Confidence Band System: Items are automatically classified into confidence bands based on AI analysis, allowing high-confidence matches to be auto-processed while routing uncertain cases to human reviewers.

• Escalation Management: Items can be escalated individually or in bulk to supervisors when they require higher-level review, with full tracking of escalation reasons and routing.

• Reviewer Analytics: Track reviewer performance including decision speed, accuracy rates, overturn rates, and specialisations to support quality calibration and workload optimisation.

• Bulk Operations: Perform batch actions including bulk escalation, priority updates, and expiration of old items for efficient queue management at scale.

Use Cases#

Human-in-the-loop review queues are critical wherever automated systems produce outputs that carry legal, safety, or financial consequences. Key industries include law enforcement, financial intelligence, and defence.

• Entity Resolution: Review AI-suggested entity matches that fall below the auto-merge confidence threshold, confirming or rejecting proposed links between records.

• Case Approval: Route investigation findings through a structured approval process with required justification and evidence before final disposition.

• Quality Assurance: Monitor reviewer calibration metrics to identify drift in decision patterns and maintain consistent standards across the review team.

• Compliance Review: Ensure sensitive actions pass through required human review gates with full documentation and audit trails.

Integration#

The Review Queue domain connects with other platform capabilities to support end-to-end workflows:

• Investigation Management: Review items linked to active cases for contextual decision-making
• Entity Resolution: AI-generated match candidates feed directly into the review queue
• AI Analysis: Confidence scores from AI models drive automatic band classification
• Observability: Queue depth and throughput metrics for operational monitoring

Open Standards#

• GraphQL (June 2018 specification): All queue operations, including item retrieval, decision submission, assignment management, and calibration analytics, are exposed exclusively through a typed GraphQL API with strongly typed enums, input types, and output types.
• JSON Web Token (RFC 7519) / OAuth 2.0 Bearer Token (RFC 6750): Every GraphQL request must carry a signed JWT as an HTTP Bearer token; the domain validates the token for identity and tenant-scoped authorisation before any operation executes.
• UUID (RFC 4122): All review items, decisions, reviewer profiles, audit log entries, and snapshot identifiers are generated as version-4 UUIDs, guaranteeing collision-resistant global uniqueness.
• ISO 8601 / RFC 3339 Timestamps: All temporal fields, created_at, assigned_at, expires_at, completed_at, and period boundaries, are stored as timezone-aware timestamps conforming to ISO 8601 / RFC 3339, including UTC offset.
• Role-Based Access Control (NIST RBAC model, ANSI/INCITS 359): The domain enforces permission scopes such as review:read and audit:read against roles assigned to authenticated users, with strict tenant isolation between organisations.
• Cohen's Kappa inter-rater reliability statistic: Reviewer calibration metrics expose a Cohen's Kappa coefficient (target >0.75) to quantify agreement consistency across reviewers, enabling data-driven quality management and calibration training.
• JSON (RFC 8259): All audit log state snapshots, webhook notification payloads, and metadata fields are serialised as RFC 8259-compliant JSON, with explicit json.dumps serialisation at the persistence boundary.

Last Reviewed: 2026-02-05 Last Updated: 2026-04-14