Data Source Domain

Overview#

An intelligence team relies on three external data providers. One is accurate but slow to update. Another updates frequently but has a known completeness issue for certain entity types. The third is excellent for financial data but weak on biographical records. Without a systematic way to track and compare these quality dimensions, analysts cannot make informed decisions about which source to trust for a given query.

The Data Source domain makes that quality assessment continuous and quantitative. Every data source has a trust score built from four measured dimensions, accuracy, timeliness, completeness, and consistency, updated with each synchronisation cycle. Analysts can see which sources are performing well and which are degrading before that degradation affects investigation quality.

Key Features#

Data source registration and lifecycle management.
Data quality trust scoring based on accuracy, timeliness, completeness, and consistency.
Data mapping configuration for transforming external data to internal domains.
Sync operation management with error logging and retry logic.
Source health monitoring with rate-based alerting.
Organisation-scoped data source access control.
Trust score trending for data quality monitoring.
Configurable sync schedules with exponential backoff on failures.

Use Cases#

Intelligence analysts use trust score data to understand the reliability of information sourced from specific external providers, giving appropriate weight to findings based on the source's measured accuracy and timeliness.

Platform administrators monitor trust score trends across registered data sources, identifying degrading sources early and engaging with providers before data quality problems affect operational investigations.

Data quality teams configure data transformations for ingesting external feeds, mapping provider-specific field formats to the platform's internal data model for each of the platform's 153 third-party integrations.

Operations teams track sync operation health across all data sources, with exponential backoff on failures preventing repeated failed requests from overwhelming degraded providers while maintaining retry attempts.

Integration#

Integrates with data ingestion, entity management, and quality monitoring domains for comprehensive data source lifecycle management.

Open Standards#

GraphQL (June 2018 Specification): the entire data source API surface, including all queries and mutations for source registration, trust scoring, and integration management, is exposed as a typed GraphQL schema.
JSON Web Token (RFC 7519) with RS256: access to all data source operations is gated by JWT bearer tokens verified against a JWKS endpoint using asymmetric RS256 signatures, ensuring only authenticated principals can read or modify source records.
JSON Web Key Set (RFC 7517 / RFC 7518): the platform publishes its public signing keys as a JWKS document, which the middleware fetches at runtime to validate incoming JWTs without embedding static secrets.
OAuth 2.0 (RFC 6749): the authorisation framework underpinning the bearer-token permission model; each GraphQL resolver enforces an IsAuthenticated check aligned with OAuth 2.0 bearer token usage.
JSON (RFC 8259): connection configurations, schema mapping rules, sync configurations, and trust-score trend histories are all serialised and persisted as JSON documents.
ISO 8601: all datetime fields, including sync timestamps, trust score evaluation times, and audit created/updated times, are represented and serialised in ISO 8601 UTC format.
RFC 4122 (UUID): every data source, trust record, and integration record is assigned a version-4 UUID as its stable, globally unique identifier.

Last Reviewed: 2026-02-05 Last Updated: 2026-04-14