Overview#
As an organisation's intelligence platform matures, the number of data connections grows. OSINT providers, financial data feeds, sanctions lists, threat intelligence platforms, each one needs to be registered, categorised, and kept current as APIs evolve and providers retire old versions. The Datasource domain provides the ecosystem management layer that keeps all of that organised: provider registration, capability declarations, status tracking, and a central catalog that gives every team a consistent view of what is available.
Key Features#
- Datasource definition and provider registration.
- Data source ecosystem organisation and categorisation.
- Provider lifecycle management with status tracking.
- Centralised datasource discovery and catalog browsing.
- Organisation-scoped datasource management.
- Provider capability declarations.
- Integration with OSINT and external data providers.
- Datasource metadata management and versioning.
Use Cases#
Platform administrators organise and categorise the organisation's full data source ecosystem, giving analysts a browsable catalog rather than requiring them to know in advance which providers are available for different data types.
Integration teams register new data providers with structured capability declarations, ensuring that routing logic can direct the right query types to the right sources based on declared capabilities rather than manual configuration.
Data engineers browse the datasource catalog to understand which integrations already exist before commissioning new connector development, avoiding duplication across the platform's growing provider library.
Operations teams manage datasource lifecycle from registration through retirement, maintaining versioned metadata that tracks which provider versions are current and which are approaching end-of-life.
Integration#
Integrates with data source catalog, OSINT providers, and external data management domains for unified datasource lifecycle management.
Open Standards#
- GraphQL (June 2018 specification): all datasource domain queries, mutations, and health or analytics operations are exposed exclusively via a GraphQL API, with typed schemas defined for every subdomain including catalog, mapping, correlation, and enrichment.
- OAuth 2.0 (RFC 6749): the generic REST connector supports the OAuth 2.0 client credentials flow to authenticate against external data providers, with cached token management and Bearer token injection into outbound requests.
- OASIS STIX 2.1: connectors registered in the catalog must declare an open-standard output contract; STIX 2.1 is the primary supported standard, ensuring threat intelligence data emitted by connectors conforms to the structured threat information expression format.
- CVE / CVSS / CPE (NIST NVD): the NVD connector fetches, normalises, and stores Common Vulnerabilities and Exposures records including Common Vulnerability Scoring System scores and Common Platform Enumeration identifiers, enabling vulnerability data to be catalogued alongside other intelligence sources.
- Web Linking (RFC 5988): the generic REST connector parses HTTP
Linkheaders to handle cursor-based and page-based pagination when ingesting data from external REST APIs that implement the RFC 5988 link relation model. - OpenAPI 3.0: at least one provider connector (FlexManager V8) is integrated via its OpenAPI 3.0.3 contract, and the catalog's REST fallback API is described using the same specification to ensure interoperability with standard HTTP clients.
- ISO 8601: date and datetime values are formatted and parsed in ISO 8601 format throughout the connector layer, including NVD date-range parameters and FlexManager epoch-to-ISO conversion utilities.
- OASIS Common Alerting Protocol (CAP) 1.2: registered alongside STIX 2.1 and NIEM as a recognised output-contract standard in the connector registry, allowing datasources that emit emergency or alerting data to declare CAP conformance as their primary schema contract.
Last Reviewed: 2026-02-05 Last Updated: 2026-04-14