{"id":"open-standard-connector-architecture","slug":"open-standard-connector-architecture","title":"Open-Standard Connector Architecture","description":"Intelligence platforms ingest data from dozens of heterogeneous sources simultaneously. When the extraction logic inside each connector is tightly coupled to the platform's internal type model, two problems emerge: addin","category":"data-integration","tags":["data-integration","ai","geospatial"],"lastModified":"2026-04-14","source_ref":"content/modules/open-standard-connector-architecture.md","url":"/developers/open-standard-connector-architecture","htmlPath":"/developers/open-standard-connector-architecture","jsonPath":"/api/docs/modules/open-standard-connector-architecture","markdownPath":"/api/docs/modules/open-standard-connector-architecture?format=markdown","checksum":"ccd61fa4330cc879bb9b6f69128c7f9110cf3031cec607f8c0100e8bb56f20ab","headings":[{"id":"overview","text":"Overview","level":2},{"id":"key-features","text":"Key Features","level":2},{"id":"open-standards-compliance","text":"Open Standards Compliance","level":2},{"id":"use-cases","text":"Use Cases","level":2},{"id":"integration","text":"Integration","level":2}],"markdown":"# Open-Standard Connector Architecture\n\n## Overview\n\nIntelligence platforms ingest data from dozens of heterogeneous sources simultaneously. When the extraction logic inside each connector is tightly coupled to the platform's internal type model, two problems emerge: adding a new data source requires understanding platform internals before writing a single line of connector code, and it becomes difficult to verify that a connector's output is correct independent of the platform. The Argus open-standard connector architecture solves both problems by requiring every connector to produce a well-known open international standard as its primary output, and separating that extraction step entirely from the Argus type mapping step.\n\nThe architecture separates connector operation into two distinct phases. In Phase 1 the connector extracts data and serialises it as a STIX 2.1, NIEM, or CAP v1.2 object graph: formats defined by OASIS and US federal interoperability bodies, with published JSON schemas and independent validation tooling. In Phase 2 a structural mapping layer converts those validated open-standard objects to Argus entity format using compile-time GraphQL type definitions. No runtime ontology is consulted during either phase. The connector author works entirely within the open standard; Argus enforces the internal type contract at compile time, not at extraction time.\n\n```mermaid\ngraph LR\n    A[Data Source] --> B[Connector\\nimplements STIX / NIEM / CAP]\n    B --> C[Open Standard Validation\\nOASIS JSON Schema]\n    C --> D[Argus Type Mapping\\nStructural GraphQL check]\n    D --> E[Entity Storage\\nPostgreSQL + Neo4j]\n```\n\n**Last Reviewed:** 2026-04-14\n**Last Updated:** 2026-04-14\n\n## Key Features\n\n- **Open-Standard Primary Output**: Every connector declares the open international standard it implements via the openStandard field in its manifest. Permitted values are STIX 2.1, NIEM, CAP 1.2, STANAG 4607, AIS (ITU-R M.1371), and custom-open (with a citationUrl pointing to the published specification). Extraction logic inside the connector is guided solely by the declared open standard, not by the Argus type system.\n\n- **Phase 1 Open-Standard Validation**: After extraction, the connector's output is validated against the published open-standard JSON schema before any Argus-specific processing occurs. The `STIXConnectorBase`, `NIEMConnectorBase`, and `CAPConnectorBase` classes provide `validate_stix()`, `validate_niem()`, and `validate_cap()` methods that run jsonschema validation against structural subsets of the respective specifications. A `ConnectorValidationResult` object records pass/fail status, per-object error messages, and entity count.\n\n- **Phase 2 Structural Type Mapping**: Only after Phase 1 validation passes does the mapping layer convert open-standard objects to Argus entity format. This mapping is driven by compile-time GraphQL type definitions declared in Argus schema files. The mapping is a static field-name translation; it does not receive ontology parameters at runtime, does not consult a property graph during field resolution, and does not validate transformation scripts using runtime ontology parameters.\n\n- **Connector Manifest Standard Declaration**: The connector registry model (`ConnectorDefinition`), the `RegisterConnectorInput` GraphQL mutation, and the Python and TypeScript SDK `ConnectorManifest` types all include a required openStandard field. This field is recorded at registration time and surfaced through the `ConnectorDefinitionType` GraphQL type, making the standard declaration auditable across the connector fleet.\n\n- **Abstract Base Classes**: The `app/integrations/base/` package provides `STIXConnectorBase`, `NIEMConnectorBase`, and `CAPConnectorBase`. Each exposes an abstract `extract()` method (Phase 1), a `validate_*()` method (Phase 1 gate), and a `to_argus_entities()` method (Phase 2). Connector authors subclass the appropriate base and implement only `extract()`, receiving validation and mapping for free.\n\n- **Graceful Degradation**: When the `jsonschema` library is unavailable (for example in lightweight edge environments), the validation methods log a warning and return a passed result with an explanatory error message rather than raising an exception, ensuring the pipeline remains operational.\n\n## Open Standards Compliance\n\nThe Argus connector architecture is built on three OASIS and federal open standards:\n\n**STIX 2.1** (Structured Threat Information eXpression) is an OASIS open standard for representing and sharing cyber threat intelligence. Connectors producing threat indicators, malware records, attack patterns, threat actors, and vulnerability data use STIX 2.1 as their output contract. Full specification: [https://docs.oasis-open.org/cti/stix/v2.1/](https://docs.oasis-open.org/cti/stix/v2.1/os/stix-v2.1-os.html). Normative JSON schemas: [https://github.com/oasis-open/cti-stix2-json-schemas](https://github.com/oasis-open/cti-stix2-json-schemas).\n\n**NIEM** (National Information Exchange Model) is a US federal interoperability standard for information exchange in justice, emergency management, immigration, and defence domains. Connectors producing person, organisation, incident, or case management data use NIEM as their output contract. Reference: [https://niem.github.io/reference/](https://niem.github.io/reference/). NIEM JSON Specification v5.0: [https://niem.github.io/NIEM-JSON-Spec/v5.0/niem-json-spec.html](https://niem.github.io/NIEM-JSON-Spec/v5.0/niem-json-spec.html).\n\n**CAP v1.2** (Common Alerting Protocol) is an OASIS standard for emergency public alerting used by meteorological agencies, civil protection authorities, and emergency management organisations. Connectors ingesting public safety alerts, natural disaster warnings, or CBRN notifications use CAP v1.2 as their output contract. Full specification: [https://docs.oasis-open.org/emergency/cap/v1.2/CAP-v1.2-os.html](https://docs.oasis-open.org/emergency/cap/v1.2/CAP-v1.2-os.html).\n\n**Two-phase architecture**: In Phase 1 the connector's `extract()` method produces objects that conform to the declared open standard. Extraction logic is authored against the open-standard specification; the schema that guides which fields are populated and how values are structured is the open standard's published JSON schema, not the Argus GraphQL schema or any runtime ontology parameter. In Phase 2 the Argus type mapping layer converts those validated objects to Argus entity format. The mapping is driven by compile-time GraphQL type definitions: field names and value types are determined at application startup when the GraphQL schema is compiled, not by consulting an ontology at runtime.\n\n## Use Cases\n\nThe connector architecture is used across cybersecurity operations, emergency management, and defence interoperability programmes.\n\n- **Threat Intelligence Ingestion**: STIX 2.1 connectors for MISP, OpenCTI, TAXII feeds, CISA KEV, and NVD produce validated indicator and vulnerability objects before Argus type mapping, enabling independent validation of connector output against the OASIS STIX 2.1 specification.\n- **Sanctions and Regulatory Data**: NIEM-based connectors for sanctions lists, corporate registries, and court filing systems produce JSON-LD objects conforming to NIEM namespaces, making connector output independently verifiable by any NIEM-aware tool.\n- **Emergency and Civil Protection Alerts**: CAP v1.2 connectors for national meteorological services, CECIS, and ERCC produce alerts conforming to the OASIS CAP standard, allowing the same connector to feed both Argus and any other CAP-capable alerting platform.\n- **Multi-Standard Fleet Management**: The openStandard field in the connector registry allows operators to query, filter, and audit the connector fleet by standard, identifying which connectors are producing independently verifiable open-standard output versus custom-open connectors with a cited specification.\n- **EDF/PESCO Interoperability**: Open-standard connector output can be shared with allied national platforms that consume STIX 2.1 or NIEM data natively, supporting EDF cloud-of-clouds federated data sharing without requiring partner platforms to understand Argus internal types.\n\n## Integration\n\n- **Connector Registry**: Each connector registration carries the openStandard and optional citationUrl fields, queryable through the `connectors` GraphQL query and visible in the `ConnectorDefinitionType`.\n- **Ingestion Pipeline**: The ingestion pipeline service documents the two-phase boundary in its module docstring, making the architectural separation explicit for developers instrumenting or extending the pipeline.\n- **Intelligence Correlation**: After Phase 2 mapping, Argus entities flow into the correlation engine for cross-source entity resolution using the same RBAC and tenant-scoping controls as all other Argus data.\n- **SDK**: Both the Python (`argus-connector-sdk`) and TypeScript (`@argus/connector-sdk`) SDKs expose `OpenStandard` as a typed enum and include it in `ConnectorManifest`, so connector authors declare their standard at manifest authoring time.\n- **External Contract**: The openStandard, citationUrl, `OpenStandardEnum`, and `ConnectorValidationResult` types are included in the versioned external partner contract bundle at `docs/external-contract/v2026.1/external.graphql`.\n"}