Data Synchronisation

Overview#

A joint operations centre connecting intelligence feeds from three national agencies faces a specific synchronisation problem: each agency's system holds the authoritative version of its own records, but analysts at the centre need a unified view across all three. When one agency updates a record, that change needs to reach the joint system within seconds, not hours. When two agencies update the same entity concurrently based on different field-level information, both updates need to be preserved and merged correctly rather than one silently overwriting the other.

The Argus Data Synchronisation platform handles continuous, conflict-aware replication across distributed systems. It supports multiple replication topologies (multi-master, hub-and-spoke, peer-to-peer, hierarchical), automatic conflict resolution using configurable strategies, and delta synchronisation to minimise bandwidth over constrained links. Real-time synchronisation is delivered via persistent connections; periodic pull and event-driven modes are available for environments where persistent connections are not feasible. For intelligence agencies, financial crime units, critical infrastructure operators, and government data registries with distributed or federated deployments, this capability is what makes shared situational awareness possible.

Key Features#

• Real-Time Synchronisation: Propagate data changes across distributed nodes with low latency using persistent connections, with automatic reconnection and fallback protocols for resilient delivery over unreliable links.
• Multiple Sync Modes: Choose from real-time push, periodic pull, event-driven pub/sub, or hybrid multi-modal synchronisation depending on network conditions, security constraints, and use case requirements.
• Automatic Conflict Resolution: Resolve concurrent modifications automatically using configurable strategies: last-write-wins, operational transformation, conflict-free replicated data types (CRDTs), or custom business logic rules.
• Flexible Replication Topologies: Deploy multi-master full mesh, hub-and-spoke, peer-to-peer, or hierarchical tree topologies optimised for different scale, sovereignty, and consistency requirements. Topology can be changed as deployment needs evolve.
• Delta Synchronisation: Transmit only changes rather than full documents, significantly reducing bandwidth consumption and enabling efficient synchronisation over constrained or metered network links.
• Change Tracking and Audit Trails: Record every synchronisation operation with high-precision timestamps. Full change history is available for compliance, debugging, and time-travel queries. All records are written to PostgreSQL with organisation scoping.
• Time-Travel Queries: Query data as it existed at any point in the past by accessing the comprehensive change log. Relevant for post-incident investigation and regulatory review.
• Automatic Schema Migration: Handle schema evolution across version boundaries so that nodes running different software versions can continue to synchronise without manual intervention.
• Zero Data Loss: Persistent change logs ensure complete recovery from network partitions, node failures, and other disruptions. No operation is discarded due to a transient failure.
• Multi-Platform SDKs: Integrate synchronisation into applications using SDKs available for multiple programming languages and platforms.

Use Cases#

• Federated Multi-Agency Operations: Keep data consistent across agencies in a joint operations environment with low-latency replication, appropriate conflict resolution, and compliance with data residency and sovereignty requirements per jurisdiction.
• Disconnected Operations: Enable field systems that lose connectivity to continue operating and synchronise all changes when the link is restored. Conflict resolution handles the case where the same record was updated in both places during the disconnection.
• Collaborative Case Management: Support concurrent editing of case records by multiple analysts with operational transformation or CRDT-based conflict resolution ensuring that all contributions are preserved.
• Edge and Forward-Deployed Systems: Synchronise data between central infrastructure and edge or forward-deployed nodes with bandwidth-efficient delta sync and graceful handling of intermittent connectivity.
• Distributed Microservices: Keep data consistent across service boundaries using event-driven synchronisation with guaranteed delivery and automatic conflict detection.

Integration#

The Data Synchronisation platform integrates with the broader Argus data infrastructure through event-driven architecture and connects with the ingestion pipeline, entity graph, and audit trail systems. PostgreSQL is the primary data store for all synchronised records. Real-time change events are available to subscribers via GraphQL subscriptions. SDKs are available for embedding synchronisation capabilities directly into applications.

Open Standards#

• GraphQL (June 2018 Specification): The synchronisation platform exposes real-time change events via GraphQL subscriptions, allowing clients to receive push notifications of completed sync operations over a persistent connection using the graphql-transport-ws sub-protocol.
• RFC 6455 (WebSocket): Persistent WebSocket connections are the transport layer for real-time synchronisation push and collaborative state-sync; the implementation explicitly references RFC 6455 §11.3.4 for sub-protocol negotiation.
• Apache Avro Schema Compatibility Rules: Automatic schema migration across nodes during version-boundary synchronisation is governed by Avro BACKWARD, FORWARD, and FULL compatibility modes, preventing sync failures when nodes run different software versions.
• JSON Schema Draft 2020-12: Entity type schemas are validated against JSON Schema Draft 2020-12 during the schema evolution checks that gate cross-node synchronisation, ensuring structural correctness before propagation.
• ISO 8601: All synchronisation timestamps, change-log entries, and time-travel query boundaries are expressed as ISO 8601 datetime strings, providing unambiguous ordering of distributed events across nodes in different time zones.
• RFC 7519 (JSON Web Token): Authentication of synchronisation requests and offline sync policy tokens uses JWT validation; scopes and claims are verified before any replication or conflict-resolution operation is authorised.

Last Reviewed: 2026-02-05 Last Updated: 2026-04-14