External Data Polling

Overview#

An investigator tracking a sanctions evasion network cannot manually check flight databases, vessel tracking services, and blockchain explorers every hour for every subject under investigation. Missing a single departure or a key transaction link can set an operation back by weeks. The Argus External Data Polling system automates that monitoring, scheduling checks against aviation databases, maritime AIS feeds, blockchain explorers, and sanctions lists at configured intervals, then correlating what it finds directly against active investigation records.

Rather than requiring analysts to visit multiple external platforms, the system continuously monitors configured sources and delivers relevant updates into investigation workflows the moment they appear.

Key Features#

Data Collection#

• Automated data collection from aviation tracking, maritime AIS, blockchain explorers, sanctions lists, and custom integrations
• Configurable polling intervals from real-time 60-second checks through daily synchronisation
• Support for multiple data formats and protocols across diverse external source types
• Automatic error recovery with retry logic, backoff strategies, and status tracking
• Source health monitoring with alerts when external services become unavailable or degrade
• Priority-based polling queues ensuring critical intelligence sources are checked most frequently
• Bandwidth-aware scheduling that adjusts polling frequency based on network capacity and load

Caching and Deduplication#

• Intelligent result caching with TTL management, deduplication, and version tracking
• Historical data retention enabling trend analysis and change detection over time
• Delta processing that identifies and highlights changes between polling cycles
• Storage optimisation through configurable retention policies and data lifecycle management
• Source reliability scoring based on historical uptime, data quality, and freshness metrics

Correlation and Analysis#

• Cross-source data correlation with confidence scoring, temporal alignment, entity linking, and anomaly detection
• Correlation types including temporal, spatial, entity matching, financial, and behavioural pattern matching
• Automated alerting when polled data matches investigation subjects or watchlist entries
• Trend analysis across polling history identifying patterns and behavioural changes in monitored subjects
• Polling performance dashboards showing success rates, latency, and data volume by source

Administration and Governance#

• Source configuration management with connection testing and validation tools
• Polling schedule optimisation recommending efficient intervals based on data change frequency
• Cost tracking for paid external data sources with usage monitoring and budget alerts
• Audit trail of all polling activities, data retrievals, and correlation results
• Access controls governing which investigators can configure and view specific polling sources
• Compliance documentation for data sharing agreements and terms of service with external providers
• Multi-tenant source isolation ensuring polling configurations remain segregated between organisations

Use Cases#

Flight Tracking. Monitor specific aircraft, track route changes, receive arrival and departure alerts, conduct historical flight analysis, and cross-reference passenger information for active investigations. Detect unusual flight patterns or deviations that may indicate smuggling or other illicit activity.

Blockchain Monitoring. Track cryptocurrency wallet addresses, receive transaction alerts, trace fund flows through multiple hops, screen against sanctions lists, and detect suspicious patterns across blockchain networks. Maintain continuous monitoring of addresses linked to subjects of interest.

Vessel Tracking. Monitor vessel positions via AIS data, track port calls and anchorage activity, detect route anomalies and dark periods, and correlate cargo information for maritime investigations and sanctions enforcement.

Sanctions and Watchlist Screening. Continuously poll updated sanctions lists and watchlist databases to screen existing records and new entries against the latest designations. Receive immediate alerts when matches are detected, enabling rapid compliance response.

Integration#

• Connects with aviation data providers for flight and aircraft tracking
• Integrates with maritime AIS networks for vessel monitoring and port activity
• Links to blockchain analysis platforms for cryptocurrency investigations
• Works with sanctions and watchlist databases for automated screening and compliance
• Feeds correlated data into investigation and alert management workflows
• Compatible with entity resolution systems for matching external data to internal records
• Supports export of collected data for reporting and evidentiary purposes
• Connects with geospatial analysis platforms for location-based data enrichment

Open Standards#

• ITU-R Recommendation M.1371 (AIS): Maritime vessel tracking polls AIS position and static-data messages (MMSI, NavigationStatus, PositionReport, VesselStaticData) from AISStream, AISHub, and NOAA AIS feeds, all of which carry data encoded to the ITU-R M.1371 Automatic Identification System specification.
• ICAO 24-bit Aircraft Address / ADS-B (Annex 10 / Doc 9684): Aviation polling identifies aircraft by their ICAO24 hex transponder address and ingests position data from ADS-B Exchange and OpenSky Network, both of which distribute Mode S / ADS-B Out transmissions standardised under ICAO Annex 10 and Doc 9684.
• IMO Ship Identification Number Scheme (IMO Resolution A.1078(28)): Vessels are indexed and correlated using their seven-digit IMO number alongside MMSI, in accordance with the IMO mandatory ship identification scheme.
• UN/LOCODE (UNECE Recommendation No. 16): Port reference data is synced and keyed using UN/LOCODE identifiers (e.g., USNYC, CNSHA), allowing maritime polling results to be resolved to standardised port locations defined by the United Nations Centre for Trade Facilitation and Electronic Business.
• FollowTheMoney (FtM) Data Model: The sanctions polling service downloads and processes OpenSanctions bulk datasets in the FollowTheMoney newline-delimited JSON entity format (entities.ftm.json), which is the open graph schema used by OpenSanctions to represent persons, companies, and sanction entries across OFAC SDN, UN Security Council, EU FSF, and UK HMT lists.
• IATA / ICAO Airport Codes: Aviation source polling resolves departure and arrival airports using both IATA three-letter codes and ICAO four-letter location indicators, as defined by the International Air Transport Association and the International Civil Aviation Organisation respectively.
• RFC 7519 JSON Web Tokens (JWT): API access to all external polling source integrations and the internal GraphQL query layer is authenticated using Bearer JWTs, conforming to the IETF RFC 7519 token format.

Last Reviewed: 2026-02-05 Last Updated: 2026-04-14