Data Sources Management

Overview#

The Data Sources Admin Panel provides centralized management for all external data integrations feeding into the platform. Administrators can configure, monitor, and troubleshoot connections to threat intelligence feeds, security tools, OSINT sources, cloud services, and custom APIs from a single interface.

Key Features#

Data Source Catalog - Browse and manage available integrations organized by category: threat intelligence, security tools, external databases, cloud services, and custom APIs. Each source displays its status, record count, and last sync time.
Connection Management - Configure authentication credentials (API keys, OAuth 2.0, basic auth, certificates), proxy settings, and connection parameters for each data source. Built-in connection testing validates credentials and connectivity before enabling a source.
Flexible Sync Configuration - Choose from real-time (WebSocket/SSE), continuous polling, scheduled, or on-demand synchronization. Support for full sync, incremental sync, delta sync, and selective sync based on filters.
Real-Time Monitoring - Live dashboards showing sync health, record counts, success/failure rates, and data freshness for all connected sources. Color-coded status indicators (green/yellow/red) provide at-a-glance health visibility.
Data Transformation - Map source fields to platform fields, apply transformations, set default values, handle missing data, and perform type conversions during sync. Pre- and post-sync filtering, enrichment, and deduplication are available.
Error Tracking and Retry - Comprehensive error logging with automatic retry logic and exponential backoff. View error messages, failed queries, rate limiting events, and timeout details for each source.
Data Quality Monitoring - Track completeness, accuracy, freshness, consistency, and reliability metrics for each data source. Automated quality checks include required field validation, format checks, range validation, and duplicate detection.

Supported Data Sources#

Threat Intelligence Feeds#

AlienVault OTX, MISP, ThreatConnect, Recorded Future, IBM X-Force, Anomali ThreatStream, RiskIQ, VirusTotal, and more.

SIEM Integrations#

Splunk, QRadar, ArcSight, LogRhythm, Elastic SIEM, Microsoft Sentinel, Sumo Logic, Chronicle.

OSINT Sources#

Social media APIs, domain registrars (WHOIS), certificate transparency logs, DNS records, IP geolocation, and company registries.

Custom APIs#

Connect any REST or GraphQL endpoint with configurable authentication, field mapping, and pagination strategies.

Use Cases#

Threat intelligence aggregation by connecting multiple threat feeds and correlating indicators across sources.
Security tool integration by syncing alerts, events, and case data from SIEM and SOAR platforms.
OSINT collection with automated, scheduled ingestion from public data sources with privacy compliance controls.
Custom data integration by connecting proprietary or internal APIs through the flexible REST and GraphQL integration framework.

Getting Started#

Navigate to Admin > Data Sources to browse the source catalog.
Select a data source and click Configure to enter credentials and sync settings.
Click Test Connection to verify connectivity and data access.
Configure sync frequency, data filters, and retention settings.
Click Enable Source to begin data ingestion.

Best Practices#

Use incremental sync whenever possible to reduce bandwidth and processing overhead.
Rotate API keys regularly and use the built-in credential management for secure storage.
Set up proactive alerts for sync failures and data quality degradation.
Monitor rate limit usage against provider limits and adjust sync frequency accordingly.
Implement deduplication rules to maintain clean, reliable data.

Availability#

Enterprise Plan: Included (all source types, unlimited connections)
Professional Plan: Core integrations included; premium threat intelligence and custom API integrations available as add-on

Last Reviewed: 2026-02-05

Metadatos del modulo

Documentacion renderizada