Data Archival

Overview#

A financial crime unit with a seven-year record retention obligation does not simply store everything indefinitely in hot storage. The cost becomes prohibitive, query performance degrades as tables grow, and auditors still expect that a transaction record from six years ago can be retrieved within minutes when a case reopens. The challenge is moving data through storage tiers automatically, without breaking the access patterns that analysts and case officers depend on.

The Data Archival module handles this lifecycle automatically. It moves data between hot, warm, cold, and frozen tiers based on configured rules tied to age, access frequency, regulatory schedules, or storage cost thresholds. All archived data is encrypted at rest using AES-256-GCM. Retrieval is transparent: analysts query the same interface they always use, and the system fetches from whichever tier holds the data. For intelligence agencies, healthcare data controllers, government registries, and financial crime units, this means retention obligations are met without manual intervention and without performance trade-offs on active operational data.

Key Features#

Automated Archival Policies: Configure rules based on data age, access frequency, storage size, cost thresholds, regulatory requirements, or custom business logic to automatically identify and archive eligible data without manual intervention.
Multi-Tier Storage: Move data across hot, warm, cold, and frozen storage tiers, each optimised for different access frequency and cost profiles. Transitions happen without application changes.
Transparent Data Access: Query archived data through the same interface as live data. The system retrieves from the appropriate storage tier automatically, so analysts do not need to know where data physically lives.
Predictive Caching: ML-based models predict likely retrieval requests and pre-fetch archived data into faster tiers before users need it, reducing retrieval latency for anticipated investigations.
Compression: Apply optimised compression strategies that achieve significant storage reduction while balancing retrieval performance for the target tier.
End-to-End Encryption: All archived data is encrypted at rest using AES-256-GCM with automated key rotation and compliance-certified key management. Data in transit is also encrypted.
Legal Hold Support: Instantly freeze matching data across all storage tiers when litigation or regulatory holds are required. Holds cannot be bypassed by archival policies or automated deletion workflows.
Analytics and Reporting: Monitor storage utilisation, cost trends, archival activity, retrieval patterns, and compliance status through real-time dashboards with actionable optimisation recommendations.
Bulk Retrieval Optimisation: Restore large volumes of archived data using parallel retrieval, streaming decompression, and progressive hydration that delivers the most critical records first.
Cost Optimisation Insights: Automatically identify data eligible for lower-cost tiers, detect policy gaps, and forecast future storage requirements to support capacity planning.

Use Cases#

Regulatory Compliance Archival: Automatically archive data per regulatory retention schedules (seven or more years for financial records, for example) while maintaining prompt retrievability for audit responses. Applicable to GDPR, DORA, and sector-specific frameworks.
Storage Cost Reduction: Move infrequently accessed data to lower-cost tiers automatically, reducing storage expenditure while keeping data available when a case reopens or an audit is triggered.
Historical Data Management: Archive completed investigations, closed cases, or past-period transaction data while preserving the ability to query it on demand for reporting, re-investigation, or judicial proceedings.
Legal Hold and e-Discovery: Place immediate holds on relevant data across all storage tiers when litigation arises. No archival policy or deletion workflow can affect held data until the hold is explicitly released.
Capacity Planning: Use analytics dashboards to forecast storage growth, identify optimisation opportunities, and plan infrastructure investments based on actual usage trends rather than estimates.

Integration#

The Data Archival module integrates with major cloud storage providers and on-premises storage systems, and works alongside the platform's retention policy engine, compliance reporting, and data access layer. All archived data is persisted with organisation scoping enforced at the storage layer, and all archival and retrieval events are written to the audit trail.

Open Standards#

AES-256-GCM (NIST FIPS 197 / NIST SP 800-38D): All archived data is encrypted at rest using AES-256-GCM with a per-blob data encryption key wrapped under a platform key-encryption key, as implemented in the backup and storage services.
SHA-256 (NIST FIPS 180-4): Backup integrity is verified using SHA-256 checksums computed at write time and re-verified on retrieval, enabling tamper detection across all storage tiers.
GDPR (EU Regulation 2016/679): Article 5(1)(e) storage limitation and Article 17 right-to-erasure obligations are enforced, with retention schedules mapped to regulatory timelines and legal-hold mechanisms that override automated deletion.
S3-Compatible Object Storage API (AWS S3 REST API): Archived objects are addressed using S3-style URIs and transferred via the S3 REST protocol, enabling interoperability with any S3-compatible storage backend for warm, cold, and frozen tiers.
ISO 8601: All archival event timestamps, retention expiry dates, and retrieval records are serialised in ISO 8601 format, ensuring unambiguous interchange with audit and compliance systems.
GraphQL (June 2018 Specification): Archival queries, mutations, and retrieval requests are exposed through a GraphQL API, providing a consistent schema-typed interface for both operational tooling and compliance dashboards.
OAuth 2.0 (RFC 6749) / JSON Web Tokens (RFC 7519): All archival and retrieval operations are gated by RBAC policies enforced via bearer JWT, ensuring that only appropriately credentialled principals can trigger archival transitions, restores, or legal holds.
TLS 1.3 (RFC 8446): Data in transit between the platform and remote storage tiers is protected by TLS, complementing at-rest encryption to provide end-to-end protection throughout the data lifecycle.

Last Reviewed: 2026-02-05 Last Updated: 2026-04-14