A leading international specialty insurance group, operating across more than 100 classes of insurance in 180 countries, faced a critical inflection point in its data infrastructure. Fragmented ingestion pipelines, legacy ETL tooling, and an inability to scale to cloud-native analytics were slowing the organization's responsiveness to business and regulatory demands. They turned to Coforge.
The Challenge
As the organization expanded its global operations, its ability to act on data quickly fell behind. The organization's data landscape had grown complex. Multiple source systems carried distinct ingestion patterns, creating inconsistent data flows that complicated governance and made integration fragile. The existing stack, built on DataStage-based transformations and SQL Server, had accumulated technical debt, limiting agility.
Onboarding a new data source or supporting emerging business needs required a significant redesign of the pipeline. This introduced delays that affected the organization's ability to act on data quickly. Lengthy batch-loading processes compounded the problem, hindering timely reporting and SLA compliance. And critically, the architecture offered no clear path to leveraging cloud-native, AI-driven ETL capabilities, thereby constraining the business's analytics ambitions.
Our Approach
Coforge designed and implemented a modern, cloud-native, metadata-driven data platform on AWS, rebuilding the organization's data infrastructure from the ground up.
Cloud Migration and Data Consolidation
Coforge migrated the organization's on-premises data warehouse to a fully AWS-native architecture, consolidating diverse data sources into a unified data lake built on Amazon S3. This eliminated fragmented data flows and established a single source of truth for reporting and analytics.
Metadata-Driven Ingestion Framework
A metadata-driven framework was introduced using AWS CodeCommit, enabling new datasets to be onboarded without bespoke pipeline development. Metadata configurations are stored in Snowflake and drive ingestion logic dynamically, removing the redesign overhead that had previously delayed new business capabilities.
Automated Orchestration via AWS Glue and Step Functions
Coforge replaced the legacy DataStage and SQL Server stack with an orchestrated, cloud-native pipeline. AWS Glue manages ingestion and transformation centrally. AWS Step Functions coordinated end-to-end workflow execution, and AWS Lambda handled high-speed processing, delivering the automation and scalability the existing architecture could not support.
Analytical Warehouse and Governance
A curated analytical layer was built in Snowflake, powering dashboards and regulatory reporting through Power BI and AtScale. Role-based access controls, automated tagging, and Amazon CloudWatch monitoring provide end-to-end operational visibility and compliance assurance across the platform.
Impact to Date
30%
100%
50%
75%
Business Impact
- All data sources are now consolidated into a unified data lake, giving the organization a single, governed foundation for reporting, analytics, and compliance.
- Standardized, automated ingestion and transformation pipelines cover 100% of data ingestion workloads, replacing manual, error-prone processes with consistent, auditable flows.
- New data source onboarding, previously a resource-intensive redesign exercise, is now configuration-driven, cutting development effort by half.
- Incremental load performance improved by 75% over legacy batch processing, enabling faster operational and regulatory reporting cycles.
- Role-based access controls, automated tagging, and centralized monitoring have materially strengthened the organization's security and compliance posture.
The platform positions the organization to progressively extend cloud-driven analytics capabilities, with raw and curated datasets now readily accessible for advanced analytics and AI-led initiatives. What was once a fragmented, bottlenecked data estate is now a scalable, governed foundation built for the demands of a global specialty insurer.
