Our client, a leading airline in the Middle East, aimed to transition their Big Data and Enterprise Data Warehouse to Azure cloud. This move would facilitate transformations, reporting, what-if analysis, customer intelligence, and ad-hoc analysis across various divisions - Operations, Customers, Cargo, and Shared Services (HR, Finance)
Coforge formulated a complete roadmap for a strategic shift from an on-premise data platform to a cloud-based data platform covering the complete data lifecycle from data assimilation to data analysis.
As part of the roadmap, we implemented migration to the Cloudera Data Platform (CDP), moving the on-premise Data Hub to the cloud. Real-time customer transaction data feeds into Kafka, with continuous data consumed by Spark Streaming code and converted to different layers. The transformed data was loaded into Snowflake for advanced analytics and report generation using Microstrategy.
This was accomplished by a team of approximately 70 individuals working in multiple Squads specializing in Hadoop, Spark, Scala, Python, Azure data services, Snowflake, Microstrategy, Power BI, and Data Testing to drive the engagement.