Coforge has come up with opensource & UI based accelerators to help clients reduce cost, time and effort, improve quality & governance free of charge. One such accelerator is DLXpress which is an end-to-end accelerator that supports template-based pipeline creation for various data sources, resulting in optimal implementation effort, scalability, and enabling foundational building blocks for a governed data lake platform, Web application with self-service capabilities, Cloud Agnostic platform.
It supports various data sources with features like source/target authentication, pipeline triggering, and rapid feed setup. The platform includes intelligent schema inference, a watermarking framework for incremental/full load data sourcing, and reusable Python and PySpark templates. It ensures data standardization, data cleansing, and validation using Spark, organizing data into zones, categories, and feeds. DLXpress handles merging, truncating, and rollback operations and supports SCDs, CDC, streaming, Data Mesh, Data Fabric, data virtualization, federation queries, and Delta Lake. It is cloud-agnostic and highly scalable and integrates with DQXpress and provides detailed execution reports.
