← Back to match
Pramen
Configuration-driven framework for defining and coordinating data pipelines on Apache Spark.
PlatformWebfree
Pramen is an open-source, configuration-driven framework for defining and running data pipelines on Apache Spark, coordinating ingestion, transformation and delivery jobs through a managed metastore. It automatically resolves job dependencies, retries only what needs rerunning after a failure, and alerts on upstream schema changes or slow sourcing. It is aimed at data engineers building Spark-based data lakes and ETL or ML pipelines who want less manual pipeline babysitting.
Categories
Data PipelineData EngineeringBig Data

