dagster-io/dagster
An orchestration platform for the development, production, and observation of data assets.
A star delta has been computed for 62,252 of 63,107 tracked repositories in this period, and only those can be ranked in it.
A repository missing here has not been measured far enough back yet. That does not mean it did not move.
Star counts measured between 07:51 and 12:01 (BKK).
An orchestration platform for the development, production, and observation of data assets.
Incremental engine for long horizon agents 🌟 Star if you like it!
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
A high-performance observability data pipeline.
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
Scalable and efficient data transformation framework - backwards compatible with dbt.
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
Flowfile is a visual ETL tool and Python library combining drag-and-drop workflows with Polars dataframes. Build data pipelines visually, define flows programmatically with a Polars-like API, and export to standalone Python code. Perfect for fast, intuitive data processing from development to production.
Pluggable data replication with checkpointing, batching, and integrity events
Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.
Support agile Ontology DataOps Based on Flink, DataX and Flink-CDC with Web-UI
Maestro: Netflix’s Workflow Orchestrator
Hop Orchestration Platform
An open source, standard data file format for graph data storage and retrieval.
Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
Privacy and Security focused Segment-alternative, in Golang and React
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
ApeCloud's Data Transfer Suite, written in Rust. Provides ultra-fast data replication between MySQL, PostgreSQL, Redis, MongoDB, Kafka and ClickHouse, ideal for disaster recovery (DR) and migration scenarios.
High-performance, time-indexed object storage for robotics and industrial IoT
A dbt adapter for Databricks.
The Fastest Unified Streaming SQL Engine in a Single C++ Binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute real-time context from streams, logs, metrics, events, and CDC.
Postgres, MySQL, or MariaDB to Elasticsearch/OpenSearch sync
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.
Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊
Flink CDC is a streaming data integration tool
Actively maintained successor to Alibaba DataX — a fast, versatile, open-source ETL tool for 30+ RDBMS and NoSQL data sources.
Database Reporting Tool and Tasks (.Net)
Showing 30 of 49 repositories