OpenDCAI/DataFlow
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
A star delta has been computed for 62,678 of 63,382 tracked repositories in this period, and only those can be ranked in it.
A repository missing here has not been measured far enough back yet. That does not mean it did not move.
Star counts measured between 08:09 and 23:57 (BKK).
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
Incremental engine for long horizon agents 🌟 Star if you like it!
Python Stream Processing
Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷
Scalable data pre processing and curation toolkit for LLMs
Apache Wayang is the first cross-platform data processing system.
A lightweight data processing framework built on DuckDB and 3FS.
The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure
Unified querying, transformation, and modification of JSON, TOML, YAML, XML, INI, HCL, KDL and CSV.
High-performance AI pipeline engine with a C++ core and 50+ Python-extensible nodes. Build, debug, and scale LLM workflows with 13+ model providers, 8+ vector databases, and agent orchestration, all from your IDE. Includes VS Code extension, TypeScript/Python SDKs, and Docker deployment.
A light-weight, flexible, and expressive statistical data testing library
A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.
Kubernetes-native platform to run massively parallel data/streaming jobs
Tangle is a web app that allows the users to build and run Machine Learning pipelines without having to set up development environment.
🦾 Mech is a programming language for building data-driven systems like robots, games, and interfaces. Start here!
ESA Earth Observation Toolbox and Java Development Platform
Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon
Desktop GUI for SNAP based on NetBeans Platform
SQL Runtime without any database
An AI-native distributed data plane built in Rust that supports high performance RPC, KV Cache, Message Queue, and File & Object Acceleration
DocWire SDK: local-first C++20 data-processing infrastructure for deterministic, auditable, on-premise workflows. Supports 100+ formats, OCR, local AI, and opt-in cloud AI pipelines
Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
That's all 23 repositories in this period.