apache/spark
Apache Spark - A unified analytics engine for large-scale data processing
A star delta has been computed for 61,707 of 62,640 tracked repositories in this period, and only those can be ranked in it.
A repository missing here has not been measured far enough back yet. That does not mean it did not move.
Star counts measured between 00:28 and 06:42 (BKK).
Apache Spark - A unified analytics engine for large-scale data processing
LLM speculative inference server for heterogeneous hardware & consumer GPUs
Python SQL Parser and Transpiler
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
Apache Wayang is the first cross-platform data processing system.
Server for the ListenBrainz project, including the front-end (javascript/react) code that it serves and all of the data processing components that LB uses.
An open source, standard data file format for graph data storage and retrieval.
The Open Control Plane for Real-Time Data
Apache DataFusion Comet Spark Accelerator
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
LakeSoul is an end-to-end, realtime cloud-native Lakehouse framework for fast data ingestion, concurrent updates, incremental analytics, multimodal data processing and vector search — powering next-generation BI and AI workloads.
Apache Amoro(incubating) is a Lakehouse management system built on open data lake formats.
REST API for Apache Spark on K8S or YARN
Official code repository for GATK versions 4 and up
A better compressed bitset in Java: used by Apache Spark, Netflix Atlas, Apache Pinot, Tablesaw, and many others
Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.
Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.
NVIDIA cuDF for Apache Spark plugin - accelerate Apache Spark with GPUs
Apache Auron is an accelerator for big data engines, leveraging native vectorized execution to accelerate query processing.
Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learn...
Nessie: Transactional Catalog for Data Lakes with Git-like semantics
Uniffle is a high performance, general purpose Remote Shuffle Service.
Apache Spark Kubernetes Operator
YTsaurus is a scalable and fault-tolerant open-source big data platform.
Firely and Incendi's open source FHIR server
Dingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool
Next-generation decentralized data lakehouse and a multi-party stream processing network
Skardi is a self-improving context framework for ai agents
Showing 30 of 48 repositories