apache/spark

More in data

Ranked by stars gained in the last 24 hours.

• 

  firecrawl/firecrawl

  Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

  TypeScript195.7k+372/dautomationdata
• 

  ShayanAbbas1/dbdelve

  Fast and lightweight DB Client made in Rust and GPUI.

  Rust542+285/d111% growthdatadevtools
• 

  storytold/figcraft

  FigCraft: interface design and prototyping; an open-source, clean-room reimplementation of collaborative design workflows in pure Rust

  Rust3.6k+189/d6% growthdata
• 

  vorssaint/vorssaint-utils

  Free and open-source macOS menu bar toolkit.

  Swift30.1k+186/ddata
• 

  supabase/supabase

  The Postgres development platform. Supabase gives you a dedicated Postgres database to build your web, mobile, and AI applications.

  TypeScript114k+109/ddataweb-backend
• 

  t8y2/dbx

  25 MB lightweight cross-platform database client for 100+ databases, including MySQL, PostgreSQL, SQLite, Redis, MongoDB, DuckDB, SQL Server, and Dameng. Built-in AI, MCP Server, CLI, desktop and Docker. | 轻量级跨平台数据库管理工具，支持 MySQL、PostgreSQL、SQLite、Redis、MongoDB、达梦等 100+ 数据库，提供桌面端、Docker、CLI、内置 AI 助手和 MCP。

  Rust27.8k+78/ddatadevtools
• 

  opendatalab/MinerU

  Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.

  Python82.5k+63/ddata
• 

  malisper/pgrust

  Postgres rewritten in Rust, now faster than Postgres and Clickhouse

  Rust5.5k+60/d1% growthdata

Reddit (fetched when this page is opened)

A Reddit search for github.com/apache/spark runs while this page is rendered, at most once every 6 hours per repository. What it finds is stored in our database (25 posts per fetch, at most 200 repositories a day), so the numbers on this site can be reproduced later. Reddit's feed has no scores or comment counts, so none are shown.

Reddit answered 429 (limit resets in 6s); next request in 20s.

Star history

Weekly, from GitHub's stargazers history, over a 30-day window.

Shares topics with

Repositories that share at least two topics with this one: most shared topics first, then stars gained in the last 24 hours.

• 

  apache/flink

  Apache Flink

  Shared: big-data, java, python, scala +1 more

  Java26.4k+2/ddata
• 

  apache/wayang

  Apache Wayang is the first cross-platform data processing system.

  Shared: big-data, java, jdbc, scala +1 more

  Java2780/ddata
• 

  trinodb/trino

  Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

  Shared: big-data, java, jdbc, sql

  Java13.4k+4/ddata
• 

  apache/beam

  Apache Beam is a unified programming model for Batch and Streaming data processing.

  Shared: big-data, java, python, sql

  Java8.7k+4/ddata
• 

  lakehq/sail

  Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

  Shared: big-data, python, spark, sql

  Rust3.5k+3/ddata

1. 28 Jan 2026Hacker News threadread the thread
2. 22 Sep 2026First seen on GitHub Trending · rank 12· 237 days later
3. 30 Sep 2026Entered our index· 8 days later

Stars: 44.1k when it entered our index, 44.3k on the last capture we hold (26 Oct 2026), so +244 in that span. That span is our own measurement window, not a claim about what happened before it.

Its most recent thread before Trending is more than 90 days older than that day, so the two are not the same story and we do not claim lead time.

We hold 8 mentions for it, the oldest dated 15 Sep 2020 and the first written into our database on 27 Oct 2026. Thread dates come from the source, not from when we fetched them.

8 items so far: 8 on Hacker News, 0 on Reddit; 0 in the last 7 days, 0 in the last 30. Checked by the pipeline, which re-scans trending repositories every few hours. Last check 18 hr ago.

• Add Support for PyCapsule to PysparkHNstory · 3 points · 0 comments · by devin-petersohn · 28 Jan 2026
• Pandas 3.0HNcomment · by devin-petersohn · 28 Jan 2026

  Funny enough, I actually just (2 weeks ago) added support for streaming from Pyspark to Polars/DuckDB/etc through Arrow PyCapsule. By streaming, I mean actually streaming, not collecting all data at once. It won't be released probably until May/June but it's there: https://github.com/apache/spark/commit/ecf179c3485ba8bac72af...

• Bazel 8.0HNcomment · by lihaoyi · 9 Dec 2024

  Bazel does a lot of non-obvious things that other build tools don't, that become important in larger codebases (100-10,000 active developers): - Automatic caching of everything. In most build tools, caching is opt in, so the core build/compile steps usually end up getting cached but everything else is not and ends up being wastefully recomputed all the time. In Bazel, caching is the default so ev…

• Spark SQL is getting pipe syntaxHNstory · 4 points · 0 comments · by tanelpoder · 17 Sep 2024
• Accidentally exponential behavior in SparkHNcomment · by IvanVergiliev · 21 Jun 2021

  Good question, the simplified example doesn't make this clear. The real implementation has a mutable `builder` argument used to gradually build the converted filter. If we perform the `transform().isDefined` call directly on the "main" builder, but the subtree turns out to not be convertible, we can mess up the state of the builder. The second example from the post would look roughly like this: v…