unclecode/crawl4ai
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf
A star delta has been computed for 62,252 of 63,108 tracked repositories in this period, and only those can be ranked in it.
A repository missing here has not been measured far enough back yet. That does not mean it did not move.
Star counts measured between 07:49 and 10:06 (BKK).
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf
Free Google Maps scraper for lead generation. Exports phones, emails, websites and socials to CSV. Runs locally in Docker, no API keys. Claude Code skill included.
🚀 自托管的抖音、TikTok、B 站数据采集与无水印视频下载 API，支持 AI agent 通过 MCP 调用与离线语音转文字，Docker 一键部署。| Self-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.
Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
Turn any website into a structured API. Extract, automate, search and monitor the web.
Free email OSINT tool, 5000+ platforms, identity clustering, breach detection. No API keys required. pip install mailaccess
开箱即用的本地 JAV/视频 刮削、管理、播放软件，支持命令行一键安装和 Docker 部署。只需简单添加目录，即可打造你的私人 JAV/视频 媒体库，带给你顶级的浏览体验，懒人必备。| Your local JAV/video manager.
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
Free open-source desktop app that scrapes JAV metadata and generates NFO + cover art for Jellyfin, Emby & Kodi. No Docker, no CLI — one-click install on Windows & macOS. 8 built-in sources + optional Metatube federation (30+ providers), actress collections, cross-language tag aliases, and a REST API for AI agents.
Client-side media downloader app and CLI built with Capacitor and Node.js.
Self-hosted scraping engine — bypasses any JS challenge & captcha: Cloudflare, Turnstile, reCAPTCHA, hCaptcha, GeeTest. FlareSolverr & Byparr alternative and drop-in replacement for your *arr stack.
Boss直聘爬虫 / BOSS直聘职位数据抓取工具，基于 Chrome CDP 协议复用真实登录态，绕过字体反爬，输出明文薪资 JSON/CSV + 薪资技能分析。A Chrome-CDP-based BOSS Zhipin job scraper/crawler.
TikTok LIVE API for Node.js: An unofficial library to receive live stream events (comments, gifts, etc.) in realtime from TikTok LIVE.
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
JAV 影片元数据刮削与媒体库整理工具 | JAV metadata scraper for Emby/Jellyfin/Kodi: NFO, auto-rename, Desktop + WebUI/Docker
Free proxies that actually work: 700+ sources, every hit verified (honeypots, injected scripts, TLS), a live list every hour, a rotating proxy server and an MCP server for AI agents.
抖音API、抖音接口、抖音爬虫示例。Douyin API examples via JustOneAPI（付费第三方API）。
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
Python library and CLI for X/Twitter scraping with multi-account rotation and built-in rate-limit handling.
A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。
一个管理电脑内资源，并提供游戏化的综合管理器
Scalable Python web scraping scripts for +40 popular domains
An ergonomic, privacy-aware Rust HTTP Client
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
Turn any webpage into structured data using LLMs
Download website to local directory (including all css, images, js, etc.)
SLG黄游之王：自由挑选你喜欢的SLG游戏，机器学习你的xp，方便快捷的管理你的本地黄油
An ergonomic, privacy-aware Python HTTP Client
An open database of international sanctions data, persons of interest and politically exposed persons
(InstaTools 2.0 soon to come 👀) 🧰 A collection of automation tools for Instagram 📱| Written in Python 🐍 | Don't forget to ⭐ the repo !
Showing 30 of 59 repositories