harbor-framework/harbor
Framework for evaluating and improving agents
A star delta has been computed for 61,707 of 62,629 tracked repositories in this period, and only those can be ranked in it.
A repository missing here has not been measured far enough back yet. That does not mean it did not move.
Star counts measured between 00:31 and 02:34 (BKK).
Framework for evaluating and improving agents
Mastra is the modern TypeScript framework for AI-powered applications and agents.
Observability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement.
AI Observability & Evaluation
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
The platform for continuously improving AI agents.
Low-level agentic design patterns. Turning old engineering wisdom into reflexes your agent reaches for on its own—on any agent.
Laminar - open-source observability platform purpose-built for AI agents. YC S24.
Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.
Evaluation and Tracking for LLM Experiments and AI Agents
Agent evals and guardrails as Jev decisions: one request per trace, a fraction of a cent, fast enough for the agent loop. Runs locally with Kev or Laya.
Testing and evaluation platform to chat, inspect, and debug MCP servers, MCP apps, and ChatGPT apps.
AI observability platform for production LLM and agent systems.
AgentEval is the comprehensive .NET toolkit for AI agent evaluation—tool usage validation, RAG quality metrics, stochastic evaluation, and model comparison—built first for Microsoft Agent Framework (MAF) and Microsoft.Extensions.AI. What RAGAS, PromptFoo and DeepEval do for Python, AgentEval does for .NET
Waku Waku! Waku Agent is a local-first AI agent harness you actually own, including loop, memory, eval, all in code built to stay legible as it grows.
Run your real agent with and without your skills, MCPs, and rules. See which ones actually help, and what they cost in tokens. Supports Claude Code, Codex, Pi, and Hermes.
Research infra for creating RL environments, post-training, and evals.
Open-source skills for Claude Code and Codex, with an MCP server, a PyPI package and Suede Awesome Everything, a directory that scores AI agent skills 0 to 99. Created by Jason Colapietro, founder and CEO of Suede AI.
Calibrate Jev questions against your own labels: tune criteria on labelled examples, confirm on a held-out set, get a verdict per question. Unofficial.
RL environments + evals for AI agents. Define once, train anything.
Cloud-synced dashboards for OpenCode and Claude Code. Track sessions, search with semantic lookup, export eval datasets.
Make AI agents aim higher without losing control: a Claude Code skill with two bars per task, a cheap gate check before expensive work, and an independent verifier that never sees the worker's reasoning. Plain markdown, EN + RU.
Build agentic systems. Run them with confidence. Orchestrate agents, automate business processes, inspect every execution, and keep humans in control. Deploy Heym on your own infrastructure.
That's all 23 repositories in this period.