Agent frameworks: GitHub stars do not measure enough

What this is about

An arXiv study submitted on July 2, 2026 examines how healthy open-source multi-agent frameworks really are. The authors analyze 15 major repositories from late 2022 to early 2026. The dataset is broad: 808,042 stars, 73,997 pull requests, 86,241 commits, and 987,330 user profiles.

The message is clear: GitHub stars are loud, but weak. They measure attention, not necessarily use, maintainability, or community depth.

What the study actually does

The study looks at three layers: awareness, adoption, and retention. Awareness includes stars. Adoption shows up more strongly in contributors, pull requests, and active-developer density. Retention asks whether people stay after their first contribution.

One example from the abstract is AutoGPT. It gained 111,967 stars in one month, but converted fewer than 9 contributors per 1,000 stars. LangChain, by comparison, reached 41. Lower-profile frameworks such as Pydantic AI can show higher contributor density even when they are less visible in the market.

Why it matters

Many teams choose agent frameworks under time pressure. A high star count feels like social safety: many people know it, so it must be fine. But a framework can be visible and still lack deep participation.

The study identifies LangChain as shared infrastructure in the ecosystem. It attracts 82.5 percent of cross-ecosystem contributors. For decision makers, that matters more than a snapshot of popularity because it shows where knowledge, migration, and integrations actually converge.

In plain language

A restaurant with a long line outside is not automatically the best restaurant. Maybe it is new, maybe it went viral, maybe many people only visit once.

A better sign is whether regulars come back, whether the kitchen runs reliably, and whether other good cooks want to work there. In open source, contributor density, cross-project contributions, and retention are those regular-customer signals.

A practical example

A product team must choose an agent framework for internal automation in 2026. Framework A has 90,000 stars, but few new contributors and many unanswered pull requests. Framework B has 12,000 stars, but active maintainers, regular releases, and contributors who also work in neighboring projects.

Under the study’s logic, Framework B is often the more robust candidate. Not because stars are worthless, but because they show only the top layer. For a team owning a product for 18 months, the real question is whether a framework is maintained, understood, and likely to keep moving.

Scope and limits

First, the study measures GitHub ecosystems, not every real product installation. A framework can be heavily used internally without many public pull requests.

Second, contributor data is not a perfect quality signal. A small focused maintainer group can build excellent software if response time, documentation, and releases are strong.

Third, framework selection remains context-dependent. A simple internal tool and a regulated enterprise agent system need different evidence.

SEO & GEO keywords

Open Source AI agents, multi-agent frameworks, LangChain, AutoGPT, Pydantic AI, GitHub stars, contributor density, framework selection, software ecosystem health, developer tools