Monarchic LLC
RepoIntel MCP
Repository
Intelligence Bench
RepoIntel is a repository intelligence MCP runtime for indexing code, symbols, evidence records, generated wiki pages, and change-aware retrieval. This page reports the current RepoIntel-Bench run across localization, explanation, patch planning, patch generation, and review.
Current RepoIntel Run
Overall score
0.9979
Tasks
59
Zero-score tasks
0
Review
1.0000
Localization
0.9938
Explanation
1.0000
Patch plan
1.0000
Patch generation
1.0000
RepoIntel-Bench
Score by Task Type
| Task Type | Tasks | Mean Score | Interpretation |
|---|---|---|---|
| Review | 5 | 1.0000 | Seeded defect findings are detected with strong file and citation coverage. |
| Localization | 20 | 0.9938 | Source-aware evidence ranking recovers target files and high-value symbols across fixture and OSS repos. |
| Explanation | 14 | 1.0000 | Evidence-backed answers have strong file and citation support in the strict retrieval path. |
| Patch plan | 10 | 1.0000 | Plans are evidence-backed with high file and citation coverage. |
| Patch generation | 10 | 1.0000 | A constrained semantic Python-service patcher emits diffs that apply and pass public and hidden tests. |
Score by Repository
fixture-small-api
51 tasks
0.9975
express
2 tasks
1.0000
flask
2 tasks
1.0000
fastapi
2 tasks
1.0000
requests
2 tasks
1.0000
What This Measures
The benchmark combines generated fixture tasks with pinned public open-source tasks from FastAPI, Express, Requests, and Flask. It scores file recall, symbol recall, citation quality, patch application, hidden tests, minimality, seeded review findings, and task-type specific metrics.
Current Ceiling
The strict public-corpus run is strong across review, explanation, localization, patch planning, and the current Python-service patch tasks. Patch generation should still be read narrowly: it is a constrained semantic patcher, not a broad autonomous coding system.
Sources and Scope
- RepoIntel artifact: local RepoIntel-Bench public run with 59 tasks and evaluated patch outputs.
- Latest verified score: 0.9979 from a pinned public-corpus RepoIntel-Bench run.
- Runtime note: benchmark corpus and adapter state are stored on the large data disk; private local calibration repos, prompt-path hints, and fixture patch templates are excluded from this public score. Patch generation uses a constrained semantic Python-service patcher.
RepoIntel results are benchmark-scoped. The current score should be read as a RepoIntel-Bench adapter result, not a general claim about all repository-intelligence workloads. A clean fully indexed run can update this page when the benchmark environment is pinned for publication.
Product Link
Use RepoIntel
The product page connects this RepoIntel-Bench result to the hosted MCP plan, Developer Bundle inclusion, and checkout path.