Reverify checks AI agent claims against the real artifact

What this is about

Reverify is an open-source verification tool for AI agents. Instead of accepting a technically convincing statement, it checks the statement against a real artifact: bytes in a binary, machine instructions, or the behavior of two implementations. The result is VERIFIED, REFUTED, or INCONCLUSIVE, accompanied by evidence.

The project appeared on GitHub in late August 2026 and is available as a CLI, Python package, and MCP server. Its main focus is authorized binary analysis. The underlying idea is broader: a language model may propose a hypothesis, but a deterministic tool decides whether it is correct.

What Reverify actually does

Reverify reads PE, ELF, and Mach-O files, disassembles x86, x64, ARM, and ARM64 code, and can check patterns, strings, imports, exports, or memory regions. Optional libraries including Capstone, Unicorn, LIEF, Z3, and angr extend analysis, emulation, and proofs; without them, a pure-Python core remains usable.

A claim is supplied as a structured request. The verifier compares it with the artifact and returns the observed value, addresses, engines used, and a file hash. False claims produce a failing status, allowing the workflow to act as a gate in CI pipelines. For ordinary source code, there is also an equivalence test: a reference and candidate implementation run on shared inputs and their outputs are compared.

For agents, Reverify exposes these functions through MCP. A local ledger stores confirmed and refuted statements for each file. After a context reset, an agent can therefore recover verified facts instead of treating a free-form summary as truth.

Why it matters

In reverse engineering, a plausible explanation is not enough. An invented structure size, wrong offset, or imagined function boundary can invalidate everything that follows. Reverify moves the final decision out of the language model and into reproducible tools.

The maintainer publishes a reproducible benchmark and CI artifacts. According to the project documentation, an unaided model answer was wrong for 97 percent of 71 Windows system files, while Reverify accepted none of those wrong claims. This is a project claim, not an independent product study. The useful part is that scripts, result files, and verification runs are publicly inspectable. OpenSSF Scorecard and Codecov add further, though limited, visibility into supply-chain practices and test coverage.

The tool is most useful to malware analysts, security researchers, CTF teams, and developers who want to compare an agent-generated implementation with a reference. Its MIT license makes internal adaptation straightforward.

In plain language

Reverify works like a kitchen scale when baking bread. An assistant can claim that the dough contains 500 grams of flour. The scale does not argue; it measures and shows the value. It still cannot decide whether the recipe will taste good, because it checks only the concrete, measurable claim.

A practical example

A security team examines 20 suspicious Windows files. An AI agent claims that one file contains a particular sequence of three machine instructions at offset 4096. Reverify reads that exact region and refutes the claim with the bytes it actually finds. The agent proposes a new hypothesis, which is then verified.

The team records the verified statement in the ledger and includes only confirmed facts in its report. In a second task, the agent reconstructs a small Python function from binary code. The equivalence test runs the reference and candidate with the same 100 inputs. A single differing output blocks approval. A convincing answer becomes an auditable workflow.

Scope and limits

First, Reverify can prove only claims for which a suitable verifier exists. Meaning, intent, and business logic cannot automatically be inferred from individual bytes. INCONCLUSIVE is therefore an important result, not a malfunction.

Second, the striking benchmark numbers come from the project itself. The open setup makes reproduction easier but does not replace an independent comparative study. Teams should test the workflow with their own files and known ground truth.

Third, binary analysis remains security-sensitive. Unknown files belong in an isolated environment, while emulation and optional analysis libraries expand the attack surface. Reverify is explicitly intended for authorized analysis and replaces neither legal approval nor experienced human review.

SEO & GEO keywords

Reverify, AI agent verification, reverse engineering, binary analysis, MCP server, deterministic verification, code equivalence, LLM hallucinations, open-source AI, malware analysis, agent security