Provable AI inference on a gaming GPU

When an AI system answers, how do you know which model actually produced the answer?

Anthropic's open-source research prototype FlashProve gives a rigorous answer: a zero-knowledge proof that a reply came from a specific committed set of model weights, which anyone can check on an ordinary CPU. Upstream supports one prover platform: a dedicated NVIDIA H100.

Two other teams have already moved it off the H100: FlashProve-L4 on a datacenter L4, and an Apple Silicon rewrite. We asked a narrower question: what does it take, and what fits, on a store-bought gaming card in an ordinary Windows PC?

The short answer

12 small patches to the prover (+34 / −28 lines), and none to the verifier, protocol or proof format.
On a GeForce RTX 5070 Ti (16 GB, consumer Blackwell, also driving the monitor), under Windows' Linux subsystem:
- the production build proves a real pretrained model's next token (Karpathy's TinyStories260K, in FlashProve-L4's integer form). The answer token is 310 in every run, the same as L4's published result and the floating-point model's own choice;
- layers at Llama 4 Maverick's full width (5,120 wide, 40 heads) commit in about 43 s and prove in 3–6 s;
- the prover serves full batches of 64 requests, the format's maximum, with GPU memory peaking at 9.3 GiB.
770 of 770 fresh proofs were accepted by upstream's unmodified verifier, built separately. 72 of 72 tampered inputs were rejected.

What fits in 16 GB

	Small synthetic model	Maverick-width layers (4 layers)
Commit	15–26 s	~43 s (it deliberately takes the free GPU memory)
Prove one request	~2 s	3–6 s
Batch of 64: GPU peak while proving	6.7 GiB	9.3 GiB
Steady rate at batch 64 (rough)	~110 tokens/s	~295 tokens/s

Not yet measured: full depth (48 layers), the full vocabulary and 1,024-token windows. Upstream ships no tool to make configurations at new sizes, so that is our next problem.

Why a patched prover does not weaken the proof, and the one place to look harder

In FlashProve, soundness lives in the verifier: a modified prover can only produce proofs that are rejected. That is why every proof is judged by the untouched upstream verifier, and why we publish tamper tests.

Zero-knowledge (hiding the weights) lives in the prover. A YES says nothing about it. Four of our patches touch the module that does this hiding work.

Argued patch by patch: the changes alter where data is stored and which equivalent kernel runs, never the random masks. Upstream states that such choices “do not change the proof's bytes”.
Tested so far: upstream's own test of that module passes on our card, and all 18 of its cheating-prover cases are refused.
Not yet verified: byte-for-byte equality with unmodified upstream. We showed our build is byte-deterministic under fixed seeds, so the next step is to compare against unmodified upstream on an H100. Until then we say so plainly.

What this does not prove

Not a statement about any served or large model. The real-model proof is about committed integers (TinyStories, 260K parameters).
Not formally verified. Upstream's Lean-proved verifier accepts only its full-size format. Our YES comes from the tested C++ verifier.
One machine, WSL2 only, and not yet independently replicated. RTX 30/40 cards and native Linux are untested. A runbook is in the repository.
No performance claim. The rates come from a few intervals on a desktop machine.
Not “the first”. The L4 port came first with a real model. We found no earlier completed proof on a consumer GeForce card, but code search cannot see forks, so we claim nothing more.

Why WHY cares

AI agents now act: they call tools, move data and approve changes. The record of what they did is mostly logs. We are building evidence that says how much it proves: observed, signed, hardware-attested, or computation-proven. Each claim is graded by its weakest link and checked by software that never trusts the issuer. Small guard models that approve or block an agent's action are exactly the size proofs reach today, on hardware like this.

The next note does exactly that: a guard model's decision, proved for one exact agent action.

Have an RTX 30/40/50 card? Replicate this Talk to us