Nebius × NVIDIA Global AI Hackathon

Receipts

Papers claim numbers. Receipts re-runs them. Paste an arXiv link and an agent extracts the headline claim, finds the code, runs the reproduction over many seeds in a sandbox, and returns a verdict with a receipt anyone can re-run: the executed code, the numbers, and the reasoning.

NVIDIA Nemotron Nano · Super · Ultra Nebius Token Factory inference Tavily repo, errata, replications 4 verdicts incl. "inconclusive"

Watch it work

Recorded replays of real runs. Pick a case to replay it. Cases tagged SYNTHETIC are planted-error tests with a known right answer, used to measure the agent honestly.

How it works

Nemotron Super1. Extract the claimReads the paper and picks the headline number that can actually be re-run.

Tavily2. Find code, data, errataSearches for the repo, datasets, corrections and prior replication attempts. Nano filters out pages that only mention the paper.

Nemotron Super3. Write the reproductionGenerates a script that runs the repo's own evaluation, and repairs it if the first run breaks. Scripts that hard-code the claimed number are rejected.

Sandbox containers4. Run many seeds in parallelEach seed runs in a locked-down container (capabilities dropped, resource limits). Same image works as a Nebius Serverless Job.

Statistics5. Decide honestlyCompares the claim with the distribution over seeds. If runs break, the verdict is inconclusive, never an accusation. A shrunken run can never count against a paper.

Nano + Ultra6. Explain the gapNano summarises logs. Ultra classifies why a number differs (leakage, metric, hyperparameters...) using the computed variants as evidence.

Run it yourself

python -m receipts run https://arxiv.org/abs/XXXX.XXXXX --runner docker --seeds 10
python -m receipts bench bench/planted.json --seeds 8
python -m receipts verify out/receipt-<id>.json   # re-executes a receipt, checks its hash