A Minimum Viable Protocol for Reproducing AI Papers

Stage 0: define the reproduction claim

Choose the exact table row, curve, qualitative behavior, or failure mode to recover. State the allowed tolerance and which original resources are available.

• Do not promise the entire paper.
• Choose a baseline and one target result.
• Write the acceptance rule before execution.

Stage 1: recover the environment

Record operating system, accelerator, package versions, seeds, data snapshot, preprocessing, and command sequence. A successful install is an environment result, not yet a scientific result.

Stage 2: run a smoke-scale test

Use a small data slice or reduced schedule to verify shapes, losses, logging, checkpoints, and evaluation. This stage catches implementation errors cheaply but cannot validate the reported performance.

Stage 3: match the target result

Run the smallest configuration capable of testing the stated claim. Preserve raw logs and compare against the predeclared tolerance. Report deviations even when they make the reproduction less impressive.

Stage 4: test one boundary

Change one meaningful condition: seed, dataset slice, model size, evaluator, or prompt. The goal is not exhaustive robustness, but evidence about whether the result is brittle.

• One variable per boundary test.
• Keep the original run as reference.
• Label transfer experiments as new claims.

Report the outcome precisely

Use outcome labels such as environment recovered, smoke test passed, target matched, partially matched, not matched, or blocked by missing artifacts. This page defines a protocol; it does not claim that Fineuralab has reproduced a particular paper.