OMW DEPLOYMENT BENCHMARK · PROTOCOL v1.0.0
Measurements you can reproduce.
Benchmarks are accepted only when tied to an exact model revision, runtime version, hardware environment and public evidence artifact. No marketing numbers and no extrapolation.
Accepted results0
Warmup runs2
Measured runs5
Aggregationmedian
CORE METRICS
Deployment performance, not a quality leaderboard.
time to first token msms
output tokens per secondtokens/s
peak device memory gbGB
peak host memory gbGB
ACCEPTED EVIDENCE
Observed benchmark runs.
Every row must link back to public run evidence.
No accepted benchmark measurements yet.
The protocol and schema are live first. Measurements will appear only after they include exact model revision, runtime version, hardware, raw configuration and public evidence.
Protocol rules
- No extrapolated or vendor-marketing numbers.
- Every accepted result must identify the exact source repository revision and runtime version.
- Results from different precisions, quantizations, runtimes or hardware are separate observations.
- Weight-only memory estimates from the registry are not benchmark measurements.
- Raw command/config and a public evidence artifact URL are required for accepted community submissions.