The benchmark didn't lie. The library changed underneath it.

Nothing about the math changed. The labels did — and if you were diffing benchmark output by algorithm name instead of by what the algorithm actually does, you’d have silently lost your own historical series and never noticed.

The thing static numbers can’t survive

Here’s what happened on our own box. liboqs 0.15 shipped, and SLH-DSA’s two most-used parameter sets stopped being called what they used to be called. They now expose as SLH_DSA_PURE_SHAKE_128S and SLH_DSA_PURE_SHAKE_128F — the PURE prefix and the underscored form are new. Nobody emailed us. Nothing in the release notes was framed as a breaking change for a benchmark pipeline. We just started getting output that didn’t match the shape of last month’s, and had to go figure out why.

That’s a small, boring, entirely typical event in the life of a cryptography library. It’s also the whole argument for this post. A report commissioned in March and delivered in June cannot know that the algorithm it measured got renamed in between — worse, it can’t know whether the numbers moved too, because nobody re-ran it to check. A dashboard that runs daily catches the rename the morning it ships, because the parser either finds the name it expects or it doesn’t.

Why this matters more in PQC than anywhere else

Every FIPS-approved post-quantum algorithm is new enough that its implementations are still moving targets. Silicon changes — we run the same suite on Intel Xeon and on AWS Graviton3 and get meaningfully different pictures. Compilers change. And the reference libraries themselves are still being actively developed: liboqs ships new releases with performance regressions, performance wins, and cosmetic renames, on a cadence measured in months.

The standards bodies gave you three approved signature schemes and told you they’re all valid. They did not tell you that the roughly seven-and-a-half-thousand-fold gap between the fastest and slowest of them is a number that will look different again next quarter — not because the cryptography changed, but because the library implementing it did. A benchmark that can’t tell you when its numbers were true is a benchmark you can’t rely on to make a deployment decision.

There’s a compliance clock underneath all of this, which is what turns interesting engineering trivia into someone’s actual deadline. NIST IR 8547, the EU’s NIS-CG roadmap, and the UK NCSC’s migration timeline all converged independently on the same shape: deprecation of RSA and ECC starting 2030, full disallowance by 2035. Three regulatory bodies, one target, arrived at separately. That’s a real deadline, which means the algorithm-selection decisions being made against it are real decisions — and a real decision deserves a number that’s still true when someone checks it.

This is the case for why eBACS is our friend rather than a competitor. eBACS and the liboqs speed tools are the reference the field already trusts for raw cycle counts. We don’t try to beat those numbers or replace them — every Q-Shield run is cross-checked against them as a sanity floor. What we add is the part a point-in-time reference table structurally cannot provide: the same measurement, repeated on the same hardware, every day, so the drift becomes visible instead of invisible. Cite-don’t-fight is the posture, because the alternative — quietly disagreeing with the field’s own reference numbers without saying so — is how you lose the argument before anyone reads your data.

The numbers, as of that morning

The run of 2026-07-26, 08:18 UTC, commit d8df129, put ML-DSA-65 signing at 185.1µs against SLH-DSA-SHAKE-128s at 1.397s — a 7,500× gap, both FIPS-approved, both defensible, and the standard doesn’t pick between them for you. Widen the lens and SLH-DSA’s own fast-versus-small tradeoff is 20.6×, while ML-DSA’s smallest-to-largest parameter set spans 2.02× on signing alone. None of these are exotic edge cases — they’re the first three numbers anyone comparing FIPS 204 against FIPS 205 will hit.

Cross-architecture, the picture gets more specific than “PQC is faster” or “PQC is slower.” Pure ML-KEM-768 key exchange beat X25519 by 62.5% on x86 and by 47.6% on Graviton3 — both real, both current, and notably different from each other. But the number every TLS-facing team actually needs is the hybrid figure, because nobody ships pure ML-KEM alone yet: X25519 combined with ML-KEM-768, as an actual TLS handshake, ran 38.3% slower than plain X25519 on x86 and 52.3% slower on ARM — because a hybrid handshake pays for both algorithms, not instead of the classical one. That’s the honest answer to “what does turning on hybrid PQC cost me,” and it’s a different question from “is ML-KEM fast.”

It’s not an X25519 quirk either. Swap the classical side for SecP256r1 — the NIST curve rather than the Bernstein one — and the hybrid handshake still ran slower than the classical baseline alone: 48.9% on ARM, 37.5% on x86. Different curve, same shape of cost. If your stack is standardized on P-256 because of internal policy or an HSM constraint, that’s the figure that applies to you, and it’s not the one most people quote.

“Continuous” isn’t a slogan here — it’s a scheduled job, running daily, writing to a public run history that supersedes rather than overwrites. Every number on the dashboard links back to the run that produced it. That’s the mechanical difference between an instrument and a snapshot: a snapshot is a document someone emails you once; an instrument is something you can check again next Tuesday and get an answer, including “it changed, and here’s exactly when.”

What this means if you’re the one signing off

If you’re choosing between ML-DSA and SLH-DSA for a system that has to survive an auditor’s questions in eighteen months, the number you cite today needs to still be true when someone checks it. That means two things in practice. First: don’t copy a PQC performance figure out of a vendor deck or an analyst report into an architecture decision record without a date and a source attached — the figure has a shelf life, and the record should say so. Second: prefer a source that re-runs itself. Not because continuity is a feature we’re selling, but because a number that can’t be re-verified against a live, dated run is functionally a rumour, no matter how official the document it’s printed in looks.

The practical test is simple. Ask whoever handed you a PQC benchmark number: what commit, what date, what hardware, and does it still match if you ran it again this morning? If the answer is a shrug, you don’t have a measurement — you have a press release with a chart attached.

If you’re the one who has to explain the choice upward rather than make it, the same test just moves up a level. “Why this algorithm” is a question an auditor will eventually ask in exactly that form, and “a vendor told us” doesn’t survive it. “Here’s the run, here’s the commit, here’s the date, and here’s what changed since” does.

The takeaway

A library update quietly renamed two of the algorithms we benchmark daily, and the only reason it cost us an hour instead of costing someone else a bad migration decision is that we were looking the same week it happened, not the same quarter. That’s not a flattering story about foresight. It’s an honest one about why a snapshot and an instrument are not the same kind of thing — and receipts, not press releases only means something if the receipts are dated.