Request for technical feedback on reproducible ML-DSA cross-implementation verification

[mokkunsuzuki-code]

Hello Open Quantum Safe community,

I am looking for technical feedback on a narrowly scoped ML-DSA independent-reproduction workflow.

I maintain an open verification project called QSP.

Current Stage391 repository:

https://github.com/mokkunsuzuki-code/stage391

Public verification page:

https://mokkunsuzuki-code.github.io/stage391/

This is not a vulnerability report, certification request, or request for endorsement.

The part I would most value OQS feedback on is the ML-DSA verification and cross-implementation reproduction path developed in the preceding QSP stages and carried into Stage391.

The current work includes:

• ML-DSA signature verification evidence,
• independent verifier / multiple-implementation checks,
• deterministic reproduction,
• exact SHA-256 evidence binding,
• explicit environment metadata,
• Fail-Closed rejection of contradictory assessment claims,
• separation between developer self-tests and genuine independent third-party reproduction.

Exact Stage391 assessment commit:

a74b7f47f8caabf37a910ea9f4da2b87c0cd5f15

Canonical Stage391 result SHA-256:

ed644d11bd49f67f89cfda50364d619066b4da3a36bf1fb26b38e111b6092b23

The authoritative Stage391 state is intentionally:

third_party_submission_pending

verification_status:

waiting_for_external_submission

No independent external assessment has been accepted yet.

My question is deliberately limited:

From the perspective of people familiar with liboqs / ML-DSA implementations, is this type of cross-implementation and reproducible verification methodology technically meaningful?

I would especially appreciate any of the following:

1. identification of an invalid ML-DSA assumption,
2. identification of a missing interoperability test,
3. identification of a reproducibility weakness,
4. an independent reproduction of the ML-DSA-related path,
5. advice on a better way to compare implementations or verification outputs,
6. or direction to a more appropriate OQS venue or maintainer.

A full review of QSP is not required.

Negative results are fully acceptable. A disagreement, mismatch, or concrete methodological criticism is as useful as agreement.

A self-contained assessor handoff package is prepared, with:

• exact commit binding,
• SHA-256 manifest,
• environment schema,
• reproduction result schema,
• agreement / disagreement / incomplete outcome format.

I am not attaching the archive unsolicited, but I can provide it if someone wants to reproduce the evidence.

One upstream state also remains intentionally unresolved:

Stage389 = dual_timestamp_pending

Stage389 records RFC3161 verification as verified, while OpenTimestamps / Bitcoin verification remains pending.

QSP does not promote that pending state to success.

Related external requests already opened:

mldsa-native:
pq-code-package/mldsa-native#1393

PQCA Readiness Tracking WG:
PQCA/wg-readiness-tracking#41

Thank you for any technical criticism, reproduction result, or direction.

Best regards,
Motohiro Suzuki

[Kxrma47]

The methodology is meaningful as an evidence-preservation and reproducibility workflow, but it should be careful not to equate “two implementations produced the same result” with an independent validation of ML-DSA correctness.

For ML-DSA, I would separate four claims and test each explicitly:

1. Known-answer conformance: run the official ACVP/NIST vectors for key generation, signing, and verification, including deterministic and hedged signing where supported.
2. Cross-implementation interoperability: generate keys and signatures in implementation A and verify in B, then reverse the direction for every supported parameter set. Include ML-DSA-44, 65, and 87 rather than only one profile.
3. Negative behavior: mutate the public key, message, context, signature hint, challenge, and length fields; verification must reject. Include truncated/extended encodings and non-canonical values. Agreement on valid fixtures alone misses a large class of parser and validation errors.
4. Reproduction integrity: pin source commits, compiler/toolchain, flags, CPU architecture, dependencies, test-vector revision, and container/base-image digest. Hashes prove which bytes were assessed, but the manifest must also describe how those bytes were produced.

Two additional assumptions deserve explicit treatment:

• ML-DSA contexts are part of the signed input. Test empty, maximum-length, and mismatched contexts where the APIs expose them.
• Randomized/hedged signing should not be compared by expecting identical signatures. Compare successful cross-verification and record the randomness mode; byte equality is appropriate only for a deterministic, identically specified path.

For independence, avoid counting two wrappers around the same underlying implementation as two implementations. Record the implementation lineage (for example, whether both ultimately call the same native code). A stronger matrix uses separately maintained codebases and includes at least one implementation not derived from the same upstream source.

The fail-closed state model and explicit third_party_submission_pending status are good practices. I would add a machine-readable claim table in which every claim points to its exact command, inputs, output, exit status, and artifact hashes. That makes partial reproduction possible without forcing an assessor to endorse the entire project.

Finally, RFC 3161/OpenTimestamps evidence establishes ordering or existence of artifacts, not cryptographic correctness. Keeping timestamp status separate from ML-DSA conformance status, as you describe, is the correct boundary.

Relevant upstream starting points:

• https://csrc.nist.gov/pubs/fips/204/final
• https://github.com/open-quantum-safe/liboqs
• https://github.com/pq-code-package/mldsa-native

The most useful next deliverable would be a small public interoperability corpus: versioned keys/messages/contexts/signatures, expected accept/reject outcomes, provenance for each producer, and a runner that executes the same corpus against every implementation.

[mokkunsuzuki-code]

Thank you — this is extremely helpful. The distinction between agreement and correctness is especially important. I’ll make sure not to treat two implementations producing the same result as independent validation of ML-DSA correctness. I’m going to separate the work into the four claims you outlined: known-answer conformance, cross-implementation interoperability, negative behavior, reproduction integrity. I also want to incorporate the other boundaries you pointed out: implementation lineage, context handling, deterministic vs. hedged signing, and keeping timestamp evidence separate from ML-DSA conformance. The machine-readable claim table is particularly useful. I like the idea that an assessor should be able to reproduce and agree or disagree with one specific claim without having to endorse the whole project. I’ll also work toward the small public interoperability corpus you suggested, with versioned inputs, expected accept/reject outcomes, producer provenance, and a common runner. One question as I start designing the corpus: do you think it would be better to keep the format implementation-neutral, and reference ACVP/NIST vectors as one conformance source, rather than structuring the corpus itself around the ACVP format? Thanks again for taking the time to give such detailed feedback.

…

2026/08/31 8:42、Mahidul Haque ***@***.***>のメール: The methodology is meaningful as an evidence-preservation and reproducibility workflow, but it should be careful not to equate “two implementations produced the same result” with an independent validation of ML-DSA correctness. For ML-DSA, I would separate four claims and test each explicitly: Known-answer conformance: run the official ACVP/NIST vectors for key generation, signing, and verification, including deterministic and hedged signing where supported. Cross-implementation interoperability: generate keys and signatures in implementation A and verify in B, then reverse the direction for every supported parameter set. Include ML-DSA-44, 65, and 87 rather than only one profile. Negative behavior: mutate the public key, message, context, signature hint, challenge, and length fields; verification must reject. Include truncated/extended encodings and non-canonical values. Agreement on valid fixtures alone misses a large class of parser and validation errors. Reproduction integrity: pin source commits, compiler/toolchain, flags, CPU architecture, dependencies, test-vector revision, and container/base-image digest. Hashes prove which bytes were assessed, but the manifest must also describe how those bytes were produced. Two additional assumptions deserve explicit treatment: ML-DSA contexts are part of the signed input. Test empty, maximum-length, and mismatched contexts where the APIs expose them. Randomized/hedged signing should not be compared by expecting identical signatures. Compare successful cross-verification and record the randomness mode; byte equality is appropriate only for a deterministic, identically specified path. For independence, avoid counting two wrappers around the same underlying implementation as two implementations. Record the implementation lineage (for example, whether both ultimately call the same native code). A stronger matrix uses separately maintained codebases and includes at least one implementation not derived from the same upstream source. The fail-closed state model and explicit third_party_submission_pending status are good practices. I would add a machine-readable claim table in which every claim points to its exact command, inputs, output, exit status, and artifact hashes. That makes partial reproduction possible without forcing an assessor to endorse the entire project. Finally, RFC 3161/OpenTimestamps evidence establishes ordering or existence of artifacts, not cryptographic correctness. Keeping timestamp status separate from ML-DSA conformance status, as you describe, is the correct boundary. Relevant upstream starting points: https://csrc.nist.gov/pubs/fips/204/final https://github.com/open-quantum-safe/liboqs https://github.com/pq-code-package/mldsa-native The most useful next deliverable would be a small public interoperability corpus: versioned keys/messages/contexts/signatures, expected accept/reject outcomes, provenance for each producer, and a runner that executes the same corpus against every implementation. — Reply to this email directly, view it on GitHub <#2533?email_source=notifications&email_token=BYN6ZAWUP2CTLXGILDJBBU35MS3VJA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOBSGEYDSNBVUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-18210945>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BYN6ZAS373SUDCJKJLPA2AD5MS3VJAVCNFSNUABHKJSXA33TNF2G64TZHM3DKNJRGQZDANJ3IRUXGY3VONZWS33OHMYTANRWGAZDQNNBOYBA>. Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/BYN6ZATKGQQEQNUUCQ7QF6D5MS3VJA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOBSGEYDSNBVUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVJTG633UMVZF62LPOM> and Android <https://github.com/notifications/mobile/android/BYN6ZASBTTYKONBP6ULTD2L5MS3VJA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOBSGEYDSNBVUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVZTG633UMVZF6YLOMRZG62LE>. Download it today! You are receiving this because you authored the thread.

[Kxrma47]

Yes—keep the corpus format implementation-neutral, and treat ACVP/NIST vectors as one provenance-tagged conformance source rather than making the whole corpus an ACVP-shaped wrapper.

I would give each case a small stable envelope containing: schema version; algorithm and parameter set; operation; message, key, signature, and context bytes in an explicit encoding; deterministic or hedged mode; expected accept/reject result; producer implementation and pinned commit; source/provenance; and hashes of every artifact. Preserve imported ACVP vectors unchanged and reference their source identifiers, with a separate adapter that maps them into this envelope. That makes it possible to verify the mapping and avoids silently changing the authoritative vector.

Then keep runner adapters implementation-specific. Each adapter consumes the same neutral case and emits a normalized result containing the implementation version, command, exit status, verification outcome, and output hashes. Negative cases and cross-produced signatures fit the same schema, while ACVP-specific metadata can live in an optional provenance object rather than becoming mandatory for every case.

This separation matters because ACVP answers “does this implementation conform to these standardized vectors?”, while the shared corpus also needs to express interoperability, malformed-input rejection, implementation lineage, and reproduction metadata. It also lets you add Wycheproof or project-generated regression cases later without revising the core format. liboqs itself follows a similar separation by consuming NIST ACVP vectors in its test harness rather than defining its public API around ACVP: https://github.com/open-quantum-safe/liboqs and https://github.com/usnistgov/ACVP-Server/tree/master/gen-val/json-files.

I would version the neutral schema independently, publish a JSON Schema plus a few canonical examples, and require that adapters preserve the original fixture hash. That gives assessors a compact common contract without confusing corpus interoperability with NIST conformance.