You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The base64 primitives are duplicated across modules — a legacy artifact of the six-crate→one-crate merge (700f5fb, "copy CESR/KERI sources from agency"). Each original crate (cesr-utils, cesr-core, cesr-stream, keri-core) carried its own copy; post-merge they were never deduplicated. This blocks P1.2 (#29): optimizing base64 is premature while there are 3–5 copies to optimize.
This is duplication, not a correctness bug — every copy uses the identical URL-safe alphabet and passes its own keripy vectors (gate is green). The risk is maintenance drift and bug-surface, not current wrongness.
What is genuinely duplicated (keripy has ONE of each)
stream/util.rs::int_to_b64 + b64_to_int (u64, own alphabet + own char match)
core/indexer/indexer.rs::int_to_b64 (u32, own alphabet)
Consumers are inconsistent: stream/unwrap.rs + indexer/builder.rs already call crate::utils::*, but stream/message.rs + indexer/indexer.rs call the local copies.
URL-safe alphabet / reverse table — 5 copies:utils/utils.rs (B64_URL_CHARS+B64_DECODE_INDEX), stream/util.rs, stream/binary.rs, core/indexer/indexer.rs, utils/b64.rs (added on the #29 branch — to be folded in or reverted).
What is NOT duplication (legitimate keripy domain structure — keep separate)
keripy intentionally has two domains: text (qb64) and binary (qb2), via _infil/_exfil (qb64) and _binfil/_bexfil (qb2). So:
stream/binary.rs (qb64_to_qb2/qb2_to_qb64) legitimately mirrors keripy codeB64ToB2/_bexfil — the binary domain. It stays a distinct codec.
The raw↔qb64 codec is the text domain. Distinct from qb2.
We do NOT merge the text and binary domains. Consolidation only dedupes the shared low-level primitives (one alphabet table, one int↔b64) that both domains draw from.
Proposed scope
One canonical alphabet + reverse table in utils (the single source of truth).
One generic int↔b64 in utils; delete stream/util.rs + indexer copies, repoint callers.
Keep stream/binary.rs qb2 codec (rebuilt on the canonical alphabet table, logic unchanged).
Prerequisite for #29 (P1.2 base64 fast path). #29 is paused pending this; its baseline bench + spec + plan live on branch perf/p1.2-base64-baseline-bench. Once consolidated, #29's fast-path optimizes the single canonical module.
Summary
The base64 primitives are duplicated across modules — a legacy artifact of the six-crate→one-crate merge (
700f5fb, "copy CESR/KERI sources from agency"). Each original crate (cesr-utils,cesr-core,cesr-stream,keri-core) carried its own copy; post-merge they were never deduplicated. This blocks P1.2 (#29): optimizing base64 is premature while there are 3–5 copies to optimize.This is duplication, not a correctness bug — every copy uses the identical URL-safe alphabet and passes its own keripy vectors (gate is green). The risk is maintenance drift and bug-surface, not current wrongness.
What is genuinely duplicated (keripy has ONE of each)
Integer ↔ B64 (keripy
intToB64/b64ToInt) — 3 copies:utils/encode.rs::encode_int+utils/decode.rs::decode_to_int(genericPrimInt)stream/util.rs::int_to_b64+b64_to_int(u64, own alphabet + own char match)core/indexer/indexer.rs::int_to_b64(u32, own alphabet)Consumers are inconsistent:
stream/unwrap.rs+indexer/builder.rsalready callcrate::utils::*, butstream/message.rs+indexer/indexer.rscall the local copies.URL-safe alphabet / reverse table — 5 copies:
utils/utils.rs(B64_URL_CHARS+B64_DECODE_INDEX),stream/util.rs,stream/binary.rs,core/indexer/indexer.rs,utils/b64.rs(added on the #29 branch — to be folded in or reverted).What is NOT duplication (legitimate keripy domain structure — keep separate)
keripy intentionally has two domains: text (
qb64) and binary (qb2), via_infil/_exfil(qb64) and_binfil/_bexfil(qb2). So:stream/binary.rs(qb64_to_qb2/qb2_to_qb64) legitimately mirrors keripycodeB64ToB2/_bexfil— the binary domain. It stays a distinct codec.We do NOT merge the text and binary domains. Consolidation only dedupes the shared low-level primitives (one alphabet table, one
int↔b64) that both domains draw from.Proposed scope
utils(the single source of truth).int↔b64inutils; deletestream/util.rs+indexercopies, repoint callers.stream/binary.rsqb2 codec (rebuilt on the canonical alphabet table, logic unchanged).utils/b64.rs(the P1.2 · Base64 fast path (CESR inner loop) #29 scalar codec) — fold into the canonical module or revert.Constraints / acceptance
keripy_diffdifferential vectors stay green.nix flake checkgreen, incl.cesr-nostd/cesr-wasm; unsafe-free; no lint relaxation.Relationship to #29
Prerequisite for #29 (P1.2 base64 fast path). #29 is paused pending this; its baseline bench + spec + plan live on branch
perf/p1.2-base64-baseline-bench. Once consolidated, #29's fast-path optimizes the single canonical module.