refactor(experimental): add prefixCodec to @solana/codecs-core

[lorisleiva]

This PR adds a new prefixCodec primitive to @solana/codecs-core. This primitives accepts any codec and prefixes its size using a NumberCodec provided as a second argument.

For instance:

const codec = prefixCodec(getBase58Codec(), getU32Codec());

codec.encode('hello world');
// 0x0b00000068656c6c6f20776f726c64
//   |       └-- Our encoded base-58 string.
//   └-- Our encoded u32 size prefix.

This PR also updates the getStringCodec and getBytesCodec as they can now use the prefixCodec primitive to simplify their internal logic.

P.S.: Now that getStringCodec and getBytesCodec are simple aggregator of fixCodec and prefixCodec, we should consider removing them or adjusting them further. (Pros: no default values => better tree-shakeability, Cons: slightly less friendly API?).

// With getStringCodec.
getStringCodec();
getStringCodec({ encoding: getBase58Codec() });
getStringCodec({ size: 32 });
getStringCodec({ size: getU16Codec() });
getStringCodec({ size: "variable" });

// Without getStringCodec.
prefixCodec(getUtf8Codec(), getU32Codec());
prefixCodec(getBase58Codec(), getU32Codec());
fixCodec(getUtf8Codec(), 32);
prefixCodec(getUtf8Codec(), getU16Codec());
getUtf8Codec();

[changeset-bot]

🦋 Changeset detected

Latest commit: 2a1b158

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 35 packages

Name	Type
@solana/codecs-data-structures	Patch
@solana/codecs-strings	Patch
@solana/codecs-core	Patch
@solana/codecs	Patch
@solana/transactions	Patch
@solana/accounts	Patch
@solana/addresses	Patch
@solana/keys	Patch
@solana/rpc-api	Patch
@solana/rpc-graphql	Patch
@solana/rpc-types	Patch
@solana/transaction-confirmation	Patch
@solana/codecs-numbers	Patch
@solana/options	Patch
@solana/web3.js-experimental	Patch
@solana/sysvars	Patch
@solana/compat	Patch
@solana/web3.js-legacy-sham	Patch
@solana/programs	Patch
@solana/rpc-subscriptions-api	Patch
@solana/signers	Patch
@solana/instructions	Patch
@solana/rpc-parsed-types	Patch
@solana/rpc	Patch
@solana/rpc-subscriptions	Patch
@solana/rpc-transformers	Patch
@solana/assertions	Patch
@solana/errors	Patch
@solana/functional	Patch
@solana/rpc-spec-types	Patch
@solana/rpc-spec	Patch
@solana/rpc-subscriptions-spec	Patch
@solana/rpc-subscriptions-transport-websocket	Patch
@solana/rpc-transport-http	Patch
@solana/webcrypto-ed25519-polyfill	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[lorisleiva]

• refactor(experimental): rename prefixCodecSize to addCodecSizePrefix #2417
• refactor(experimental): nit: rename Endian enum variants #2414
• refactor(experimental): remove getStringCodec in favour of fixCodecSize and prefixCodecSize #2413
• refactor(experimental): remove the size option of getBytesCodec #2412
• refactor(experimental): rename fixCodec and prefixCodec to fixCodecSize and prefixCodecSize #2411
• refactor(experimental): add getZeroableNullableCodec and getZeroableOptionCodec #2410
• refactor(experimental): refactor getNullableCodec and getOptionCodec to use getUnionCodec #2409
• refactor(experimental): add getHiddenPrefixCodec and getHiddenSuffixCodec to @solana/codecs-data-structures #2401
• refactor(experimental): add getConstantCodec to @solana/codecs-data-structures #2400
• refactor(experimental): refactor getDiscriminatedUnionCodec by using new getUnionCodec primitive #2399
• refactor(experimental): add getUnionCodec to @solana/codecs-data-structures #2398
• refactor(experimental): add prefixCodec to @solana/codecs-core #2397 👈
• refactor(experimental): nit: extract any types to local codec types #2396
• refactor(experimental): support more scalar values in getDiscriminatedUnionCodec #2395
• refactor(experimental): add getLiteralUnionCodec to @solana/codecs-data-structures #2394
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @lorisleiva and the rest of your teammates on Graphite

[steveluscher]

tl;dr unless I missed it, I think you need something to prevent the actual size of the data from being larger than that which you can represent using the length prefix. We either need to throw when that's the case, or truncate the data to the actual length that will eventually be decoded.

[steveluscher]

[lorisleiva]

Merge activity

• Apr 2, 6:48 PM EDT: @lorisleiva started a stack merge that includes this pull request via Graphite.
• Apr 2, 6:53 PM EDT: Graphite rebased this pull request as part of a merge.
• Apr 2, 6:54 PM EDT: @lorisleiva merged this pull request with Graphite.

[github-actions]

🎉 This PR is included in version 1.91.4 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[github-actions]

Because there has been no activity on this PR for 14 days since it was merged, it has been automatically locked. Please open a new issue if it requires a follow up.