Consider encoding: Safebase

[DonaldTsang]

https://github.com/kstenerud/safe-encoding has safe16, safe32, safe64, safe80 and safe85 for HTML/XML, JSON, URL and POSIX file names.

[DonaldTsang]

https://github.com/tbaumgard/hybrid64 also has a better version of "zbase64"

[DonaldTsang]

@Stebalien @lidel so, what alternate encoding can we bring on to the table to make multibase more diverse?

[DonaldTsang]

I also kept a list of other bases in kstenerud/safe-encoding#3 and kstenerud/safe-encoding#5 and kstenerud/safe-encoding#6

[Stebalien]

We have a very limited namespace here (there are only so many prefix characters we can use) so making multibase "more diverse" isn't really something I'd like to do (we already have several useless encodings I'd like to drop).

However, I do like those encodings. Are there any projects currently using them?

[DonaldTsang]

@Stebalien sadly right now there are no projects reported that are using those encodings.

But for diversity's sake, is it possible to "hack" the namespaces by using two-byte blocks? I would hope that such safe encodings could gradually phase out RFC-like encoding for some use cases.

[Stebalien]

But for diversity's sake, is it possible to "hack" the namespaces by using two-byte blocks?

We've thought about this a bit. We could, e.g., allocate the s prefix for "safe" encodings. I'd be fine with this from a spec standpoint, but implementing it will be a bit of work.

Regardless, I'm not sure how to break the adoption/implementation chicken and egg situation. For us, the order-preserving property really isn't a sufficient motivation to implement this across the board and switch to it unless there's some kind of critical mass.

[DonaldTsang]

@Stebalien the other option would be to instead create a human-oriented translation layer between safebase hash encodings used externally and actual RFC encoded hashes used internally.

[Stebalien]

Not sure I follow.

[DonaldTsang]

@Stebalien external IO and human input would be done in safebase, then there will be a translation layer between the multibase itself and the input to convert it to zbase32 and RFC-compliant base64, such that the user does not need to handle RFC-compliant zbase32 or base64 or other encodings within multibase.

[DonaldTsang]

@lidel sorry, maybe this sound weird but it is a Christmas wish to see this through.

[lidel]

Internally, in IPFS/libp2p/IPLD, we use unwrapped, binary multihash, so base does not matter outside userland. Creating translation layer for text representations sounds like reinvention of CIDv1, which already allows a single multihash to be represented in multiple bases when in text form.

Looking at it from pragmatic perspective, adding safebase support to CID means not only adding it to the list in this repo, but to every library implementing it, such as go-cid, js-cid etc. This is a significant effort and given the fact that safebases are not being used in real world won't happen until there are libraries, community need or desired characteristic that is hard to ignore.

For now, in contexts where safehash provides real value, one can use it "in userland": extract binary multihash from CID (binary_multihash = new CID('QmHash').multihash) and do safehash(binary_multihash).

[mikeal]

Internally, in IPFS/libp2p/IPLD, we use unwrapped, binary multihash, so base does not matter outside userland.

Mostly, yes. dag-json uses base encoded CID’s internally in its format but uses a consistent base encoding and should probably be updated to not support alternative base encodings in order to maintain consistency.