/git

[SurfingNerd]

git support

Hello, i wanted to ask about support for git for multiaddress.
Something like this:
/git/5a179104a6f9e9a2c95562496c88bf514871a1d3
multiaddr would point to a commit at this repository: 5a17910 .
a client would receive the fetched data as a git repository, and even can verify the sha-1 or sha-256 of the repository.

A server implementation would require either one of those solutions

• brute force through all his stored repositories to find this hash.
• hold mapping information between commit hashes and repositories for faster lookups.
• store all git repositories within one merkle tree (?? i don't know enough details about this if this is possible and benefits the most - especially de-duplication ?!)

The Server would then either respond with the verifiable content, or with Not Found.
A P2P extension would be possible.

Security: Sha-256 vs Sha1

Since Sha1 (20 byte) is considered vulnerable and sha-256 (32 byte) is considered to be much more secure, Sha-256 seems the way to go.
Further Informations

• https://git-scm.com/docs/hash-function-transition/
• https://devclass.com/2023/08/22/git-2-42-released-sha-256-repositories-no-longer-an-experimental-curiosity/

But the reality is, that sha-1 is used in most repositories, therefore sha-1 would be required for compatibility support.

Every concern about backward compatibility would be gone, if Github and Co. would manage either go into the direction of dual hashing, or SHA-1 fades out as deprecated - but we know from other tech: this will take years...

sha-1 backward compatibility with sha-256 security extension.

the address scheme could get extended so sacrifice additional 32 bytes for delivering the sha-256 hash

i don't know exactly about the chaining rules of multiaddr, but it could look something like this:
/git/5a179104a6f9e9a2c95562496c88bf514871a1d3/123456789012345678901234567890123456789012345678901234567890abcd
(the provided hash in this example is incorrect, grateful if someone can provide the correct hash)

If you have a trusted platform that knows about the repository you want to secure,
it is possible to provide mappings for this in a multiaddr way.

Compatibility mapping for git: /git-sha1-sha256/5a179104a6f9e9a2c95562496c88bf514871a1d3 would result into a hash that could also be used for the /git/ addressing.

this concept could also support other secure hash functions, but with sha-256 it's a step towards the general adoption of sha-256.

Also it is flexible extendable for other ciphers. example; sha3 with 512 bit security: /git-sha1-sha3_512/5a179104a6f9e9a2c95562496c88bf514871a1d3

Mapping for IPFS Content Address that provides this repository: /git-ipfs/5a179104a6f9e9a2c95562496c88bf514871a1d3

[Stebalien]

Multiaddrs are for addressing endpoints (machines, processes, etc.), not data. A multiaddr is a sequence of recursive instructions for how to connect to some endpoint. For git commits, we'd just use a CID (there's even a multicodec for that). cc @aschmahmann who has played with this kind of thing.

Also see multiformats/multiformats#55 (which I unfortunately no longer have the bandwidth to work on). There's no reason /git/CID can't be a thing (it really should be a thing), it just isn't a multiaddr (it would be a "namespace" like /ipfs, /ipns, etc...). One option is to register a git-namespace (search https://github.com/multiformats/multicodec/blob/master/table.csv for "namespace" to see what I mean).

[aschmahmann]

Thanks @Stebalien for the callout. @SurfingNerd the below is longer than I intended, but I wanted to both try and disambiguate a few of the options here (for the /git case) as well as put a reference for others involved in multiformats and multiaddr to give their thoughts on some of the more general questions this brings up.

From what I can tell there are three main ways you might want to reference git data:

• Machine addressing (i.e. refer to where a machine is):
  • You want to refer to the machine you want to interact with along with some information about the protocol you're using. For example, as an alternative to git clone git@github.com:multiformats/multiaddr perhaps having something roughly along the lines of git clone /dns/github.com/git/multiformats/git-path/multiaddr (and similar could be said for the various Git transfer protocols https://git-scm.com/book/en/v2/Git-Internals-Transfer-Protocols)
• Resource addressing (i.e. refer to a resource including how to get it):
  • You want to refer to the machine you want to download the specific git object from in addition to the protocol you'd like to use to do so. For example, you want to download the single git object with SHA-1 hash 45a179104a6f9e9a2c95562496c88bf514871a1d3 from the machine 1.2.3.4 with a specific protocol. Maybe it's the "dumb git protocol" (their words not mine) that IIUC looks like curl https://1.2.3.4/multiformats/multiaddr/objects/45/a179104a6f9e9a2c95562496c88bf514871a1d3, the "smart protocol", or the new one you've proposed. Either way it ends up looking something like /dns/myendpoint.tld/<the-protocol-id-and-parameters> which for the dumb protocol are just HTTP, for the smart one likely would require a /git or /git-protocol name, and maybe the new one you've proposed would get another
• Content addressing (i.e. refer to content independent of where it is):
  • You want to refer to the data by its hash and download + validate it. In this case something like /ipfs/f017811145a179104a6f9e9a2c95562496c88bf514871a1d3 would suffice (note: /ipfs is the namespace here not a multiaddr). You can see the breakdown of the CID example I listed at https://cid.ipfs.tech/#f017811145a179104a6f9e9a2c95562496c88bf514871a1d3.
  • A few notes here just to make sure people are on the same page about using /ipfs:
    • The IPFS namespace doesn't care about things like the hash function, data transfer protocols, or content routing systems in use. You can, for example, choose to refer to existing Git, BitTorrent or Ethereum data using IPFS CIDs.
      • I see the linked issue with Radicle, if they were interested in exposing objects, blobs, trees, etc. using their hashes I suspect the IPFS namespace would be fine even though they're using different data transfer and discovery protocols than others using the IPFS namespace today. If there were any issues here they could likely be tackled in https://github.com/ipfs/specs.
    • If someone came around with a new content addressing format and wanted a code in the table IMO that'd be fine assuming it made sense. I'm calling out IPFS here as an option because it means there's no need to introduce a new code, there's already a bunch of tooling here, etc. As a disclaimer I work on various IPFS implementations.

---

On Machine vs Resource Addressing

I don't recall seeing the distinction of "machine" vs "resource" addressing elsewhere and perhaps there are more standard words to use here but I don't think the concept is new. The point I'm trying to get across is the difference between referring to something you can use in a variety of ways (e.g. given an IP address there's all sorts of things you can ask the machine to do) vs something that has really one use (e.g. download the pile of bytes over HTTP from 1.2.3.4 using path /foo/bar).

From what I can tell historically multiaddr has focused primarily on machine addressing, although it seems like the idea of HTTP multiaddrs has been around for a long time and is now in use in a few places and so maybe resource addressing is really designed to have a place here if people want it. Something I've noticed is that being able to introduce new protocols by name rather than a global number in the table makes building new protocols much easier and so using multiaddrs for things lower down the stack like ip, dns, tcp, udp, onion, libp2p, etc. feels better than doing so for protocols closer to the end user. As an example, the system where these "transports" were introduced multiformats/multicodec#260 currently has limited ability to deal with new "transports" not in the code table which is a pain for experimentation.

Some thoughts on what IMO currently makes sense

• Use the /ipfs namespace for git hashes since it's already doable (and allows for SHA1, SHA2-256, etc.)
• If someone is building an application that wants machine or resource addresses for Git so they can live alongside other multiaddrs then let's figure out how to make this work based on what's needed. Some examples will help make sure this ends up sensible.
• If, as you alluded to, you're interested in a (new) Git retrieval protocol that works across many repos but allows hash-based access to the data I'd probably start a repo somewhere with a spec + basic implementation and probably talk to some folks in this space (e.g. the communities of Git, Radicle, IPFS, etc.) and reference this issue so people like me can follow along 😄. That spec will also come in handy if you end up wanting a code in the table later on.
  • If you just want some Git objects by hash as opposed to something more optimized (e.g. bulk requests for parts of the tree of git objects) you might consider using https://specs.ipfs.tech/http-gateways/trustless-gateway/. For fancier things a deeper discussion may be needed.

Hopefully this was helpful and I didn't miss too much of @SurfingNerd's point in filing this issue or go off on too much of a tangent 😅

[SurfingNerd]

Thanks a lot, for your time and very detailed answers, it took me quite some time to get through it. 🥵
I did not known that IPFS is already supporting git, i will give it a try.
Fetching a repo, by only knowing the hash is already a good thing.
I will close the issue, since the demand for it is very questionable.