Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial process upgrades for Multiformats registry group and org #65

Merged
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
bf8c973
initial process upgrade for multiformats repo
Jul 14, 2023
77007bb
list out well-known registry policies from RFC8126
Jul 14, 2023
5a365ef
minor tweaks and github templates
Jul 17, 2023
5eab8d0
add new issue template
bumblefudge Jul 17, 2023
6a703d7
update issues
bumblefudge Jul 17, 2023
924721b
yml typos how embarassing
bumblefudge Jul 17, 2023
fc4f4e8
yml typos how embarassing
bumblefudge Jul 17, 2023
8b00470
yml typos how embarassing
bumblefudge Jul 17, 2023
9e53696
oh wait it was gh specific syntax
bumblefudge Jul 17, 2023
05887e9
oh man
bumblefudge Jul 17, 2023
f322d3e
oh man
bumblefudge Jul 17, 2023
ff1f407
last tweak
bumblefudge Jul 17, 2023
6c19228
Update contributing.md
bumblefudge Jul 21, 2023
1ab4da9
Update contributing.md
bumblefudge Jul 21, 2023
2ea1271
Update README.md
bumblefudge Jul 21, 2023
87d1427
change example per vmx suggestion
bumblefudge Jul 24, 2023
0d5ccdc
Update contributing.md
bumblefudge Jul 24, 2023
f8b38ef
Update contributing.md
bumblefudge Jul 24, 2023
f37b62e
Update .github/ISSUE_TEMPLATE/BUG-REPORT.yml
bumblefudge Jul 24, 2023
dab4d59
add implementation announcement issue template
bumblefudge Jul 24, 2023
f75eb98
reverse hardwraps as per vmx request
bumblefudge Jul 24, 2023
e4064af
move out some templates to discussions
bumblefudge Aug 31, 2023
40bff07
move over templates from pr#65
bumblefudge Aug 31, 2023
edecea2
template updates
bumblefudge Aug 31, 2023
4644edd
oops wrong branch
bumblefudge Aug 31, 2023
8e6d769
remove two templates
bumblefudge Oct 24, 2023
9bd83ec
address last remaining comments
bumblefudge Oct 24, 2023
963024d
address missing links
bumblefudge Oct 24, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/ISSUE_TEMPLATE/BUG-REPORT.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: "Bug Report - documentation or registries"
description: Report possible bugs in multiformats spec, process docs, and/or the mf-wide registry.
bumblefudge marked this conversation as resolved.
Show resolved Hide resolved
title: "🐛 [DOC/PROCESS BUG] - <title>"
labels: [
"bug"
]
body:
- type: textarea
id: description
attributes:
label: "Description"
description: Please enter an explicit description of your issue,
placeholder: Short and explicit description of your incident, ideally with commit-specific link to lines
validations:
required: true
- type: input
id: reprod-url
attributes:
label: "Reproduction URL"
description: Please enter your GitHub URL to provide a reproduction of the issue
placeholder: ex. https://github.com/multiformats/multiformats/
validations:
required: true
- type: textarea
id: context
attributes:
label: "Context"
description: Please provide additional context
placeholder: "Context or external links needed to explain the possible mistake"
validations:
required: true
61 changes: 61 additions & 0 deletions .github/ISSUE_TEMPLATE/NEW-PROJECT.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: "Propose New Multi- Project"
description: Express interest in possible new multiformats project
title: "📚 [NEW PROJECT] - <title>"
labels: [
"ideation"
]
body:
- type: input
id: name
attributes:
label: "Name"
description: Name this multi- in a word, if you can
placeholder: Multi-thingy for now
validations:
required: false
- type: checkboxes
attributes:
label: "Have read contributing"
description: I have read the [contributing](./contributing.md) document
options:
- label: I read it!
validations:
required: true
- type: textarea
id: problem_statement
attributes:
label: "Description of problem solved"
description: Please describe the problem solved by a mini-registry within multiformats
placeholder: Feel free to provide links for context and use-case descriptions, and how this problem is not solved by existing multi-formats entries or mini-registries
validations:
required: true
- type: textarea
id: prior_art
attributes:
label: "Description of relevant prior art and status quo"
description: Please describe relevant prior art and how this is use-case functions today
placeholder: Links welcome
validations:
required: true
- type: textarea
id: solution_and_rationale
attributes:
label: "Proposed solution and rationale"
description: Please describe at a high level what a new multi- registry or new tooling for existing registries would enable and discuss tradeoffs and rationale (for new sub-registry in particular)
placeholder: Detail welcome
validations:
required: true
- type: checkboxes
attributes:
label: "Have checked table"
description: I have reviewed the [multiformats mega-table](https://github.com/multiformats/multicodec/blob/master/table.csv) to assess viable sub-namespace for a registry if applicable
options:
- label: I read it!
- type: textarea
id: registry_design
attributes:
label: "Proposed sub-namespace"
description: Please describe the range of entries your project would need and any considerations about the greater multiformats namespace
placeholder: Registration space need not be contiguous and one entry in the macro-registry can be annotated differently in multiple sub-registries
validations:
required: true
21 changes: 21 additions & 0 deletions .github/ISSUE_TEMPLATE/TRANSLATION-ANNOUNCEMENT.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: "Translation Announcement"
description: Announcement of intent to translate
title: "📚 [TRANSLATION/LOCALIZATION] - <title>"
labels: [
"community"
]
body:
- type: textarea
id: description
attributes:
label: "Language or Localization offered"
description: Please describe the language and/or alphabet/localization standard you would like to target for a translation and/or localization of this specification.
placeholder: ISO codes always nice but not required!
validations:
required: true
- type: checkboxes
attributes:
label: "Want help"
description: Check this box if you'd like potential helpers tagged into the thread for review.
options:
- label: Would collaborate!
5 changes: 5 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
blank_issues_enabled: true
contact_links:
- name: Protocol Labs Vulnerability Disclosure Team
url: mailto:security@ipfs.io
about: Please do NOT open issues related to security of implementations or spec here without contacting the IPFS security team first.
113 changes: 75 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,19 @@

> The main repository for discussing multiformats

Currently, we have the following formats. More formats are being discussed and worked on.
Multiformats is a set of self-describing protocol values. These values are
foundational in that they are low-level building blocks for both data and
network layers of the composable protocols making up IPFS, IPLD, libp2p, and
many other decentralized data systems. This repository's issues and pull
requests are currently the primary venue for the coordination between the
various registries making up the group, each of which is separately being
hardened as specifications and public, formal registries over time. See
[contributing.md][./contributing.md] for more details on governance and process.

## Current Registries

Currently, we have the following formats, each of which corresponds to a
specification and a registry. More formats are being discussed and worked on.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The table below is badly outdated, it even refers to things that were renamed a long time ago. Also "captain" was used in early Protocol Labs days. I think only multiaddr, multibase and mutlihash should be listed here. multicodec these days it just the "multicodec table".

This might be out of scope for this PR, but I thought I mention it though :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: would it make sense to just MOVE the multicodec table into this repo and call it the multiformats table or call it the Registry Group table, since that's the IETF term for that kind of meta-registry? I'm not sure there should even be a "multicodec" repo or spec if the multiformats WG manages all the repos/registries and the multiforamts spec is the meta-registry spec...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might also be good as part of this clean-up to enumerate the values available for the tag column, if the process for new registries moves into an IANA-friendly spec here... I'm having trouble understanding what is what there

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: would it make sense to just MOVE the multicodec table into this repo and call it the multiformats table or call it the Registry Group table, since that's the IETF term for that kind of meta-registry?

Removing the "multicodec table" would be a huge change. Many people would need to be asked if they think it's a good idea. In case we do that, we would need a new name for the concept the multicodec table represents. Would it then be "Multiformats identifier"?

I'd say such a change is outside of this PE (as this is more of a cleanup) and should be discussed in a separate issue/PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might also be good as part of this clean-up to enumerate the values available for the tag column, if the process for new registries moves into an IANA-friendly spec here... I'm having trouble understanding what is what there

Yes it's totally unclear. The tags grew organically, some things even have the wrong tag/some tags don't even make sense.


| Repo | Captain | Status | IETF |
|------|---------|--------|------|
Expand All @@ -26,54 +38,79 @@ See the project directory, below, for implementations and other related reposito

## Table of Contents

- [Background](#background)
- [An Example: Multihash](#an-example-multihash)
- [A note on the word Multiformats](#a-note-on-the-word-multiformats)
- [Project Directory](#project-directory)
- [Protocols](#protocols)
- [Protocols in Progress](#protocols-in-progress)
- [Implementations](#implementations)
- [Go Implementations](#go-implementations)
- [JavaScript Implementations](#javascript-implementations)
- [Other Implementations](#other-implementations)
- [Other Repositories](#other-repositories)
- [Contribute](#contribute)
- [License](#license)
- [multiformats](#multiformats)
bumblefudge marked this conversation as resolved.
Show resolved Hide resolved
- [Current Registries](#current-registries)
- [Table of Contents](#table-of-contents)
- [Background](#background)
- [A note on the word Multiformats](#a-note-on-the-word-multiformats)
- [Project Directory](#project-directory)
- [Implementations](#implementations)
- [Multiaddr Implementations](#multiaddr-implementations)
- [Multihash Implementations](#multihash-implementations)
- [Multicodec Implementations](#multicodec-implementations)
- [Other Implementations](#other-implementations)
- [Other Repositories](#other-repositories)
- [Maintainers](#maintainers)
bumblefudge marked this conversation as resolved.
Show resolved Hide resolved
- [Contribute](#contribute)
- [License](#license)

## Background

Every choice in computing has a tradeoff. This includes formats, algorithms, encodings, and so on. And even with a great deal of planning, decisions may lead to breaking changes down the road, or to solutions which are no longer optimal. Allowing systems to evolve and grow is important.
Every choice in computing has a tradeoff. This includes formats, algorithms,
encodings, and so on. And even with a great deal of planning, decisions may lead
to breaking changes down the road, or to solutions which are no longer optimal.
Allowing systems to evolve and grow is important.

Multiformats is a collection of protocols which aim to future-proof systems, today. They do this mainly by allowing data to be self-describable. This allows interoperability, protocol agility, and helps us avoid lock in. Currently, our protocols (both works in progress and implemented) cover the following areas:
Multiformats is a collection of protocols which aim to future-proof systems,
today. They do this mainly by allowing data to be self-describable. This allows
interoperability, protocol agility, and helps us avoid various forms of lock-in.
Currently, these interlocking protocols (both works in progress and implemented)
cover the following areas:

- [multiaddr](https://github.com/multiformats/multiaddr): network addresses
- [multibase](https://github.com/multiformats/multibase): base encodings
- [multicodec](https://github.com/multiformats/multicodec): serialization codes
- [multigram](https://github.com/multiformats/multigram): protocol negotiation and multiplexing over datagrams
- [multigram](https://github.com/multiformats/multigram): protocol negotiation
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💭 this is the first time I hear about this 😅

the linked repository is archived and ipfs/specs#123 (comment) suggests we should not include it here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'd be +1 to removing this if no one speaks up soon. note that the IETF WG proposal does NOT include any mention of it, and it has an explicit clause that "no new multiformats" can be added at least until re-chartering...
https://datatracker.ietf.org/wg/multi/about/

and multiplexing over datagrams
- [multihash](https://github.com/multiformats/multihash): cryptographic hashes
- [multikey](https://github.com/ipfs/specs/issues/58): cryptographic keys and artifacts
- [multistream](https://github.com/multiformats/multistream): stream wire protocols
- [multistream-select](https://github.com/multiformats/multistream-select): Friendly protocol multiplexing.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

multistream-select might still be a thing, so it might make sense to keep it here. @marten-seemann or @mxinden might know more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the ping. I suggest not standardizing multistream-select. I think it is very libp2p specific. If one would want to standardize a protocol negotiation protocol, I suggest starting from scratch similar to libp2p/specs#349.


The self-describing aspects of the protocols have a few stipulations:

- they must be in the value (not passed in out of band, in function calls, implicit choices, or documentation);
- they must be compact and have a binary-packed representation (as opposed to a sparser encoding) or they will hinder performance;
- they must have a human readable representation.

Several of the multiformats are stable, and we're working on the others. We are trying to prioritize their usage as soon as possible. What they offer -- protocol interoperability and future-proofing -- would have real-world consequences.

Towards that end, we are encouraging implementations of these protocols; if you know of any, please link them here (or add them to the organization!).
- [multistream-select](https://github.com/multiformats/multistream-select):
Copy link
Member

@lidel lidel Jul 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we include this as a "standard" here, since there is wip to supersede it?
libp2p/specs#349

cc @mxinden / @marten-seemann for thoughts:

  • if we should include "multistream-select" on this "multiformats" list at all? my intuition is it is libp2p-specific and just happens to have "multi" in name.
    • if it is generic enough to include here, should we also include "protocol-select" also/instead, since it aims to supersede the former?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that "multistream" and "multistream-select" are NOT named in the charter and could NOT be added later without a re-charter if the current WG charter draft is ratified next month as currently written:
https://datatracker.ietf.org/wg/multi/about/

I hope someone speaks up if multistream should be put back in scope, or if any other multi- project that I don't know about needs to be in-scope for the WG to protect its range of values in the mega-table. If there is no protocol spec BUT there are multistream-select entries in multicodec, we gotta put our heads together and figure out how to keep them there with a minimally-viable spec or placeholder for one!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be missing some context. I don't think multistream-select should get standardized through the IETF. I don't think it is worth the effort, both given that it is very libp2p specific and given that its design is dated and worth a complete redesign.

Friendly protocol multiplexing.

Several of the multiformats are stable, and work on the others is ongoing.
Implementers and refiners of the drafts of any one of these registries or their
tooling are welcome to [contribute][./contributing.md], without needing to
bumblefudge marked this conversation as resolved.
Show resolved Hide resolved
understand deeply or track progress on the others. Across these otherwise quite
different use-cases and mechanisms, the self-describing aspects of the protocols
have a few stipulations in common:

- the "prefixes" use to self-describe a value must be inline with the value (not
passed out-of-band, in function calls, implicit choices, or documentation);
- they must be compact and have a binary-packed representation (as opposed to a
sparser encoding) or they will hinder performance;
- they must have a human-readable representation.

### A note on the word Multiformats

Multiformats is the name for the organization, but it can also be used to refer to protocols; for instance, in the sentence "Use one of the multiformats". Formats is interchangeable with protocols, here. We try to capitalize Multiformats when it refers to the organization, on GitHub.
`Multiformats` is the name for the community (and the "organization" in GitHub's
actor model), but `multiformats` can also be used to refer to protocols; for
instance, in the sentence "Use one of the multiformats". Formats is
interchangeable with protocols, here, as each format is designed in tandem with
one or more protocols which handle those self-describing values centrally. We
try to capitalize Multiformats when it refers to the organization.

## Project Directory

Below, a list of all of the projects in the Multiformats organization is listed.

**Captains** are the active leads for each project. Their responsibilities are to make sure that issues and pull requests are attended to in a timely manner, and general upkeep. If you have questions about a repository, or need feedback, please contact them as appropriate.
**Captains** are the active leads for each project. Their responsibilities are
to make sure that issues and pull requests are attended to in a timely manner,
and general upkeep. If you have questions about a repository, or need feedback,
please contact them as appropriate. If any of the specifications defining these
formats are formalized and finalized in a standards body, these captains may
continue on as Registrars of the table of entries which can keep growing after
stabilizing the syntax and tooling interfaces.

### Implementations
bumblefudge marked this conversation as resolved.
Show resolved Hide resolved

Expand Down Expand Up @@ -155,15 +192,15 @@ As well as specifications, we also have some implementations in the organization

## Contribute

Please contribute! [Dive into the issues](https://github.com/multiformats/multiformats/issues)!

Check out our [contributing document](contributing.md) for more information on how we work, and about contributing in general. Please be aware that all interactions related to multiformats are subject to the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).

If you're interested in proposing a new format (in addition to multihash, multistream, and so on), [open an issue](https://github.com/multiformats/multiformats/issues/new) in this repository, explaining the name of the format, what it is for, and why you think it should become one of the multiformats. Pointing to any existing implementations would also be great, if there are any. If you want to implement a multiformat in a new language, open an issue in the main repository for the relevant multiformat: for instance, if you want to write rust-multicodec, open an issue in the multiformats/multicodec repository. This will allow others to know that you're working on it, and potentially join in the effort.

Small note: If editing the README, please conform to the [standard-readme](https://github.com/RichardLitt/standard-readme) specification.
Check out our [contributing document](contributing.md) for more information on
how we work, and about contributing in general.

## License

This repository is only for documents. All of these are licensed under the [CC-BY-SA 3.0](https://ipfs.io/ipfs/QmVreNvKsQmQZ83T86cWSjPu2vR3yZHGPm5jnxFuunEB9u) license © 2016 Protocol Labs Inc. Any code is under a [MIT](LICENSE) © 2016 Protocol Labs Inc.
This repository is only for documents. All of these are licensed under the
[CC-BY-SA
3.0](https://ipfs.io/ipfs/QmVreNvKsQmQZ83T86cWSjPu2vR3yZHGPm5jnxFuunEB9u)
license © 2016 Protocol Labs Inc. Any code is under a [MIT](LICENSE) © 2016
Protocol Labs Inc.

[Code of Conduct]: https://github.com/ipfs/community/blob/master/code-of-conduct.md