New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ERC-1319 Smart Contract Package Registry Interface #1319

Open
cgewecke opened this Issue Aug 13, 2018 · 10 comments

Comments

Projects
None yet
7 participants
@cgewecke
Copy link
Contributor

cgewecke commented Aug 13, 2018

Simple Summary

A standard interface for smart contract package registries.

Abstract

This EIP specifies an interface for publishing to and retrieving assets from smart contract package registries. It is a companion EIP to 1123 which defines a standard for smart contract package manifests.

Motivation

The goal is to establish a framework that allows smart contract publishers to design and deploy code registries with arbitrary business logic while exposing a set of common endpoints that tooling can use to retrieve assets for contract consumers.

A clear standard would help the existing EthPM Package Registry evolve from a centralized, single-project community resource into a decentralized multi-registry system whose constituents are bound together by the proposed interface. In turn, these registries could be ENS name-spaced, enabling installation conventions familiar to users of npm and other package managers.

Examples

$ ethpm install packages.zeppelin.eth/Ownership
const SimpleToken = await web3.packaging
                              .registry('packages.ethpm.eth')
                              .getPackage('simple-token')
                              .getVersion('^1.1.5');

Specification

The specification describes a small read/write API whose components are mandatory. It allows registries to manage versioned releases using the conventions of semver without imposing this as a requirement. It assumes registries will share the following structure and conventions:

  • a registry is a deployed contract which manages a collection of packages.
  • a package is a collection of releases
  • a package is identified by a unique string name and unique bytes32 packageId within a given registry
  • a release is identified by a bytes32 releaseId which must be unique for a given package name and release version string pair.
  • a releaseId maps to a set of data that includes a manifestURI string which describes the location of an EIP 1123 package manifest. This manifest contains data about the release including the location of its component code assets.
  • a manifestURI is a URI as defined by RFC3986 which can be used to retrieve the contents of the package manifest. In addition to validation against RFC3986, each manifestURI must also contain a hash of the content as specified in the EIP 1123.

Examples

Package Names / Release Versions

"simple-token" # package name
"1.0.1"        # version string

Release IDs

Implementations are free to choose any scheme for generating a releaseId. A common approach would be to hash the strings together as below:

// Hashes package name and a release version string
function generateReleaseId(string packageName, string version)
  public
  view
  returns (bytes32)
  {
    return keccak256(abi.encodePacked(packageName, version));
  }

Implementations must expose this id generation logic as part of their public read API so
tooling can easily map a string based release query to the registry's unique identifier for that release.

Manifest URIs

Any IPFS or Swarm URI meets the definition of manifestURI.

Another example is content on GitHub addressed by its SHA-1 hash. The Base64 encoded content at this hash can be obtained by running:

$ curl https://api.github.com/repos/:owner/:repo/git/blobs/:file_sha

# Example
$ curl https://api.github.com/repos/rstallman/hello/git/blobs/ce013625030ba8dba906f756967f9e9ca394464a

The string "hello" can have its GitHub SHA-1 hash independently verified by comparing it to the output of:

$ printf "blob 6\000hello\n" | sha1sum
> ce013625030ba8dba906f756967f9e9ca394464a

Write API Specification

The write API consists of a single method, release. It passes the registry the package name, a
version identifier for the release, and a URI specifying the location of a manifest which
details the contents of the release.

function release(string packageName, string version, string manifestURI) public
  returns (bytes32 releaseId);

Read API Specification

The read API consists of a set of methods that allows tooling to extract all consumable data from a registry.

// Retrieves a slice of the list of all unique package identifiers in a registry.
// `offset` and `limit` enable paginated responses / retrieval of the complete set.  (See note below)
function getAllPackageIds(uint offset, uint limit) public view
  returns (
    bytes32 packageIds,
    uint offset
  );

// Retrieves the unique string `name` associated with a package's id.
function getPackageName(bytes32 packageId) public view returns (string name);

// Retrieves the registry's unique identifier for an existing release of a package.
function getReleaseId(string packageName, string version) public view returns (bytes32);

// Retrieves a slice of the list of all release ids for a package.
// `offset` and `limit` enable paginated responses / retrieval of the complete set. (See note below)
function getAllReleaseIds(string packageName, uint offset, uint limit) public view
  returns (
    bytes32[] ids,
    uint offset
  );

// Retrieves package name, release version and URI location data for a release id.
function getReleaseData(bytes32 releaseId) public view
  returns (
    string packageName,
    string version,
    string manifestURI
  );

// Retrieves the release id a registry *would* generate for a package name and version pair
// when executing a release.
function generateReleaseId(string packageName, string version) 
  public
  view
  returns (bytes32);

Pagination

getAllPackageIds and getAllReleaseIds support paginated requests because it's possible that the return values for these methods could become quite large. The methods should return an offset that is a pointer to the next available item in a list of all items such that a caller can use it to pick up from where the previous request left off. (See here for a discussion of the merits and demerits of various pagination strategies.) The limit parameter defines the maximum number of items a registry should return per request.

Rationale

The proposal hopes to accomplish the following:

  • Define the smallest set of inputs necessary to allow registries to map package names to a set of
    release versions while allowing them to use any versioning schema they choose.
  • Provide the minimum set of getter methods needed to retrieve package data from a registry so that registry aggregators can read all of their data.
  • Define a standard query that synthesizes a release identifier from a package name and version pair so that tooling can resolve specific package version requests without needing to query a registry about all of a package's releases.

Registries may offer more complex read APIs that manage requests for packages within a semver range or at latest etc. This EIP is agnostic about how tooling or registries might implement these. It recommends that registries implement EIP 165 and avail themselves of resources to publish more complex interfaces such as EIP 926.

Backwards Compatibility

No standard exists for package registries. The package registry currently deployed by EthPM would not comply with the standard since it implements only one of the method signatures described in the specification.

Implementation

A reference implementation of this proposal is in active development at the EthPM organization on Github here.

Copyright

Copyright and related rights waived via CC0.

@cgewecke cgewecke changed the title Smart Contract Package Registry Interface ERC1319 Smart Contract Package Registry Interface Aug 13, 2018

@cgewecke cgewecke changed the title ERC1319 Smart Contract Package Registry Interface ERC-1319 Smart Contract Package Registry Interface Aug 13, 2018

@xinbenlv

This comment has been minimized.

Copy link
Contributor

xinbenlv commented Aug 13, 2018

Thanks for composing this ERC, this is very interesting. I look forward to more updates on the draft. Cheers!

@tjayrush

This comment has been minimized.

Copy link

tjayrush commented Aug 14, 2018

This is gorgeous. Thanks for posting it. Does it easily expand out to plain old-fashioned software distribution as well (or any digital content for that matter), or is there some reason why it would only apply to smart contracts? Asking out of ignorance more than anything else.

@gnidan

This comment has been minimized.

Copy link
Contributor

gnidan commented Aug 14, 2018

@tjayrush in theory, EthPM works just fine with traditional software distribution, but it's overkill and not the use case for which it was designed.

You would include sources in your manifest file, as described by EIP-1123 (discussion: #1123), but leave out contract_types and deployments. See also: EthPM Use Cases for descriptions and links to examples.

Here's a basic manifest file you might use, as example:

{
  "manifest_version": "2",
  "version": "1.0.0",
  "package_name":"cool-program",
  "sources":{
    "./bin/a.out": "ipfs://Qma8ti...",
    "./src/index.cpp": "ipfs://QmRJHL..."
    /* ... other files to include in the package ... */
  }
}
@Hackdom

This comment has been minimized.

Copy link
Contributor

Hackdom commented Aug 17, 2018

@gnidan unless we're migrating software registries to blockchain as whole right? I think a case can be made about this serving as a foundation to a much larger ecosystem. :)

What is the status of any of the web3js or any other current web3 api adopting this if we know? Are they going to wait for this to move up in status beforehand? Also, I haven't looked, but just want to verify that the ethpm registry is live on mainnet?

@cgewecke

This comment has been minimized.

Copy link
Contributor Author

cgewecke commented Aug 18, 2018

Hi @Hackdom :)

  • A Web3.js module is part of the work EthPrize is sponsoring at EthPM. We've talked to the maintainers at Web3.js and have their encouragement as well as advice about how to structure the API and integrate with an ENS library they're rolling out. There should be substantial progress on that over the next month.

  • There's active work at EthPM on a reference implementation for a registry whose interface matches this ERC's (hopefully completed by next week). That project includes deployment and temporary ENS namespace registration scripts that should allow contract publishers to quickly deploy a registry and/or experiment with whatever registry model they're interested in. The standard proposed here is meant to make it possible for tooling to pull packages down from a multiple-registry ecosystem with the minimum fuss.

  • Finally there's ongoing work at EthPM developing Python and JS libraries to facilitate package publication and installation in accordance with the package manifest standard proposed by EIP #1123, tying all these pieces together.

In short there's WIP for all of this but ultimately community agreement about a reasonable interface is necessary before those implementations can be locked down.

The current EthPM Registry (which has been active for more than a year) is deployed to Ropsten at 0x8011df4830b4f696cd81393997e5371b93338878.

@hiddentao

This comment has been minimized.

Copy link

hiddentao commented Sep 14, 2018

When publishing a new release it would be good if multiple manifest URIs could be provided, as fallbacks. So perhaps a function which allows you to associate additional URIs with a release id:

function addManifestUri(bytes32 releaseId, string manifestURI) public;

Then for reading, since we can't yet return an array of strings:

function getReleaseData(bytes32 releaseId) public view
  returns (
    string packageName,
    string version,
    string manifestURI, /* the first one */
    uint totalManifestURIs /* no. of URIs in total */
  );

/* Pass in 0-based index to get the URI at that index in the array */
function getManifestUri(bytes32 releaseId, uint index) public view return (string)
@cgewecke

This comment has been minimized.

Copy link
Contributor Author

cgewecke commented Sep 15, 2018

@hiddentao Thanks for this comment.

  • Could you provide more information about why you believe fallbacks would be helpful?
  • Is there an analog to this approach in other resource request patterns you know of?
@hiddentao

This comment has been minimized.

Copy link

hiddentao commented Sep 17, 2018

@cgewecke

It's purely from a redundancy point of view. Also note that they wouldn't be mandatory - the slots are there if a package manager wishes to makes use of them.

I know that Maven allows for mirror repositories (essentially the same thing) -> https://maven.apache.org/guides/mini/guide-mirror-settings.html. And they list some good reasons:

  • There is a synchronized mirror on the internet that is geographically closer and faster
  • You want to replace a particular repository with your own internal repository which you have greater control over
  • You want to run a repository manager to provide a local cache to a mirror and need to use its URL instead
@pipermerriam

This comment has been minimized.

Copy link
Member

pipermerriam commented Oct 5, 2018

Looks like the return types for getAllPackageIds is incorrectly a bytes32 instead of a bytes32[].

@pipermerriam

This comment has been minimized.

Copy link
Member

pipermerriam commented Oct 5, 2018

@hiddentao since all releases are content addressed, and we are expecting to see mostly ipfs/swarm uris, I don't see a lot of value in this. I understand there is some extra benefit in being able to provide redundancy, but for this use case, submitting both a swarm url and an IPFS url is not necessary because you can perform a transform between the two urls.

Another problem I see with this is if the different urls have different content hashes, which results in both ambiguity as to which is the right one, as well as additional load on package managers to do something like validate that all listed URLs have the same content hash.

Also, I believe that this can be accomplished by layer-2 solutions, especially since the content hash is required.

I don't think it has zero value by any means, but I don't think the value added is worth the extra complexity. 👎 from me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment