Additional contract metadata #299

ascjones · 2019-12-12T11:43:46Z

Robbepop · 2019-12-12T12:25:00Z

For version we might just use the one we already have in the ink! macro header and its value should be a string with the following formatting:

version: "0.1.0"

Also we might be interested in the underlying compiler, so we might want to include the Rust version into version somehow.
For language we might want to use ink! for an ink! project. Its value is simply a string:

language: "ink!"

Not sure how we are going to compute the hashes but we should standardize a specific hash for that first.
The authors field should be an array of string where string should have the following formatting (as in Cargo):

username <user@email.domain>

So we end up having something like:

authors:  [ foo <foo@bar.baz>, qed <qed.boom@meow.woof> ]

Also we maybe even want the date of creation of the metadata:

date: "2019-12-31"

Maybe even a description of the whole contract might be handy:

description: "This is my description of what this contract does and does not."

We should be able to extract the contract description from the contract module doc comment in ink!.

seanyoung · 2019-12-12T14:09:35Z

How about including the compiler version in the string which identifies the compiler.

compiler: "solang 0.1.0 (llvm 8.0.1)"

which could also be:

compiler: "ink! 2.0 (rust 1.33.0)"

seanyoung · 2019-12-12T14:12:57Z

I think some sort of link back to source code would be great. We could include the repo, but also the commit and filename? Or should that be part of the link itself.

src = [ "https://github.com/foo/bar/baz.sol",  "https://github.com/foo/bar/bax.sol" ]

Robbepop · 2020-06-29T13:47:21Z

We just had an audio call with the following extensional metadata support aggrement:

Smart contract metadata is JSON encoded.

In root:
- metadata_version: Required semantic version for the smart contract metadata.
In source section:
- hash: Required of the associated Wasm file using BLAKE2 256-bit
- language: e.g. Solidity, AssemblyScript or Rust (including version X.Y.Z (semantic versioning))
- compiler: e.g. Solang, ink!/Rust (including version X.Y.Z (semantic versioning))
In contract section:
- name: Required identifier for the smart contract name. (https://crates.io/crates/unicode-xid)
- version: Required semantic version field for the smart contract.
- authors: Required non-empty sequence of strings.
- description: Optional short description about the smart contract.
- documentation: Optional URL to link to API-level documentation of the smart contract.
- repository: Optional URL to link to the repository.
- homepage: Optional URL to link to the smart contract's associated homepage.
- license: Optional field to identify the license of the smart contract given in SPDX license list 3.6 format OR a URL to a user provided non-standard license.
Add user section for custom metadata fields that are generally ignored by standard tools.
Rename current contract section to spec.
- Concerning registry section:
  - Remove strings stable, we want to no longer compress strings in JSON.
  - Move types into spec section.
- layout section: Static storage layout of the smart contract.

Watch list for additional metadata fields:

version_lookup: Optional URL for new version query.

Example

{
    "metadata_version": "0.1.0",
    "source": {
        "hash": "0x...",
        "language": "ink! 3.0/Rust",
        "compiler": "rustc 1.45"
    },
    "contract": {
        "name": "Flipper",
        "version": "1.0.0",
        "authors": [ "Parity Technologies <admin@parity.io>" ],
        "description": "Very simple contract to flip a boolean value.",
        "documentation": "https://my-documentation.com",
        "repository": "https://github.com/examples/flipper",
        "homepage": "https://flipper-homepage.com",
        "license": "MIT OR APACHE-2",
    },
    "user": {
        "some-user-provided-field": "and-its-value",
        "more-user-provided-fields": ["and", "their", "values"],
    },
    "spec": {
        "types": { ... },
        "events": [ ... ],
        "messages:" [ ... ],
        "constructors:" [ ... ],
        "layout": { ... },
    },
}

ascjones · 2020-07-03T15:19:14Z

First auto-generated json of the new format (with hardcoded values)

{
  "metadata_version": "0.1.0",
  "source": {
    "hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "language": "ink! 2.1.0",
    "compiler": "ink! 2.1.0 (rustc 1.46.0)"
  },
  "contract": {
    "name": "testing",
    "version": "0.1.0",
    "authors": [
      "author@example.com"
    ],
    "documentation": "http://example.com/"
  },
  "spec": {
    "types": [
      {
        "def": {
          "primitive": "i32"
        }
      }
    ],
    "storage": {
      "Struct": {
        "fields": [
          {
            "name": "value",
            "layout": {
              "Cell": {
                "key": "0x0000000000000000000000000000000000000000000000000000000000000000",
                "ty": 1
              }
            }
          }
        ]
      }
    },
    "spec": {
      "name": "incrementer",
      "constructors": [
        {
          "name": "new",
          "selector": "0x41e691fc",
          "args": [
            {
              "name": "init_value",
              "type": {
                "id": 1,
                "displayName": [
                  "i32"
                ]
              }
            }
          ],
          "docs": []
        },
        {
          "name": "default",
          "selector": "0xcfee7c08",
          "args": [],
          "docs": []
        }
      ],
      "messages": [
        {
          "name": "inc",
          "selector": "0x37b82f69",
          "mutates": true,
          "args": [
            {
              "name": "by",
              "type": {
                "id": 1,
                "displayName": [
                  "i32"
                ]
              }
            }
          ],
          "returnType": null,
          "docs": []
        },
        {
          "name": "get",
          "selector": "0x6817c00f",
          "mutates": false,
          "args": [],
          "returnType": {
            "id": 1,
            "displayName": [
              "i32"
            ]
          },
          "docs": []
        }
      ],
      "events": [],
      "docs": []
    }
  }
}

ascjones · 2020-07-06T09:12:52Z

New updated metadata:

{
  "metadata_version": "0.1.0",
  "source": {
    "hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "language": "ink! 2.1.0",
    "compiler": "rustc 1.46.0"
  },
  "contract": {
    "name": "testing",
    "version": "0.1.0",
    "authors": [
      "author@example.com"
    ],
    "documentation": "http://example.com/"
  },
  "storage": {
    "name": "Incrementer",
    "layout": {
      "Struct": {
        "fields": [
          {
            "name": "value",
            "layout": {
              "Cell": {
                "key": "0x0000000000000000000000000000000000000000000000000000000000000000",
                "ty": 1
              }
            }
          }
        ]
      }
    }
  },
  "spec": {
    "constructors": [
      {
        "name": "new",
        "selector": "0x41e691fc",
        "args": [
          {
            "name": "init_value",
            "type": {
              "id": 1,
              "displayName": [
                "i32"
              ]
            }
          }
        ],
        "docs": []
      },
      {
        "name": "default",
        "selector": "0xcfee7c08",
        "args": [],
        "docs": []
      }
    ],
    "messages": [
      {
        "name": "inc",
        "selector": "0x37b82f69",
        "mutates": true,
        "args": [
          {
            "name": "by",
            "type": {
              "id": 1,
              "displayName": [
                "i32"
              ]
            }
          }
        ],
        "returnType": null,
        "docs": []
      },
      {
        "name": "get",
        "selector": "0x6817c00f",
        "mutates": false,
        "args": [],
        "returnType": {
          "id": 1,
          "displayName": [
            "i32"
          ]
        },
        "docs": []
      }
    ],
    "events": [],
    "docs": []
  }
}

Robbepop · 2020-07-06T09:21:15Z

Looking good so far!

Reminder: That name on storage (here Incrementer) is actually an ink! detail information.
In ink! it is kind of nice to know about it but I am not sure about this for Solidity (Solang) or other languages.
Maybe we need to encode this into user but I will talk to @seanyoung first.

ascjones · 2020-07-06T09:27:39Z

Could make it an optional field?

Robbepop · 2020-07-06T09:29:34Z

Wait for action until we resolved this with @seanyoung please.

seanyoung · 2020-07-06T10:04:23Z

The name on storage isn't something that necessarily makes sense with Solidity. It could be populated with the contract name (so same as contract.name).

ascjones · 2020-07-10T14:34:01Z

Generated for incrementer with no hardcoded values:

{
  "metadata_version": "0.1.0",
  "source": {
    "hash": "0x4aa82d1faa733477c06424b3d67d9fa0a40f5aa7a34877f811652ab33e8b5d1b",
    "language": "ink! 2.1.0",
    "compiler": "rustc 1.46.0-nightly"
  },
  "contract": {
    "name": "incrementer",
    "version": "2.1.0",
    "authors": [
      "Parity Technologies <admin@parity.io>"
    ],
    "documentation": "http://docs.rs/",
    "homepage": "http://homepage.rs/"
  },
  "user": {
    "more-user-provided-fields": [
      "and",
      "their",
      "values"
    ],
    "some-user-provided-field": "and-its-value"
  },
  "spec": {
    "constructors": [
      {
        "args": [
          {
            "name": "init_value",
            "type": {
              "displayName": [
                "i32"
              ],
              "id": 1
            }
          }
        ],
        "docs": [],
        "name": "new",
        "selector": "0x41e691fc"
      },
      {
        "args": [],
        "docs": [],
        "name": "default",
        "selector": "0xcfee7c08"
      }
    ],
    "docs": [],
    "events": [],
    "messages": [
      {
        "args": [
          {
            "name": "by",
            "type": {
              "displayName": [
                "i32"
              ],
              "id": 1
            }
          }
        ],
        "docs": [],
        "mutates": true,
        "name": "inc",
        "returnType": null,
        "selector": "0x37b82f69"
      },
      {
        "args": [],
        "docs": [],
        "mutates": false,
        "name": "get",
        "returnType": {
          "displayName": [
            "i32"
          ],
          "id": 1
        },
        "selector": "0x6817c00f"
      }
    ]
  },
  "storage": {
    "layout": {
      "Struct": {
        "fields": [
          {
            "layout": {
              "Cell": {
                "key": "0x0000000000000000000000000000000000000000000000000000000000000000",
                "ty": 1
              }
            },
            "name": "value"
          }
        ]
      }
    },
    "name": "Incrementer"
  },
  "types": [
    {
      "def": {
        "primitive": "i32"
      }
    }
  ]
}

Robbepop · 2020-07-10T15:40:04Z

Very nice!

How was the user data provided in this case?

"user": {
    "more-user-provided-fields": [
      "and",
      "their",
      "values"
    ],
    "some-user-provided-field": "and-its-value"
  },

Also due to the upcoming trait support we might have the need to find some ground rules or changes for message and constructor names since they now can overlap for ink!.

ascjones · 2020-07-10T15:47:15Z

[package.metadata.contract.user]
some-user-provided-field = "and-its-value"
more-user-provided-fields = ["and", "their", "values"]

ascjones · 2020-07-10T15:57:16Z

Continuing a conversation I had with @Robbepop use-ink/cargo-contract#62 (comment).

The question is: where the data structures which define the extended metadata (generated by cargo-contract), and combine it with the contract metadata (generated by ink_lang), should live.

Current status

ink_metadata defines a struct which contains contract metadata about the ABI and the storage layout:

/// An entire ink! project for metadata file generation purposes.
#[derive(Debug, Serialize)]
pub struct InkProject {
    #[serde(flatten)]
    registry: Registry,
    #[serde(rename = "storage")]
    layout: layout2::StorageLayout<CompactForm>,
    spec: ContractSpec<CompactForm>,
}

ink_lang generates an instance of this which is serialized and written to a file by an auto-generated binary in cargo-contract: https://github.com/paritytech/cargo-contract/blob/master/templates/tools/generate-metadata/main.rs

Dependency on `ink!`

cargo-contract does not currently have an explicit dependency on ink! itself. When generating the metadata-gen binary package, it copies over the ink_metadata dependency from the target contract's manifest. This means that metadata generation will work for contracts with any version of ink! which exposes a compatible __ink_generate_metadata extern:

extern "Rust" {
	fn __ink_generate_metadata() -> ink_metadata::InkProject;
}

Combining with extended metadata

For this issue we want to combine the extended metadata with the contract metadata itself. We can define a (pseudo) data structure like so:

#[derive(Serialize)]
pub struct CombinedMetadata {
    ink: ink_metadata::InkProject,
    extended: ExtendedMetadata,
}

Where should such a data structure be defined?

Options

1, Define in `ink_metadata`

Pros

All metadata defined in the same place

Cons

Need to generate code in cargo-contract to instantiate ExtendedMetadata, increasing the coupling to ink!

2. Define in `cargo-contract`

Pros

Forwards and backwards compatibility with different versions of ink! (Assuming compatible __ink_generate_metadata extern)
Extended metadata defined where it is generated.
Can update extended metadata with a release of cargo-contract instead of requiring both an ink_metadata release and a compatible cargo-contract release.

Cons

No single crate/data structure to depend on for the whole strongly typed metadata
Multiple places to look for the different parts of the metadata definition

Initially I implemented the definitions in ink_metadata, but the current working implementation has the definition in cargo-contract. I believe the reduction in coupling is a strong enough reason to do it this way.

ascjones · 2020-07-15T11:20:26Z

@seanyoung an idea that @Robbepop and I discussed: we could extract the types which define the extended metadata into their own crate, which solang could use. This would avoid duplication and ensure consistency.

seanyoung · 2020-07-15T13:24:37Z

@ascjones that is a great solution, I like it very much!

There is some precedent for a crate which just defines types. For example https://github.com/gluon-lang/lsp-types does this for the Language Server protocol.

Robbepop · 2020-07-15T16:23:15Z

sorry 🙈

Robbepop · 2020-10-08T14:44:02Z

This has been implemented and is available with ink! 3.0. Closed.

ascjones added C-discussion An issue for discussion for a given topic. A-ink_metadata [ink_metadata] Work item labels Dec 12, 2019

ascjones mentioned this issue Dec 12, 2019

New metadata format #296

Merged

Robbepop added this to ToDo in ink! 3.0 Jun 27, 2020

ascjones mentioned this issue Jun 29, 2020

Remove string table, inline strings paritytech/scale-info#17

Merged

ascjones self-assigned this Jul 1, 2020

ascjones moved this from ToDo to In Progress in ink! 3.0 Jul 1, 2020

This was referenced Jul 2, 2020

[metadata] update to scale-info 0.3, removes string interning #467

Merged

Generate extended contract metadata use-ink/cargo-contract#62

Merged

This was referenced Jul 15, 2020

Extend contract metadata with cargo metadata use-ink/cargo-contract#57

Closed

Extract extended contract metadata types to separate crate use-ink/cargo-contract#65

Closed

Robbepop closed this as completed Jul 15, 2020

Robbepop reopened this Jul 15, 2020

ascjones mentioned this issue Aug 5, 2020

Extract contract metadata to its own crate use-ink/cargo-contract#69

Merged

ii-ii-ii mentioned this issue Aug 27, 2020

Supports parsing the additional metadata polkadot-js/api#2524

Closed

Robbepop closed this as completed Oct 8, 2020

Robbepop moved this from In Progress to Done in ink! 3.0 Oct 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Additional contract metadata #299

Additional contract metadata #299

ascjones commented Dec 12, 2019 •

edited

Robbepop commented Dec 12, 2019 •

edited

seanyoung commented Dec 12, 2019

seanyoung commented Dec 12, 2019

Robbepop commented Jun 29, 2020 •

edited

ascjones commented Jul 3, 2020

ascjones commented Jul 6, 2020

Robbepop commented Jul 6, 2020

ascjones commented Jul 6, 2020

Robbepop commented Jul 6, 2020

seanyoung commented Jul 6, 2020

ascjones commented Jul 10, 2020

Robbepop commented Jul 10, 2020

ascjones commented Jul 10, 2020

ascjones commented Jul 10, 2020 •

edited

ascjones commented Jul 15, 2020

seanyoung commented Jul 15, 2020

Robbepop commented Jul 15, 2020

Robbepop commented Oct 8, 2020

Additional contract metadata #299

Additional contract metadata #299

Comments

ascjones commented Dec 12, 2019 • edited

todo:

Robbepop commented Dec 12, 2019 • edited

seanyoung commented Dec 12, 2019

seanyoung commented Dec 12, 2019

Robbepop commented Jun 29, 2020 • edited

Example

ascjones commented Jul 3, 2020

ascjones commented Jul 6, 2020

Robbepop commented Jul 6, 2020

ascjones commented Jul 6, 2020

Robbepop commented Jul 6, 2020

seanyoung commented Jul 6, 2020

ascjones commented Jul 10, 2020

Robbepop commented Jul 10, 2020

ascjones commented Jul 10, 2020

ascjones commented Jul 10, 2020 • edited

Current status

Dependency on ink!

Combining with extended metadata

Options

1, Define in ink_metadata

2. Define in cargo-contract

ascjones commented Jul 15, 2020

seanyoung commented Jul 15, 2020

Robbepop commented Jul 15, 2020

Robbepop commented Oct 8, 2020

ascjones commented Dec 12, 2019 •

edited

Robbepop commented Dec 12, 2019 •

edited

Robbepop commented Jun 29, 2020 •

edited

ascjones commented Jul 10, 2020 •

edited

Dependency on `ink!`

1, Define in `ink_metadata`

2. Define in `cargo-contract`