feat: txwrapper-template & CHAIN_BUILDER guide (#35)

* feat: txwrapper-template & CHAIN_BUILDER guide

* Add EOF

* Update wording

* Update template example README'

* Add todos

* Update comment

* Lint

* Update tsconfig

* Apply suggestions from code review

Co-authored-by: David <dvdplm@gmail.com>

* Apply suggestions from code review

Co-authored-by: David <dvdplm@gmail.com>

* Update README and example

* Update BUILDER_GUIDE based on David's feedback

* Apply suggestions from code review

Co-authored-by: David <dvdplm@gmail.com>

* Update CHAIN_BUILDER.md

Co-authored-by: Andrew Plaza <aplaza@liquidthink.net>

Co-authored-by: David <dvdplm@gmail.com>
Co-authored-by: Andrew Plaza <aplaza@liquidthink.net>

CHAIN_BUILDER.md

@@ -1,4 +1,4 @@
-# [WIP] txwrapper creation guide for chain builders
+# txwrapper creation guide for chain builders
 
 This guide walks through the steps that a chain builder will need to take in order to create, publish, and maintain a chain specific txwrapper package.
 
@@ -8,23 +8,121 @@ Creating a txwrapper package will expand the offline signing options for users o
 
 ## Note
 
-This guide has very specific instructions on how to structure the public API of a chain's txwrapper package. This approach is taken by txwrapper-core's maintainers so users of existing txwrappers can quickly integrate with newly created txwrappers for `FRAME`-based chains. Feel free to open up a github issue to discuss any of these aspects.
+This guide has very specific instructions on how to structure the public API of a chain's txwrapper package. This approach is taken so existing txwrapper users can easily integrate new txwrappers. Feel free to open a github issue in this repo to discuss any of these aspects.
 
 ## Template project
 
-(coming soon)
+The [txwrapper-template](packages/txwrapper-template) directory is provided to use as a starting point for your own package.
 
-## Steps*
+## Steps
 
-*These steps all need to be expanded into detailed sections and reviewed
+**Note:** Prior to building a txwrapper for your own chain we highly encourage you to take a look at the [txwrapper-examples](/packages/txwrapper-examples/README.md). Make sure you understand at least the Polkadot example and take a look at the `txwrapper-core` methods an end user is expected to use (see `decode`, `construct.{signingPayload, signedTx, txHash}`). You package will be re-exporting these so take care to understand the public API you will create.
 
-- Create repo
-- Re-export all of txwrapper-core (particularly transaction construction and decoding)
-- Re-export `getRegistry` (and optionally `knownChainProperties`) from txwrapper-registry
-- Re-export methods for pallets included in the chains runtime from packages like txwrapper-substrate, txwrapper-orml
-- Create any methods that don't already exist in existing packages like txwrapper-substrate, txwrapper-orml. (Consider a PR to add the methods if they are relevant to substrate or ORML pallets)
-- Optionally Create constants or other utilities like enums that are connivent (i.e. `TokenSymbol` in the txwrapper-acala examles)
-- Create an end-to-end example for users similar to [txwrapper-examples](packages/txwrapper-examples/src/mandala.ts) (Feel free to use the existing examples as a template)
-- Add tests for methods that are exported to ensure stable serialization/deserialization with your chains metadata. This can used testing utilities exposed by txwrapper-core and should use metadata static metadata from the chain. (TBD if this should be recommended in most circumstances).
-- Publish package
-- Keep it up to date!
+1) **Create a repo from the template**
+For this demo we will assume you have copied the [txwrapper-template](packages/txwrapper-template) directory, are using the yarn package manager, and have the repos remote on github. In the repo we give you the basics of a typescript package near ready for being published to NPM. The exports show some methods that are relevant to a `FRAME` based chain using at least the `balances`, `proxy`, and `utility` pallets. We re-export all of txwrapper-core at the top level to give the user access to its tools.
+
+2) **Update the template**
+Update `package.json` to reflect your chains information. You need to modify the following fields with relevant information: `name`, `author`, `description`, `repository`, `bugs`, `homepage`, and `private` (mark as false). Additionally, you will need to add the following field to give publishing permission:
+
+    ```JSON
+      "publishConfig": {
+        "access": "public"
+      },
+    ```
+
+3) **Choose relevant methods to re-export**
+You will need to choose what pallet methods you want your txwrapper to expose. We recommend choosing methods that are likely to be signed by keys stored offline. If you just need methods from Substrate or ORML pallets, checkout [txwrapper-substrate](packages/txwrapper-substrate/README.md) and [txwrapper-orml](packages/txwrapper-orml/README.md) to see if the methods are already defined.
+    - If a method is already defined in txwrapper-substrate or txwrapper-orml you can simply re-export methods.{pallet name} as a property in the `method` object export. (This will re-export all the methods defined for that pallet, which should be fine as long as you are not using a modified version of the pallet).
+    - If a Substrate or ORML pallet method you need is not already defined you can either make a github issue or submit a PR in this repo for the new method.
+    - If you need methods that do not exist in a Substrate or ORML pallet you have to add what you need directly to your package. See the [Adding a method](#adding-a-method) section for details.
+    - The template already imports @substrate/txwrapper-substrate, but if you do not need any methods from Substrate pallets feel free to remove that dependency.
+
+4) **Create a working example**
+Create an end-to-end example so users have a clear understanding of the full flow for offline transaction generation for your chain. A good example can ease user friction and reduce workload for maintainers.
+In the template we provide an example directory that has all the pieces you need to create a running example. You need to rename `template-example.ts` to something appropriate to your chain and update all the sections in the file marked `TODO`. The file `examples/README.md` will need to be updated as well in the sections marked `TODO`. Finally, make sure the example is fully runable using a development node for your chain.
+
+5) **Publish**
+  Prior to publishing, make sure the package works locally, ([npm pack](https://docs.npmjs.com/cli/v6/commands/npm-pack) command may be useful) and that the versioning makes sense. Once you complete those tasks go ahead and [publish your package to NPM](https://docs.npmjs.com/cli/v6/commands/npm-publish).
+
+6) **Maintain**
+Keep dependencies up to date, paying special attention to security vulnerability warnings.
+If polkadot-js API types for your chain are coming from @polkadot/apps-config you may need to update the `package.json` [resolutions](https://classic.yarnpkg.com/en/docs/selective-version-resolutions/) with the @polkadot/apps-config version that includes your chain's latest types.
+Also keep in mind that if method signatures change (e.g. arguments were added/removed or the return value changed), the corresponding txwrapper method definition will need to be updated as well.
+
+## Adding a method
+
+In some circumstances you may want to add a method for a pallet that does not exist in Substrate or ORML. As an example we will build and test the [ORML `Currencies` pallet's `transfer` method](https://github.com/open-web3-stack/open-runtime-module-library/blob/de2c87064161595b5f6cc9d3e163e576247ff88f/currencies/src/lib.rs#L143-L169).
+
+Note: you will need to create unit tests as well to ensure consistent serialization/deserialization. Creating unit tests is a bit more involved and we do not go into detail here. Take a look at [packages/txwrapper-orml/src/methods/currencies/transfer.spec.ts](packages/txwrapper-orml/src/methods/currencies/transfer.spec.ts) for an example of how unit testing is done using `jest`. Among other things, you will need static metadata stored in the repo to use as the `metadataRpc`, which is used to create the type registry, and passed in as options to the method. In general, the unit tests follow a consistent pattern and should be understandable by looking at all the components.
+
+```typescript
+// src/methods/currencies/transfer.ts
+
+import {
+  BaseTxInfo,
+  defineMethod,
+  OptionsWithMeta,
+  UnsignedTransaction,
+} from '@substrate/txwrapper-core';
+
+// We take care to build an explicit interface with docstrings that will make it easy for users to
+// understand the arguments they need to provide (docstrings also play nice with IDEs).
+//
+// For choosing what type to give each argument it is very helpful to look at the auto-generated
+// typing that polkadot-js typegen creates for each method. In this case, we can refer to the types generated
+// for Acala since the runtime uses the ORML currencies pallets. We find the definitions here:
+// https://github.com/AcalaNetwork/acala.js/blob/4fe5881058003fdb5bc9e3b0c505f3846426db96/packages/types/src/interfaces/augment-api-tx.ts#L213
+//
+// Note: An `augment-api-tx.ts` file (where we find this definitions) is generated by polkadot-js typegen: 
+// https://polkadot.js.org/docs/api/examples/promise/typegen/
+export interface CurrenciesTransferArgs {
+  /**
+   * The recipient address, SS58 encoded.
+   */
+  dest: string;
+  /**
+   * The amount to send.
+   */
+  amount: number | string;
+  /**
+   * The `CurrencyId` of the token to send.
+   */
+  currencyId: string | { Token: string } | { DEXShare: string };
+}
+
+// Define the method
+export function transfer(
+  args: CurrenciesTransferArgs,
+  info: BaseTxInfo,
+  options: OptionsWithMeta
+): UnsignedTransaction {
+  return defineMethod(
+    {
+      method: {
+        args,
+        name: 'transfer',
+        pallet: 'currencies',
+      },
+      ...info,
+    },
+    options
+  );
+}
+```
+
+```typescript
+// src/methods/currencies/index.ts
+
+// export the method, effectively making available under the `currencies` namespace
+export * from './transfer';
+```
+
+```typescript
+// src/methods
+
+// Export everything from within `methods`, including the `currencies` namespace, making it so we can
+// access the method via `methods.currencies.transfer`
+export * as methods from './methods';
+```
+
+Once it is successfully passing unit tests and it works against a dev node it should be good to go.

README.md

@@ -23,14 +23,15 @@ For example, those looking to construct a transaction offline on Polkadot would
 
 - [@substrate/txwrapper-polkadot](/packages/txwrapper-polkadot/README.md) Helper functions for Polkadot, Kusama, Rococo and Westend offline transaction generation.
 - [@substrate/txwrapper-core](/packages/txwrapper-core/README.md) The essentials for creating a chain specific txwrapper lib.
-- [@substrate/txwrapper-registry](/packages/txwrapper-registry/README.md) Registry creation support, catering to chains with types in [@polkadot/apps-config](https://github.com/polkadot-js/apps/tree/master/packages/apps-config/README.md)
-- [@substrate/txwrapper-substrate](/packages/txwrapper-substrate/README.md) Selected dispatchables of Substrate pallets, to be re-exported by txwrappers (e.g. @substrate/txwrapper-polkadot)
-- [@substrate/txwrapper-orml](/packages/txwrapper-orml/README.md) Selected dispatchables of ORML pallets, to be re-exported by txwrappers (e.g. txwrapper-acala)
+- [@substrate/txwrapper-registry](/packages/txwrapper-registry/README.md) Registry creation support, catering to chains with types in [@polkadot/apps-config](https://github.com/polkadot-js/apps/tree/master/packages/apps-config/README.md).
+- [@substrate/txwrapper-substrate](/packages/txwrapper-substrate/README.md) Selected dispatchables of Substrate pallets, to be re-exported by txwrappers (e.g. @substrate/txwrapper-polkadot).
+- [@substrate/txwrapper-orml](/packages/txwrapper-orml/README.md) Selected dispatchables of ORML pallets, to be re-exported by txwrappers (e.g. txwrapper-acala).
 
 #### Non-published
 
-- [@substrate/txwrapper-example](/packages/txwrapper-example/README.md) Usage examples including how to construct, sign, and send an extrinsic with @substrate/txwrapper-polkadot
-- [txwrapper-acala](/packages/txwrapper-acala/README.md) PoC of how a `FRAME`-based chain can leverage txwrapper packages to create a txwrapper. **N.B.** This is only for PoC usage and not intended to be published by txwrapper-core repo maintainers
+- [@substrate/txwrapper-example](/packages/txwrapper-example/README.md) Usage examples including how to construct, sign, and decode an extrinsic with @substrate/txwrapper-polkadot.
+- [@substrate/txwrapper-template](/packages/txwrapper-template/README.md) Template package for chain builders.
+- [txwrapper-acala](/packages/txwrapper-acala/README.md) PoC of how a `FRAME`-based chain can leverage txwrapper packages to create a txwrapper. **N.B.** This is only for PoC usage and not intended to be published by txwrapper-core repo maintainers.
 
 ## End user examples
 

packages/txwrapper-core/README.md

@@ -1,7 +1,7 @@
 <br /><br />
 
 <h1 align="center">@substrate/txwrapper-core</h1>
-<h4 align="center">Core components for creating a chain specific txwrapper lib.</h4>
+<h4 align="center">Core components for creating a txwrapper lib.</h4>
 
 <p align="center">
   <a href="https://www.npmjs.com/package/@substrate/txwrapper-orml">

packages/txwrapper-core/package.json

@@ -2,7 +2,7 @@
   "name": "@substrate/txwrapper-core",
   "version": "0.1.0",
   "author": "Parity Technologies <admin@parity.io>",
-  "description": "Tools for substrate chain builders to build chain specific offline signing lib",
+  "description": "Core components for creating a txwrapper lib.",
   "files": [
     "lib"
   ],

packages/txwrapper-orml/package.json

@@ -2,7 +2,7 @@
   "name": "@substrate/txwrapper-orml",
   "version": "0.1.0",
   "author": "Parity Technologies <admin@parity.io>",
-  "description": "Helper functions for offline transaction generation for selected ORML pallets",
+  "description": "Selected dispatchables of ORML pallets, to be re-exported by txwrappers.",
   "files": [
     "lib"
   ],

packages/txwrapper-polkadot/package.json

@@ -2,7 +2,7 @@
   "name": "@substrate/txwrapper-polkadot",
   "version": "0.1.0",
   "author": "Parity Technologies <admin@parity.io>",
-  "description": "Helper functions for offline transaction generation for Polkadot and Kusama.",
+  "description": "Helper functions for Polkadot, Kusama, Rococo and Westend offline transaction generation.",
   "files": [
     "lib"
   ],

packages/txwrapper-substrate/package.json

@@ -2,7 +2,7 @@
   "name": "@substrate/txwrapper-substrate",
   "version": "0.1.0",
   "author": "Parity Technologies <admin@parity.io>",
-  "description": "Helper functions for offline transaction generation for selected Substrate pallets.",
+  "description": "Selected dispatchables of Substrate pallets, to be re-exported by txwrappers.",
   "files": [
     "lib"
   ],

packages/txwrapper-template/README.md

@@ -0,0 +1,3 @@
+# txwrapper-template
+
+This directory is a template for creating a chain specific txwrapper package, in the vein of `txwrapper-polkadot`. For guidance on how to use this template consult the [CHAIN_BUILDER.md guide](../../CHAIN_BUILDER.md)

packages/txwrapper-template/examples/README.md

@@ -0,0 +1,36 @@
+# How to use `txwrapper-[TODO]`
+
+Here's a mini-tutorial on how `txwrapper-[TODO]` can interact with a Substrate chain. We're using a [TODO LINK TO YOUR CHAINS REPO] dev chain:
+
+## Get Started
+
+1. Fetch the latest [TODO YOUR CHAINS NAME] node from the above link. Follow instructions to build it, and start a dev chain.
+
+    ```bash
+
+    [TODO YOUR CHAINS BINARY NAME/PATH] --dev
+    ```
+
+2. Run the example script in this folder. It will interact with your local node.
+
+    ```bash
+    ./node_modules/.bin/ts-node examples/[TODO].ts
+    ```
+
+## Expected Output
+
+Here's a sample output of the above script, using a [TODO YOUR CHAINS NAME] node. Your payload to sign and signature will of course differ from this example.
+
+```
+[TODO PUT OUTPUT FROM SCRIPT HERE]
+```
+
+## Offline vs. Online
+
+In the examples, the `rpcToLocalNode` function is the only function that needs to be called with internet access. Everything else can be performed offline. In particular, this example shows how to perform the following operations offline:
+
+- Generate a tx,
+- Create its signing payload,
+- Sign the signing payload,
+- Calculate the tx hash,
+- Decode at various levels of the tx lifecycle.