Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add graph init with --from-example and --from-contract modes #224

Merged
merged 26 commits into from
Mar 14, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
6004a61
commands/init, scaffold: Add subgraph scaffolding using address + Eth…
Feb 22, 2019
9cf69da
commands/init: Make Etherscan ABI download work with testnets
Feb 22, 2019
66680f8
scaffold: Avoid semicolons in the generated mapping
Feb 22, 2019
1c3979e
commands/init: Make output prettier
Feb 22, 2019
281470f
commands/init: Handle ABIs without events
Feb 23, 2019
f8e278b
commands/init: Fix loading ABIs from local files
Feb 23, 2019
48d51ad
scaffold: Handle unnamed event parameters correctly
Feb 23, 2019
611a717
commands/init: Avoid duplicating event counting logic
Feb 23, 2019
8599321
codegen: Implement getting value types from AssemblyScript types
Feb 23, 2019
6e41e2b
scaffold: Handle all Ethereum event parameter types properly
Feb 23, 2019
4d75ed4
commands/init, command-helpers: Add --from-example and --from-contract
Mar 6, 2019
01c15d7
tests/cli: Make testCli more flexible (allow CLI command and cwd to b…
Mar 7, 2019
aebb8f0
tests/cli/validation: Pass 'codegen' command to cliTest
Mar 7, 2019
25fc7e0
NPM, Yarn: Add strip-ansi to development dependencies
Mar 7, 2019
550de17
tests/cli/init: Add positive tests for non-interactive `graph init`
Mar 7, 2019
6ca943d
NPM, Yarn: Add missing node-fetch dependency
Mar 7, 2019
e56d994
NPM, Yarn: Bump jest to v24.3.0
Mar 7, 2019
d20f09f
README: Update commands, refer to the official docs for getting started
Mar 12, 2019
9b3af7c
command-helpers/gluegun: Improve fixParameters and add documentation
Mar 14, 2019
54edc8a
commands/init: Handle fixParameter errors
Mar 14, 2019
6dc67d8
commands: Use fixParameters in all commands where it makes sense
Mar 14, 2019
5987a6c
commands/init: Add comment for when the form is cancelled
Mar 14, 2019
efcf205
commands/init: Don't fail silently in --from-example
Mar 14, 2019
f43024d
scaffold: Remove outdated comment
Mar 14, 2019
1bf7373
commands/init: Add comment for removing .git in --from-example mode
Mar 14, 2019
be125ee
scaffold: Automatically use the latest graph-cli version
Mar 14, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 25 additions & 90 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,117 +5,52 @@

## The Graph Command Line Interface

As of today, the command line interface consists of five commands:
As of today, the command line interface supports the following commands:

- `graph codegen` — Generates TypeScript code for smart contract ABIs used in subgraphs.
- `graph build` — Compiles subgraphs to WebAssembly and deploys them to IPFS.
- `graph deploy` — Deploys subgraphs to a [Graph Node](https://github.com/graphprotocol/graph-node).
- `graph remove` — Removes subgraphs from a [Graph Node](https://github.com/graphprotocol/graph-node).
- `graph auth` — Saves access token for [Graph Node](https://github.com/graphprotocol/graph-node) to the system's keychain.
- `graph init` — Creates a new subgraph project from an example or an existing contract.
- `graph create` — Registers a subgraph name with a Graph Node.
- `graph remove` — Unregisters a subgraph name with a Graph Node.
- `graph codegen` — Generates AssemblyScript types for smart contract ABIs and the subgraph schema.
- `graph build` — Compiles a subgraph to WebAssembly.
- `graph deploy` — Deploys a subgraph to a [Graph Node](https://github.com/graphprotocol/graph-node).
- `graph auth` — Stores a [Graph Node](https://github.com/graphprotocol/graph-node) access token in the system's keychain.

## How It Works

`graph` takes a subgraph manifest (defaults to `subgraph.yaml`) with references to:
The Graph CLI takes a subgraph manifest (defaults to `subgraph.yaml`) with references to:

- A GraphQL schema,
- Smart contract ABIs, and
- Mappings written in TypeScript/AssemblyScript.
- Mappings written in AssemblyScript.

It compiles the mappings to WebAssembly, builds a ready-to-use version of the subgraph saved to IPFS or a local directory for debugging, and deploys the subgraph to a [Graph Node](https://github.com/graphprotocol/graph-node).

## Usage
## Installation

Subgraphs for The Graph are set up like a typical TypeScript project. We recommend installing `graph-cli` as a local dependency via `package.json` and use `npm` scripts for code generation and building.
The Graph CLI can be installed with `npm` or `yarn`:

If you are just getting started with creating a subgraph, read the [Getting Started](https://github.com/graphprotocol/graph-node/blob/master/docs/getting-started.md) document. Eventually, that guide will lead you back here.
```sh
# NPM
npm install -g @graphprotocol/graph-cli

For clarity, an example of the setup below can be found in the [ENS subgraph repository](https://github.com/graphprotocol/ens-subgraph).
# Yarn
npm add --global @graphprotocol/graph-cli
```

### On Linux

`libsecret` is used for storing access tokens, so you may need to install it before getting started. Use one of the following commands depending on your distribution:

- Debian/Ubuntu: `sudo apt-get install libsecret-1-dev`
- Red Hat: `sudo yum install libsecret-devel`
- Arch Linux: `sudo pacman -S libsecret`

### Steps

1. Create a project for the subgraph with a `package.json`.
2. Add a `subgraph.yaml` subgraph manifest with a GraphQL schema.
3. Add `@graphprotocol/graph-cli` and `@graphprotocol/graph-ts` dependencies with either NPM or Yarn.

```bash
# NPM
npm install --save-dev
@graphprotocol/graph-cli \
@graphprotocol/graph-ts

# Yarn
yarn add --dev \
@graphprotocol/graph-cli \
@graphprotocol/graph-ts
```

4. Add the following `tsconfig.json`:
```json
{
"extends": "./node_modules/@graphprotocol/graph-ts/tsconfig.json",
"compilerOptions": {
"types": ["@graphprotocol/graph-ts"]
}
}
```
5. Add the following to `package.json`:
```json
{
"scripts": {
"codegen": "graph codegen --output-dir types/",
"build": "graph build",
"build-ipfs": "graph build --ipfs /ip4/127.0.0.1/tcp/5001",
"deploy":
"graph deploy --ipfs /ip4/127.0.0.1/tcp/5001 --node http://127.0.0.1:8020 --subgraph-name <SUBGRAPH_NAME>"
}
}
```
**Note:** Replace the IP addresses and ports with any [Graph Node](https://github.com/graphprotocol/graph-node) you want to deploy the subgraph to.
6. Generate type definitions for contract ABIs used in the subgraph with:
```bash
yarn codegen
```
This creates the `types/` folder. This folder does not need to be uploaded to GitHub, and the files within it should not be edited. With these types created, you can get auto complete for the types while writing the mappings.

7. Develop your `mapping.ts` against these generated types. If you are new to this process, read the [Getting Started](https://github.com/graphprotocol/graph-node/blob/master/docs/getting-started.md#34-write-your-mappings) document for a beginner-friendly walkthrough of The Graph.

8. Deploy your subgraph to a [Graph Node](https://github.com/graphprotocol/graph-node). The following command builds and deploys the subgraph continuously as you are making changes to it:
```sh
graph \
deploy \
--watch \
--verbosity debug \
--node http://127.0.0.1:8020/ \
--ipfs /ip4/127.0.0.1/tcp/5001 \
--subgraph-name <SUBGRAPH_NAME>
```
**Note:** If the Graph Node you are deploying to requires authorization, make sure to authorize the node using `graph auth http://127.0.0.:8020 <ACCESS_TOKEN>` before deploying.

You can also use the deploy script added into `package.json`:

```sh
yarn deploy --debug
```

This will deploy the subgraph. If you want it to continuously upgrade, also pass the `--watch` flag.

`yarn deploy` will create the `dist/` folder. Within this folder are the compiled WASM files from the AssemblyScript mappings. Since this folder is generated, it does not need to be tracked in version control.


To remove a subgraph from the [Graph Node](https://github.com/graphprotocol/graph-node), use:
```sh
graph \
remove \
--node http://127.0.0.1:8020/ \
--subgraph-name <SUBGRAPH_NAME>
```
## Getting Started

The Graph CLI can be used with a local or self-hosted [Graph Node](https://github.com/graphprotocol/graph-node) or with the [Hosted Service](https://thegraph.com/explorer/). To help you get going, there are [quick start guides](https://thegraph.com/docs/quick-start) available for both.

If you are ready to dive into the details of building a subgraph from scratch, there is a [detailed walkthrough](https://thegraph.com/docs/define-a-subgraph) for that as well, along with API documentation for the [AssemblyScript API](https://thegraph.com/docs/assemblyscript-api).

## License

Copyright &copy; 2018-2019 Graph Protocol, Inc. and contributors.
Expand Down
7 changes: 5 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,18 @@
"jayson": "^2.0.6",
"js-yaml": "^3.12.0",
"keytar": "^4.3.0",
"node-fetch": "^2.3.0",
"pkginfo": "^0.4.1",
"prettier": "^1.13.5",
"request": "^2.88.0"
},
"bin": {
"graph": "bin/graph"
},
"devDependencies": {
"jest": "^23.6.0",
"spawn-command": "^0.0.2-1"
"jest": "^24.3.0",
"spawn-command": "^0.0.2-1",
"strip-ansi": "^5.0.0"
},
"scripts": {
"test": "jest --verbose"
Expand Down
6 changes: 4 additions & 2 deletions src/codegen/types/conversions.js
Original file line number Diff line number Diff line change
Expand Up @@ -161,22 +161,24 @@ const VALUE_TO_ASSEMBLYSCRIPT = [
const ASSEMBLYSCRIPT_TO_VALUE = [
// Arrays

['Array<Address>', '[Bytes]', code => `Value.fromBytesArray(${code})`],
['Array<Bytes>', '[Bytes]', code => `Value.fromBytesArray(${code})`],
['Array<boolean>', '[Boolean]', code => `Value.fromBooleanArray(${code})`],
['Array<i32>', '[Int]', code => `Value.fromI32Array(${code})`],
['Array<BigInt>', '[BigInt]', code => `Value.fromBigIntArray(${code})`],
['Array<string>', '[ID]', code => `Value.fromStringArray(${code})`],
['Array<string>', '[String]', code => `Value.fromStringArray(${code})`],
['Array<string>', '[ID]', code => `Value.fromStringArray(${code})`],
['Array<string>', /\[.*\]/, code => `Value.fromStringArray(${code})`],

// Scalar values

['Address', 'Bytes', code => `Value.fromBytes(${code})`],
['Bytes', 'Bytes', code => `Value.fromBytes(${code})`],
['boolean', 'Boolean', code => `Value.fromBoolean(${code})`],
['i32', 'Int', code => `Value.fromI32(${code})`],
['BigInt', 'BigInt', code => `Value.fromBigInt(${code})`],
['string', 'ID', code => `Value.fromString(${code})`],
['string', 'String', code => `Value.fromString(${code})`],
['string', 'ID', code => `Value.fromString(${code})`],
['string', /.*/, code => `Value.fromString(${code})`],
]

Expand Down
4 changes: 4 additions & 0 deletions src/codegen/types/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,9 @@ const ethereumValueFromAsc = (code, ethereumType) =>
const ascTypeForValue = valueType =>
findConversionFromType('Value', 'AssemblyScript', valueType).getIn(['to', 'type'])

const valueTypeForAsc = ascType =>
findConversionFromType('AssemblyScript', 'Value', ascType).getIn(['to', 'type'])

const valueToAsc = (code, valueType) =>
findConversionFromType('Value', 'AssemblyScript', valueType).get('convert')(code)

Expand All @@ -107,6 +110,7 @@ module.exports = {

// Value <-> AssemblyScript
ascTypeForValue,
valueTypeForAsc,
valueToAsc,
valueFromAsc,
}
51 changes: 51 additions & 0 deletions src/command-helpers/gluegun.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
// Workaround for https://github.com/infinitered/gluegun/pull/464.
//
// There is currently no way in Gluegun to define command-line options
// that take no arguments (like `--watch`). As a consequence, boolean
// options like `--watch` will consume the immediately following argument,
// leading to confusing behavior.
//
// E.g. `graph deploy --watch subgraph/name
//
// Will result in
// ```
// toolbox.parameters.options === { watch: 'subgraph/name' }
// toolbox.parameters.first === undefined
// toolbox.parameters.array === []
// ```
// where what we really want is
// ```
// toolbox.parameters.options === { watch: true }
// toolbox.parameters.first = 'subgraph/name'
// toolbox.parameters.array = ['subgraph/name']
// ```
//
// The `fixParameters` function checks if any of the provided boolean
// options has a string value; if so, it pushes it to the front of the
// parameters array and returns the result of that.
//
const fixParameters = (parameters, booleanOptions) => {
let unexpectedStringOptions = Object.keys(booleanOptions)
.filter(key => typeof booleanOptions[key] === 'string')
.map(key => ({ key, value: booleanOptions[key] }))

let optionNames = unexpectedStringOptions
.map(({ key }) => `--` + key.replace(/([A-Z])/, '-$1').toLowerCase())
.join(', ')

if (unexpectedStringOptions.length > 1) {
throw new Error(
`Unexpected value provided for one or more of ${optionNames}. See --help for more information.`
)
} else if (unexpectedStringOptions.length == 1) {
let params = parameters.array
params.unshift(unexpectedStringOptions[0].value)
return params
} else {
return parameters.array
}
}

module.exports = {
fixParameters,
}
21 changes: 18 additions & 3 deletions src/commands/build.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
const chalk = require('chalk')

const { createCompiler } = require('../command-helpers/compiler')
const { fixParameters } = require('../command-helpers/gluegun')

const HELP = `
${chalk.bold('graph build')} [options] ${chalk.bold('[<subgraph-manifest>]')}
Expand Down Expand Up @@ -41,13 +42,27 @@ module.exports = {
outputFormat = outputFormat || t
watch = watch || w

let manifest
try {
;[manifest] = fixParameters(toolbox.parameters, {
h,
help,
w,
watch,
})
} catch (e) {
print.error(e.message)
process.exitCode = 1
return
}

// Fall back to default values for options / parameters
outputFormat =
outputFormat && ['wasm', 'wast'].indexOf(outputFormat) >= 0 ? outputFormat : 'wasm'
outputDir = outputDir && outputDir !== '' ? outputDir : filesystem.path('build')
let manifest =
toolbox.parameters.first !== undefined && toolbox.parameters.first !== ''
? toolbox.parameters.first
manifest =
manifest !== undefined && manifest !== ''
? manifest
: filesystem.resolve('subgraph.yaml')

// Show help text if requested
Expand Down
22 changes: 19 additions & 3 deletions src/commands/codegen.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
const chalk = require('chalk')

const TypeGenerator = require('../type-generator')
const { fixParameters } = require('../command-helpers/gluegun')

const HELP = `
${chalk.bold('graph codegen')} [options] ${chalk.bold('[<subgraph-manifest>]')}
Expand All @@ -25,14 +27,28 @@ module.exports = {
outputDir = outputDir || o
watch = watch || w

let manifest
try {
;[manifest] = fixParameters(toolbox.parameters, {
h,
help,
w,
watch,
})
} catch (e) {
print.error(e.message)
process.exitCode = 1
return
}

// Fall back to default values for options / parameters
outputDir =
outputDir !== undefined && outputDir !== ''
? outputDir
: filesystem.path('generated')
let manifest =
toolbox.parameters.first !== undefined && toolbox.parameters.first !== ''
? toolbox.parameters.first
manifest =
manifest !== undefined && manifest !== ''
? manifest
: filesystem.resolve('subgraph.yaml')

// Show help text if requested
Expand Down
28 changes: 21 additions & 7 deletions src/commands/deploy.js
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
const chalk = require('chalk')

const { identifyAccessToken } = require('../command-helpers/auth')
const { createCompiler } = require('../command-helpers/compiler')
const { fixParameters } = require('../command-helpers/gluegun')
const { createJsonRpcClient } = require('../command-helpers/jsonrpc')
const { identifyAccessToken } = require('../command-helpers/auth')
const { validateSubgraphName } = require('../command-helpers/subgraph')
const { validateNodeUrl } = require('../command-helpers/node')
const { withSpinner } = require('../command-helpers/spinner')
const { validateSubgraphName } = require('../command-helpers/subgraph')

const HELP = `
${chalk.bold('graph deploy')} [options] ${chalk.bold('<subgraph-name>')} ${chalk.bold(
Expand All @@ -15,7 +16,7 @@ ${chalk.bold('graph deploy')} [options] ${chalk.bold('<subgraph-name>')} ${chalk
Options:

--access-token <token> Graph access token
-g, --node Graph node to deploy the subgraph to
-g, --node <node> Graph node to deploy the subgraph to
-h, --help Show usage information
-i, --ipfs <node> Upload build results to an IPFS node
-o, --output-dir <path> Output directory for build results (default: build/)
Expand Down Expand Up @@ -50,12 +51,25 @@ module.exports = {
outputDir = outputDir || o
watch = watch || w

let subgraphName, manifest
try {
;[subgraphName, manifest] = fixParameters(toolbox.parameters, {
h,
help,
w,
watch,
})
} catch (e) {
print.error(e.message)
process.exitCode = 1
return
}

// Fall back to default values for options / parameters
outputDir = outputDir && outputDir !== '' ? outputDir : filesystem.path('build')
let subgraphName = toolbox.parameters.first
let manifest =
toolbox.parameters.second !== undefined && toolbox.parameters.second !== ''
? toolbox.parameters.second
manifest =
manifest !== undefined && manifest !== ''
? manifest
: filesystem.resolve('subgraph.yaml')

// Show help text if requested
Expand Down
Loading