Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NEAR CLI enhancements (#31) #31

Closed
wants to merge 13 commits into from
Closed

NEAR CLI enhancements (#31) #31

wants to merge 13 commits into from

Conversation

damons
Copy link
Contributor

@damons damons commented Feb 14, 2020

Initial draft of NEP for NEAR Shell enhancements. Comments required and appreciated.

@damons damons assigned damons and mikedotexe and unassigned damons Feb 14, 2020
@amgando
Copy link

amgando commented Feb 14, 2020

I like this interface

adding this to the mix
https://github.com/lirantal/nodejs-cli-apps-best-practices

### `near key <command>`

### `near contract <command>`
* `near contract list`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be incredibly useful. Would this information be stored from the user's history? Is there some way we can use a library/RPC to get this info?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what this is supposed to do. Will it just return a list of contract account ids, or contract metadata, or contract code itself?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the unique identifier for a contract? Do contracts have names? The idea was to list contracts for a given account. It would be good to have an ID and an contract name.

Copy link
Contributor

@mikedotexe mikedotexe Feb 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unique identifier is the account name, which is a regular, human-readable NEAR account. Thanks for bringing this up, Bowen. I am now considering removing that list of commands.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@damons what do you mean by "contracts for a given account"? There can be zero or one contract deployed on a given account. If you are talking about contracts that an account interact with, that is much harder to do because user data are stored under the contract account, not the user account.

@mikedotexe
Copy link
Contributor

Yeah, this is great. I think besides adding the validator aspect to near-shell, we can also take the approach we intend to add all functionality on the command line that can currently only be done with other, external interactions. For instance, if a person wants to give function-level access to their smart contract, you must log in via Wallet. Or if a person wants to deauthorize (aka revoke) an access key you must use Wallet, as far as I am aware.
These are the kind of examples that could be moved to near-shell as well.
I like the idea of laying it out like this:

While interacting with NEAR as a developer or validator, there is a relatively small number of atomic actions that are needed. Currently, some of these actions are tied to other NEAR offerings, such as Wallet or Explorer. The ideal future of near-shell allows a command-line interface for these in a highly secure and configurable way.

Copy link
Contributor

@mikedotexe mikedotexe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this is looking great, let's continue to iterate.

@behaviary behaviary changed the title Adding an NEP for NEAR Shell enhancements (#30) NEAR Shell enhancements (#30) Feb 26, 2020
@mikedotexe
Copy link
Contributor

Still have more work to do on this NEP, committing progressively

* `near contract status`
* `near contract add`
* `near contract remove`
* `near contract build`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just highlighting a discussion here: near/near-cli#275

Since the build process for AssemblyScript and Rust differs, it may make sense to either:

  1. Remove this and lean on the tools for building in the language ecosystems
  2. In implementation, the build process has a step where the language is determined.

@vgrichina makes good points for (1) here: near/near-cli#275 (comment)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is another important aspect to this.
We don't currently have support for multi-contract projects.

And I don't think we can easily provide anything like near build for these, as they can easily be mix of Rust / AssemblyScript / whatever else contracts which all need to be deployed together.

We need to think of some build system supporting dependencies (i.e. something like make or gulp) to put this all together and provide ability for developer to customize individual steps as necessary.

Copy link
Contributor

@mikedotexe mikedotexe Mar 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to think of some build system supporting dependencies (i.e. something like make or gulp) to put this all together

near build currently runs gulp. if the suggestion is to remove near build would we be adding this back in later?
https://github.com/nearprotocol/near-shell/blob/7d9e8601b0eeb05872406e8bf5eca3e14098224c/bin/near-cli.js#L118-L122

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should:

  1. remove near build
  2. have design for multi contract projects if we want to move away from gulp

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed near build from the spec
Haven't added a recommended method for multi-contract projects, as I think we haven't discussed the alternatives yet. Honestly, I don't have a good gut instinct for that task and would like to hear from others.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about something like lerna? We could probably abstract away all of the lerna operations with near commands.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Sladuca Lerna solves somewhat orthogonal problem, I'm talking more about stuff how you can have one project with multiple deployments (e.g. given this is shared system you may want to deploy single contract on multiple accounts).

Other than that I think it should be possible to use Lerna with near-shell as an option, no need to wrap around with near-shell commands. We want to provide dev tools that fit into different workflows (i.e. not tied to specific build tools ideally) + provide reasonable default workflows in examples.

@ilblackdragon
Copy link
Member

ilblackdragon commented Mar 31, 2020

Few comments:

  • We should use different folder from ~/.near, for example ~/.neardev
  • ~/.neardev/config.json should contain list of network configurations
  • We should deprecate and remove --nodeUrl and other overriders for networks - instead people should just add alternative networks. Right now it's confusing because there are multiple ways to do the same thing.
  • Local near-config.json in the project can override networks and provide any extra infromation
  • config.json also can specify any custom key management solution people want to use.
  • BUT! let's start with just organizing what we have right now - e.g. unecncryptedFileSystem key store. Currently it's extremely unclear to people why if you do near create_account in one folder and switch to another folder you can't do anything.
  • Make sure to NEVER to override keys.

@mikedotexe
Copy link
Contributor

mikedotexe commented Apr 15, 2020

@ilblackdragon when you say:

Make sure to NEVER to override keys.

I want to confirm that having environment variables containing private keys is not considered overriding, correct? If I'm spinning up a Docker instance on the cloud, it would be great to have the option to include these are env vars, fallback to key files in the project (cwd), fallback to key files in the home directory…
Are you wanting to avoid this?

@render
Copy link

render bot commented Apr 22, 2020

A deploy for your Render PR Server at https://nomicon-pr-31.onrender.com just failed.

View details on your dashboard at https://dashboard.render.com/static/srv-bqgav33eej5i1qn99atg.

Copy link
Member

@ilblackdragon ilblackdragon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few comments


These are key-values reflecting how `near-shell` behaves when called within a NEAR project.

Example: a developer using OS X uses `near login` with access to a browser, whereas a validator runs `near login` on a CentOS box with no UI or browser. A user prompt may ask "Is this login from a computer with a browser?" The answer is then stored in key-value format in the project-level directory so it can be referenced later, skipping the prompt in the future.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is global configuration, not per project


Example: a developer using OS X uses `near login` with access to a browser, whereas a validator runs `near login` on a CentOS box with no UI or browser. A user prompt may ask "Is this login from a computer with a browser?" The answer is then stored in key-value format in the project-level directory so it can be referenced later, skipping the prompt in the future.

Besides answers to user prompts, shell-experience settings also store information about the last version of `near-shell` used in this project. For more information, please see [Upgradability](#upgradability).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like work for package.json file not for a .<something> folder. Usually . folders are added to gitignore as they contain local configuration.

https://stackoverflow.com/questions/10065564/add-custom-metadata-or-config-to-package-json-is-it-valid


`.near-config/settings.js`

In summary, "settings" are key-value pairs that are set by `near-shell` and relate to the behavior of how it operates in a given project directory. They do not relate to the development or deployment of smart contracts, except for providing default values like `accountId`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

default app name can also go to package.json


```
─ awesome-near-project ⟵ NEAR dApp
├── .near-config ⟵ Stores shell-experience settings and connection configuration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would really prefer not to have any settings folders inside the project folder.

│ │ ├── default.js ⟵ Default configuration is for development
│ │ └── localnet.js ⟵ Custom connection added by user for localnet
│ └── settings.js ⟵ Stores near-shell settings, how shell behaves when run in this project
└── .near-credentials ⟵ Previously "neardev" this directory contains private key inforamtion
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

credentials for sure should not be here. this is both easiest way to delete them (delete & reclone the project) or expose them because forgot to add the folder into .gitignore.

```
─ awesome-near-project ⟵ NEAR dApp
├── .near-config ⟵ Stores shell-experience settings and connection configuration
│ ├── connections ⟵ Contains files used to connect and deploy a contract (except keys)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

connections should be global for the computer. In cases where developers really need their non-local and non-standard environment, we can be near-provider.js or something in project folder to extend and be similar to truffle?


As shown in the directory structure earlier, this file is located in the project directory at:

`.near-config/connections/default.js`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's stop use default, as it's confusing. we have testnet and mainnet

@render
Copy link

render bot commented Apr 28, 2020

A deploy for your Render PR Server at https://nomicon-pr-31.onrender.com just failed.

View details on your dashboard at https://dashboard.render.com/static/srv-bqgav33eej5i1qn99atg.

* `network` : The NEAR network (example, "betanet", "testnet", etc.)
* `private_key` : The plain-text private key, used when `type = "unencrypted"`

`near-shell` will look for keys in a specific order. This list is in the prioritized order and can be understood to mean, "if the key is not found here, then try the next location/store."

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about not having a complex order, but instead a simple default + a way to set this in config

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we even need complex configurable order? Seems like it always can be:

project folder -> global folder -> protected OS key store

External stuff like Ledger likely makes sense to specify explicitly (as it's generally not going to be based on account ID).

@ilblackdragon
Copy link
Member

ilblackdragon commented May 5, 2020

@mikedotexe I also wanted to mention that there are a bunch of cases, when user wouldn't want to sign transaction on the same device.

People would use custody, CloudHSM, HSMs that our shell doesn't support, etc.

E.g., near shell should be able to create transactions without signing or sending them.
And also optionally sending signed transaction (as input prob combination of data from above and signature).

We should make this at least possible and as secure as we can.

Note: that we haven't had this asked by people yet, so we shouldn't actually build this. More to make sure we have a way in our CLI APIs to add this. It's possible that people can just use near-api-js instead.

@render
Copy link

render bot commented May 8, 2020

A deploy for your Render PR Server at https://nomicon-pr-31.onrender.com just failed.

View details on your dashboard at https://dashboard.render.com/static/srv-bqgav33eej5i1qn99atg.

@behaviary
Copy link

Just wanted to x-link this request from a while back: near/near-cli#57

Copy link
Contributor

@chadoh chadoh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some more work to do here, reading through this and understanding everything at a deep enough level to help guide the spec. But I wanted to leave some thoughts now, as I might not have time to return to this for a couple weeks.

As I'm reading this, I'm also thinking about how we can make the interface for near-shell match the interface for near-api-js. I think there should be a basic set of functionality that is closely parallel in each. See near/near-api-js#341

I believe my suggestions here will help simplify the experience of near-shell on its own, and will furthermore make it simpler to have matching experiences in both shell & JS.


* `networkId` : Similar to an environment to work on (ex: 'staging')
* `nodeUrl` : URL to the NEAR node
* `contractName` : NEAR account name for the smart contract
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I doubt this should be here. It's commonly included in src/config.js today, but that doesn't seem right to me.

Let's consider your example command below for calling a contract:

near contract view near-game topPlayers --env localnet

What would having contractName in my config file give me? It doesn't make sense to call a function on a contract without specifying which contract. I doubt any sort of "default contract" makes sense.


`near contract view near-game topPlayers --env localnet`

will call the function `topPlayers` on the contract `near-game` that is deployed to localnet. The connection information will be read from the file: `.near-config/connections/localnet.js`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think these connection settings are complicated enough to each justify their own file. Here's the alternative I propose:

  • We use sensible defaults based on NEAR_ENV / NODE_ENV, no files required on local machine

  • NEAR_ENV=betanet near do thing is equivalent to near do thing --networkId=betanet (probably better to use --network-id, dash-case, for consistency with other CLIs)

  • You can override specific connection settings with near config connection.nodeUrl https://myownnode.cool. This follows the same interface as git config – by default, it sets it for the local folder. Unlike git, we probably don't actually need a whole folder for the local project, but can use a .nearconfig file. So this command would result in a ./.nearconfig that looks like this:

    [connection]
      nodeUrl = https://myownnode.cool
    

    Like git config, you can use near config --global to set the value in ~/.nearconfig instead.

  • It doesn't make sense to store contractName in this file. The name of a contract/account is needed in specific commands; it is not part of configuring a connection.

  • It doesn't make sense to store networkId in this file. Network ID is a command line switch (--network-id=x) or an environment variable (NEAR_ENV=x); giving this non-standard third way to set it adds confusion rather than clarity. This switch/env var gives you a way to select a predetermined set of connection configs; what does it mean for a network id itself to be part of the connection config that a network id is used to select?

  • You can also set each individual setting with its own env var. If you have no .nearconfig file anywhere on your machine but you have NEAR_CONNECTION_NODE_URL=https://myownnode.cool in env, then near commands will use that for connection.nodeUrl.

  • Any of these settings can be configured on a per-command basis, using a switch such as --node-url

  • Settings priority order:

    1. per-command setting switch, e.g. --node-url
    2. environment variable, e.g. NEAR_CONNECTION_NODE_URL
    3. local .nearconfig
    4. closest ancestor directory .nearconfig
    5. default for network specified in --network-id switch
    6. default for network specified in NEAR_ENV env var
    7. default for network specified in NODE_ENV env var
    8. default for testnet

This results in a wonderfully simple first-use experience with sensible defaults, with customization options that fit a variety of workflows and that can be discovered as-needed.

Since the lookup priority order could potentially cause some confusion, I suggest one more item:

  • near config default command lists current config and where the value came from. Example:

    $ NEAR_ENV=devnet NEAR_CONNECTION_NODE_URL=https://myownnode.cool near config
    connection.nodeUrl=https://myownnode.cool           (from environment variable NEAR_CONNECTION_NODE_URL)
    connection.walletUrl=https://wallet.devnet.near.org (default for NEAR_ENV=devnet)
    connection.helperUrl=https://someotherhelper.dance  (from ../../.nearconfig)
    


will call the function `topPlayers` on the contract `near-game` that is deployed to localnet. The connection information will be read from the file: `.near-config/connections/localnet.js`.

Users may add, remove, or modify connection environments using commands detailed later in this spec.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I question how often people need to manage multiple customized environments. For people who do, providing the environment-variable-based customization option allows them to use tools like dotenv or simple shell scripts to quickly switch between sets of custom settings.

@frol
Copy link
Collaborator

frol commented Oct 8, 2020

I want to dump my vision on near-cli and tooling design in general. (moving it from a private discussion to this public NEP discussion)

TL;DR: A well-designed tool should drive its own usage instead of requiring the user to learn about everything before they can even start using it.

I want to make the number of available options on each step to be as minimal as possible (as opposed to 18 top-level subcommands in near-cli JS), and I don't want to hide flows behind the scenes (near login is 100% confusing to users at first, even though the idea behind it is quite straightforward).

For example, I consider having the following command to do a transfer:

$ near-cli \
    construct-transaction \
    online \
        --network testnet \
    send-from frol4.testnet \
    send-to qq.frol4.testnet \
    transfer 3.14NEAR \
    sign-with-secret-key ed25519:...

it is quite a lengthy command, but a user is going to be guided through it as follows:

  1. Let's see what we can do:
$ near-cli
...
SUBCOMMANDS

construct-transaction
    Initiate transaction creation with follow up actions to optionally sign and submit the transactions to the network

utils
   Low-level helpers for common operations (key pair generation, transaction verification, signing, etc)
  1. Ok, let's create a transaction:
$ near-cli construct-transaction

I initially considered printing a help message here listing all the required options, but we can have it for --help, and instead opt into an interactive mode here:

To construct a transaction you will need to provide information about sender (signer) and receiver accounts, and actions that needs to be performed.

Do you want to derive some information required for transaction construction automatically querying it online?
> Yes, I keep it simple
  No, I want to work in no-network (air-gapped) environment

Select NEAR protocol RPC server:
> testnet (https://rpc.testnet.near.org)
  mainnet (https://rpc.mainnet.near.org)
  betanet (https://rpc.betanet.near.org)
  Custom (will prompt you to provide RPC URL)

What is the account ID of the sender? ... frol4.testnet

What is the account ID of the receiver? ... qq.frol4.testnet

Select an action that you want to add to the transaction:
> Transfer NEAR Tokens
  Call a Function
  Stake NEAR Tokens
  Create an Account
  Delete an Account
  Add an Access Key
  Detete an Access Key
  [Skip adding a new action]

How many NEAR Tokens do you want to transfer? ... 3.14 NEAR

Select an action that you want to add to the transaction:
...
> [Skip adding a new action]

Would you like to sign the transaction? ...
> Yes, I want to sign the transaction with my private key
  No, I want to construct the transaction and sign it somewhere else

Select your key chain: // <-- if "Yes", otherwise, skip this step
...

Some extra information needs to be filled in before we proceed signing...

The account ID has 2 access keys, select one that you will use to sign the transaction:  // <-- Only in online mode with "No" reply to signing since we don't know the access key; in offline mode, we should prompt a user to enter their public access key manually
> ed25519:11111...
  ed25519:22222...

Constructing the Transaction...
Transaction {
...
}
Base64-encoded Transaction: ...(base64 value)...
Transaction Hash (sha256): 83ffe880d749ea081c12cb7aa96b481bea3ee0c30e13650d363c2a28edfc0971

Constructing the Signed Transaction...
SignedTransaction {
...
}

Confirm submitting the transaction:
> Everything is correct, send it
  Oops! Stop it!
Offline flow example

To construct a transaction you will need to provide information about sender (signer) and receiver accounts, and actions that needs to be performed.

Do you want to derive some information required for transaction construction automatically querying it online?
 Yes, I keep it simple
>  No, I want to work in no-network (air-gapped) environment

What is the account ID of the sender? ... frol4.testnet

What is the account ID of the receiver? ... qq.frol4.testnet

Select an action that you want to add to the transaction:
> Transfer NEAR Tokens
  Call a Function
  Stake NEAR Tokens
  Create an Account
  Delete an Account
  Add an Access Key
  Detete an Access Key
  [Skip adding a new action]

How many NEAR Tokens do you want to transfer? ... 3.14 NEAR

Select an action that you want to add to the transaction:
...
> [Skip adding a new action]

Would you like to sign the transaction? ...
  Yes, I want to sign the transaction with my private key
> No, I want to construct the transaction and sign it somewhere else

Some extra information needs to be filled in before we proceed signing...

Enter transaction nonce (query the access key information with `near-cli utils view-access-key frol4.testnet ed25519:...` incremented by 1):

Enter recent block hash (see above or `near-cli inspect-data blocks latest`):

Constructing the Transaction...
Transaction {
...
}
Base64-encoded Transaction: ...(base64 value)...
Transaction Hash (sha256): 83ffe880d749ea081c12cb7aa96b481bea3ee0c30e13650d363c2a28edfc0971

If you want to sign the transaction using `near-cli` on another device, run: `near-cli utils sign-transaction ...(base64 value)...`.

Once you sign the Transaction Hash, we can proceed.
Enter the signature (base58-encoded string):
Signature is valid.

Constructing the Signed Transaction...
SignedTransaction {
...
}

Here is the Signed Transaction ready for submittion through RPC: ...(base64 value)...

Merged flows into a single scenario (just to understand the full picture)

To construct a transaction you will need to provide information about sender (signer) and receiver accounts, and actions that needs to be performed.

Do you want to derive some information required for transaction construction automatically querying it online?
> Yes, I keep it simple
  No, I want to work in no-network (air-gapped) environment

Select NEAR protocol RPC server: // <-- Only in online mode
> testnet (https://rpc.testnet.near.org)
  mainnet (https://rpc.mainnet.near.org)
  betanet (https://rpc.betanet.near.org)
  Custom (will prompt you to provide RPC URL)

What is the account ID of the sender? ... frol4.testnet

What is the account ID of the receiver? ... qq.frol4.testnet

Select an action that you want to add to the transaction:
> Transfer NEAR Tokens
  Call a Function
  Stake NEAR Tokens
  Create an Account
  Delete an Account
  Add an Access Key
  Detete an Access Key
  [Skip adding a new action]

How many NEAR Tokens do you want to transfer? ... 3.14 NEAR

Select an action that you want to add to the transaction:
...
> [Skip adding a new action]

Would you like to sign the transaction? ...
  Yes, I want to sign the transaction with my private key
  No, I want to construct the transaction and sign it somewhere else

Select your key chain: // <-- if "Yes", otherwise, skip this step
...

Some extra information needs to be filled in before we proceed signing...

The account ID has 2 access keys, select one that you will use to sign the transaction:  // <-- Only in online mode with "No" reply to signing since we don't know the access key; in offline mode, we should prompt user to enter their public access key manually
> ed25519:11111...
  ed25519:22222...

Enter transaction nonce (query the access key information with `near-cli utils view-access-key frol4.testnet ed25519:...` incremented by 1): // <-- Only for offline mode

Enter recent block hash (see above or `near-cli inspect-data blocks latest`): // <-- Only in offline mode

Constructing the Transaction...
Transaction {
...
}
Base64-encoded Transaction: ...(base64 value)...
Transaction Hash (sha256): 83ffe880d749ea081c12cb7aa96b481bea3ee0c30e13650d363c2a28edfc0971

NOTE: if you want to sign the transaction using `near-cli` on another device, run: `near-cli utils sign-transaction ...(base64 value)...`. // <-- Only in "No, I want to construct the transaction and sign it somewhere else"

// <-- Stop here if "No, I want to construct the transaction and sign it somewhere else"

Constructing the Signed Transaction...
SignedTransaction {
...
}

Here is the Signed Transaction ready for submittion through RPC: ...(base64 value)... // <-- Only in offline mode

// <-- Stop here if "offline" or "online --dry-run" modes

Confirm submitting the transaction:
> Everything is correct, send it
  Oops! Stop it!

The flow is lengthy with various branches for online/offline and early-exit scenarios, but it guides the users through quite confidently.

Here is a demo of the CLI prompts from enquirer crate:

Based on some feedback I want to clarify that the non-interactive mode will co-exist and it will just have the values prepopulated into the interactive flow, and those will not be prompted for.

There are basically four scenarios I see based on the online or offline mode of fetching nonce and recent block hash, and automatic or manual signing process:

  1. [simplest, automatic signing in online mode] Online mode with keys availble on the machine or a device attached to it [e.g. Ledger]:
$ near-cli \
    construct-transaction \
    online \
        --network testnet \
    send-from frol4.testnet \
    send-to qq.frol4.testnet \
    transfer 3.14NEAR \
    sign-with-sender-secret-key ed25519:...
  1. [secured with air-gapped device for signing] Online mode with manual signing:
$ near-cli \
    construct-transaction \
    online \
        --network testnet \
    send-from frol4.testnet \
    send-to qq.frol4.testnet \
    transfer 3.14NEAR \
    sign-manually
  1. [secured air-gapped device constructs the transaction completely] Offline mode, signing with provided keys (either via args, or in a local key store):
$ near-cli \
    construct-transaction \
    offline \
        --recent-block-hash Etd1FevM62PPi57YXF4T6r56cPPPCcmF3onJUhcuF3M3 \
        --nonce 123
    send-from frol4.testnet \
    send-to qq.frol4.testnet \
    transfer 3.14NEAR \
    sign-with-sender-secret-key ed25519:...
  1. [paranoid air-gapped device constructs the transaction, but signing is performed somewhere else] Offline mode with manual signing:
$ near-cli \
    construct-transaction \
    offline \
        --recent-block-hash Etd1FevM62PPi57YXF4T6r56cPPPCcmF3onJUhcuF3M3 \
        --nonce 123
    send-from frol4.testnet \
    send-to qq.frol4.testnet \
    transfer 3.14NEAR \
    sign-manually

Extra examples:

Ledger support:

$ near-cli \
    construct-transaction \
    online \
        --network testnet \
    send-from frol4.testnet \
    send-to qq.frol4.testnet \
    transfer 3.14NEAR \
    sign-with-ledger <path>

Having all of those as nested subcommands we can have help message on every single level with only the relevant options to choose from. For example:

$ near-cli construct-transaction online --help
`online` subcommand is a part of transaction construction which defines that the extra information required for transaction construction will get fetched automatically and directly from RPC node.

ARGUMENTS:

--dry-run Do not actually submit the final transaction to JSON RPC, just log what are going to do
--network  [testnet | mainnet | betanet] predefined set of networks with hardcoded rpc-server links; use `--rpc-server` if you want to use a custom NEAR JSON RPC server
--rpc-server HTTP URL to NEAR node serving JSON RPC providing access to access keys and transaction submission

SUBCOMMANDS:

send-from  specify the account ID of the transaction sender

The next stop would be:

$ near-cli construct-transaction online send-from --help
...

send-to    specify the account ID of the transaction receiver

Notice how you progressed from online help to send-from help guided with the only available option to choose (send-from)

/cc @khorolets @vgrichina @nearmax @ilblackdragon

@chadoh chadoh mentioned this pull request Mar 15, 2021
@frol
Copy link
Collaborator

frol commented Mar 29, 2021

FYI, I want to share that there has been great progress on my proposed design of the new near-cli. The project lacks the README and a proper show-case at the moment and lacks a few features here and there, but overall, it is on track, and it can be ready for release in April!

image

you can also skip a few steps (or even all of them) providing more arguments:

image

Full command to send a transaction to transfer tokens would be:

near-cli construct-transaction online testnet sender frol4.testnet receiver illia.testnet add-action transfer-near-tokens 1NEAR skip sign-private-key --signer-public-key ed25519:3oN25HE9tf9YUGXQyXCFB2rEtpomijG8ghVV2JvrA4zJ --signer-secret-key ed25519:...

If you split it into lines for readability:

near-cli \
  construct-transaction \
    online testnet \
    sender frol4.testnet \
    receiver illia.testnet \
    add-action transfer-near-tokens 1NEAR \
    skip \
    sign-private-key \
      --signer-public-key ed25519:3oN25HE9tf9YUGXQyXCFB2rEtpomijG8ghVV2JvrA4zJ \
      --signer-secret-key ed25519:...

Demo: https://asciinema.org/a/wPtlwPbJveOIpwWEABGaCjQeT

The project is implemented from scratch here: https://github.com/FroVolod/near-cli

@stefanopepe
Copy link

Do we still retain the possibility to use in-line command parameters instead of this interactive interface? This is an example of the many creations we have out there: https://gist.github.com/gaia/cff45baf3fa710a42c3fc4cdaafe8edc

@frol
Copy link
Collaborator

frol commented Mar 29, 2021

@jimmy3dita Look at the second screenshot, it is there 😄

@chadoh
Copy link
Contributor

chadoh commented Mar 30, 2021

This looks awesome. Thanks!

I do wonder if some of the subcommands given to construct-transaction should be flags instead. online especially seems like sort of parenthetical information, not the main text of the command. Seeing the word skip in the non-interactive version also strikes me as strange.

Maybe something like:

near-cli \
  construct-transaction \
    from frol4.testnet \
    to illia.testnet \
    --online \ # could have alias `--network` and a `--no-network` flag to accompany it
    --rpc testnet \ # RPC endpoint seems like a separate concern from `--online`
    --action transfer-near-tokens 1NEAR \ # allow adding multiple `--action`s
    sign-private-key \ # do we need this? maybe just use flags below
      --signer-public-key ed25519:3oN25HE9tf9YUGXQyXCFB2rEtpomijG8ghVV2JvrA4zJ \
      --signer-secret-key ed25519:...

@frol
Copy link
Collaborator

frol commented Apr 1, 2021

@chadoh I initially started with an idea that looks somewhat similar to yours:

image

I ended up with tons of unstructured arguments, that are hard to follow for a newbie, and it was not even feature-complete (there is no online/offline mode, network selection, and signing options). In my new proposal, I decided to try a guided step-by-step process with exhaustive parameters matching, for example, when you select online mode, you are presented with a finite number of options to select: betanet, testnet, mainnet, or custom, where custom has a required option --url, and other options don't have any mention of the --url option:

image

So I defined the constraints that I wanted to satisfy:

  1. The command line arguments should not overwhelm users when they add --help to the end. Ideally, help should guide users through the flow (one step at a time)
  2. The questionary and the CLI arguments order should be in sync, otherwise, it is confusing
  3. In questionary mode we want to be able to validate user input and immediately ask to type valid values, e.g. to validate the sender and the receiver account ids we need to confirm online/offline status first and know the RPC endpoints to communicate with, that is why in my CLI design online/offline status is so early on the list

The first point is my main pain-point with the current near-cli (in JS):

image

I find this approach much more friendly:

image

(surely help messages needs to be added, but you have a clear idea which options you need to specify at a given stage)

@frol
Copy link
Collaborator

frol commented Apr 7, 2021

construct-transaction is mostly done, though there are still some papercuts. Some utilities are already implemented as well: generate a keypair, sing a transaction with a secret key in plain text (useful for teaching low-level concepts), combine transaction with a signature (also useful for teaching the low-level concepts).

The next step would be to design a bunch of shortcuts, and I plan to group them into five categories: transfer, execute, add, delete, and view. All of them should have feature-complete CLI and interactive modes.

I want to invite everyone to review the proposed flows. The order of parameters is fixed on purpose; I believe that will contribute to a clear mental model and also can guide users nicely through the process even when they don't know where to start, and especially, where to go next. See my previous posts and examples on how construct-transaction is implemented.

Transfer NEAR [done]

near-cli transfer NEAR \
    network testnet \
    sender 'frol.testnet' \
    receiver 'illia.testnet' \
    amount  '1 NEAR' \
    sign-with-keychain \
    send

Note: I use transfer NEAR since I think we may want to support other tokens in the future.

Same command as a one-liner:

near-cli transfer NEAR network testnet sender 'frol.testnet' receiver 'illia.testnet' amount '1 NEAR' sign-with-keychain send

Execute a change method (function) [done]

Note: the reasoning for the argument order is similar to the construct-transaction design as it should have the same flexibility in online/air-gapped transaction construction, and various ways of signing it.

near-cli execute change-method \
    network testnet \
    contract 'meta.pool.testnet' \
    call 'distribute_staking' '{}' \
        --attached-deposit '0 NEAR' \
        --prepaid-gas '1 Tgas' \
    signer 'frol.testnet' \
    sign-with-keychain \
    send

Note: we may consider the following options to the call arguments syntax:

  • call 'distribute_staking' '{"account_id": "testnet"}' (as in the example)
  • call 'distribute_staking({"account_id": "testnet"})'
  • call 'distribute_staking(account_id="testnet")'

Execute a view method (read-only function) [done]

Note: to do a view method call, we don't need to create a transaction, we don't need a signer, we don't need keys etc. It is just a JSON RPC call to query endpoint with call_function type

near-cli execute view-method \
    network testnet \
    contract 'meta.pool.testnet' \
    call 'distribute_staking' '{}'

Add Access Key (and "login") [partially done]

near-cli add access-key \
    network testnet \
    account 'frol.testnet' \
    public-key 'ed25519:...' \
    grant-full-access \
    sign-with-keychain \
    send
near-cli add access-key \
    network testnet \
    account 'frol.testnet' \
    generate-keypair \
    grant-function-call-access \
        --receiver-id 'meta.pool.testnet' \
        --allowance '10 NEAR' \
        --method-names 'set_a, set_b' \
    sign-with-keychain \
    send

Here is how near-cli login could look like:

near-cli add access-key network testnet generate-keypair grant-by-near-wallet

One could argue that login is much shorter, but I argue that we would spend less time explaining to people over and over again how login actually works.

Delete Access Key [done]

near-cli delete access-key \
    network testnet \
    account 'frol.testnet' \
    public-key 'ed25519:...' \
    sign-with-keychain \
    send

Stake [planned]

near-cli add stake-proposal \
    network testnet \
    validator 'staking.poolv1.testnet' \
    amount '10_000_000 NEAR' \
    transactions-signing-public-key 'ed25519:...' \
    sign-with-keychain \
    send

Create New Sub-Account [done]

near-cli add sub-account \
    network testnet \
    owner-account 'frol.testnet' \
    sub-account 'sub.frol.testnet' \
    sub-account-full-access \
        public-key 'ed25519:...' \
    deposit '1 NEAR' \
    sign-with-keychain \
    send
near-cli add sub-account \
    network testnet \
    owner-account 'frol.testnet' \
    sub-account 'sub.frol.testnet' \
    sub-account-full-access \
        generate-keypair \
    deposit '1 NEAR' \
    sign-with-keychain \
    send

"Create" New Implicit Account [planned]

near-cli add implicit-account \
    generate-keypair

This effectively does not issue any transaction, the only action this command does is generating a new keypair, and prints the public key as in hex, which can be used as an account id for transfers. The account is not going to appear on the network with this command, but the account id can be used to receive tokens, and later issue transactions on behalf of this account.

Delete Account itself [done]

near-cli delete account \
    network testnet \
    account 'frol.testnet' \
    beneficiary 'illia.testnet' \
    sign-with-keychain \
    send

Deploy Contract Code [done]

near-cli add contract-code \
    network testnet \
    account 'frol.testnet' \
    contract-file './frol.testnet.wasm' \
    no-initialize \
    sign-with-keychain \
    send

When you need to initialize the contract:

near-cli add contract-code \
    network testnet \
    account 'frol.testnet' \
    contract-file './frol.testnet.wasm' \
    initialize 'new' '{}' \
        --attached-deposit '0 NEAR' \
        --prepaid-gas '1 TGas' \
    sign-with-keychain \
    send

View Account Details [done]

Displays the non-staked balance, staked balance, storage usage, whether a contract is deployed to it, and lists access keys:

near-cli view account-summary network testnet account 'frol.testnet'

Download Contract Code [done]

near-cli view contract-code network testnet contract 'meta.pool.testnet' download './meta.pool.testnet.wasm'

near-cli view contract-code network testnet contract 'meta.pool.testnet' hash

View Contract State [partially done]

nenar-cli view contract-state network testnet account 'meta.pool.testnet'

View Transaction Status [done]

near-cli view transaction network testnet transaction-hash '7K1sVuE2tY8kjRj9TcututW6TGJTZAmryrprjv9hYpnQ' signer 'frol4.testnet'

Note: due to sharding we need to specify the signer account id when lookup the transaction status

@mikedotexe mikedotexe changed the title NEAR Shell enhancements (#30) NEAR CLI enhancements (#30) Apr 14, 2021
@behaviary behaviary changed the title NEAR CLI enhancements (#30) NEAR CLI enhancements (#31) May 13, 2021
@frol
Copy link
Collaborator

frol commented Jul 16, 2021

https://near.cli.rs is now in a pretty usable state (it does everything that was laid out above, and has NEAR Wallet login and Ledger support)

There is also the next iteration of a discussion on where to move forward with NEAR CLI: https://gov.near.org/t/new-near-cli-design-proposal/3616

@frol frol closed this Jul 16, 2021
@chadoh chadoh deleted the near-shell-enhancements branch July 16, 2021 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet