Codius Client Command Line Interface (CLI)
Switch branches/tags
Nothing to show
Clone or download
wilsonianb fix: don't assume host uses same asset as uploader
Get host asset details from quote
Latest commit fd0bec0 Dec 3, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci feat: generate manifest and manage state via files Jul 3, 2018
bin
src fix: don't assume host uses same asset as uploader Dec 4, 2018
test
.eslintrc
.gitignore feat: generate manifest and manage state via files Jul 3, 2018
LICENSE feat: add apache 2.0 license, skip discover if --host option used Jun 5, 2018
README.md fix a typo Aug 27, 2018
npmrc-env feat: generate manifest and manage state via files Jul 3, 2018
package-lock.json fix: don't assume host uses same asset as uploader Dec 4, 2018
package.json fix: don't assume host uses same asset as uploader Dec 4, 2018

README.md

Codius CLI

Command Line Interface for Codius

NPM Package CircleCI JavaScript Style Guide Known Vulnerabilities Gitter chat

The Command Line Interface for uploading and extending pods on Codius.

Table of Contents

Overview

The Codius CLI supports uploading & extending pods on codius hosts.

Upgrade Notes

If upgrading from Codius CLI 2.0.x please note that the LevelDB dependency is removed the list of currently uploaded pods and stored manifests will be inaccessible. It's recommended that the list of pods and any manifests that are only stored in the Codius CLI be exported before upgrading.

Listing all uploaded pods

Use the command

codius pods -l

to list all pods that have been uploaded. Redirect this command to a file to save the list. For example

codius pods -l > myPodList.json

Exporting a manifest

To export a manifest that is only stored in the Codius CLI from the LevelDB use the command

codius pods -m [manifestHash]

For example the following will export the pod with the manifest hash s5e3eqgzy4eqvsejphru6y62uw325nj7tvqr7a2pqxohtkcd6wj to the file s5e3eqgz-manifest-backup.json

codius pods -m s5e3eqgzy4eqvsejphru6y62uw325nj7tvqr7a2pqxohtkcd6wjq > s5e3eqgz-manifest-backup.json

Repeat the command above for all the manifests that need to be preserved so the pods can be kept running or uploaded to a new host.

Prerequisites

  • NodeJS
  • An XRP Wallet (you will need at least 35 XRP to run a Codius Host)
    • The calculation is as follows:
      reserveBaseXRP + currentOwnerCount * reserveIncrementXRP + 1 * reserveIncrementXRP +
      OUTGOING_CHANNEL_DEFAULT_AMOUNT_XRP => 20 + 0 * 5 + 1 * 5 + 10 = 35
      
  • Moneyd

Installation

The Codius CLI can be installed globally by running the following command:

npm install -g codius

It can then be run with the command codius.

Command Reference

All codius commands have a --help parameter describing their usage, e.g. codius upload --help for help with the upload command.

upload [options]

Looks in the current directory for the files codius.json & codiusvars.json which are used to generate a manifest. The generated manifest is then used to upload the pod to host(s). By default it uploads the pod to a single random known host with a duration of 10 minutes. It's recommended that you start with a short duration and then extend it to ensure your manifest is running appropriately. Upload should only be used for new manifest where a default.codiusstate.json file does not exist.

Options Argument Type Description
--duration, -d Integer Duration (in seconds) the pod will be run on all Codius hosts, defaults to 10 mins.
--max-monthly-rate, -m Integer Max rate per month the uploader is willing to pay a Codius host to run the pod, requires --units, -u flag to be set. Defaults to 10.
--units, -u Integer The unit of currency to pay the Codius hosts with, e.g. 'XRP', requires --max-monthly-rate, -m flag to be set. Defaults to 'XRP'.
--host-count, -c Integer The number of hosts to upload the pod to. They are discovered from known hosts and selected randomly. Defaults to 1. This and --host are mutually exclusive.
--host String The public URI of a host to upload the manifest to. Can be repeated any number of times for multiple hosts. This and --host-count, -c are mutually exclusive.
--add-host-env, --a Boolean Adds a $HOST env in the manifest before upload which contains all the hosts the manifest will be uploaded to.
--codius-file String Filename or full path to codius file to be used. If not set the CLI looks in the current directory for the codius.json file.
--codius-vars-file String Filename or full path to the codius variables file to be used. If not set the CLI looks in the current directory for the codiusvars.json file.
--codius-hosts-file String Filename or full path to the codius hosts file to be used. If not set the CLI looks in the current directory for the codiushosts.json file.
--codius-state-file String Filename or full path to the codius state file to be generated. If not set the CLI will make a default.codiusstate.json file.
--overwrite-codius-state, -o Boolean Overwrite the current *.codiusstate.json file if it exists.
--assume-yes, -y Boolean Say yes to all prompts.
--debug Boolean Run this pod in debug mode with logging
--tail Boolean Tail the pods logs on upload. Requires debug mode to be enabled on the pod during upload by the --debug flag.

extend [options]

Used to manage the pod after the initial upload. It extends the duration of a running pod or uploads the pod again if the pod has expired. Extend can only be used after a default.codiusstate.json file exists in the current directory. The upload command is used to generate that file. When run successfully the *.codiusstate.json file will be updated.

Options Argument Type Description
--duration, -d Integer Duration (in seconds) the pod will be run on all Codius hosts, defaults to 10 mins.
--max-monthly-rate, -m Integer Max rate per month the uploader is willing to pay a Codius host to run the pod, requires --units, -u flag to be set. Defaults to 10.
--units, -u Integer The unit of currency to pay the Codius hosts with, e.g. 'XRP', requires --max-monthly-rate, -m flag to be set. Defaults to 'XRP'.
--codius-state-file String Filename or full path to the codius state file to be generated. If not set the CLI will make a default.codiusstate.json file.
--assume-yes, -y Boolean Say yes to all prompts.

extend-hash <manifest-hash> [options]

Extends a pod using the provided manifest hash on the host. Allows the user to extend a pod if they know the manifest hash and the host. Useful if the user is not the one who generated the original manifest but wishes to keep it running.

Arguments:

  • <manifest-hash>
    • Type: String
    • Description: The manifest hash of the manifest to extend.
Options Argument Type Description
--duration, -d Integer Duration (in seconds) the pod will be run on all Codius hosts, defaults to 10 mins.
--max-monthly-rate, -m Integer Max rate per month the uploader is willing to pay a Codius host to run the pod, requires --units, -u flag to be set. Defaults to 10.
--units, -u Integer The unit of currency to pay the Codius hosts with, e.g. 'XRP', requires --max-monthly-rate, -m flag to be set. Defaults to 'XRP'.
--codius-state-file String Filename or full path to the codius state file to be generated. If not set the CLI will make a default.codiusstate.json file.
--host String The public URI of a host to extend the manifest.
--assume-yes, -y Boolean Say yes to all prompts.

tail [manifest-hash] [options]

Tails the logs of the pod if when uploaded the --debug flag was used during upload. Useful to see what your pod is doing on a host via its logs.

Arguments:

  • [manifest-hash]
    • Type: String
    • Description: The manifest hash of the manifest to tail.
Options Argument Type Description
--host String The public URI of a host to upload the manifest to. Can be repeated any number of times for multiple hosts. This and --host-count, -c are mutually exclusive.
--codius-state-file String Filename or full path to the codius state file to be generated. If not set the CLI will make a default.codiusstate.json file.

cron [create|view|remove] [options]

Allows the user to automatically extend pods using cron jobs. This command can only be used on pods that have already been uploaded. And only one cron job can be created at a time for each pod.

NOTE: This command is not supported on Windows machines.

cron create

Create a new cron job to extend a pod.

Options Argument Type Description
--buffer-sec, -b Integer The minimum duration (in seconds) to maintain on all Codius hosts. Must be greater than 1 hr (3600 seconds). Mutually exclusive with skip-extend.
--max-monthly-rate, -m Integer Max rate per month the uploader is willing to pay a Codius host to run the pod, requires --units, -u flag to be set. Defaults to 10.
--units, -u Integer The unit of currency to pay the Codius hosts with, e.g. 'XRP', requires --max-monthly-rate, -m flag to be set. Defaults to 'XRP'.
--codius-state-file String Filename or full path to the codius state file. If not set the CLI will search the current directory for default.codiusstate.json.
--skip-extend Boolean Create the extend cron job without initially extending the pod by a buffer.
--assume-yes, -y Boolean Say yes to all prompts.

cron view

View existing cron jobs.

Options Argument Type Description
--codius-state-file String Filename or full path to the codius state file. If not set the CLI will search the current directory for default.codiusstate.json. Mutually exclusive with all.
--all, -a Boolean View all existing cron jobs. Mutually exclusive with codius-state-file.

cron remove

Remove an existing cron job.

Options Argument Type Description
--codius-state-file String Filename or full path to the codius state file. If not set the CLI will search the current directory for default.codiusstate.json. Mutually exclusive with all.
--all, -a Boolean Remove all existing cron jobs. Mutually exclusive with codius-state-file.
--assume-yes, -y Boolean Say yes to all prompts.

How to Use

Terms

  • Codius File: File named codius.json which contains the main details of the manifest along with variable place holders to be interpolated with the actual values in the codiusvars.json file. This file may contain description fields for variables defined in the manifest.vars field. The official specification can be found here.
  • Codius Vars File: File named codiusvars.json which contains the public and private variables which will be interpolated with the codius.json file to generate the full manifest. Similar to codius.json, this file may include description fields for public variables. The official specification can be found here.
  • Codius State File: File named default.codiusstate.json which is created by the upload command on successful upload to at least one host. This file contains the generated interpolated manifest, the hosts its been uploaded to, and details about when the pod expires on those hosts.

Background

Codius CLI uses a directory per manifest approach where each manifest is composed of 2 files codius.json & codiusvars.json. The upload command uses these 2 files and through interpolation generates the codius manifest for upload to a host. By default the upload command looks for those files in its current directory. Once uploaded the generated manifest is stored locally in a default.codiusstate.json file. This file contains all the information needed to extend and manage the generated manifest.

NOTE: This file is very important, it contains the generated manifest and the hash of the manifest, if deleted you will no longer be able to extend the current contract or upload it to a new host.

Manifests should be stored on a per folder basis and be broken up into a codius.json & codiusvars.json files. An example folder structure is how the Codius Examples repo is setup.

examples
├── host-list
│   ├── codius.json
│   └── codiusvars.json
├── nginx
│   ├── codius.json
│   └── codiusvars.json
├── quake
│   ├── codius.json
│   └── codiusvars.json
└── react-app
    ├── codius.json
    └── codiusvars.json

This setup allows the user to save the codius.json & codiusvars.json in any version control software. The codius.json file can then be shared with anyone without exposing the private or public variables used by your application. After upload the default.codiusstate.json file will be created in the folder which can also be added to version control to keep track of the generated manifest and where the pods are running.

After uploading the 4 apps above from the examples the folder structure would look as follows.

examples
├── host-list
│   ├── codius.json
│   ├── codiusvars.json
│   └── default.codiusstate.json
├── nginx
│   ├── codius.json
│   ├── codiusvars.json
│   └── default.codiusstate.json
├── quake
│   ├── codius.json
│   ├── codiusvars.json
│   └── default.codiusstate.json
└── react-app
    ├── codius.json
    ├── codiusvars.json
    └── default.codiusstate.json

Example Files

codius.json

{
 "manifest": {
   "name": "nginx-codius-pod",
   "version": "1.0.0",
   "machine": "small",
   "port": "80",
   "containers": [{
     "id": "app",
     "image": "nginx@sha256:3e2ffcf0edca2a4e9b24ca442d227baea7b7f0e33ad654ef1eb806fbd9bedcf0",
     "command": ["nginx", "-g", "daemon off;"],
     "workdir": "/root",
     "environment": {
       "EXAMPLE_PUBLIC_ENV": "$EXAMPLE_PUBLIC_ENV",
       "EXAMPLE_PRIVATE_ENV": "$EXAMPLE_PRIVATE_ENV"
     }
   }]
 }
}

codiusvars.json

{
  "vars": {
    "public": {
      "EXAMPLE_PUBLIC_ENV": {
        "value": "Public Env Value"
      }
    },
    "private": {
      "EXAMPLE_PRIVATE_ENV": {
        "value": "Private Env Value",
        "nonce": "jklnfgwekltfhj4io23r89o"
      }
    }
  }
}

empty codiusvars.json

{
  "vars": {
    "public": {},
    "private": {}
  }
}

Uploading a Pod

To upload a pod create a folder which contains 2 files a codius.json and codiusvars.json. The codius.json file contains the details about the container and all the main contents of the pod. The codiusvar.json file contains the variables which will be interpolated with the codius.json file to create the generated manifest.

NOTE: The codiusvars.json file is required even if there are no variables to be interpolated. When running the Codius CLI it will prompt you in the case of a missing codiusvars.json file and generate one with no variables present.

Examples of pods can be found in the Codius Examples repo.

Upload to a specific host

Description: Generates the manifest and uploads a pod to the specified host. Parameters:

  • Host: https://codius.example.com
  • Duration: 600 seconds (10 Mins)
codius upload --host https://codius.example.com -d 600

Upload to X random hosts

Description: Generates the manifest and uploads a pod 5 random hosts. Parameters:

  • Host Count: 5
  • Duration: 600 seconds (10 Mins)
codius upload -c 5 -d 600

Upload to 2+ specific hosts

Description: Generates the manifest and uploads a pod to 2 specific hosts. Parameters:

  • Hosts: https://codius.example.com, https://codius.example2.com
  • Duration: 600 seconds (10 Mins)
codius upload --host https://codius.example.com --host https://codius.example2.com -d 600

Extending a Pod

Extend supports managing the pod once it has been uploaded successfully. This is done through the default.codiusstate.json file. To extend as pod there must be a default.codiusstate.json file present. This file is generated during the upload step and contains the generated manifest, hosts uploaded to, and the status of when the current running pod expires on those hosts.

Extend with same parameters as upload

Description: Extends the pod with the same parameters as used during the initial upload or last extend. Parameters:

  • None
codius extend

Extend with different parameters

Description: Extends the pod with the passed in parameters. Parameters:

  • Duration: 600 seconds (10 Mins)
  • Max Monthly Rate: 10
  • Units: XRP
codius extend -d 600 -m 10 -u XRP

Extending a Pod by Manifest Hash

If the manifest hash and the host where the pod is running is known the pods runtime can be extended using the extend-hash command. NOTE: The pod must be running in order to be able to extend it. If its expired extend-hash will not work.

Extend by Hash & Host

Description: Extend the running pod on the host by its manifest hash. Parameters:

extend-hash hyg2qziqlhdogtbxm347spzrwkibgbzdalyj2qavqra4gzmm5jzq --host https://codius.example.com -d 600 -m 10 -u XRP

Extend by Pod Url

Description: Extends the running pod on the host using its full URL. Parameters:

  • Duration: 600 seconds (10 Mins)
  • Max Monthly Rate: 10
  • Units: XRP
extend-hash https://hyg2qziqlhdogtbxm347spzrwkibgbzdalyj2qavqra4gzmm5jzq.codius.example.com -d 600 -m 10 -u XRP

Tailing logs

Tail logs on upload

Description: Enables debug mode and tails logs on upload Parameters:

  • Upload Parameters: See above for upload params.
  • Debug: Enables debug logs on the pod.
  • Tail: Immediately tails the logs of the pod after upload.
codius upload --host https://codius.example.com -d 600 --debug --tail

Enable Debug Mode on upload

Description: Enables debug mode on upload Parameters:

  • Upload Parameters: See above for upload params.
  • Debug: Enables debug logs on the pod.
codius upload --host https://codius.example.com -d 600 --debug

Tail logs after upload using manifest hash and host

Description: Tails the logs of a pod using the manifest hash and host Parameters:

  • Host: Host where the pod is running you wish to tail.
codius tail  xb4raigiudbbbqutwp6iftxvmx6zih4sebcpiy5dpwr2w4iypcfa --host https://codius.tinypolarbear.com

Tail logs after upload using *.codiusstate.json file

Description: Tails the logs of all hosts in the *codiusstate.json file Parameters:

  • Codius State File: codius/default.codiusstate.json
codius tail --codius-state-file codius/default.codiusstate.json

Tail logs after upload using *.codiusstate.json file

Description: Tails the logs of all hosts in the *codiusstate.json file from the current working directory.

codius tail

Extending a Pod with Cron

Creating a New Extend Cron Job

Description: Creates a new cron job for a running pod. Searches the current directory for default.codiusstate.json since the codius state file is not specified. Parameters:

  • Buffer: 3600 seconds (1 hr)
  • Max Monthly Rate: 10
  • Units: XRP
cron create --buffer-sec 3600 -m 10 -u XRP

View the Cron Job for a Pod

Description: Lists the cron job associated with a running pod. Parameters:

  • Codius State File: codius/default.codiusstate.json
cron view --codius-state-file codius/default.codiusstate.json

View All Codius Cron Jobs

Description: Lists all Codius cron jobs. Parameters:

  • All: flag to view all codius jobs
cron view -a

Remove Existing Cron Job for a Pod

Description: Removes the cron job for a pod. Parameters:

  • Codius State File: codius/default.codiusstate.json
  cron remove --codius-state-file codius/default.codiusstate.json

Remove All Codius Cron Jobs

Description: Remove all Codius cron jobs.

  • All: flag to remove all codius jobs
cron remove -a

Advanced Features

Manually Editing *.codiusstate.json files

The *.codiusstate.json file consists of the following properties

  • description Used for notes about the file, not used by the Codius CLI. Can be manually modified.
  • manifestHash The manifest hash of the manifest in the generatedManifest.manifest property. DO NOT MANUALLY MODIFY
  • generatedManifest The generated manifest from the codius upload command. DO NOT MANUALLY MODIFY
  • options The options used for the last upload or extend command. Can be manually modified.
  • hostList The list of hosts that was used in the last upload or extend command. Can be manually modified to remove or add a host.
    • NOTE: If you are using the HOST environment variable this should not be modified since you cannot modify the generatedManifest section manually to add or remove a host. You must use the upload command to regenerate the manifest with any updated HOSTS since it will change the manifest hash.
  • status The current known status of the pod on the hosts it was uploaded to. It should not be manually modified but if you remove a host from the hostList you may want to remove its associated status key as well. Its used to make the Codius CLI more informative when extending a pod.

Codius Hosts File

In your manifest directory optionally a codiushosts.json file may be included which contains a list of hosts which should be used for upload. If the --host-count option is used the # of hosts will be randomly selected from those in this file. If that option is not specified then the pod will attempt to upload to all hosts in the codiushosts.json file.

codiushost.json

{
  "hosts":["https://codius.example.com","https://codius.example2.com"]
}

Migrating Existing Manifests

Existing manifest should be migrated to the new format by splitting the main manifest and the variables into the codius.json and codiusvars.json files. Unfortunately this means that the generated manifest will change slightly and so will the manifest hash. If that is not tenable please stick with using the Codius CLI 2.0.x.

To migrate the existing manifest to the new format first look at the existing manifests manifest object. Copy it into a new file codius.json and remove the vars object from the manifest object.

Make a new codiusvars.json file which looks like the example in the Example Files: section with the private and public variables that were in the original manifests vars and private objects.

Once that is complete the manifest is ready to be uploaded with the new Codius CLI. Run upload which will combine the 2 files and generate a manifest with the private variables hashed

Debugging

If the Codius CLI is not working as expected debugging can be enabled by adding DEBUG=codius-cli* to the beginning of the codius command. E.g.

DEBUG=codius-cli* codius

If that does not provide enough detail, all debug output can be enabled by using DEBUG=*.

License

Apache-2.0