Release Prep (#41)

This PR in a variety of release-oriented changes in preparation for delivering testing artifacts and moving towards more formal releases.

## Added
- Extracted `myceli` config values into config file with reasonable defaults.
- CI based docker builds and binary releases.
- Basic general testing plan in `testing/testing-plan.md`.
- Added basic for for building/running `myceli` in Docker.

## Changed
- Updated docs/readme to reflect new `myceli` configuration file.

.github/workflows/create-release.yaml

@@ -0,0 +1,75 @@
+name: release
+
+on:
+  push:
+    tags:
+      - "*"
+
+jobs:
+  start_release:
+    name: build_release
+    runs-on: ubuntu-latest
+    outputs:
+      upload_url: ${{ steps.create_release.outputs.upload_url }}
+    steps:
+      - uses: actions/checkout@v2
+      - name: Create release
+        uses: actions/create-release@v1
+        id: create_release
+        with:
+          draft: true
+          prerelease: false
+          release_name: ${{ github.ref_name }}
+          tag_name: ${{ github.ref }}
+        env:
+          GITHUB_TOKEN: ${{ github.token }}
+  build_upload_artifacts:
+    needs: [start_release]
+    name: build_upload_artifacts
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        target:
+          [
+            armv7-unknown-linux-gnueabihf,
+            aarch64-unknown-linux-gnu,
+            x86_64-unknown-linux-gnu,
+          ]
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions-rs/toolchain@v1
+        with:
+          toolchain: stable
+      - name: install cross
+        run: cargo install cross --git https://github.com/cross-rs/cross
+      - name: clean cargo
+        run: cargo clean
+      - name: myceli build
+        run: CROSS_CONFIG=Cross.toml cross build --bin myceli --target ${{ matrix.target }} --release
+      - name: upload myceli binary
+        uses: actions/upload-release-asset@v1
+        env:
+          GITHUB_TOKEN: ${{ github.token }}
+        with:
+          upload_url: ${{ needs.start_release.outputs.upload_url }}
+          asset_path: ./target/${{ matrix.target }}/release/myceli
+          asset_name: myceli-${{ matrix.target }}
+          asset_content_type: application
+  build_upload_docker:
+    needs: [start_release]
+    name: build_upload_docker
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Login to Github Container Registry
+        uses: docker/login-action@v1
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Build and publish myceli docker
+        run: |
+          docker build -f myceli.Dockerfile . --tag ghcr.io/ipfs-shipyard/myceli:latest
+          docker build -f myceli.Dockerfile . --tag ghcr.io/ipfs-shipyard/myceli:${{ github.ref_name }}
+          docker push ghcr.io/ipfs-shipyard/myceli:latest
+          docker push ghcr.io/ipfs-shipyard/myceli:${{ github.ref_name }}

Cross.toml

@@ -0,0 +1,8 @@
+[target.armv7-unknown-linux-gnueabihf]
+dockerfile = "cross-armv7-linux.Dockerfile"
+
+[target.aarch64-unknown-linux-gnu]
+dockerfile = "cross-aarch64-linux.Dockerfile"
+
+[target.x86_64-unknown-linux-gnu]
+dockerfile = "cross-x86-linux.Dockerfile"

car-utility/Cross.toml

@@ -1,2 +1,2 @@
 [target.armv7-unknown-linux-gnueabihf]
-dockerfile = "../Dockerfile"
+dockerfile = "../cross.Dockerfile"

cross-aarch64-linux.Dockerfile

@@ -0,0 +1,9 @@
+FROM ghcr.io/cross-rs/aarch64-unknown-linux-gnu:latest
+
+RUN apt-get update && \
+    apt-get install -y unzip
+
+RUN curl -Lo protoc.zip "https://github.com/protocolbuffers/protobuf/releases/download/v21.12/protoc-21.12-linux-x86_64.zip"
+RUN unzip -q protoc.zip -d /usr/local
+RUN chmod a+x /usr/local/bin/protoc
+ENV PROTOC=/usr/local/bin/protoc

cross-x86-linux.Dockerfile

@@ -0,0 +1,9 @@
+FROM ghcr.io/cross-rs/x86_64-unknown-linux-gnu:latest
+
+RUN apt-get update && \
+    apt-get install -y unzip
+
+RUN curl -Lo protoc.zip "https://github.com/protocolbuffers/protobuf/releases/download/v21.12/protoc-21.12-linux-x86_64.zip"
+RUN unzip -q protoc.zip -d /usr/local
+RUN chmod a+x /usr/local/bin/protoc
+ENV PROTOC=/usr/local/bin/protoc

docs/myceli-basic-setup.md

@@ -49,25 +49,44 @@ After `myceli` has been built for the appropriate environments it needs to be ru
 
 Use ssh to access the raspberry-pi and navigate to the `/home/pi` directory. Start a `myceli` instance with the following command:
 
-    $ ./myceli 127.0.0.1:8081
+    $ ./myceli
 
-This command assumes that the pi currently has a radio service running which is downlinking to the address `127.0.0.1:8081`, as specified in the local environment setup guide. 
+This command assumes that the pi currently has a radio service running which is downlinking to the address `127.0.0.1:8080`, as specified in the local environment setup guide. 
 
 A log message should appear indicating that `myceli` is up and listening:
 
-    $ INFO myceli::listener: Listening for messages on 127.0.0.1:8081
+    $ INFO myceli::listener: Listening for messages on 127.0.0.1:8080
 
 ### Running on the local computer
 
 Navigate to the `space/myceli` directory and run the following command:
 
-    $ cargo run -- 127.0.0.1:8081
+    $ cargo run
 
-This command assumes that the local computer has a radio service running which is downlinking to the address `127.0.0.1:8081`, as specified in the local environment setup guide.
+This command assumes that the local computer has a radio service running which is downlinking to the address `127.0.0.1:8080`, as specified in the local environment setup guide.
 
 A log message should appear indicating that `myceli` is up and listening:
 
-    $ INFO myceli::listener: Listening for messages on 127.0.0.1:8081
+    $ INFO myceli::listener: Listening for messages on 127.0.0.1:8080
+
+## Configuring Myceli
+
+`myceli` has a few configuration options which ship with default values, or can be tuned to fit system requirements.
+
+Current configuration values and defaults are:
+- `listen_address` - The network address `myceli` will listen on for incoming messages. Defaults to `127.0.0.1:8080`.
+- `retry_timeout_duration` - Timeout before `myceli` will retry a dag transfer, measured in milliseconds. The default value is 120_00 or two minutes.
+- `storage_path` - Directory path for `myceli` to use for storage. If this directory does not exist it will be created. Defaults to `storage/` in the process working directory.
+
+These configuration values can be set via a TOML config file which is passed as an argument when running `myceli`.
+
+Here is an example configuration file:
+
+    listen_address="127.0.0.1:9011"
+    retry_timeout_duration=360_000
+    storage_path="myceli_storage"
+
+If this configuration is saved to "myceli.toml", then we would run `myceli myceli.toml` to use the config file.
 
 ## Interacting with Myceli
 
@@ -79,9 +98,9 @@ Navigate to `space/app-api-cli` and run `cargo build` to build the tool we'll us
 
 One of the fundamental actions `myceli` can take is importing a file into it's internal IPFS store. Navigate to `space/app-api-cli` and run the following command to import a local file:
 
-    $ cargo run -- -l 127.0.0.1:8081 import-file Cargo.toml
+    $ cargo run -- -l 127.0.0.1:8080 import-file Cargo.toml
 
-This will send the `ImportFile` command to the local `myceli` instance listening at `127.0.0.1:8081` with the local `Cargo.toml` as the file to import. In this case we'll use the `-l` flag to listen for a response, as `myceli` will respond with the root CID if the file is correctly imported. Here is what the output may look like for a successful file import:
+This will send the `ImportFile` command to the local `myceli` instance listening at `127.0.0.1:8080` with the local `Cargo.toml` as the file to import. In this case we'll use the `-l` flag to listen for a response, as `myceli` will respond with the root CID if the file is correctly imported. Here is what the output may look like for a successful file import:
 
     Transmitting: {"ApplicationAPI":{"ImportFile":{"path":"Cargo.toml"}}}
     ApplicationAPI(FileImported { path: "Cargo.toml", cid: "bafybeicwxyav7jde73wb5svahp53qi5okq2p4bguyflfw6hsbmwbbl4bw4" })
@@ -92,19 +111,19 @@ Once a file has been imported, and the root CID is known, it is possible to ask
 
 On the local computer, in the `app-api-cli` directory, run the following command:
 
-    $ cargo run -- 127.0.0.1:8081 transmit-dag [root-cid-here] 127.0.0.1:8080
+    $ cargo run -- 127.0.0.1:8080 transmit-dag [root-cid-here] 127.0.0.1:8081 5
 
-This will send the `TransmitDag` command to the `myceli` instance listening on `127.0.0.1:8081`, which will ask it to transmit the blocks associated with the specified root CID to `127.0.0.1:8080`. After sending this command you should see several `Transmitting block ...` messages from the local computer's `myceli`, and several `Received block ...` messages from the raspberry-pi's `myceli`.
+This will send the `TransmitDag` command to the `myceli` instance listening on `127.0.0.1:8080`, which will ask it to transmit the blocks associated with the specified root CID to `127.0.0.1:8081` with `5` specified as the number of retries. After sending this command you should see several `Transmitting block ...` messages from the local computer's `myceli`, and several `Received block ...` messages from the raspberry-pi's `myceli`.
 
 ### Validating a dag
 
 After a dag has been transmitted, it must be verified that it is complete and valid at the destination. 
 
 To verify the status of the dag on the raspberry-pi, run the following app-api-cli command:
 
-    $ cargo run -- 127.0.0.1:8080 validate-dag [root-cid-here]
+    $ cargo run -- 127.0.0.1:8081 validate-dag [root-cid-here]
 
-This will send the `ValidateDag` command to the radio listening at `127.0.0.1:8080`, which will then send it to the `myceli` instance on the raspberry-pi. In this case, the response will appear in the logs for the local computer's `myceli` instance. Check over there for `ValidateDagResponse` and the `result` string. If it says `Dag is valid`, then we know the transfer was complete and valid.
+This will send the `ValidateDag` command to the radio listening at `127.0.0.1:8081`, which will then send it to the `myceli` instance on the raspberry-pi. In this case, the response will appear in the logs for the local computer's `myceli` instance. Check over there for `ValidateDagResponse` and the `result` string. If it says `Dag is valid`, then we know the transfer was complete and valid.
 
 
 ### Exporting a dag
@@ -113,7 +132,7 @@ Once a dag has been transmitted and validated, it can be exported as a file on t
 
 To export the received dag on the raspberry-pi, run the following app-api-cli command:
 
-    $ cargo run -- 127.0.0.1:8080 export-dag [root-cid-here] [/file/system/path]
+    $ cargo run -- 127.0.0.1:8081 export-dag [root-cid-here] [/file/system/path]
 
-This will send the `ExportDag` command to the radio listening at `127.0.0.1:8080`, which will send it to the `myceli` instance on the rasberry-pi. This command includes the specified root cid and path to export to. After the command has been received and executed, you should find a file at the specified path containing the dag data.
+This will send the `ExportDag` command to the radio listening at `127.0.0.1:8081`, which will send it to the `myceli` instance on the rasberry-pi. This command includes the specified root cid and path to export to. After the command has been received and executed, you should find a file at the specified path containing the dag data.
 

docs/myceli-docker.md

@@ -0,0 +1,32 @@
+# Build and running Myceli in Docker
+
+This doc contains instructions on how to build and run `myceli` in Docker
+
+### Building
+
+The file `myceli.Dockerfile` contains all the instructions needed by Docker to produce an image for running `myceli`. This image can be built by running the following command:
+
+    $ docker build -f myceli.Dockerfile . -t myceli
+
+### Pulling
+
+The `myceli` docker images are published to the Github Container registry and can be pulled with the following command:
+
+    $ docker pull ghcr.io/ipfs-shipyard/myceli:latest
+
+### Running
+
+We only suggest running `myceli` in Docker in Linux environments due to networking requirements.
+
+Example running of `myceli` in a standalone Docker container with default settings:
+
+    $ docker run --rm -v `pwd`:/myceli/ --network host -it ghcr.io/ipfs-shipyard/myceli:latest
+
+Important pieces to point out here:
+
+    - `-v pwd:/myceli/`: Mounting a local directory is necessary for `myceli`'s storage to persist
+    - `--network host`: The container running `myceli` needs to either run on the host network, or on the same network as the other services which will be communicating with it (controller CLI, ground radio bridge).
+
+Optionally you may want to pass a config file argument in with the `CONFIG_PATH` environment variable, like this:
+
+    $ docker run --rm -v `pwd`:/myceli/ --network host -e CONFIG_PATH=/myceli/config.toml -it ghcr.io/ipfs-shipyard/myceli:latest