This repository has been archived by the owner on Apr 19, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #258 from threefoldtech/development_docs
add extra docs
- Loading branch information
Showing
6 changed files
with
440 additions
and
136 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,186 +1,159 @@ | ||
| # Grid proxy server | ||
| # Tfgrid Proxy | ||
| <!-- Header --> | ||
| <div class="header" align="center"> | ||
| <h1>TFGrid Proxy</h1> | ||
| <p><strong>A RESTful API to TFGridDB.</strong></p> | ||
| <img src="https://github.com/threefoldtech/tfgridclient_proxy/actions/workflows/unit.yml/badge.svg" > <img src="https://github.com/threefoldtech/tfgridclient_proxy/actions/workflows/integration.yml/badge.svg" > <img src="https://github.com/threefoldtech/tfgridclient_proxy/actions/workflows/release.yml/badge.svg" > <img src="https://github.com/threefoldtech/tfgridclient_proxy/actions/workflows/go.yml/badge.svg" > | ||
| </div> | ||
|
|
||
|  | ||
| <!-- ToC --> | ||
|
|
||
| Interact with TFgridDB using rest APIs | ||
| ## Table of Content | ||
|
|
||
| ## Live Instances | ||
| - [Tfgrid Proxy](#tfgrid-proxy) | ||
| - [Table of Content](#table-of-content) | ||
| - [About](#about) | ||
| - [How to use the project](#how-to-use-the-project) | ||
| - [Used Technologies \& Prerequisites](#used-technologies--prerequisites) | ||
| - [Start for Development](#start-for-development) | ||
| - [Setup for Production](#setup-for-production) | ||
| - [Get and install the binary](#get-and-install-the-binary) | ||
| - [Add as a Systemd service](#add-as-a-systemd-service) | ||
|
|
||
| - Dev network: <https://gridproxy.dev.grid.tf> | ||
| - Swagger: https://gridproxy.dev.grid.tf/swagger/index.html | ||
| - Test network: <https://gridproxy.test.grid.tf> | ||
| - Swagger: https://gridproxy.test.grid.tf/swagger/index.html | ||
| - Main network: <https://gridproxy.grid.tf> | ||
| - Swagger: https://gridproxy.grid.tf/swagger/index.html | ||
| <!-- About --> | ||
|
|
||
| ## Run for Development & Testing | ||
| ## About | ||
|
|
||
| ```bash | ||
| $ make help | ||
| ``` | ||
| The TFGrid client Proxy acts as an interface to access information about the grid. It supports features such as filtering, limitation, and pagination to query the various entities on the grid like nodes, contracts and farms. Additionally the proxy can contact the required twin ID to retrieve stats about the relevant objects and performing ZOS calls. | ||
|
|
||
| To list all the available tasks for running: | ||
| The proxy is used as the backend of several threefold projects like: | ||
|
|
||
| - Database | ||
| - Server | ||
| - Tests | ||
| - [Weblets](https://github.com/threefoldtech/grid_weblets/) | ||
| - [Dashboard](https://github.com/threefoldtech/tfgrid_dashboard/) | ||
|
|
||
| <!-- Usage --> | ||
|
|
||
| ## Prerequisites | ||
| ## How to use the project | ||
|
|
||
| 1. If you need to compile the server from source code you will need Golang compiler > 1.13, otherwise you can just download the compiled binaries from github [releases](https://github.com/threefoldtech/tfgridclient_proxy/releases). | ||
| 2. Postgres connection string | ||
| If you don't want to care about setting up your instance you can use one of the live instances. each works against a different TFChain network. | ||
|
|
||
| ## Generate swagger doc files | ||
| - Dev network: <https://gridproxy.dev.grid.tf> | ||
| - Swagger: <https://gridproxy.dev.grid.tf/swagger/index.html> | ||
| - Qa network: <https://gridproxy.qa.grid.tf> | ||
| - Swagger: <https://gridproxy.qa.grid.tf/swagger/index.html> | ||
| - Test network: <https://gridproxy.test.grid.tf> | ||
| - Swagger: <https://gridproxy.test.grid.tf/swagger/index.html> | ||
| - Main network: <https://gridproxy.grid.tf> | ||
| - Swagger: <https://gridproxy.grid.tf/swagger/index.html> | ||
|
|
||
| 1. Install swag and export to exec path | ||
| Or follow the [development guide](#start-for-development) to run yours. | ||
| By default, the instance runs against devnet. to configure that you will need to config this while running the server. | ||
|
|
||
| ```bash | ||
| go install github.com/swaggo/swag/cmd/swag@latest | ||
| export PATH=$(go env GOPATH)/bin:$PATH | ||
| ``` | ||
| > Note: You may face some differences between each instance and the others. that is normal because each network is in a different stage of development and works correctly with others parts of the Grid on the same network. | ||
| 2. Run `make docs`. | ||
| <!-- Prerequisites --> | ||
| ## Used Technologies & Prerequisites | ||
|
|
||
| ## Build | ||
| 1. **GoLang**: Mainly the two parts of the project written in `Go 1.17`, otherwise you can just download the compiled binaries from github [releases](https://github.com/threefoldtech/tfgridclient_proxy/releases) | ||
| 2. **Postgresql**: Used to load the TFGrid DB | ||
| 3. **Docker**: Containerize the running services such as Postgres and Redis. | ||
| 4. **Mnemonics**: Secret seeds for adummy identity to use for the relay client. | ||
|
|
||
| ```bash | ||
| GIT_COMMIT=$(git describe --tags --abbrev=0) && \ | ||
| cd cmds/proxy_server && CGO_ENABLED=0 GOOS=linux go build -ldflags "-w -s -X main.GitCommit=$GIT_COMMIT -extldflags '-static'" -o server | ||
| ``` | ||
| For more about the prerequisites and how to set up and configure them. follow the [Setup guide](./docs/setup.md) | ||
|
|
||
| ## Production Run | ||
| <!-- Development --> | ||
|
|
||
| - Download the latest binary [here](https://github.com/threefoldtech/tfgridclient_proxy/releases) | ||
| - add the execution permission to the binary and move it to the bin directory | ||
| ## Start for Development | ||
|
|
||
| To start the services for development or testing make sure first you have all the [Prerequisites](#used-technologies--prerequisites). | ||
|
|
||
| - Clone this repo | ||
|
|
||
| ```bash | ||
| chmod +x ./gridproxy-server | ||
| mv ./gridproxy-server /usr/local/bin/gridproxy-server | ||
| git clone https://github.com/threefoldtech/tfgridclient_proxy.git | ||
| cd tfgridclient_proxy/ | ||
| ``` | ||
|
|
||
| - Add a new systemd service | ||
|
|
||
| ```bash | ||
| # create msgbus service | ||
| cat << EOF > /etc/systemd/system/gridproxy-server.service | ||
| [Unit] | ||
| Description=grid proxy server | ||
| After=network.target | ||
| [Service] | ||
| ExecStart=gridproxy-server --domain gridproxy.dev.grid.tf --email omar.elawady.alternative@gmail.com -ca https://acme-v02.api.letsencrypt.org/directory --postgres-host 127.0.0.1 --postgres-db db --postgres-password password --postgres-user postgres | ||
| Type=simple | ||
| Restart=always | ||
| User=root | ||
| Group=root | ||
| [Install] | ||
| WantedBy=multi-user.target | ||
| Alias=gridproxy.service | ||
| EOF | ||
| ``` | ||
|
|
||
| - enable the service | ||
| - The `Makefile` has all that you need to deal with Db, Explorer, Tests, and Docs. | ||
|
|
||
| ``` | ||
| systemctl enable gridproxy.service | ||
| ```bash | ||
| make help # list all the available subcommands. | ||
| ``` | ||
|
|
||
| - start the service | ||
|
|
||
| ``` | ||
| systemctl start gridproxy.service | ||
| - For a quick test explorer server. | ||
| ```bash | ||
| make restart | ||
| ``` | ||
|
|
||
| - check the status | ||
|
|
||
| ``` | ||
| systemctl status gridproxy.service | ||
| Now you can access the server at [:8080](http://loaclhost:8080) | ||
| - Run the tests | ||
|
|
||
| ```bash | ||
| make test-all | ||
| ``` | ||
|
|
||
| - The command options: | ||
| - domain: the host domain which will generate ssl certificate to. | ||
| - email: the mail used to run generate the ssl certificate. | ||
| - ca: certificate authority server url, e.g. | ||
| - let's encrypt staging: `https://acme-staging-v02.api.letsencrypt.org/directory` | ||
| - let's encrypt production: `https://acme-v02.api.letsencrypt.org/directory` | ||
| - postgre-\*: postgres connection info. | ||
|
|
||
| ## To upgrade the machine | ||
| - Generate docs. | ||
|
|
||
| - just replace the binary with the new one and apply | ||
|
|
||
| ``` | ||
| systemctl restart gridproxy-server.service | ||
| ``` | ||
| ```bash | ||
| make docs | ||
| ``` | ||
|
|
||
| - it you have changes in the `/etc/systemd/system/gridproxy-server.service` you have to run this command first | ||
| To run in development envornimnet see [here](tools/db/README.md) how to generate test db or load a db dump then use: | ||
|
|
||
| ``` | ||
| systemctl daemon-reload | ||
| ```sh | ||
| go run cmds/proxy_server/main.go --address :8080 --log-level debug -no-cert --postgres-host 127.0.0.1 --postgres-db tfgrid-graphql --postgres-password postgres --postgres-user postgres --mnemonics <insert user mnemonics> | ||
| ``` | ||
|
|
||
| ## Development Run | ||
| Then visit `http://localhost:8080/<endpoint>` | ||
|
|
||
| - To run in development envornimnet see [here](tools/db/README.md) how to generate test db or load a db dump then use: | ||
| ```sh | ||
| go run cmds/proxy_server/main.go --address :8080 --log-level debug -no-cert --postgres-host 127.0.0.1 --postgres-db tfgrid-graphql --postgres-password postgres --postgres-user postgres | ||
| ``` | ||
| - all server Options: | ||
| For more illustrations about the commands needed to work on the project. see [commands.md](./docs/commands.md). And for more about the project structure and contributions guidelines check [contributions.md](./docs/contributions.md) | ||
|
|
||
| | Option | Description | | ||
| | --- | --- | | ||
| | -address | Server ip address (default `":443"`) | | ||
| | -ca | certificate authority used to generate certificate (default `"https://acme-v02.api.letsencrypt.org/directory"`) | | ||
| | -cert-cache-dir | path to store generated certs in (default `"/tmp/certs"`) | | ||
| | -domain | domain on which the server will be served | | ||
| | -email | email address to generate certificate with | | ||
| | -log-level | log level `[debug\|info\|warn\|error\|fatal\|panic]` (default `"info"`) | | ||
| | -no-cert | start the server without certificate | | ||
| | -postgres-db | postgres database | | ||
| | -postgres-host | postgres host | | ||
| | -postgres-password | postgres password | | ||
| | -postgres-port | postgres port (default 5432) | | ||
| | -postgres-user | postgres username | | ||
| | -v | shows the package version | | ||
| <!-- Production--> | ||
|
|
||
| ## Setup for Production | ||
|
|
||
| - Then visit `http://localhost:8080/<endpoint>` | ||
| ## Get and install the binary | ||
|
|
||
| ## Dockerfile | ||
| - You can either build the project: | ||
|
|
||
| To build & run dockerfile | ||
|
|
||
| ```bash | ||
| docker build -t threefoldtech/gridproxy . | ||
| docker run --name gridproxy -e POSTGRES_HOST="127.0.0.1" -e POSTGRES_PORT="5432" -e POSTGRES_DB="db" -e POSTGRES_USER="postgres" -e POSTGRES_PASSWORD="password" threefoldtech/gridproxy | ||
| ``` | ||
|
|
||
| ## Update helm package | ||
|
|
||
| - Do `helm lint charts/gridproxy` | ||
| - Regenerate the packages `helm package -u charts/gridproxy` | ||
| - Regenerate index.yaml `helm repo index --url https://threefoldtech.github.io/tfgridclient_proxy/ .` | ||
| - Push your changes | ||
|
|
||
| ## Install the chart using helm package | ||
| ```bash | ||
| make build | ||
| chmod +x cmd/proxy_server/server \ | ||
| && mv cmd/proxy_server/server /usr/local/bin/gridproxy-server | ||
| ``` | ||
|
|
||
| - Adding the repo to your helm | ||
| - Or download a release: | ||
| Check the [releases](https://github.com/threefoldtech/tfgridclient_proxy/releases) page and edit the next command with the chosen version. | ||
|
|
||
| ```bash | ||
| helm repo add gridproxy https://threefoldtech.github.io/tfgridclient_proxy/ | ||
| wget https://github.com/threefoldtech/tfgridclient_proxy/releases/download/v1.6.7-rc2/tfgridclient_proxy_1.6.7-rc2_linux_amd64.tar.gz \ | ||
| && tar -xzf tfgridclient_proxy_1.6.7-rc2_linux_amd64.tar.gz \ | ||
| && chmod +x server \ | ||
| && mv server /usr/local/bin/gridproxy-server | ||
| ``` | ||
|
|
||
| - install a chart | ||
| ## Add as a Systemd service | ||
|
|
||
| - Create the service file | ||
|
|
||
| ```bash | ||
| helm install gridproxy/gridproxy | ||
| cat << EOF > /etc/systemd/system/gridproxy-server.service | ||
| [Unit] | ||
| Description=grid proxy server | ||
| After=network.target | ||
| [Service] | ||
| ExecStart=gridproxy-server --domain gridproxy.dev.grid.tf --email omar.elawady.alternative@gmail.com -ca https://acme-v02.api.letsencrypt.org/directory --substrate wss://tfchain.dev.grid.tf/ws --postgres-host 127.0.0.1 --postgres-db db --postgres-password password --postgres-user postgres --mnemonics <insert user mnemonics> | ||
| Type=simple | ||
| Restart=always | ||
| User=root | ||
| Group=root | ||
| [Install] | ||
| WantedBy=multi-user.target | ||
| Alias=gridproxy.service | ||
| EOF | ||
| ``` | ||
| ## Release | ||
| - Update the `appVersion` in `charts/Chart.yaml`. (push, open PR, merge) | ||
| - Draft new release with [Github UI Releaser](https://github.com/threefoldtech/tfgridclient_proxy/releases/new) | ||
| - In the tags dropdown menu write the new tag `appVersion` and create it. | ||
| - Generate release notes | ||
| - Mark as release or pre-release and publish | ||
| For more about the production environment and how the deployed instances are upgraded. see [production.md](./docs/production.md) |
Oops, something went wrong.