Skip to content

Commit

Permalink
docs: update pages-docker,cluster-deployment,
Browse files Browse the repository at this point in the history
contributors & installation
  • Loading branch information
NishaSharma14 committed Aug 8, 2023
1 parent 93cc891 commit ce67ac1
Show file tree
Hide file tree
Showing 11 changed files with 120 additions and 51 deletions.
19 changes: 13 additions & 6 deletions docs/src/deployment/cluster-deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,16 @@
:::

# Kubernetes deployment via Helm.
Cheminformatics Python Microservices comes packaged with [Helm](https://helm.sh/docs/) chart which simplifies the deployment and management of applications on [Kubernetes](https://kubernetes.io/) by providing a convenient package manager interface. By following the steps outlined in this documentation, you can easily deploy this microservice using [Helm](https://helm.sh/docs/), enabling efficient and reproducible deployments in your [Kubernetes](https://kubernetes.io/) cluster.
NMR Kit comes packaged with [Helm](https://helm.sh/docs/) chart which simplifies the deployment and management of applications on [Kubernetes](https://kubernetes.io/) by providing a convenient package manager interface. By following the steps outlined in this documentation, you can easily deploy this microservice using [Helm](https://helm.sh/docs/), enabling efficient and reproducible deployments in your [Kubernetes](https://kubernetes.io/) cluster.

Please refer to Helm’s [documentation](https://helm.sh/docs/) to get started.
The chart is packaged with below additional services as subcharts, which can be enabled or disabled via the value.yml file.
* [minio - v12.6.12](https://artifacthub.io/packages/helm/bitnami/minio)
* [redis - v17.14.6](https://artifacthub.io/packages/helm/bitnami/redis)
* [grafana - v8.4.5](https://artifacthub.io/packages/helm/bitnami/grafana)
* [prometheus - v22.6.5](https://prometheus-community.github.io/helm-charts/)
* [rdkitcatridgepostgres - v0.1.0](https://nfdi4chem.github.io/repo-helm-charts)

If you are new to Helm, please refer to Helm’s [documentation](https://helm.sh/docs/) to get started.

**Prerequisites:**

Expand All @@ -22,10 +29,10 @@ If you had already added this repo earlier, run `helm repo update` to retrieve t

* **Deploy the Chart:** To deploy the chart, use the helm install command followed by the chart package name and an optional release name:
```bash
helm install myrelease repo-helm-charts/cheminfo-microservice
helm install myrelease repo-helm-charts/nmrkit
```
The release name (myrelease in this example) is used to identify the deployment, and it must be unique within the Kubernetes cluster.
The above command with deploy the service with the default configuration provided in [values.yml](https://github.com/NFDI4Chem/repo-helm-charts/blob/main/charts/cheminfo-microservice/values.yaml) file. To overwrite the default configuration please follow this [link](https://helm.sh/docs/chart_template_guide/values_files/) to learn more.
The above command with deploy the service with the default configuration provided in [values.yml](https://github.com/NFDI4Chem/repo-helm-charts/blob/main/charts/nmrkit/values.yaml) file. To overwrite the default configuration please follow this [link](https://helm.sh/docs/chart_template_guide/values_files/) to learn more.

* Helm will install the chart and deploy the application to your Kubernetes cluster. You can view the deployed resources using kubectl commands:
```bash
Expand All @@ -34,7 +41,7 @@ kubectl get services
```
* **Upgrading and Managing Deployments:** To upgrade an existing deployment, Use the `helm upgrade` command to apply the changes to the existing release e.g.
```bash
helm upgrade myrelease repo-helm-charts/cheminfo-microservice-0.0.2
helm upgrade myrelease repo-helm-charts/nmrkit-0.0.2
```

* **Uninstalling the Chart:** To remove a deployed chart and associated resources, use the helm uninstall command:
Expand All @@ -45,7 +52,7 @@ This will delete all resources created by the chart, including pods, services, a


### Contribute or Report an issue
Thank you for your valuable assistance in enhancing our deployment process. If you would like to contribute, kindly create a pull request in our [GitHub](https://github.com/NFDI4Chem/repo-helm-charts) repository. For any issues or bugs you have encountered, please feel free to create an [issue](https://github.com/NFDI4Chem/repo-helm-charts/issues) in the same or write to us at caffeine-devs@uni-jena.de.
Thank you for your valuable assistance in enhancing our deployment process. If you would like to contribute, kindly create a pull request in our [GitHub](https://github.com/NFDI4Chem/repo-helm-charts) repository. For any issues or bugs you have encountered, please feel free to create an [issue](https://github.com/NFDI4Chem/repo-helm-charts/issues) or reach out to us via our [Help Desk](https://helpdesk.nfdi4chem.de/).
Your feedback is greatly appreciated.


24 changes: 13 additions & 11 deletions docs/src/deployment/docker.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,25 @@
:::danger Needs to be updated
:::

# Docker
The NMR Kit project utilizes the containerized microservices approach to package chemistry toolkits and deep learning tools. It comes pre-packaged with toolkits [RDKit](https://github.com/rdkit/rdkit), [CDK](https://doi.org/10.1186/s13321-017-0220-4), [OpenBabel](https://github.com/openbabel/openbabel) and deep learning tools ([DECIMER](https://github.com/Kohulan/DECIMER-Image_Transformer), [STOUT](https://github.com/Kohulan/Smiles-TO-iUpac-Translator)) for handling chemical data - OSR, format conversions, and descriptor calculation. This enables efficient handling of large data volumes and improved performance and development of cheminformatics applications that are scalable and interoperable.

It is containerized using Docker and is distributed publicly via the [Docker Hub](https://hub.docker.com/r/caffeinejena/cheminformatics-python-microservice), a cloud-based registry provided by Docker that allows developers to store, share, and distribute Docker images.
NMRKit is containerized using Docker and is distributed publicly via the [Docker Hub](https://hub.docker.com/r/nfdi4chem/nmrkit), a cloud-based registry provided by Docker that allows developers to store, share, and distribute Docker images.

To use this image:

* Make sure you have Docker installed and configured on your target deployment environment.
* Pull the image by providing the appropiate tag.
* Make sure you have [Docker](https://docs.docker.com/get-docker/) installed and configured on your target deployment environment.
* Pull the NMRKit image by providing the appropiate tag.

```bash
docker pull caffeinejena/cheminformatics-python-microservice:[tag]
docker pull nfdi4chem/nmrkit:[tag]

```
* Run the below command to run the image
* NMRKit uses [rdkit-cartridge-debian](https://hub.docker.com/r/informaticsmatters/rdkit-cartridge-debian) Postgres. Run the below command to spin up the image.

```bash
docker run -d --name postgres -e POSTGRES_PASSWORD=password -e POSTGRES_USER=nmrkit -p 5432:5432 informaticsmatters/rdkit-cartridge-debian:latest

```

* After the Postgres container is prepared to receive connections, initiate the NMRKit container by executing the following command and providing the Postgres credentials.

```bash
docker run -d -p 8080:80 --name [image-name] caffeinejena/cheminfo-microservice:[tag]
docker run -d -p 8080:80 --name nmrkit -e POSTGRES_PASSWORD=password -e POSTGRES_USER=nmrkit -e POSTGRES_SERVER=postgres -e POSTGRES_DB=nmr_predict nfdi4chem/nmrkit:[tag]

```
2 changes: 1 addition & 1 deletion docs/src/development/contributors.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Contributors

![Alt](https://repobeats.axiom.co/api/embed/f8652b486f670b74258723f278c28c29c7e233d7.svg "Repobeats analytics image")
![Alt](https://repobeats.axiom.co/api/embed/3830ca170f30ecb4e709a88c4747737874d72b46.svg "Repobeats analytics image")
126 changes: 93 additions & 33 deletions docs/src/development/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,21 +9,21 @@

4. Navigate to the desired directory: Use `cd` to navigate to the directory where you want to clone the project.

5. Clone the repository: Run the command `git clone https://github.com/Steinbeck-Lab/cheminformatics-python-microservice.git` to clone the project.
5. Clone the repository: Run the command `git clone https://github.com/NFDI4Chem/nmrkit.git` to clone the project.

6. Use `cd` to navigate into the cloned project directory.

You have successfully cloned the project from GitHub onto your local machine.

Once cloned you can either choose to run the project via Docker (recommended) or locally (need to make sure you have all the dependencies resolved).
Once cloned you can either choose to run the project via Docker-compose (recommended) or locally (need to make sure you have all the dependencies resolved).

## Docker

1. Install Docker: Install Docker on your machine by following the instructions for your specific operating system.
1. Install Docker: Install [Docker](https://img.docker.com/get-docker/) on your machine by following the instructions for your specific operating system.

2. Use `cd` to navigate into the cloned project directory.
2. Use `cd` to navigate into the cloned project directory and create a .env file which can be copied from .env.template and provide your own values for password and username.

3. You will find a docker-compose.yml file in the project. If you don't have a docker-compose.yaml file uses the following template.
3. You can use the docker-compose.yml file in the root directory, which is same as below and update accordingly if required.

```yaml
version: "3.8"
Expand All @@ -33,58 +33,118 @@ services:
build:
context: ./
dockerfile: Dockerfile
container_name: cheminformatics-python-microservice
container_name: nmrkit-api
volumes:
- ./app:/code/app
ports:
- "80:80"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:80/latest/chem"]
test: ["CMD", "curl", "-f", "http://localhost:80/latest/ping"]
interval: 1m30s
timeout: 10s
retries: 20
start_period: 60s
Prometheus:
image: prom/Prometheus
container_name: Prometheus
env_file:
- ./.env
prometheus:
image: prom/prometheus
container_name: nmrkit_prometheus
ports:
- 9090:9090
volumes:
- ./Prometheus_data/Prometheus.yml:/etc/Prometheus/Prometheus.yml
- ./prometheus_data/prometheus.yml:/etc/prometheus/prometheus.yml
command:
- '--config.file=/etc/Prometheus/Prometheus.yml'
Grafana:
image: Grafana/Grafana
container_name: Grafana
- '--config.file=/etc/prometheus/prometheus.yml'
grafana:
image: grafana/grafana
container_name: nmrkit_grafana
ports:
- 3000:3000
volumes:
- Grafana_data:/var/lib/Grafana
- grafana_data:/var/lib/grafana
redis:
image: "redis:alpine"
ports:
- "${FORWARD_REDIS_PORT:-6379}:6379"
volumes:
- "redis:/data"
networks:
- default
healthcheck:
test: ["CMD", "redis-cli", "ping"]
retries: 3
timeout: 5s
pgsql:
image: "informaticsmatters/rdkit-cartridge-debian"
ports:
- "${FORWARD_DB_PORT:-5432}:5432"
env_file:
- ./.env
volumes:
- "pgsql:/var/lib/postgresql/data"
networks:
- default
healthcheck:
test:
[
"CMD",
"pg_isready",
"-q",
"-d",
"${POSTGRES_DB}",
"-U",
"${POSTGRES_USER}",
]
retries: 3
timeout: 5s
minio:
image: 'minio/minio:latest'
ports:
- '${FORWARD_MINIO_PORT:-9001}:9001'
- '${FORWARD_MINIO_CONSOLE_PORT:-8900}:8900'
environment:
MINIO_ROOT_USER: 'sail'
MINIO_ROOT_PASSWORD: 'password'
volumes:
- 'minio:/data/minio'
networks:
- default
command: minio server /data/minio --console-address ":8900"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9001/minio/health/live"]
retries: 3
timeout: 5s
volumes:
Prometheus_data:
prometheus_data:
driver: local
driver_opts:
o: bind
type: none
device: ./Prometheus_data
Grafana_data:
device: ./prometheus_data
grafana_data:
driver: local
driver_opts:
o: bind
type: none
device: ./Grafana_data
device: ./grafana_data
redis:
driver: local
minio:
driver: local
pgsql:
driver: local
networks:
default:
name: cpm_fastapi
name: nmrkit_vpc
```
4. Run Docker Compose: Execute the command ```docker-compose up``` to start the containers defined in the Compose file.
4. Run Docker Compose: Execute the command ```docker-compose up -d``` to start the containers defined in the Compose file.

5. Wait for the containers to start: Docker Compose will start the containers and display their logs in the terminal or command prompt.

Unicorn will start the app and display the server address (usually `http://localhost:80`) and Grafana dashboard can be accessed at `http://localhost:3000`

You may update the docker-compose file to disable or add additional services but by default, the docker-compose file shipped with the project has the web (cheminformatics-python-microservice FAST API app), Prometheus and Grafana (logging and visualisation of metrics) services and associated volumes shared via a network.
You may update the docker-compose file to disable or add additional services but by default, the docker-compose file shipped with the project has the web (nmrkit FAST API app), [rdkit-cartridge-debian](https://hub.docker.com/r/informaticsmatters/rdkit-cartridge-debian), [Prometheus](https://prometheus.io/img/introduction/overview/) and [Grafana](https://prometheus.io/img/introduction/overview/) (logging and visualisation of metrics), [Minio](https://min.io/img/minio/linux/index.html), [Redis](https://redis.io/img/) services.

## Standalone

Expand Down Expand Up @@ -137,7 +197,7 @@ CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80", "--workers"
The following instructions are based on the blog post - https://dev.to/ken_mwaura1/getting-started-monitoring-a-fastapi-app-with-Grafana-and-Prometheus-a-step-by-step-guide-3fbn
To learn more about using Grafana in general, see the official [Prometheus](https://prometheus.io/docs/introduction/overview/) and [Grafana](https://grafana.com/docs/) documentation, or check out our other monitoring tutorials.
To learn more about using Grafana in general, see the official [Prometheus](https://prometheus.io/img/introduction/overview/) and [Grafana](https://grafana.com/img/) documentation, or check out our other monitoring tutorials.
:::
Expand Down Expand Up @@ -171,28 +231,28 @@ Grafana login
Enter the default username and password (admin/admin) and click "Log In". You should be prompted to change the password. Enter a new password and click "Save". You should see the following screen:
<p align="center">
<img align="center" src="/docs/grafana_login.jpeg" alt="Logo" style="filter: drop-shadow(0px 0px 10px rgba(0, 0, 0, 0.5));" width="auto">
<img align="center" src="/img/grafana_login.jpeg" alt="Logo" style="filter: drop-shadow(0px 0px 10px rgba(0, 0, 0, 0.5));" width="auto">
</p>
Grafana home
Click on the "Create your first data source" button. You should see the following screen:
<p align="center">
<img align="center" src="/docs/grafana.png" alt="Logo" style="filter: drop-shadow(0px 0px 10px rgba(0, 0, 0, 0.5));" width="auto">
<img align="center" src="/img/grafana.png" alt="Logo" style="filter: drop-shadow(0px 0px 10px rgba(0, 0, 0, 0.5));" width="auto">
</p>
Grafana add the data source
<p align="center">
<img align="center" src="/docs/grafana_ds.png" alt="Logo" style="filter: drop-shadow(0px 0px 10px rgba(0, 0, 0, 0.5));" width="auto">
<img align="center" src="/img/grafana_ds.png" alt="Logo" style="filter: drop-shadow(0px 0px 10px rgba(0, 0, 0, 0.5));" width="auto">
</p>
Click on the "Prometheus" button. You should see the following screen:
<p align="center">
<img align="center" src="/docs/prometheus.png" alt="Logo" style="filter: drop-shadow(0px 0px 10px rgba(0, 0, 0, 0.5));" width="auto">
<img align="center" src="/img/prometheus.png" alt="Logo" style="filter: drop-shadow(0px 0px 10px rgba(0, 0, 0, 0.5));" width="auto">
</p>
Enter the following information:
Expand All @@ -211,22 +271,22 @@ TLS CA Certificate: None <br/>
Click on the "Save & Test" button. You should see the following screen:
<p align="center">
<img align="center" src="/docs/grafana_ds_saved.png" alt="Logo" style="filter: drop-shadow(0px 0px 10px rgba(0, 0, 0, 0.5));" width="auto">
<img align="center" src="/img/grafana_ds_saved.png" alt="Logo" style="filter: drop-shadow(0px 0px 10px rgba(0, 0, 0, 0.5));" width="auto">
</p>
Click on the "Dashboards" button. You should see the following screen:
<p align="center">
<img align="center" src="/docs/grafana_db.png" alt="Logo" style="filter: drop-shadow(0px 0px 10px rgba(0, 0, 0, 0.5));" width="auto">
<img align="center" src="/img/grafana_db.png" alt="Logo" style="filter: drop-shadow(0px 0px 10px rgba(0, 0, 0, 0.5));" width="auto">
</p>
Click on the ""New Dashboard" button. You should see the following screen:
<p align="center">
<img align="center" src="/docs/grafana_db_new.png" alt="Logo" style="filter: drop-shadow(0px 0px 10px rgba(0, 0, 0, 0.5));" width="auto">
<img align="center" src="/img/grafana_db_new.png" alt="Logo" style="filter: drop-shadow(0px 0px 10px rgba(0, 0, 0, 0.5));" width="auto">
</p>
Download the Cheminformatics Microservice dashboard template (JSON) here - https://github.com/Steinbeck-Lab/cheminformatics-python-microservice/blob/main/cpm-dashboard.json
Download the NMRKit dashboard template (JSON) here - https://github.com/NFDI4Chem/nmrkit/blob/main/api-dashboard.json
## Benchmarking / Stress testing
Expand Down Expand Up @@ -281,7 +341,7 @@ We recommend using flake8 and Black to perform linting and formatting in Python
2. Linting with flake8: flake8 is a popular Python linter that checks your code for style and potential errors. Run flake8 by executing the following command in your project directory:

```bash
flake8 --ignore E501,W503 $(git ls-files '*.py')
flake8 --per-file-ignores="__init__.py:F401" --ignore E402,E501,W503 $(git ls-files '*.py') .
```

flake8 will analyze your code and provide feedback on any style violations or issues found.
Expand Down
Binary file added docs/src/img/grafana.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/src/img/grafana_db.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/src/img/grafana_db_new.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/src/img/grafana_ds.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/src/img/grafana_ds_saved.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/src/img/grafana_login.jpeg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/src/img/prometheus.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit ce67ac1

Please sign in to comment.