Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"tabWidth": 2,
"useTabs": false,
"singleQuote": false
}
74 changes: 55 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ To test the MINT services on your local machine, we recommend using the [microk8

### Microk8s installation

Tested on MacOS (arm64) with microk8s 1.28/stable.
Tested on MacOS (arm64) and microk8s 1.28/stable.
Tested on Ubuntu 22.04 (amd64) and microk8s 1.28/stable.

Install the microk8s using the following command:

Expand All @@ -26,7 +27,7 @@ Support for 'multipass' needs to be set up. Would you like to do that now? [y/N]
Enable required addons:

```bash
$ microk8s enable dashboard hostpath-storage
$ microk8s enable dashboard hostpath-storage ingress
```

Configure the kubectl to use the microk8s cluster
Expand All @@ -50,35 +51,64 @@ $ kubectl get nodes
### MINT services installation

If you using MacOS Silicon, you need to install the `arm64` version of the postgresql database. Add the following lines in the `values.yaml` file
WARNING: The arm64 image is not tested and may not work as expected. Please use it at your own risk.

**WARNING:** The arm64 image has not been tested and may not work as expected. Please use it at your own risk.

```
arm_support: true
```

Use the following commands to install the MINT services:

````bash
```bash
$ helm install -f ./values.yaml mint ./helm --namespace mint --create-namespace
```

or

Helm will returns the URL to access the MINT services. You can use the following command to get the URL:

```txt
The MINT system has been installed!

Please remember to edit your `/etc/hosts/`

After that, you can access to the MINT services


http://mint.local
http://ensemble-manager.mint.local
http://cromo.mint.local
http://mic.mint.local
http://datacatalog.mint.local
```

If you are using microk8s on a VM, need to get the IP address of the VM to access the MINT services. You can use the following command to get the IP address of the VM:

```bash
export UI_PORT=$(kubectl get --namespace mint -o jsonpath="{.spec.ports[0].nodePort}" services mint-ui)
export CROMO_PORT=$(kubectl get --namespace mint -o jsonpath="{.spec.ports[0].nodePort}" services mint-cromo)
export DATA_CATALOG_PORT=$(kubectl get --namespace mint -o jsonpath="{.spec.ports[0].nodePort}" services mint-data-catalog)
export HASURA_PORT=$(kubectl get --namespace mint -o jsonpath="{.spec.ports[0].nodePort}" services mint-hasura)
export MODEL_CATALOG_API_PORT=$(kubectl get --namespace mint -o jsonpath="{.spec.ports[0].nodePort}" services mint-model-catalog)
export MODEL_CATALOG_DATABASE_PORT=$(kubectl get --namespace mint -o jsonpath="{.spec.ports[1].nodePort}" services mint-model-catalog)
export NODE_IP=$(kubectl get nodes --namespace mint -o jsonpath="{.items[0].status.addresses[0].address}")

echo "MINT User Interface: http://$NODE_IP:$UI_PORT"
echo "MINT Model Catalog API: http://$NODE_IP:$MODEL_CATALOG_API_PORT/v1.8.0/docs"
echo "MINT Data Catalog: http://$NODE_IP:$DATA_CATALOG_PORT"
echo "MINT Constraint Reasoning Over MOdels (CROMO): http://$NODE_IP:$CROMO_PORT"
echo "MINT Database: http://$NODE_IP:$HASURA_PORT"
echo "MINT Model Catalog Database: http://$NODE_IP:$MODEL_CATALOG_DATABASE_PORT"
$ microk8s kubectl get node -o json | jq '.items[].status.addresses[] | select(.type=="InternalIP") | .address'
10.211.59.16
```

Then, edit the `/etc/hosts` file and add the following lines:

```bash
$ sudo vim /etc/hosts
```

Add the following lines (replace the IP address with the IP address of the VM):

```text
10.211.59.16 mint.local
10.211.59.16 cromo.mint.local
10.211.59.16 mic.mint.local
10.211.59.16 api.mic.mint.local
10.211.59.16 datacatalog.mint.local
10.211.59.16 endpoint.models.mint.local
10.211.59.16 graphql.mint.local
10.211.59.16 models.mint.local
10.211.59.16 api.models.mint.local
10.211.59.16 endpoint.models.mint.local
```

### MINT services removal
Expand All @@ -91,6 +121,12 @@ $ kubectl delete pvc --all -n mint
$ kubectl delete jobs --all -n mint
```

or

```bash
bash uninstall.sh
```

## Developer documentation

This repository contains the helm charts for the MINT services. The helm charts are used to deploy the services in a Kubernetes cluster. The helm charts are organized in the following folders:
Expand All @@ -106,6 +142,6 @@ To release a new version of the MINT charts, follow the following steps:

```bash
make release
````
```

3. Commit the changes and push them to the repository.
54 changes: 22 additions & 32 deletions helm/templates/NOTES.txt
Original file line number Diff line number Diff line change
@@ -1,35 +1,25 @@
The MINT system has been installed!
The MINT system has been installed!

1. Get the application URL by running these commands:
{{- if .Values.components.ui.ingress.enabled }}
{{ with .Values.components.ui }}
http{{ if .ingress.tls }}s{{ end }}://{{ with (first .ingress.hosts ) }}{{ .host }};{{ end }}
{{- end }}
{{- else if contains "NodePort" .Values.service.type }}
export UI_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "mint.prefix" . }}-ui)
export CROMO_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "mint.prefix" . }}-cromo)
export DATA_CATALOG_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "mint.prefix" . }}-data-catalog)
export HASURA_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "mint.prefix" . }}-hasura)
export MODEL_CATALOG_API_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "mint.prefix" . }}-model-catalog)
export MODEL_CATALOG_DATABASE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[1].nodePort}" services {{ include "mint.prefix" . }}-model-catalog)
export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
Please remember to edit your `/etc/hosts/`

echo "MINT User Interface: http://$NODE_IP:$UI_PORT"
echo "MINT Model Catalog API: http://$NODE_IP:$MODEL_CATALOG_API_PORT/{{ .Values.components.model_catalog_api.api_version }}/docs"
echo "MINT Data Catalog: http://$NODE_IP:$DATA_CATALOG_PORT"
echo "MINT Constraint Reasoning Over MOdels (CROMO): http://$NODE_IP:$CROMO_PORT"
echo "MINT Database: http://$NODE_IP:$HASURA_PORT"
echo "MINT Model Catalog Database: http://$NODE_IP:$MODEL_CATALOG_DATABASE_PORT"

{{- else if contains "LoadBalancer" .Values.service.type }}
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "mint.prefix" . }}-ui'
export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "mint.prefix" . }}-ui --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
UI address:
echo http://$SERVICE_IP:{{ .Values.service.port }}
{{- else if contains "ClusterIP" .Values.service.type }}
export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "helm.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
{{- end }}

After that, you can access to the MINT services

{{ if .Values.components.ui.ingress.enabled }}
{{- with .Values.components.ui }}
http{{ if .ingress.tls }}s{{ end }}://{{ with (first .ingress.hosts ) }}{{ .host }}{{ end }}
{{- end }}
{{- with .Values.components.ensemble_manager }}
http{{ if .ingress.tls }}s{{ end }}://{{ with (first .ingress.hosts ) }}{{ .host }}{{ end }}
{{- end }}
{{- with .Values.components.cromo }}
http{{ if .ingress.tls }}s{{ end }}://{{ with (first .ingress.hosts ) }}{{ .host }}{{ end }}
{{- end }}
{{- with .Values.components.mic_ui }}
http{{ if .ingress.tls }}s{{ end }}://{{ with (first .ingress.hosts ) }}{{ .host }}{{ end }}
{{- end }}
{{- with .Values.components.data_catalog }}
http{{ if .ingress.tls }}s{{ end }}://{{ with (first .ingress.hosts ) }}{{ .host }}{{ end }}
{{- end }}
{{- end }}
4 changes: 2 additions & 2 deletions helm/templates/ui-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ data:
{{ if .enabled }}
window.REACT_APP_ENSEMBLE_MANAGER_API = "http{{ if .ingress.tls }}s{{ end }}://{{ with (first .ingress.hosts ) }}{{ .host }}{{ end }}/{{ .api_version }}";
//Execution - Local
window.REACT_APP_EXECUTION_ENGINE = "localex";
window.REACT_APP_EXECUTION_ENGINE = "tapis";
window.REACT_APP_LOCALEX_CODEDIR = "{{ .environment.data_dir }}/code";
window.REACT_APP_LOCALEX_DATADIR = "{{ .environment.data_dir }}/data";
window.REACT_APP_LOCALEX_TEMPDIR = "{{ .environment.data_dir }}/temp";
Expand Down Expand Up @@ -74,4 +74,4 @@ data:
window.REACT_APP_VISUALIZATION_URL = "http://dev.viz.mint.isi.edu";
window.REACT_APP_INGESTION_API = "http://node1.ingestion.mint.isi.edu/v1.3.0";
window.REACT_APP_AIRFLOW_API = "https://airflow.mint.isi.edu/api/v1";
window.REACT_APP_AIRFLOW_DAG_DOWNLOAD_THREAD_ID = "download_thread"
window.REACT_APP_AIRFLOW_DAG_DOWNLOAD_THREAD_ID = "download_thread"
66 changes: 33 additions & 33 deletions helm/values.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
welcome_message: Welcome to MINT
default_user: mint@isi.edu
service:
type: NodePort
port: 30000
type: ClusterIP
port: 80
hostname: localhost
arm_support: false
components:
Expand All @@ -15,8 +15,8 @@ components:
tag: 3c75586989aedf2573c37f5352f960c294377931
pullPolicy: IfNotPresent
ingress:
enabled: false
className: ''
enabled: true
className: ""
annotations: {}
hosts:
- host: cromo.mint.local
Expand All @@ -34,8 +34,8 @@ components:
api_url: null
airflow_url: https://airflow.mint.isi.edu/api/v1
ingress:
enabled: false
className: ''
enabled: true
className: ""
annotations: {}
hosts:
- host: mic.mint.local
Expand All @@ -50,8 +50,8 @@ components:
tag: ee71a1a364fc3d384f85243684ad95d37913b049
pullPolicy: IfNotPresent
ingress:
enabled: false
className: ''
enabled: true
className: ""
annotations: {}
hosts:
- host: api.mic.mint.local
Expand All @@ -66,12 +66,12 @@ components:
pullPolicy: IfNotPresent
persistence:
enabled: true
storageClass: ''
storageClass: ""
accessModes:
- ReadWriteOnce
size: 10Gi
dataSource: {}
existingClaim: ''
existingClaim: ""
selector: {}
annotations:
helm.sh/resource-policy: keep
Expand All @@ -82,8 +82,8 @@ components:
tag: 0e6b32b73228ef268065694ddf0aec47d28bf58e
pullPolicy: IfNotPresent
ingress:
enabled: false
className: ''
enabled: true
className: ""
annotations: {}
hosts:
- host: datacatalog.mint.local
Expand All @@ -98,12 +98,12 @@ components:
pullPolicy: IfNotPresent
persistence:
enabled: true
storageClass: ''
storageClass: ""
accessModes:
- ReadWriteOnce
size: 10Gi
dataSource: {}
existingClaim: ''
existingClaim: ""
selector: {}
annotations:
helm.sh/resource-policy: keep
Expand All @@ -119,8 +119,8 @@ components:
prefix: https://w3id.org/okn/i/mint/
backup_file: /fuseki-base/seeds/model-catalog.trig
ingress:
enabled: false
className: ''
enabled: true
className: ""
annotations: {}
hosts:
- host: endpoint.models.mint.local
Expand All @@ -130,12 +130,12 @@ components:
tls: []
persistence:
enabled: true
storageClass: ''
storageClass: ""
accessModes:
- ReadWriteOnce
size: 10Gi
dataSource: {}
existingClaim: ''
existingClaim: ""
selector: {}
annotations:
helm.sh/resource-policy: keep
Expand All @@ -150,7 +150,7 @@ components:
enable_dev_mode: false
unauthorized_role: anonymous
ingress:
enabled: false
enabled: true
className: null
annotations: {}
hosts:
Expand All @@ -170,12 +170,12 @@ components:
pullPolicy: IfNotPresent
persistence:
enabled: true
storageClass: ''
storageClass: ""
accessModes:
- ReadWriteOnce
size: 10Gi
dataSource: {}
existingClaim: ''
existingClaim: ""
selector: {}
annotations:
helm.sh/resource-policy: keep
Expand All @@ -186,8 +186,8 @@ components:
tag: 0b2f9f0a9124076aeb492add2f123d0757066f6b
pullPolicy: IfNotPresent
ingress:
enabled: false
className: ''
enabled: true
className: ""
annotations: {}
hosts:
- host: models.mint.local
Expand All @@ -206,10 +206,10 @@ components:
context_dir: contexts/
queries_dir: queries/
ingress:
enabled: false
className: ''
enabled: true
className: ""
annotations:
nginx.ingress.kubernetes.io/enable-cors: 'true'
nginx.ingress.kubernetes.io/enable-cors: "true"
hosts:
- host: api.models.mint.local
paths:
Expand All @@ -223,8 +223,8 @@ components:
tag: 858ce054d1f757d0f6ec3765e878ec54091bb110
pullPolicy: IfNotPresent
ingress:
enabled: false
className: ''
enabled: true
className: ""
annotations: {}
hosts:
- host: mint.local
Expand All @@ -244,8 +244,8 @@ components:
data_url: http://localhost:30010/data
parallel: 1
ingress:
enabled: false
className: ''
enabled: true
className: ""
annotations: {}
hosts:
- host: ensemble-manager.mint.local
Expand Down Expand Up @@ -290,12 +290,12 @@ auth:
realm: production
ui_client_id: mint-ui
imagePullSecrets: []
nameOverride: ''
fullnameOverride: ''
nameOverride: ""
fullnameOverride: ""
serviceAccount:
create: true
annotations: {}
name: ''
name: ""
podAnnotations: {}
podSecurityContext: {}
securityContext: {}
Expand Down
Loading