Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/kubernetes-charts-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ defaults:
jobs:
helm-lint-test:
runs-on: ubuntu-latest
if: github.event.pull_request.user.login != 'dependabot[bot]' && github.repository == 'CogStack/cogstack-platform-toolkit'
if: github.event.pull_request.user.login != 'dependabot[bot]' && github.repository == 'CogStack/cogstack-platform'
steps:
- name: Checkout
uses: actions/checkout@v5.0.0
Expand Down Expand Up @@ -130,7 +130,7 @@ jobs:
run: |
helm package ./medcat-service-helm --version ${{ steps.version.outputs.chart_version }}
helm package ./medcat-trainer-helm --version ${{ steps.version.outputs.chart_version }} --dependency-update
helm package ./cogstack-helm-ce --version ${{ steps.version.outputs.chart_version }} --dependency-update
helm package ./cogstack-ce-helm --version ${{ steps.version.outputs.chart_version }} --dependency-update

- name: Helm OCI login to Docker Hub
run: helm registry login registry-1.docker.io -u ${{ secrets.DOCKERHUB_USERNAME }} -p ${{ secrets.DOCKERHUB_TOKEN }}
Expand All @@ -139,7 +139,7 @@ jobs:
run: |
helm push ./medcat-service-helm-${{ steps.version.outputs.chart_version }}.tgz oci://registry-1.docker.io/cogstacksystems
helm push ./medcat-trainer-helm-${{ steps.version.outputs.chart_version }}.tgz oci://registry-1.docker.io/cogstacksystems
helm push ./cogstack-helm-ce-${{ steps.version.outputs.chart_version }}.tgz oci://registry-1.docker.io/cogstacksystems
helm push ./cogstack-ce-helm-${{ steps.version.outputs.chart_version }}.tgz oci://registry-1.docker.io/cogstacksystems

- name: Release
uses: softprops/action-gh-release@v2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/observability-docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ on:
jobs:
docker:
runs-on: ubuntu-latest
if: github.event.pull_request.user.login != 'dependabot[bot]' && github.repository == 'CogStack/cogstack-platform-toolkit'
if: github.event.pull_request.user.login != 'dependabot[bot]' && github.repository == 'CogStack/cogstack-platform'
strategy:
matrix: # This should be replaced with Docker Bake instead
include:
Expand Down
5 changes: 2 additions & 3 deletions docs/docs/cogstack-ce/tutorial/quickstart-installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The install should take around 15 minutes, and by the end of this tutorial you w
Run:

```sh
helm install cogstack oci://registry-1.docker.io/cogstacksystems/cogstack-helm-ce --timeout 15m
helm install cogstack oci://registry-1.docker.io/cogstacksystems/cogstack-ce-helm --timeout 15m
```

This command will install Cogstack community edition with all the default values.
Expand Down Expand Up @@ -55,8 +55,7 @@ If you use a custom namespace or Helm release name, add the namespace or replace

2. Open JupyterHub by opening http://127.0.0.1:8000 in a web browser

Jupyter should start up, and present a login screen.

Jupyter should start up, and present a login screen.

## Next step: run the bundled notebook

Expand Down
6 changes: 3 additions & 3 deletions docs/docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,10 @@ Deploy the CogStack Community Edition on an existing Kubernetes cluster using he

```sh
$ helm install \
cogstack oci://registry-1.docker.io/cogstacksystems/cogstack-helm-ce \
cogstack oci://registry-1.docker.io/cogstacksystems/cogstack-ce-helm \
--timeout=15m0s
---> 100%
Pulled: registry-1.docker.io/cogstacksystems/cogstack-helm-ce:0.0.1
Pulled: registry-1.docker.io/cogstacksystems/cogstack-ce-helm:0.0.1
Digest: sha256:02e8ad3df7173270f7fdeb3e1ed5133427cec06ffc15b4ce763fa9bb062c8df1

NAME: cogstack
Expand Down Expand Up @@ -66,7 +66,7 @@ The AI and Data Engineering layer comprises of:
- MLOps tooling for model training and validation

!!! tip
Many of these apps and tools are open source and available on GitHub (subject to the licensing in each project), in the [CogStack GitHub](https://github.com/CogStack).
Many of these apps and tools are open source and available on GitHub (subject to the licensing in each project), in the [CogStack GitHub](https://github.com/CogStack).

The public documentation on this page covers these open source community offerings.

Expand Down
5 changes: 2 additions & 3 deletions docs/docs/platform/deployment/helm/charts/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,11 @@ The Helm charts for CogStack are published to Docker Hub, which is an OCI-compli
https://hub.docker.com/r/cogstacksystems/medcat-trainer-helm

- **CogStack CE (Community Edition, umbrella chart):**
https://hub.docker.com/r/cogstacksystems/cogstack-helm-ce
https://hub.docker.com/r/cogstacksystems/cogstack-ce-helm

- [MedCAT Service Helm](medcat-service-helm.md)
- [MedCAT Trainer Helm](medcat-trainer-helm.md)

- [CogStack CE Helm](cogstack-ce-helm.md)
- [CogStack Community Edition Helm](cogstack-ce-helm.md)

## Chart Publishing

Expand Down
4 changes: 2 additions & 2 deletions docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ nav:
- MedCAT: https://docs.cogstack.org/projects/nlp
- MedCAT Trainer: https://docs.cogstack.org/projects/medcat-trainer/
- Tutorials:
- Medcat Service Tutorial: platform/cogstack-ai/medcat-service-tutorial.ipynb
- Medcat Service Tutorial: platform/cogstack-ai/medcat-service-tutorial.ipynb
- Data Engineering:
- Data Engineering: data-engineering/index.md
- Data pipelines: overview/Data pipelines.md
Expand All @@ -69,7 +69,7 @@ nav:
- Charts Overview: platform/deployment/helm/charts/_index.md
- MedCAT Service Helm: platform/deployment/helm/charts/medcat-service-helm.md
- MedCAT Trainer Helm: platform/deployment/helm/charts/medcat-trainer-helm.md
- Cogstack CE Helm: platform/deployment/helm/charts/cogstack-helm-ce.md
- Cogstack CE Helm: platform/deployment/helm/charts/cogstack-ce-helm.md
- Examples:
- Examples: platform/deployment/examples/_index.md
- AWS Kubernetes EKS: platform/deployment/examples/aws-kubernetes-eks.md
Expand Down
4 changes: 2 additions & 2 deletions docs/scripts/copy_files_from_repo.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@
# Add more entries here to copy additional static files into documentation.
COPY_SPECS = [
{
"sourceFilePath": "helm-charts/cogstack-helm-ce/charts/jupyterhub/examples/medcat-service-tutorial.ipynb",
"sourceFilePath": "helm-charts/cogstack-ce-helm/charts/jupyterhub/examples/medcat-service-tutorial.ipynb",
"outputFilePath": "platform/cogstack-ai/medcat-service-tutorial.ipynb",
},
{
"sourceFilePath": "helm-charts/cogstack-helm-ce/charts/jupyterhub/examples/medcat-opensearch-e2e.ipynb",
"sourceFilePath": "helm-charts/cogstack-ce-helm/charts/jupyterhub/examples/medcat-opensearch-e2e.ipynb",
"outputFilePath": "cogstack-ce/tutorial/medcat-opensearch-e2e.ipynb",
},
]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
apiVersion: v2
name: cogstack-helm-ce
name: cogstack-ce-helm
description: CogStack Community Edition Helm Chart

# A chart can be either an 'application' or a 'library' chart.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# CogStack Helm Community Edition
# CogStack Community Edition Helm Chart

This is a all in one helm chart that runs CogStack on Kubernetes

Expand All @@ -11,6 +11,9 @@ This chart is an umbrella chart that deploys:
| **MedCAT** | Medical Concept Annotation Tool — NER and linking for clinical text. |
| **AnonCAT** | De-identification service (MedCAT in DEID mode) for anonymising text. |
| **MedCAT Trainer** | Training and model management for MedCAT, with Solr and PostgreSQL. |
| **OpenSearch** | Search and analytics engine powering logs and document indexing for CogStack. |
| **OpenSearch Dashboards** | Visualization and analytics UI for OpenSearch data. |
| **JupyterHub** | Multi-user notebook server for interactive data science workflows. |

## Prerequisites

Expand All @@ -20,7 +23,7 @@ This chart is an umbrella chart that deploys:
## Installation

```sh
helm install cogstack oci://registry-1.docker.io/cogstacksystems/cogstack-helm-ce
helm install cogstack oci://registry-1.docker.io/cogstacksystems/cogstack-ce-helm
```

## Configuration
Expand Down Expand Up @@ -115,7 +118,7 @@ kubectl delete namespace cogstack
| medcat-trainer.env.CSRF_TRUSTED_ORIGINS | string | `"http://localhost:8080"` | CSRF trusted origins for the MedCAT Trainer frontend (set for your deployment/port-forward). |
| medcat-trainer.image.tag | string | `"latest@sha256:103215a7540ad614c32866f4cb00ddd91e7aff37cea9abc25dc226c577f9506d"` | MedCAT Trainer image tag (can be a digest-pinned tag). |
| medcat-trainer.provisioning.enabled | bool | `true` | Enable provisioning of projects and models on startup. |
| medcat-trainer.provisioning.existingConfigMap.name | string | `"cogstack-helm-ce-example-trainer-provisioining"` | Existing ConfigMap name containing the provisioning configuration. |
| medcat-trainer.provisioning.existingConfigMap.name | string | `"cogstack-ce-helm-example-trainer-provisioining"` | Existing ConfigMap name containing the provisioning configuration. |
| nameOverride | string | `""` | This is to override the chart name. |
| opensearch-dashboards.enabled | bool | `true` | Deploy an opensearch-dashboards instance |
| opensearch.enabled | bool | `true` | Deploy an opensearch cluster |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# CogStack Helm Community Edition
# CogStack Community Edition Helm Chart

This is a all in one helm chart that runs CogStack on Kubernetes

Expand All @@ -11,6 +11,10 @@ This chart is an umbrella chart that deploys:
| **MedCAT** | Medical Concept Annotation Tool — NER and linking for clinical text. |
| **AnonCAT** | De-identification service (MedCAT in DEID mode) for anonymising text. |
| **MedCAT Trainer** | Training and model management for MedCAT, with Solr and PostgreSQL. |
| **OpenSearch** | Search and analytics engine powering logs and document indexing for CogStack. |
| **OpenSearch Dashboards** | Visualization and analytics UI for OpenSearch data. |
| **JupyterHub** | Multi-user notebook server for interactive data science workflows. |


## Prerequisites

Expand All @@ -20,7 +24,7 @@ This chart is an umbrella chart that deploys:
## Installation

```sh
helm install cogstack oci://registry-1.docker.io/cogstacksystems/cogstack-helm-ce
helm install cogstack oci://registry-1.docker.io/cogstacksystems/cogstack-ce-helm
```

## Configuration
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ A Helm chart for CogStack Jupyter Hub using official JupyterHub chart

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| jupyterhub.cull.enabled | bool | false, so pods stay running indefinitely d | Enable culling of user pods |
| jupyterhub.cull.enabled | bool | false, so pods stay running indefinitely | Enable culling of user pods |
| jupyterhub.hub.config.Authenticator.admin_users[0] | string | `"admin"` | |
| jupyterhub.hub.config.DummyAuthenticator.password | string | `"SuperSecret"` | |
| jupyterhub.hub.config.JupyterHub.authenticator_class | string | `"dummy"` | |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "cogstack-helm-ce.name" -}}
{{- define "cogstack-ce-helm.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}

Expand All @@ -10,7 +10,7 @@ Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "cogstack-helm-ce.fullname" -}}
{{- define "cogstack-ce-helm.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
Expand All @@ -26,16 +26,16 @@ If release name contains chart name it will be used as a full name.
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "cogstack-helm-ce.chart" -}}
{{- define "cogstack-ce-helm.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Common labels
*/}}
{{- define "cogstack-helm-ce.labels" -}}
helm.sh/chart: {{ include "cogstack-helm-ce.chart" . }}
{{ include "cogstack-helm-ce.selectorLabels" . }}
{{- define "cogstack-ce-helm.labels" -}}
helm.sh/chart: {{ include "cogstack-ce-helm.chart" . }}
{{ include "cogstack-ce-helm.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
Expand All @@ -45,17 +45,17 @@ app.kubernetes.io/managed-by: {{ .Release.Service }}
{{/*
Selector labels
*/}}
{{- define "cogstack-helm-ce.selectorLabels" -}}
app.kubernetes.io/name: {{ include "cogstack-helm-ce.name" . }}
{{- define "cogstack-ce-helm.selectorLabels" -}}
app.kubernetes.io/name: {{ include "cogstack-ce-helm.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

{{/*
Create the name of the service account to use
*/}}
{{- define "cogstack-helm-ce.serviceAccountName" -}}
{{- define "cogstack-ce-helm.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "cogstack-helm-ce.fullname" .) .Values.serviceAccount.name }}
{{- default (include "cogstack-ce-helm.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if and (index .Values "medcat-trainer" "provisioning" "enabled") (eq (index .Values "medcat-trainer" "provisioning" "existingConfigMap" "name") "cogstack-helm-ce-example-trainer-provisioining") }}
{{- if and (index .Values "medcat-trainer" "provisioning" "enabled") (eq (index .Values "medcat-trainer" "provisioning" "existingConfigMap" "name") "cogstack-ce-helm-example-trainer-provisioining") }}
apiVersion: v1
kind: ConfigMap
metadata:
Expand All @@ -15,7 +15,7 @@ data:
description: "Example project for medcat trainer using MedCAT Service"
annotationGuidelineLink: "https://docs.google.com/document/d/1xxelBOYbyVzJ7vLlztP2q1Kw9F5Vr1pRwblgrXPS7QM/edit?usp=sharing"
useModelService: true
modelServiceUrl: "http://{{ include "cogstack-helm-ce.fullname" . }}-medcat-service:5000"
modelServiceUrl: "http://{{ include "cogstack-ce-helm.fullname" . }}-medcat-service:5000"
- dataset:
name: "Example Dataset - psych.csv - deid"
url: "https://raw.githubusercontent.com/CogStack/cogstack-nlp/be9825f151da2d3b6faac80d49e5be91a1629c68/medcat-trainer/notebook_docs/example_data/psych.csv"
Expand All @@ -25,5 +25,5 @@ data:
description: "Example project for medcat trainer using AnonCAT Service"
annotationGuidelineLink: "https://docs.google.com/document/d/1xxelBOYbyVzJ7vLlztP2q1Kw9F5Vr1pRwblgrXPS7QM/edit?usp=sharing"
useModelService: true
modelServiceUrl: "http://{{ include "cogstack-helm-ce.fullname" . }}-anoncat-service:5000"
modelServiceUrl: "http://{{ include "cogstack-ce-helm.fullname" . }}-anoncat-service:5000"
{{- end }}
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@ apiVersion: v1
kind: Pod
metadata:
labels:
{{- include "cogstack-helm-ce.labels" . | nindent 4 }}
{{- include "cogstack-ce-helm.labels" . | nindent 4 }}
annotations:
"helm.sh/hook": post-install,post-upgrade
"helm.sh/hook-weight": "1"
"helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation
#"helm.sh/hook-delete-policy": before-hook-creation
name: {{ include "cogstack-helm-ce.fullname" . }}-opensearch-provisioning
name: {{ include "cogstack-ce-helm.fullname" . }}-opensearch-provisioning
spec:
restartPolicy: OnFailure
containers:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Default values for cogstack-helm-ce.
# Default values for cogstack-ce-helm.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

Expand Down Expand Up @@ -49,7 +49,7 @@ medcat-trainer:
enabled: true
existingConfigMap:
# -- Existing ConfigMap name containing the provisioning configuration.
name: "cogstack-helm-ce-example-trainer-provisioining"
name: "cogstack-ce-helm-example-trainer-provisioining"
env:
# -- CSRF trusted origins for the MedCAT Trainer frontend (set for your deployment/port-forward).
CSRF_TRUSTED_ORIGINS: "http://localhost:8080"
Expand Down
6 changes: 3 additions & 3 deletions helm-charts/local_dev_startup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ helm test medcat-service --logs

## helm install trainer-registry oci://registry-1.docker.io/cogstacksystems/medcat-trainer-helm --wait --timeout 5m0s

helm upgrade cogstack-helm-ce ./cogstack-helm-ce --install
helm upgrade cogstack-ce-helm ./cogstack-ce-helm --install

# Stop any running port forwards
# ps -ef | grep '[p]ort-forward' | awk '{print $2}' | xargs -r kill
# Port forward all services in cogstack-helm-ce
# helm get notes cogstack-helm-ce | bash
# Port forward all services in cogstack-ce-helm
# helm get notes cogstack-ce-helm | bash
Loading