Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions .github/workflows/medcat-trainer-v1_ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: medcat-trainer-v1 ci-build

on:
pull_request:
paths:
- 'v1/medcat-trainer/**'
- '.github/workflows/medcat-trainer-v1**'

defaults:
run:
working-directory: ./v1/medcat-trainer

jobs:
# Test and build client library
test-client:
runs-on: ubuntu-latest
steps:
- name: Checkout main
uses: actions/checkout@v4
with:
ref: ${{ github.ref }}

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install requests pytest build

- name: Install client package in development mode
run: |
cd client
pip install -e .

- name: Run client tests
run: |
cd client
python -m pytest tests/ -v

- name: Build client package
run: |
cd client
python -m build

# Build and test webapp container
build-and-push:
Comment on lines +16 to +49

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 3 months ago

To fix the problem, add a permissions block to the workflow file to explicitly set the minimal required permissions for the GITHUB_TOKEN. Since the workflow only checks out code, installs dependencies, runs tests, and builds packages/containers, it does not require any write permissions. The minimal required permission is contents: read, which allows the workflow to read repository contents. This block should be added at the top level of the workflow file (after the name: and before on:), so it applies to all jobs in the workflow. No changes to the jobs or steps are needed.


Suggested changeset 1
.github/workflows/medcat-trainer-v1_ci.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/medcat-trainer-v1_ci.yml b/.github/workflows/medcat-trainer-v1_ci.yml
--- a/.github/workflows/medcat-trainer-v1_ci.yml
+++ b/.github/workflows/medcat-trainer-v1_ci.yml
@@ -1,3 +1,5 @@
+permissions:
+  contents: read
 name: medcat-trainer-v1 ci-build
 
 on:
EOF
@@ -1,3 +1,5 @@
permissions:
contents: read
name: medcat-trainer-v1 ci-build

on:
Copilot is powered by AI and may make mistakes. Always verify output.
runs-on: ubuntu-latest
needs: test-client
steps:
- name: Checkout main
uses: actions/checkout@v4
with:
ref: ${{ github.ref }}

- name: Build
env:
IMAGE_TAG: ${{ env.RELEASE_VERSION }}
run: |
docker build -t cogstacksystems/medcat-trainer:dev-latest webapp/.
- name: Run Django Tests
env:
IMAGE_TAG: ${{ env.RELEASE_VERSION }}
run: |
# run tests
docker run --rm cogstacksystems/medcat-trainer:dev-latest python manage.py test
Comment on lines +50 to +68

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 3 months ago

To fix the problem, add a permissions block to the workflow file .github/workflows/medcat-trainer-v1_ci.yml. The block should be placed at the top level, just after the name and before the on key, so that it applies to all jobs in the workflow. The minimal required permission for this workflow is contents: read, as none of the jobs require write access to the repository. No additional imports or definitions are needed; this is a YAML configuration change only.

Suggested changeset 1
.github/workflows/medcat-trainer-v1_ci.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/medcat-trainer-v1_ci.yml b/.github/workflows/medcat-trainer-v1_ci.yml
--- a/.github/workflows/medcat-trainer-v1_ci.yml
+++ b/.github/workflows/medcat-trainer-v1_ci.yml
@@ -1,3 +1,5 @@
+permissions:
+  contents: read
 name: medcat-trainer-v1 ci-build
 
 on:
EOF
@@ -1,3 +1,5 @@
permissions:
contents: read
name: medcat-trainer-v1 ci-build

on:
Copilot is powered by AI and may make mistakes. Always verify output.
82 changes: 82 additions & 0 deletions .github/workflows/medcat-trainer-v1_qa.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
name: medcat-trainer-v1 qa-build

on:
push:
branches: [ main ]

defaults:
run:
working-directory: ./v1/medcat-trainer

jobs:
# Test and build client library
test-client:
runs-on: ubuntu-latest
steps:
- name: Checkout main
uses: actions/checkout@v4
with:
ref: 'main'

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install requests pytest build

- name: Install client package in development mode
run: |
cd client
pip install -e .

- name: Run client tests
run: |
cd client
python -m pytest tests/ -v

- name: Build client package
run: |
cd client
python -m build

# - name: Publish dev distribution to Test PyPI
# uses: pypa/gh-action-pypi-publish@v1.4.2
# with:
# password: ${{ secrets.MEDCAT_TRAINER_TEST_PYPI_API_TOKEN }}
# repository_url: https://test.pypi.org/legacy/
# packages_dir: v1/medcat-trainer/client/dist

# Build and test webapp container
build-and-push:
Comment on lines +14 to +54

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 3 months ago

To fix the problem, add a permissions block to the workflow file to explicitly restrict the GITHUB_TOKEN permissions. The best way to do this is to add the block at the top level of the workflow, so it applies to all jobs unless overridden. In this case, the jobs only need to read repository contents (for checkout), so set contents: read. This change should be made near the top of the file, after the name: and before the on: block. No additional imports or definitions are needed.

Suggested changeset 1
.github/workflows/medcat-trainer-v1_qa.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/medcat-trainer-v1_qa.yml b/.github/workflows/medcat-trainer-v1_qa.yml
--- a/.github/workflows/medcat-trainer-v1_qa.yml
+++ b/.github/workflows/medcat-trainer-v1_qa.yml
@@ -1,3 +1,5 @@
+permissions:
+  contents: read
 name: medcat-trainer-v1 qa-build
 
 on:
EOF
@@ -1,3 +1,5 @@
permissions:
contents: read
name: medcat-trainer-v1 qa-build

on:
Copilot is powered by AI and may make mistakes. Always verify output.
runs-on: ubuntu-latest
needs: test-client
steps:
- name: Checkout main
uses: actions/checkout@v4
with:
ref: 'main'

- name: Build
run: |
docker build -t cogstacksystems/medcat-trainer:v1-latest webapp/.

- name: Run Django Tests
env:
IMAGE_TAG: latest
run: |
# run tests
docker run --rm cogstacksystems/medcat-trainer:v1-latest python manage.py test

- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}

- name: Push to DockerHub
run: |
docker push cogstacksystems/medcat-trainer:v1-latest
Comment on lines +55 to +82

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 3 months ago

To fix the problem, you should add a permissions block to the workflow file. This can be done at the root level (applies to all jobs) or at the job level (for more granular control). Since neither job appears to require write access to repository contents, issues, or pull requests, the minimal permission of contents: read is appropriate. This change should be made at the top level of the workflow file, immediately after the name: and before the on: block, to ensure all jobs inherit these permissions unless overridden. No additional imports or definitions are required.

Suggested changeset 1
.github/workflows/medcat-trainer-v1_qa.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/medcat-trainer-v1_qa.yml b/.github/workflows/medcat-trainer-v1_qa.yml
--- a/.github/workflows/medcat-trainer-v1_qa.yml
+++ b/.github/workflows/medcat-trainer-v1_qa.yml
@@ -1,3 +1,5 @@
+permissions:
+  contents: read
 name: medcat-trainer-v1 qa-build
 
 on:
EOF
@@ -1,3 +1,5 @@
permissions:
contents: read
name: medcat-trainer-v1 qa-build

on:
Copilot is powered by AI and may make mistakes. Always verify output.
95 changes: 95 additions & 0 deletions .github/workflows/medcat-trainer-v1_release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
name: medcat-trainer-v1 release-build

on:
push:
tags:
- 'medcat-trainer/v1.*.*'

defaults:
run:
working-directory: ./v1/medcat-trainer

jobs:
# Test, build and publish client library
test-and-publish-client:
runs-on: ubuntu-latest
steps:
- name: Checkout main
uses: actions/checkout@v4
with:
ref: "main"

- name: Release Tag
# If GITHUB_REF=refs/tags/medcat-trainer/v0.1.2, this returns v0.1.2. Note it's including the "v" though it probably shouldnt
run: echo "RELEASE_VERSION=${GITHUB_REF##refs/*/}" >> $GITHUB_ENV

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install requests pytest build twine

- name: Install client package in development mode
run: |
cd client
pip install -e .

- name: Run client tests
run: |
cd client
python -m pytest tests/ -v

- name: Build client package
run: |
cd client
python -m build

- name: Publish production distribution to PyPI
if: startsWith(github.ref, 'refs/tags') && ! github.event.release.prerelease
uses: pypa/gh-action-pypi-publish@v1.4.2
with:
# TODO CU-869a25n7e Use Trusted Platform Publisher based PyPI release
password: ${{ secrets.PYPI_API_TOKEN }}
packages_dir: v1/medcat-trainer/client/dist

# Build and test webapp container
build-and-push:
Comment on lines +15 to +60

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 3 months ago

To fix the problem, add a permissions block to the workflow file to explicitly set the minimum required permissions for the jobs. The best way to do this is to add the permissions key at the top level of the workflow (just after the name and before on), which will apply to all jobs unless overridden. Since the jobs do not appear to require any write access to repository contents, the minimal permission of contents: read is sufficient. No changes to the jobs themselves are necessary.

Steps:

  • Insert the following at the top of .github/workflows/medcat-trainer-v1_release.yml, after the name field and before on:
    permissions:
      contents: read
  • No additional imports, methods, or definitions are needed.
Suggested changeset 1
.github/workflows/medcat-trainer-v1_release.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/medcat-trainer-v1_release.yml b/.github/workflows/medcat-trainer-v1_release.yml
--- a/.github/workflows/medcat-trainer-v1_release.yml
+++ b/.github/workflows/medcat-trainer-v1_release.yml
@@ -1,3 +1,5 @@
+permissions:
+  contents: read
 name: medcat-trainer-v1 release-build
 
 on:
EOF
@@ -1,3 +1,5 @@
permissions:
contents: read
name: medcat-trainer-v1 release-build

on:
Copilot is powered by AI and may make mistakes. Always verify output.
runs-on: ubuntu-latest
needs: test-and-publish-client
steps:
- name: Checkout main
uses: actions/checkout@v4
with:
ref: "main"

- name: Release Tag
# If GITHUB_REF=refs/tags/medcat-trainer/v0.1.2, this returns v0.1.2. Note it's including the "v" though it probably shouldnt
run: echo "RELEASE_VERSION=${GITHUB_REF##refs/*/}" >> $GITHUB_ENV
- name: Build
env:
IMAGE_TAG: ${{ env.RELEASE_VERSION }}
run: |
docker build -t cogstacksystems/medcat-trainer:$IMAGE_TAG webapp/.
- name: Run Django Tests
env:
IMAGE_TAG: ${{ env.RELEASE_VERSION }}
run: |
# run tests
docker run --rm cogstacksystems/medcat-trainer:$IMAGE_TAG python manage.py test

- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}

- name: Push to DockerHub
env:
IMAGE_TAG: ${{ env.RELEASE_VERSION }}
run: |
docker push cogstacksystems/medcat-trainer:$IMAGE_TAG
Comment on lines +61 to +94

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 3 months ago

To fix the problem, add an explicit permissions block to the workflow file. The block should be placed at the top level (applies to all jobs) or at the job level (for each job that does not need write access). Since neither job in this workflow appears to require write access to the repository via the GITHUB_TOKEN, the minimal required permission is contents: read. This should be added near the top of the file, after the name and before or after the on block. No other code changes are needed.


Suggested changeset 1
.github/workflows/medcat-trainer-v1_release.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/medcat-trainer-v1_release.yml b/.github/workflows/medcat-trainer-v1_release.yml
--- a/.github/workflows/medcat-trainer-v1_release.yml
+++ b/.github/workflows/medcat-trainer-v1_release.yml
@@ -1,5 +1,8 @@
 name: medcat-trainer-v1 release-build
 
+permissions:
+  contents: read
+
 on:
   push:
     tags:
EOF
@@ -1,5 +1,8 @@
name: medcat-trainer-v1 release-build

permissions:
contents: read

on:
push:
tags:
Copilot is powered by AI and may make mistakes. Always verify output.

6 changes: 6 additions & 0 deletions v1/medcat-trainer/.env-example
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# MedCAT
SPACY_MODELS="en_core_web_sm en_core_web_md en_core_web_lg"

# Ports
MCTRAINER_PORT=8001
SOLR_PORT=8983
51 changes: 51 additions & 0 deletions v1/medcat-trainer/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
#Directories to be ignored fully
/books/
/articles/
/other/
/output/
/graphics/
/webapp/models/*
data/
tmp/
*_tmp/
.idea

/webapp/frontend/dist/*
/webapp/api/media/*
/webapp/api/static/*

# Configuration
.env

# Keep folders with this
!.keep

#tmp and similar files
.nfs*
*.pyc
*.out
*.swp
*.swn
tmp_*
t_*
tmp_*
*_tmp
*.swo
*.lyx.emergency
*.lyx#
*~
*.log
*hidden*
db.sqlite3
nohup.out
tmp.py

# docs outputs
docs/_build

# macOS system files
.DS_Store
*/.DS_Store

# Jupyter Notebook checkpoints
*/.ipynb_checkpoints/*
17 changes: 17 additions & 0 deletions v1/medcat-trainer/.readthedocs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# .readthedocs.yaml
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

version: 2

build:
os: ubuntu-20.04
tools:
python: "3.9"

sphinx:
configuration: medcat-trainer/docs/conf.py

python:
install:
- requirements: medcat-trainer/docs/requirements.txt
Loading
Loading