Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[![Build Status](https://github.com/CogStack/cogstack-nlp/actions/workflows/medcat-v2_main.yml/badge.svg?branch=main)](https://github.com/CogStack/cogstack-nlp/actions/workflows/medcat-v2_main.yml/badge.svg?branch=main)
[![Documentation Status](https://readthedocs.org/projects/cogstack-nlp/badge/?version=latest)](https://readthedocs.org/projects/cogstack-nlp/badge/?version=latest)
[![Latest release](https://img.shields.io/github/v/release/CogStack/MedCAT2)](https://github.com/CogStack/MedCAT2/releases/latest)
[![Latest release](https://img.shields.io/github/v/release/CogStack/cogstack-nlp?filter=medcat/*)](https://github.com/CogStack/cogstack-nlp/releases/latest)
<!-- [![pypi Version](https://img.shields.io/pypi/v/medcat.svg?style=flat-square&logo=pypi&logoColor=white)](https://pypi.org/project/medcat/) -->

Cogstack Natural Language Processing is for analysing clinical data using AI to draw insights from text in or documents in an Electronic Health Records.
Expand Down
4 changes: 2 additions & 2 deletions anoncat-demo-app/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Deidentify app

Demo for AnonCAT. It uses [MedCAT](https://github.com/CogStack/MedCAT), an advanced natural language processing tool, to identify and classify sensitive information, such as names, addresses, and medical terms.
Demo for AnonCAT. It uses [MedCAT](https://github.com/CogStack/cogstack-nlp/tree/main/medcat-v1), an advanced natural language processing tool, to identify and classify sensitive information, such as names, addresses, and medical terms.

## Example

Expand All @@ -22,7 +22,7 @@ MODEL_NAME = '<NAME OF MODEL HERE.zip>'

### Build your own model

To build your own models please follow the tutorials outlined in [MedCATtutorials](https://github.com/CogStack/MedCATtutorials)
To build your own models please follow the tutorials outlined in [MedCATtutorials](https://github.com/CogStack/cogstack-nlp/tree/main/medcat-v1-tutorials)

*__Note:__ This is currently under development*

Expand Down
2 changes: 1 addition & 1 deletion anoncat-demo-app/app/frontend/src/App.vue
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<br>
<p>Please DO NOT test with any real sensitive PHI data.</p>
<br>
<p>Local validation and fine-tuning available via <a href="https://github.com/CogStack/MedCATtrainer">MedCATtrainer</a>.
<p>Local validation and fine-tuning available via <a href="https://github.com/CogStack/cogstack-nlp/tree/main/medcat-trainer">MedCATtrainer</a>.
Email us, <a href="mailto:contact@cogstack.org">contact@cogstack.org</a>, to discuss model access, model performance, and your use case.
</p>
<br>
Expand Down
3 changes: 1 addition & 2 deletions medcat-trainer/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
# Medical <img src="https://github.com/CogStack/cogstack-nlp/blob/main/media/cat-logo.png?raw=true" width=45>oncept Annotation Tool Trainer

[![Build Status](https://github.com/CogStack/cogstack-nlp/actions/workflows/medcat-trainer_ci.yml/badge.svg?branch=main)](https://github.com/CogStack/cogstack-nlp/actions/workflows/medcat-trainer_ci.yml?query=branch%3Amain)
[![Build Status](https://github.com/CogStack/cogstack-nlp/actions/workflows/medcat-trainer_qa.yml/badge.svg?branch=main)](https://github.com/CogStack/cogstack-nlp/actions/workflows/medcat-trainer_qa.yml?query=branch%3Amain)
[![Build Status](https://github.com/CogStack/cogstack-nlp/actions/workflows/medcat-trainer_release.yml/badge.svg)](https://github.com/CogStack/cogstack-nlp/actions/workflows/medcat-trainer_release.yml)
[![Documentation Status](https://readthedocs.org/projects/cogstack-nlp-medcat-trainer/badge/?version=latest)](https://readthedocs.org/projects/cogstack-nlp-medcat-trainer/badge/?version=latest)
[![Latest release](https://img.shields.io/github/v/release/CogStack/cogstack-nlp)](https://github.com/CogStack/cogstack-nlp/releases/latest)
[![Latest release](https://img.shields.io/github/v/release/CogStack/cogstack-nlp?filter=medcat-trainer/*)](https://github.com/CogStack/cogstack-nlp/releases/latest)

MedCATTrainer is an interface for building, improving and customising a given Named Entity Recognition
and Linking (NER+L) model (MedCAT) for biomedical domain text.
Expand Down
6 changes: 3 additions & 3 deletions medcat-trainer/docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ MedCATtrainer is a docker-compose packaged Django application.
## Download from Dockerhub
Clone the repo, run the default docker-compose file and default env var:
```shell
$ git clone https://github.com/CogStack/MedCATtrainer
$ cd MedCATtrainer
$ git clone https://github.com/CogStack/cogstack-nlp
$ cd cogstack-nlp/medcat-trainer
$ docker-compose up
```

Expand Down Expand Up @@ -61,7 +61,7 @@ Environment variables are used to configure the app:

|Parameter|Description|
|---------|-----------|
|MEDCAT_CONFIG_FILE|MedCAT config file as described [here](https://github.com/CogStack/MedCAT/blob/master/medcat/config.py)|
|MEDCAT_CONFIG_FILE|MedCAT config file as described [here](https://github.com/CogStack/cogstack-nlp/blob/main/medcat-v2/medcat/config/config.py)|
|BEHIND_RP| If you're running MedCATtrainer, use 1, otherwise this defaults to 0 i.e. False|
|MCTRAINER_PORT|The port to run the trainer app on|
|EMAIL_USER|Email address which will be used to send users emails regarding password resets|
Expand Down
2 changes: 1 addition & 1 deletion medcat-trainer/docs/maintenance.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ be upgrading to the latest release.

The latest stable releases update the `docker-compose.yml` and `docker-compose-prod.yml` files.

To update these docker compose files, either copy them directly from the [repo](https://github.com/CogStack/MedCATtrainer)
To update these docker compose files, either copy them directly from the [repo](https://github.com/CogStack/cogstack-nlp/tree/main/medcat-trainer)
or update the cloned files via:

```shell
Expand Down
2 changes: 1 addition & 1 deletion medcat-trainer/docs/project_admin.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ Admin users have extra options on the home screen:
3. Save Model - Write the in memory model to disk - to save the current in memory model state. This option is generally not advised as full model training should be done outside the trainer instance ideally.

### Notes
- Example Concept and Vocab databses are freely available on MedCAT [github](https://github.com/CogStack/MedCAT).
- Example Concept and Vocab databses are freely available on MedCAT [github](https://github.com/CogStack/cogstack-nlp/tree/main/medcat-v2).
Note. UMLS and SNOMED-CT are licensed products so only these smaller trained concept / vocab databases are made available currently.
- More documentation on the creation of UMLS / SNOMED-CT CDBs from respective source data will be released soon.
- Tasks allow for the creation of meta-annotations and their associated set of values an annotator can use.
Expand Down
4 changes: 2 additions & 2 deletions medcat-trainer/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ DOCKER_COMP_FILE=dc.yml
ENV_FILE=/tmp/mc_trainer/envs/env

echo "Downloading docker-compose.yml and default env vars"
curl https://cdn.githubraw.com/CogStack/MedCATtrainer/9fbd517e/docker-compose.yml > $DOCKER_COMP_FILE
curl https://raw.githubusercontent.com/CogStack/MedCATtrainer/9fbd517e/envs/env > $ENV_FILE
curl https://raw.githubusercontent.com/CogStack/cogstack-nlp/09e32429/medcat-trainer/docker-compose.yml > $DOCKER_COMP_FILE
curl https://raw.githubusercontent.com/CogStack/cogstack-nlp/09e32429/medcat-trainer/envs/env > $ENV_FILE
trap 'rm -rf /tmp/mc_trainer && cd $CURR_LOC' EXIT

echo "Starting MedCATtrainer containers"
Expand Down
2 changes: 1 addition & 1 deletion medcat-trainer/notebook_docs/API_Examples.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@
"### Create CDBs and Vocabularies\n",
"The MedCAT models used by MedCATTrainer are output by MedCAT instances of classesmedcat.cdb.CDB, medcat.utils.vocab.Vocabulary. Calling save_dict('\\<file location\\>') will write a file that can be loaded in another instance of MedCAT (via load_dict()), or within MedCATTrainer.\n",
"\n",
"Examples models are provided on the MedCAT Repository: https://github.com/CogStack/MedCAT"
"Examples models are provided on the MedCAT Repository: https://github.com/CogStack/cogstack-nlp/tree/main/medcat-v1"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion medcat-trainer/notebook_docs/Generate_CUI_Filters.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Relation file from SNOMED-CT Preprocessing tutorial: https://github.com/CogStack/MedCATtutorials/blob/main/notebooks/specialised/Preprocessing_SNOMED_CT.ipynb, \n",
"Relation file from SNOMED-CT Preprocessing tutorial: https://github.com/CogStack/cogstack-nlp/blob/main/medcat-v1-tutorials/notebooks/specialised/Preprocessing_SNOMED_CT.ipynb, \n",
"\n",
"In section \"SNOMED Relationships\"\n"
]
Expand Down
4 changes: 2 additions & 2 deletions medcat-trainer/notebook_docs/Processing_Annotations.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -143,11 +143,11 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load the annotations downloaded - as described: https://github.com/CogStack/MedCATtrainer/blob/main/README.md#download-annos\n",
"# Load the annotations downloaded - as described: https://docs.cogstack.org/projects/medcat-trainer/en/latest/project_admin.html#downloading-annotations\n",
"projs = json.load(open('example_data/MedCAT_Export_With_Text_2020-05-22_10_34_09.json'))['projects']"
]
},
Expand Down
3 changes: 2 additions & 1 deletion medcat-trainer/webapp/scripts/load_examples.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,8 @@ def main(port=8000,
with open(vocab_tmp_file, 'wb') as f:
f.write(vocab_file.content)
print("Downloading example dataset")
ds = requests.get('https://raw.githubusercontent.com/CogStack/MedCATtrainer/main/notebook_docs/example_data/ortho.csv')
ds = requests.get(
'https://raw.githubusercontent.com/CogStack/cogstack-nlp/09e32429/medcat-trainer/notebook_docs/example_data/ortho.csvv')
with open(dataset_tmp_file, 'w') as f:
f.write(ds.text)

Expand Down
2 changes: 1 addition & 1 deletion medcat-v1/docs/main.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ A demo application is available at [MedCAT](https://medcat.sites.er.kcl.ac.uk).
A guide on how to use MedCAT is available at [MedCAT Tutorials](https://github.com/CogStack/MedCATtutorials). Read more about MedCAT on [Towards Data Science](https://towardsdatascience.com/medcat-introduction-analyzing-electronic-health-records-e1c420afa13a).

## Related Projects
- [MedCATtrainer](https://github.com/CogStack/MedCATtrainer/) - an interface for building, improving and customising a given Named Entity Recognition and Linking (NER+L) model (MedCAT) for biomedical domain text.
- [MedCATtrainer](https://github.com/CogStack/cogstack-nlp/tree/main/medcat-trainer/) - an interface for building, improving and customising a given Named Entity Recognition and Linking (NER+L) model (MedCAT) for biomedical domain text.
- [MedCATservice](https://github.com/CogStack/MedCATservice) - implements the MedCAT NLP application as a service behind a REST API.
- [iCAT](https://github.com/CogStack/iCAT) - A docker container for CogStack/MedCAT/HuggingFace development in isolated environments.

Expand Down
14 changes: 7 additions & 7 deletions medcat-v2-tutorials/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,25 +6,25 @@

| Part | Title |
| ---- |-----------------------------------------------------------------------------|
| 1. | [Migrate a v1 model to v2](https://github.com/CogStack/MedCATv2tutorials/blob/main/notebooks/introductory/migration/1._Migrate_v1_model_to_v2.ipynb) |
| 1. | [Migrate a v1 model to v2](https://github.com/CogStack/cogstack-nlp/blob/main/notebooks/introductory/migration/1._Migrate_v1_model_to_v2.ipynb) |

### Basic (regex-tokenizer) tutorials

| Part | Title |
| ---- |-----------------------------------------------------------------------------|
| 1. | [Building a Concept Database and a Vocab](https://github.com/CogStack/MedCATv2tutorials/blob/main/notebooks/introductory/basic/1._Build_a_Concept_Database_and_a_Vocabulary.ipynb) |
| 2. | [Unsupervised training on model](https://github.com/CogStack/MedCATv2tutorials/blob/main/notebooks/introductory/basic/2._Unsupervised_training_on_model.ipynb) |
| 3. | [Supervised training on model](https://github.com/CogStack/MedCATv2tutorials/blob/main/notebooks/introductory/basic/3._Supervised_training_on_model.ipynb) |
| 1. | [Building a Concept Database and a Vocab](https://github.com/CogStack/cogstack-nlp/blob/main/notebooks/introductory/basic/1._Build_a_Concept_Database_and_a_Vocabulary.ipynb) |
| 2. | [Unsupervised training on model](https://github.com/CogStack/cogstack-nlp/blob/main/notebooks/introductory/basic/2._Unsupervised_training_on_model.ipynb) |
| 3. | [Supervised training on model](https://github.com/CogStack/cogstack-nlp/blob/main/notebooks/introductory/basic/3._Supervised_training_on_model.ipynb) |

### MetaCAT (meta-annotation) tutorials

| Part | Title |
| ---- |-----------------------------------------------------------------------------|
| 1. | [Add a MetaCAT to a Model](https://github.com/CogStack/MedCATv2tutorials/blob/main/notebooks/introductory/meta/1._Add_a_MetaCat_to_a_Model.ipynb) |
| 1. | [Add a MetaCAT to a Model](https://github.com/CogStack/cogstack-nlp/blob/main/notebooks/introductory/meta/1._Add_a_MetaCat_to_a_Model.ipynb) |

## Advanced tutorials

| Part | Title |
| ---- |-----------------------------------------------------------------------------|
| 1. | [Creating and using a custom tokenizer](https://github.com/CogStack/MedCATv2tutorials/blob/main/notebooks/advanced/1._Creating_and_using_a_custom_tokenizer.ipynb) |
| 2. | [Create and use component](https://github.com/CogStack/MedCATv2tutorials/blob/main/notebooks/advanced/2._Create_and_use_component.ipynb) |
| 1. | [Creating and using a custom tokenizer](https://github.com/CogStack/cogstack-nlp/blob/main/notebooks/advanced/1._Creating_and_using_a_custom_tokenizer.ipynb) |
| 2. | [Create and use component](https://github.com/CogStack/cogstack-nlp/blob/main/notebooks/advanced/2._Create_and_use_component.ipynb) |
Loading
Loading