Skip to content

Commit

Permalink
Models hub internal (#104)
Browse files Browse the repository at this point in the history 
* 2023-03-29-icd10cm_resolver_pipeline_en (#44)

* Add model 2023-03-29-icd10cm_resolver_pipeline_en

* Add model 2023-03-29-icd10cm_resolver_pipeline_en

---------

Co-authored-by: Ahmetemintek <ahmetemin.tek.66@gmail.com>

* 2023-03-29-oncology_general_pipeline_en (#46)

* Add model 2023-03-29-oncology_general_pipeline_en

* Add model 2023-03-29-oncology_diagnosis_pipeline_en

* Add model 2023-03-29-oncology_biomarker_pipeline_en

* Add model 2023-03-29-icd10_icd9_mapping_en

* Add model 2023-03-29-icd10cm_snomed_mapping_en

* Add model 2023-03-29-icd10cm_umls_mapping_en

* Add model 2023-03-29-icd9_resolver_pipeline_en

* Add model 2023-03-29-icdo_snomed_mapping_en

* Add model 2023-03-29-mesh_umls_mapping_en

* Add model 2023-03-29-rxnorm_ndc_mapping_en

* Add model 2023-03-29-rxnorm_umls_mapping_en

* Add model 2023-03-29-snomed_icd10cm_mapping_en

* Add model 2023-03-29-snomed_icd10cm_mapping_en

* Add model 2023-03-29-snomed_icdo_mapping_en

* Add model 2023-03-29-oncology_therapy_pipeline_en

* Add model 2023-03-29-snomed_umls_mapping_en

---------

Co-authored-by: Cabir40 <cabir4006@gmail.com>

* 2023-03-29-icd10cm_resolver_pipeline_en (#45)

* Add model 2023-03-29-icd10cm_resolver_pipeline_en

* Add model 2023-03-29-umls_drug_resolver_pipeline_en

---------

Co-authored-by: Ahmetemintek <ahmetemin.tek.66@gmail.com>

* Add model 2023-03-30-text_generator_biomedical_biogpt_en (#48)

Co-authored-by: Cabir40 <cabir4006@gmail.com>

* 2023-03-30-summarizer_generic_jsl_en (#49)

* Add model 2023-03-30-summarizer_generic_jsl_en

* Update 2023-03-30-summarizer_generic_jsl_en.md

---------

Co-authored-by: Cabir40 <cabir4006@gmail.com>
Co-authored-by: Cabir C <64752006+Cabir40@users.noreply.github.com>

* Add model 2023-03-30-summarizer_clinical_jsl_augmented_en (#51)

* Update 2023-03-30-summarizer_generic_jsl_en.md

* Update 2023-03-30-text_generator_biomedical_biogpt_en.md

* Update 2023-03-30-summarizer_clinical_jsl_augmented_en.md

* 2023-04-02-ner_jsl_limited_80p_for_benchmarks_en (#58)

* Add model 2023-04-02-ner_jsl_limited_80p_for_benchmarks_en

* Update 2023-04-02-ner_jsl_limited_80p_for_benchmarks_en.md

* Update 2023-04-02-ner_jsl_limited_80p_for_benchmarks_en.md

---------

Co-authored-by: Damla-Gurbaz <dml.grbz.01@gmail.com>
Co-authored-by: Cabir C <64752006+Cabir40@users.noreply.github.com>
Co-authored-by: Veysel Kocaman <vkocaman@gmail.com>

* 2023-03-30-ner_vop_wip_en (#52)

* Add model 2023-03-30-ner_vop_wip_en

* Update 2023-03-30-ner_vop_wip_en.md

---------

Co-authored-by: mauro-nievoff <mauro.nievasoffidani@gmail.com>
Co-authored-by: Cabir C <64752006+Cabir40@users.noreply.github.com>

* 2023-03-30-umls_clinical_findings_resolver_pipeline_en (#47)

* Add model 2023-03-30-umls_clinical_findings_resolver_pipeline_en

* Add model 2023-03-30-cvx_resolver_pipeline_en

* Add model 2023-03-30-umls_disease_syndrome_resolver_pipeline_en

* Add model 2023-03-30-umls_major_concepts_resolver_pipeline_en

* Add model 2023-03-30-umls_drug_substance_resolver_pipeline_en

* Add model 2023-03-31-medication_resolver_pipeline_en

* Add model 2023-03-31-medication_resolver_transform_pipeline_en

* Add model 2023-03-31-spellcheck_clinical_pipeline_en

---------

Co-authored-by: Ahmetemintek <ahmetemin.tek.66@gmail.com>

* Add model 2023-04-03-ner_oncology_limited_80p_for_benchmarks_en (#61)

* 2023-04-03-text_generator_biomedical_biogpt_base_en (#60)

* Delete 2023-03-30-text_generator_biomedical_biogpt_en.md

* Update 2023-04-03-text_generator_generic_flan_base_en.md

* Update 2023-03-30-summarizer_clinical_jsl_augmented_en.md

* Update 2023-03-30-summarizer_generic_jsl_en.md

* Add model 2023-04-04-text_generator_generic_flan_t5_large_en (#64)

* 2023-04-07-embeddings_clinical_medium_en (#75)

* Add model 2023-04-07-embeddings_clinical_medium_en

* Update 2023-04-07-embeddings_clinical_medium_en.md

---------

Co-authored-by: Cabir40 <cabir4006@gmail.com>
Co-authored-by: Cabir C <64752006+Cabir40@users.noreply.github.com>

* Add model 2023-04-07-embeddings_clinical_large_en (#79)

* 2023-04-10-genericclassifier_sdoh_mental_health_embeddings_clinical_en (#81)

* Add model 2023-04-10-genericclassifier_sdoh_mental_health_embeddings_clinical_en

* Add model 2023-04-10-genericclassifier_sdoh_under_treatment_sbiobert_cased_mli_en

* Add model 2023-04-10-genericclassifier_sdoh_housing_insecurity_sbiobert_cased_mli_en

* added missing md files

* Add model 2023-04-10-genericclassifier_sdoh_mental_health_clinical_en

* Delete 2023-04-10-genericclassifier_sdoh_mental_health_embeddings_clinical_en.md

* Update 2023-04-10-genericclassifier_sdoh_under_treatment_sbiobert_cased_mli_en.md

* Update 2023-04-10-genericclassifier_sdoh_mental_health_clinical_en.md

* Update 2023-04-10-genericclassifier_sdoh_housing_insecurity_sbiobert_cased_mli_en.md

* Update 2023-04-10-genericclassifier_sdoh_under_treatment_sbiobert_cased_mli_en.md

* Update 2023-04-10-genericclassifier_sdoh_mental_health_clinical_en.md

---------

Co-authored-by: hsaglamlar <hsaglamlar@gmail.com>
Co-authored-by: Cabir ÇELİK <ogr.cabir.celik@ahievran.edu.tr>
Co-authored-by: Cabir C <64752006+Cabir40@users.noreply.github.com>

* 2023-04-10-medication_resolver_pipeline_en (#83)

* Add model 2023-04-10-medication_resolver_pipeline_en

* Add model 2023-04-11-medication_resolver_transform_pipeline_en

---------

Co-authored-by: SKocer <samedkocer22@gmail.com>

* 2023-04-11-umls_clinical_findings_resolver_pipeline_en (#89)

* Add model 2023-04-11-umls_clinical_findings_resolver_pipeline_en

* Add model 2023-04-11-umls_drug_substance_resolver_pipeline_en

---------

Co-authored-by: Cabir40 <cabir4006@gmail.com>

* 2023-04-12-ner_vop_emb_clinical_large_wip_en (#100)

* Add model 2023-04-12-ner_vop_emb_clinical_large_wip_en

* Add model 2023-04-12-ner_vop_emb_clinical_medium_wip_en

---------

Co-authored-by: mauro-nievoff <mauro.nievasoffidani@gmail.com>

* Add model 2023-04-12-biogpt_chat_jsl_en (#103)

Co-authored-by: Cabir40 <cabir4006@gmail.com>

* 2023-04-12-ner_oncology_emb_clinical_large_en (#101)

* Add model 2023-04-12-ner_oncology_emb_clinical_large_en

* Add model 2023-04-12-ner_oncology_emb_clinical_medium_en

---------

Co-authored-by: Meryem1425 <vildansarikaya25@gmail.com>

* 2023-04-12-ner_posology_emb_clinical_large_en (#99)

* Add model 2023-04-12-ner_posology_emb_clinical_large_en

* Update 2023-04-12-ner_posology_emb_clinical_large_en.md

Fixed the given sample text

* Add model 2023-04-12-ner_posology_emb_clinical_medium_en

* Update 2023-04-12-ner_posology_emb_clinical_medium_en.md

Corrected thr code block view

* Add model 2023-04-12-ner_deid_large_emb_clinical_large_en

* Update 2023-04-12-ner_deid_large_emb_clinical_large_en.md

Corrected the code block view

* Add model 2023-04-12-ner_deid_large_emb_clinical_medium_en

* Update 2023-04-12-ner_deid_large_emb_clinical_medium_en.md

corrected the python code block view

* Update 2023-04-12-ner_posology_emb_clinical_medium_en.md

Fixed scala code

* Update 2023-04-12-ner_posology_emb_clinical_large_en.md

Fixed scala code

---------

Co-authored-by: gokhanturer <mgturer@gmail.com>
Co-authored-by: Gökhan <81560784+gokhanturer@users.noreply.github.com>

* 2023-04-12-ner_jsl_emb_clinical_medium_en (#98)

* Add model 2023-04-12-ner_jsl_emb_clinical_medium_en

* Add model 2023-04-12-ner_jsl_emb_clinical_large_en

---------

Co-authored-by: Damla-Gurbaz <dml.grbz.01@gmail.com>

* 2023-04-12-ner_sdoh_emb_clinical_medium_wip_en (#96)

* Add model 2023-04-12-ner_sdoh_emb_clinical_medium_wip_en

* Add model 2023-04-12-ner_sdoh_emb_clinical_large_wip_en

---------

Co-authored-by: hsaglamlar <hsaglamlar@gmail.com>

---------

Co-authored-by: jsl-models <74001263+jsl-models@users.noreply.github.com>
Co-authored-by: Ahmetemintek <ahmetemin.tek.66@gmail.com>
Co-authored-by: Damla-Gurbaz <dml.grbz.01@gmail.com>
Co-authored-by: Veysel Kocaman <vkocaman@gmail.com>
Co-authored-by: mauro-nievoff <mauro.nievasoffidani@gmail.com>
Co-authored-by: Damla Gurbaz <81505007+Damla-Gurbaz@users.noreply.github.com>
Co-authored-by: hsaglamlar <hsaglamlar@gmail.com>
Co-authored-by: Cabir ÇELİK <ogr.cabir.celik@ahievran.edu.tr>
Co-authored-by: SKocer <samedkocer22@gmail.com>
Co-authored-by: Meryem1425 <vildansarikaya25@gmail.com>
Co-authored-by: gokhanturer <mgturer@gmail.com>
Co-authored-by: Gökhan <81560784+gokhanturer@users.noreply.github.com>
  • Loading branch information
@Cabir40 @jsl-models
@Ahmetemintek @Damla-Gurbaz @vkocaman @mauro-nievoff @hsaglamlar @SKocer @Meryem1425 @gokhanturer
13 people committed Apr 12, 2023
1 parent 4e68a10 commit 1fb8fe8
Show file tree
Hide file tree
Showing 18 changed files with 3,424 additions and 0 deletions.
94 changes: 94 additions & 0 deletions docs/_posts/Cabir40/2023-04-11-umls_clinical_findings_resolver_pipeline_en.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
---
layout: model
title: Clinical Findings to UMLS Code Pipeline
author: John Snow Labs
name: umls_clinical_findings_resolver_pipeline
date: 2023-04-11
tags: [licensed, clinical, en, umls, pipeline]
task: Chunk Mapping
language: en
edition: Healthcare NLP 4.3.2
spark_version: 3.0
supported: true
annotator: PipelineModel
article_header:
type: cover
use_language_switcher: "Python-Scala-Java"
---

## Description

This pretrained pipeline maps entities (Clinical Findings) with their corresponding UMLS CUI codes. You’ll just feed your text and it will return the corresponding UMLS codes.

{:.btn-box}
<button class="button button-orange" disabled>Live Demo</button>
<button class="button button-orange" disabled>Open in Colab</button>
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/umls_clinical_findings_resolver_pipeline_en_4.3.2_3.0_1681216655167.zip){:.button.button-orange}
[Copy S3 URI](s3://auxdata.johnsnowlabs.com/clinical/models/umls_clinical_findings_resolver_pipeline_en_4.3.2_3.0_1681216655167.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}

## How to use



<div class="tabs-box" markdown="1">
{% include programmingLanguageSelectScalaPythonNLU.html %}
```python
from sparknlp.pretrained import PretrainedPipeline

pipeline = PretrainedPipeline("umls_clinical_findings_resolver_pipeline", "en", "clinical/models")

text = 'HTG-induced pancreatitis associated with an acute hepatitis, and obesity'

result = pipeline.annotate(text)
```
```scala
import com.johnsnowlabs.nlp.pretrained.PretrainedPipeline

val pipeline = new PretrainedPipeline("umls_clinical_findings_resolver_pipeline", "en", "clinical/models")

val text = "HTG-induced pancreatitis associated with an acute hepatitis, and obesity"

val result = pipeline.annotate(text)
```
</div>

## Results

```bash
+------------------------+---------+---------+
|chunk |ner_label|umls_code|
+------------------------+---------+---------+
|HTG-induced pancreatitis|PROBLEM |C1963198 |
|an acute hepatitis |PROBLEM |C4750596 |
|obesity |PROBLEM |C1963185 |
+------------------------+---------+---------+
```

{:.model-param}
## Model Information

{:.table-model}
|---|---|
|Model Name:|umls_clinical_findings_resolver_pipeline|
|Type:|pipeline|
|Compatibility:|Healthcare NLP 4.3.2+|
|License:|Licensed|
|Edition:|Official|
|Language:|en|
|Size:|4.3 GB|

## Included Models

- DocumentAssembler
- SentenceDetectorDLModel
- TokenizerModel
- WordEmbeddingsModel
- MedicalNerModel
- NerConverterInternalModel
- ChunkMapperModel
- ChunkMapperModel
- ChunkMapperFilterer
- Chunk2Doc
- BertSentenceEmbeddings
- SentenceEntityResolverModel
- ResolverMerger
89 changes: 89 additions & 0 deletions docs/_posts/Cabir40/2023-04-11-umls_drug_substance_resolver_pipeline_en.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
---
layout: model
title: Drug Substance to UMLS Code Pipeline
author: John Snow Labs
name: umls_drug_substance_resolver_pipeline
date: 2023-04-11
tags: [licensed, clinical, en, umls, pipeline, drug, subtance]
task: Chunk Mapping
language: en
edition: Healthcare NLP 4.3.2
spark_version: 3.0
supported: true
annotator: PipelineModel
article_header:
type: cover
use_language_switcher: "Python-Scala-Java"
---

## Description

This pretrained pipeline maps entities (Drug Substances) with their corresponding UMLS CUI codes. You’ll just feed your text and it will return the corresponding UMLS codes.

{:.btn-box}
<button class="button button-orange" disabled>Live Demo</button>
<button class="button button-orange" disabled>Open in Colab</button>
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/umls_drug_substance_resolver_pipeline_en_4.3.2_3.0_1681217098344.zip){:.button.button-orange}
[Copy S3 URI](s3://auxdata.johnsnowlabs.com/clinical/models/umls_drug_substance_resolver_pipeline_en_4.3.2_3.0_1681217098344.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}

## How to use



<div class="tabs-box" markdown="1">
{% include programmingLanguageSelectScalaPythonNLU.html %}
```python
from sparknlp.pretrained import PretrainedPipeline

pipeline = PretrainedPipeline("umls_drug_substance_resolver_pipeline", "en", "clinical/models")

result = pipeline.annotate("The patient was given metformin, lenvatinib and Magnesium hydroxide 100mg/1ml")
```
```scala
import com.johnsnowlabs.nlp.pretrained.PretrainedPipeline

val pipeline = PretrainedPipeline("umls_drug_substance_resolver_pipeline", "en", "clinical/models")

val result = pipeline.annotate("The patient was given metformin, lenvatinib and Magnesium hydroxide 100mg/1ml")
```

{:.nlu-block}
```python
+-----------------------------+---------+---------+
|chunk |ner_label|umls_code|
+-----------------------------+---------+---------+
|metformin |DRUG |C0025598 |
|lenvatinib |DRUG |C2986924 |
|Magnesium hydroxide 100mg/1ml|DRUG |C1134402 |
+-----------------------------+---------+---------+
```
</div>

{:.model-param}
## Model Information

{:.table-model}
|---|---|
|Model Name:|umls_drug_substance_resolver_pipeline|
|Type:|pipeline|
|Compatibility:|Healthcare NLP 4.3.2+|
|License:|Licensed|
|Edition:|Official|
|Language:|en|
|Size:|5.1 GB|

## Included Models

- DocumentAssembler
- SentenceDetector
- TokenizerModel
- WordEmbeddingsModel
- MedicalNerModel
- NerConverter
- ChunkMapperModel
- ChunkMapperModel
- ChunkMapperFilterer
- Chunk2Doc
- BertSentenceEmbeddings
- SentenceEntityResolverModel
- ResolverMerger
93 changes: 93 additions & 0 deletions docs/_posts/Cabir40/2023-04-12-biogpt_chat_jsl_en.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
layout: model
title: Clinical QA BioGPT (JSL)
author: John Snow Labs
name: biogpt_chat_jsl
date: 2023-04-12
tags: [licensed, en, clinical, text_generation, biogpt, tensorflow]
task: Text Generation
language: en
edition: Healthcare NLP 4.3.2
spark_version: 3.0
supported: true
engine: tensorflow
annotator: MedicalTextGenerator
article_header:
type: cover
use_language_switcher: "Python-Scala-Java"
---

## Description

This model is based on BioGPT finetuned with medical conversations happening in a clinical settings and can answer clinical questions related to symptoms, drugs, tests, and diseases.

{:.btn-box}
<button class="button button-orange" disabled>Live Demo</button>
<button class="button button-orange" disabled>Open in Colab</button>
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/biogpt_chat_jsl_en_4.3.2_3.0_1681319163583.zip){:.button.button-orange}
[Copy S3 URI](s3://auxdata.johnsnowlabs.com/clinical/models/biogpt_chat_jsl_en_4.3.2_3.0_1681319163583.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}

## How to use



<div class="tabs-box" markdown="1">
{% include programmingLanguageSelectScalaPythonNLU.html %}
```python

document_assembler = DocumentAssembler() \
.setInputCol("text") \
.setOutputCol("documents")

gpt_qa = MedicalTextGenerator.pretrained("biogpt_chat_jsl", "en", "clinical/models")\
.setInputCols("documents")\
.setOutputCol("answer")

pipeline = Pipeline().setStages([document_assembler, gpt_qa])

data = spark.createDataFrame([["How to treat asthma ?"]]).toDF("text")

pipeline.fit(data).transform(data)

```
```scala

val document_assembler = new DocumentAssembler()
.setInputCol("text")
.setOutputCol("document")

val summarizer = MedicalTextGenerator.pretrained("biogpt_chat_jsl", "en", "clinical/models")
.setInputCols("documents")
.setOutputCol("answer")

val pipeline = new Pipeline().setStages(Array(document_assembler, summarizer))

val text = "How to treat asthma ?"

val data = Seq(Array(text)).toDS.toDF("text")

val result = pipeline.fit(data).transform(data)

```
</div>

## Results

```bash

['Asthma is itself an allergic disease due to cold or dust or pollen or grass etc. irrespective of the triggering factor. You can go for pulmonary function tests if not done. Treatment is mainly symptomatic which might require inhalation steroids, beta agonists, anticholinergics as MDI or rota haler as a regular treatment. To decrease the inflammation of bronchi and bronchioles, you might be given oral antihistamines with mast cell stabilizers (montelukast) and steroids (prednisolone) with nebulization and frequently steam inhalation. To decrease the bronchoconstriction caused by allergens, you might be given oral antihistamines with mast cell stabilizers (montelukast) and steroids (prednisolone) with nebulization and frequently steam inhalation. The best way to cure any allergy is a complete avoidance of allergen or triggering factor. Consult your pulmonologist for further advise.']

```

{:.model-param}
## Model Information

{:.table-model}
|---|---|
|Model Name:|biogpt_chat_jsl|
|Compatibility:|Healthcare NLP 4.3.2+|
|License:|Licensed|
|Edition:|Official|
|Language:|en|
|Size:|1.4 GB|
|Case sensitive:|true|
Loading

0 comments on commit 1fb8fe8

Please sign in to comment.