Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2023-06-13-recognize_entities_posology_en #378

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
d87a9e0
Add model 2023-06-13-recognize_entities_posology_en
Cabir40 Jun 13, 2023
fbcfaf2
Add model 2023-06-13-snomed_icd10cm_mapping_en
Cabir40 Jun 13, 2023
7295c81
Add model 2023-06-13-icd10cm_umls_mapping_en
Cabir40 Jun 13, 2023
fe0d042
Add model 2023-06-13-mesh_umls_mapping_en
Cabir40 Jun 13, 2023
8583fc9
Add model 2023-06-13-clinical_deidentification_de
Cabir40 Jun 13, 2023
0d91ba5
Add model 2023-06-13-clinical_deidentification_es
Cabir40 Jun 13, 2023
c9bb8f4
Add model 2023-06-13-snomed_umls_mapping_en
Cabir40 Jun 13, 2023
037cf28
Add model 2023-06-13-rxnorm_umls_mapping_en
Cabir40 Jun 13, 2023
315d576
Add model 2023-06-13-clinical_deidentification_glove_en
Cabir40 Jun 13, 2023
892630d
Add model 2023-06-13-ner_deid_generic_augmented_pipeline_en
Cabir40 Jun 13, 2023
dc67f98
Add model 2023-06-13-clinical_deidentification_fr
Cabir40 Jun 13, 2023
bd9df17
Add model 2023-06-13-icd10_icd9_mapping_en
Cabir40 Jun 13, 2023
daaca0d
Add model 2023-06-13-bert_token_classifier_ner_jsl_pipeline_en
Cabir40 Jun 13, 2023
ce45bb1
Add model 2023-06-13-clinical_deidentification_augmented_es
Cabir40 Jun 13, 2023
948f2d5
Add model 2023-06-13-rxnorm_mesh_mapping_en
Cabir40 Jun 13, 2023
ce665f6
Add model 2023-06-13-re_bodypart_directions_pipeline_en
Cabir40 Jun 13, 2023
3ed1396
Add model 2023-06-13-clinical_deidentification_it
Cabir40 Jun 13, 2023
fb5c6f5
Add model 2023-06-13-re_bodypart_proceduretest_pipeline_en
Cabir40 Jun 13, 2023
a725eab
Add model 2023-06-13-re_human_phenotype_gene_clinical_pipeline_en
Cabir40 Jun 13, 2023
d01f974
Add model 2023-06-13-re_temporal_events_clinical_pipeline_en
Cabir40 Jun 13, 2023
79af5ae
Add model 2023-06-13-re_temporal_events_enriched_clinical_pipeline_en
Cabir40 Jun 13, 2023
ec1d24e
Add model 2023-06-13-re_test_problem_finding_pipeline_en
Cabir40 Jun 13, 2023
d437f5b
Add model 2023-06-13-re_test_result_date_pipeline_en
Cabir40 Jun 13, 2023
060eeef
Add model 2023-06-13-clinical_deidentification_pt
Cabir40 Jun 13, 2023
8d2d8f3
Add model 2023-06-13-bert_sequence_classifier_binary_rct_biobert_pipe…
Cabir40 Jun 13, 2023
9b64565
Add model 2023-06-13-icd10cm_snomed_mapping_en
Cabir40 Jun 13, 2023
69bdcb6
Add model 2023-06-13-icdo_snomed_mapping_en
Cabir40 Jun 13, 2023
08a1250
Add model 2023-06-13-rxnorm_ndc_mapping_en
Cabir40 Jun 13, 2023
1958eda
Add model 2023-06-13-clinical_deidentification_ro
Cabir40 Jun 13, 2023
3a7bf25
Add model 2023-06-13-clinical_deidentification_slim_en
Cabir40 Jun 13, 2023
d882580
Add model 2023-06-13-ner_medication_pipeline_en
Cabir40 Jun 13, 2023
2ec8c9c
Add model 2023-06-13-snomed_icdo_mapping_en
Cabir40 Jun 13, 2023
90d1eae
Add model 2023-06-13-ner_deid_subentity_pipeline_ar
Cabir40 Jun 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
---
layout: model
title: Pipeline to Detect Clinical Entities (BertForTokenClassifier)
author: John Snow Labs
name: bert_token_classifier_ner_jsl_pipeline
date: 2023-06-13
tags: [ner_jsl, ner, berfortokenclassification, en, licensed]
task: Named Entity Recognition
language: en
edition: Healthcare NLP 4.4.4
spark_version: 3.4
supported: true
annotator: PipelineModel
article_header:
type: cover
use_language_switcher: "Python-Scala-Java"
---

## Description

This pretrained pipeline is built on the top of [bert_token_classifier_ner_jsl](https://nlp.johnsnowlabs.com/2022/03/21/bert_token_classifier_ner_jsl_en_2_4.html) model.

## Predicted Entities



{:.btn-box}
<button class="button button-orange" disabled>Live Demo</button>
<button class="button button-orange" disabled>Open in Colab</button>
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/clinical/models/bert_token_classifier_ner_jsl_pipeline_en_4.4.4_3.4_1686675054357.zip){:.button.button-orange.button-orange-trans.arr.button-icon.hidden}
[Copy S3 URI](s3://auxdata.johnsnowlabs.com/clinical/models/bert_token_classifier_ner_jsl_pipeline_en_4.4.4_3.4_1686675054357.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}

## How to use

<div class="tabs-box" markdown="1">
{% include programmingLanguageSelectScalaPythonNLU.html %}
```python
from sparknlp.pretrained import PretrainedPipeline

pipeline = PretrainedPipeline("bert_token_classifier_ner_jsl_pipeline", "en", "clinical/models")

text = '''The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby-girl also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature.'''

result = pipeline.fullAnnotate(text)
```
```scala
import com.johnsnowlabs.nlp.pretrained.PretrainedPipeline

val pipeline = new PretrainedPipeline("bert_token_classifier_ner_jsl_pipeline", "en", "clinical/models")

val text = "The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby-girl also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature."

val result = pipeline.fullAnnotate(text)
```


{:.nlu-block}
```python
import nlu
nlu.load("en.classify.bert_token_ner_jsl.pipeline").predict("""The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby-girl also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature.""")
```

</div>

<div class="tabs-box" markdown="1">
{% include programmingLanguageSelectScalaPythonNLU.html %}
```python
from sparknlp.pretrained import PretrainedPipeline

pipeline = PretrainedPipeline("bert_token_classifier_ner_jsl_pipeline", "en", "clinical/models")

text = '''The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby-girl also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature.'''

result = pipeline.fullAnnotate(text)
```
```scala
import com.johnsnowlabs.nlp.pretrained.PretrainedPipeline

val pipeline = new PretrainedPipeline("bert_token_classifier_ner_jsl_pipeline", "en", "clinical/models")

val text = "The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby-girl also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature."

val result = pipeline.fullAnnotate(text)
```

{:.nlu-block}
```python
import nlu
nlu.load("en.classify.bert_token_ner_jsl.pipeline").predict("""The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby-girl also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature.""")
```
</div>

## Results

```bash
Results


| | ner_chunk | begin | end | ner_label | confidence |
|---:|:---------------------------------|--------:|------:|:-------------|-------------:|
| 0 | 21-day-old | 17 | 26 | Age | 0.999456 |
| 1 | Caucasian male | 28 | 41 | Demographics | 0.9901 |
| 2 | congestion | 62 | 71 | Symptom | 0.997918 |
| 3 | mom | 75 | 77 | Demographics | 0.999013 |
| 4 | yellow discharge | 99 | 114 | Symptom | 0.998663 |
| 5 | nares | 135 | 139 | Body_Part | 0.998609 |
| 6 | she | 147 | 149 | Demographics | 0.999442 |
| 7 | mild problems with his breathing | 168 | 199 | Symptom | 0.930385 |
| 8 | perioral cyanosis | 237 | 253 | Symptom | 0.99819 |
| 9 | retractions | 258 | 268 | Symptom | 0.999783 |
| 10 | One day ago | 272 | 282 | Date_Time | 0.999386 |
| 11 | mom | 285 | 287 | Demographics | 0.999835 |
| 12 | tactile temperature | 304 | 322 | Symptom | 0.999352 |
| 13 | Tylenol | 345 | 351 | Drug | 0.999762 |
| 14 | Baby-girl | 354 | 362 | Age | 0.980529 |
| 15 | decreased p.o. intake | 382 | 402 | Symptom | 0.998978 |
| 16 | His | 405 | 407 | Demographics | 0.999913 |
| 17 | breast-feeding | 416 | 429 | Body_Part | 0.99954 |
| 18 | his | 493 | 495 | Demographics | 0.999661 |
| 19 | respiratory congestion | 497 | 518 | Symptom | 0.834984 |
| 20 | He | 521 | 522 | Demographics | 0.999858 |
| 21 | tired | 555 | 559 | Symptom | 0.999516 |
| 22 | fussy | 574 | 578 | Symptom | 0.997592 |
| 23 | over the past 2 days | 580 | 599 | Date_Time | 0.994786 |
| 24 | albuterol | 642 | 650 | Drug | 0.999735 |


{:.model-param}
```

{:.model-param}
## Model Information

{:.table-model}
|---|---|
|Model Name:|bert_token_classifier_ner_jsl_pipeline|
|Type:|pipeline|
|Compatibility:|Healthcare NLP 4.4.4+|
|License:|Licensed|
|Edition:|Official|
|Language:|en|
|Size:|405.0 MB|

## Included Models

- DocumentAssembler
- SentenceDetectorDLModel
- TokenizerModel
- MedicalBertForTokenClassifier
- NerConverterInternalModel
Loading