healthcare-nlp/01.0.Clinical_Named_Entity_Recognition_Model.ipynb

{"cells":[{"cell_type":"markdown","metadata":{"id":"I08sFJYCxR0Z"},"source":["![JohnSnowLabs](https://nlp.johnsnowlabs.com/assets/images/logo.png)"]},{"cell_type":"markdown","metadata":{"id":"FwJ-P56kq6FU"},"source":["[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/healthcare-nlp/01.0.Clinical_Named_Entity_Recognition_Model.ipynb)"]},{"cell_type":"markdown","metadata":{"id":"nWu_IMfY4L1N"},"source":["# Clinical Named Entity Recognition (NER)"]},{"cell_type":"markdown","metadata":{"id":"Niy3mZAjoayg"},"source":["##  NER Model Implementation in Spark NLP\n","\n","  The deep neural network architecture for NER model in Spark NLP is BiLSTM-CNN-Char framework. a slightly modified version of the architecture proposed by Jason PC Chiu and Eric Nichols ([Named Entity Recognition with Bidirectional LSTM-CNNs](https://arxiv.org/abs/1511.08308)). It is a neural network architecture that automatically detects word and character-level features using a hybrid bidirectional LSTM and CNN architecture, eliminating the need for most feature engineering steps.\n","\n","  In the original framework, the CNN extracts a fixed length feature vector from character-level features. For each word, these vectors are concatenated and fed to the BLSTM network and then to the output layers. They employed a stacked bi-directional recurrent neural network with long short-term memory units to transform word features into named entity tag scores. The extracted features of each word are fed into a forward LSTM network and a backward LSTM network. The output of each network at each time step is decoded by a linear layer and a log-softmax layer into log-probabilities for each tag category. These two vectors are then simply added together to produce the final output. In the architecture of the proposed framework in the original paper, 50-dimensional pretrained word embeddings is used for word features, 25-dimension character embeddings is used for char features, and capitalization features (allCaps, upperInitial, lowercase, mixedCaps, noinfo) are used for case features."]},{"cell_type":"markdown","metadata":{"id":"huGcLHSP6oVb"},"source":["## Blogposts and videos:\n","\n","- [How to Setup Spark NLP for HEALTHCARE on UBUNTU - Video](https://www.youtube.com/watch?v=yKnF-_oz0GE)\n","\n","- [Named Entity Recognition (NER) with BERT in Spark NLP](https://towardsdatascience.com/named-entity-recognition-ner-with-bert-in-spark-nlp-874df20d1d77)\n","\n","- [State of the art Clinical Named Entity Recognition in Spark NLP - Youtube](https://www.youtube.com/watch?v=YM-e4eOiQ34)\n","\n","- [Named Entity Recognition for Healthcare with SparkNLP NerDL and NerCRF](https://medium.com/spark-nlp/named-entity-recognition-for-healthcare-with-sparknlp-nerdl-and-nercrf-a7751b6ad571)\n","\n","- [Named Entity Recognition for Clinical Text](https://medium.com/atlas-research/ner-for-clinical-text-7c73caddd180)"]},{"cell_type":"markdown","metadata":{"id":"okhT7AcXxben"},"source":["## Colab Setup"]},{"cell_type":"code","source":["# Install the johnsnowlabs library to access Spark-OCR and Spark-NLP for Healthcare, Finance, and Legal.\n","! pip install -q johnsnowlabs"],"metadata":{"id":"QnEWoyREND8D"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["from google.colab import files\n","print('Please Upload your John Snow Labs License using the button below')\n","license_keys = files.upload()"],"metadata":{"id":"uJ9NsgmSNHTy"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["from johnsnowlabs import nlp, medical\n","\n","# After uploading your license run this to install all licensed Python Wheels and pre-download Jars the Spark Session JVM\n","nlp.settings.enforce_versions=True\n","nlp.install(refresh_install=True)"],"metadata":{"id":"FfxtAw1kNKox"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["from johnsnowlabs import nlp, medical\n","import pandas as pd\n","\n","# Automatically load license data and start a session with all jars user has access to\n","spark = nlp.start()"],"metadata":{"id":"kEq2r009NZXW","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1712934899359,"user_tz":-180,"elapsed":7853,"user":{"displayName":"Monster C","userId":"08787989274818793476"}},"outputId":"04db2979-49b2-48fe-9afb-b386359967fa"},"execution_count":4,"outputs":[{"output_type":"stream","name":"stdout","text":["👌 Detected license file /content/5.3.1.spark_nlp_for_healthcare.json\n","👌 Launched \u001b[92mcpu optimized\u001b[39m session with with: 🚀Spark-NLP==5.3.1, 💊Spark-Healthcare==5.3.1, running on ⚡ PySpark==3.4.0\n"]}]},{"cell_type":"code","execution_count":5,"metadata":{"id":"ELqzaf32MT6E","colab":{"base_uri":"https://localhost:8080/","height":237},"executionInfo":{"status":"ok","timestamp":1712934899992,"user_tz":-180,"elapsed":638,"user":{"displayName":"Monster C","userId":"08787989274818793476"}},"outputId":"51ca3f36-e009-477b-c055-4d58a0fe6ce1"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["<pyspark.sql.session.SparkSession at 0x79d292aed9f0>"],"text/html":["\n","            <div>\n","                <p><b>SparkSession - in-memory</b></p>\n","                \n","        <div>\n","            <p><b>SparkContext</b></p>\n","\n","            <p><a href=\"http://bba1c37f35a2:4040\">Spark UI</a></p>\n","\n","            <dl>\n","              <dt>Version</dt>\n","                <dd><code>v3.4.0</code></dd>\n","              <dt>Master</dt>\n","                <dd><code>local[*]</code></dd>\n","              <dt>AppName</dt>\n","                <dd><code>John-Snow-Labs-Spark-Session 🚀 with Jars for: 🚀Spark-NLP==5.3.1, 💊Spark-Healthcare==5.3.1, running on ⚡ PySpark==3.4.0</code></dd>\n","            </dl>\n","        </div>\n","        \n","            </div>\n","        "]},"metadata":{},"execution_count":5}],"source":["spark"]},{"cell_type":"code","execution_count":6,"metadata":{"id":"GiigSdavGaeg","executionInfo":{"status":"ok","timestamp":1712934899992,"user_tz":-180,"elapsed":4,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["from pyspark.sql import DataFrame\n","import pyspark.sql.functions as F\n","import pyspark.sql.types as T\n","import pyspark.sql as SQL\n","from pyspark import keyword_only"]},{"cell_type":"markdown","metadata":{"id":"zutgrNB3uuZT"},"source":["\n","\n","\n","\n","## Clinical NER Models\n","|index|model|index|model|index|model|index|model|\n","|-----:|:-----|-----:|:-----|-----:|:-----|-----:|:-----|\n","| 1| [jsl_ner_wip_clinical](https://nlp.johnsnowlabs.com/2021/03/31/jsl_ner_wip_clinical_en.html)  | 2| [jsl_ner_wip_greedy_clinical](https://nlp.johnsnowlabs.com/2021/03/31/jsl_ner_wip_greedy_clinical_en.html)  | 3| [jsl_ner_wip_modifier_clinical](https://nlp.johnsnowlabs.com/2021/04/01/jsl_ner_wip_modifier_clinical_en.html)  | 4| [jsl_rd_ner_wip_greedy_clinical](https://nlp.johnsnowlabs.com/2021/04/01/jsl_rd_ner_wip_greedy_clinical_en.html)  |\n","| 5| [ner_abbreviation_clinical](https://nlp.johnsnowlabs.com/2021/12/30/ner_abbreviation_clinical_en.html)  | 6| [ner_abbreviation_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/05/12/ner_abbreviation_emb_clinical_large_en.html)  | 7| [ner_abbreviation_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/05/12/ner_abbreviation_emb_clinical_medium_en.html)  | 8| [ner_ade_clinical](https://nlp.johnsnowlabs.com/2021/04/01/ner_ade_clinical_en.html)  |\n","| 9| [ner_ade_clinical_langtest](https://nlp.johnsnowlabs.com/2023/07/31/ner_ade_clinical_langtest_en.html)  | 10| [ner_ade_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/05/21/ner_ade_emb_clinical_large_en.html)  | 11| [ner_ade_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/05/21/ner_ade_emb_clinical_medium_en.html)  | 12| [ner_ade_healthcare](https://nlp.johnsnowlabs.com/2021/04/01/ner_ade_healthcare_en.html)  |\n","| 13| [ner_anatomy](https://nlp.johnsnowlabs.com/2021/03/31/ner_anatomy_en.html)  | 14| [ner_anatomy_coarse](https://nlp.johnsnowlabs.com/2021/03/31/ner_anatomy_coarse_en.html)  | 15| [ner_anatomy_coarse_en](https://nlp.johnsnowlabs.com/2020/11/04/ner_anatomy_coarse_en.html)  | 16| [ner_anatomy_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/05/15/ner_anatomy_emb_clinical_large_en.html)  |\n","| 17| [ner_anatomy_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/05/15/ner_anatomy_emb_clinical_medium_en.html)  | 18| [ner_anatomy_en](https://nlp.johnsnowlabs.com/2020/04/22/ner_anatomy_en.html)  | 19| [ner_aspect_based_sentiment](https://nlp.johnsnowlabs.com/2021/03/31/ner_aspect_based_sentiment_en.html)  | 20| [ner_bacterial_species](https://nlp.johnsnowlabs.com/2021/04/01/ner_bacterial_species_en.html)  |\n","| 21| [ner_bacterial_species_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/05/23/ner_bacterial_species_emb_clinical_large_en.html)  | 22| [ner_bacterial_species_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/05/23/ner_bacterial_species_emb_clinical_medium_en.html)  | 23| [ner_bacterial_species_langtest](https://nlp.johnsnowlabs.com/2023/10/15/ner_bacterial_species_langtest_en.html)  | 24| [ner_biomarker](https://nlp.johnsnowlabs.com/2021/11/26/ner_biomarker_en.html)  |\n","| 25| [ner_biomarker_langtest](https://nlp.johnsnowlabs.com/2023/10/10/ner_biomarker_langtest_en.html)  | 26| [ner_biomedical_bc2gm](https://nlp.johnsnowlabs.com/2022/05/11/ner_biomedical_bc2gm_en_2_4.html)  | 27| [ner_bionlp](https://nlp.johnsnowlabs.com/2021/03/31/ner_bionlp_en.html)  | 28| [ner_bionlp_en](https://nlp.johnsnowlabs.com/2020/01/30/ner_bionlp_en.html)  |\n","| 29| [ner_bionlp_langtest](https://nlp.johnsnowlabs.com/2023/10/10/ner_bionlp_langtest_en.html)  | 30| [ner_cancer_genetics](https://nlp.johnsnowlabs.com/2021/03/31/ner_cancer_genetics_en.html)  | 31| [ner_cellular](https://nlp.johnsnowlabs.com/2021/03/31/ner_cellular_en.html)  | 32| [ner_cellular_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/05/24/ner_cellular_emb_clinical_large_en.html)  |\n","| 33| [ner_cellular_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/05/24/ner_cellular_emb_clinical_medium_en.html)  | 34| [ner_cellular_en](https://nlp.johnsnowlabs.com/2020/04/22/ner_cellular_en.html)  | 35| [ner_cellular_langtest](https://nlp.johnsnowlabs.com/2023/10/15/ner_cellular_langtest_en.html)  | 36| [ner_chemd_clinical](https://nlp.johnsnowlabs.com/2021/11/04/ner_chemd_clinical_en.html)  |\n","| 37| [ner_chemicals](https://nlp.johnsnowlabs.com/2021/04/01/ner_chemicals_en.html)  | 38| [ner_chemicals_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/06/02/ner_chemicals_emb_clinical_large_en.html)  | 39| [ner_chemicals_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/06/02/ner_chemicals_emb_clinical_medium_en.html)  | 40| [ner_chemprot_clinical](https://nlp.johnsnowlabs.com/2020/09/21/ner_chemprot_clinical_en.html)  |\n","| 41| [ner_chemprot_clinical_langtest](https://nlp.johnsnowlabs.com/2023/10/12/ner_chemprot_clinical_langtest_en.html)  | 42| [ner_chexpert](https://nlp.johnsnowlabs.com/2021/09/30/ner_chexpert_en.html)  | 43| [ner_clinical](https://nlp.johnsnowlabs.com/2021/03/31/ner_clinical_en.html)  | 44| [ner_clinical_abbreviation_langtest](https://nlp.johnsnowlabs.com/2024/01/25/ner_clinical_abbreviation_langtest_en.html)  |\n","| 45| [ner_clinical_en](https://nlp.johnsnowlabs.com/2020/01/30/ner_clinical_en.html)  | 46| [ner_clinical_langtest](https://nlp.johnsnowlabs.com/2023/09/26/ner_clinical_langtest_en.html)  | 47| [ner_clinical_large](https://nlp.johnsnowlabs.com/2021/03/31/ner_clinical_large_en.html)  | 48| [ner_clinical_large_en](https://nlp.johnsnowlabs.com/2020/05/23/ner_clinical_large_en.html)  |\n","| 49| [ner_clinical_large_langtest](https://nlp.johnsnowlabs.com/2023/10/10/ner_clinical_large_langtest_en.html)  | 50| [ner_clinical_trials_abstracts](https://nlp.johnsnowlabs.com/2022/06/22/ner_clinical_trials_abstracts_en_3_0.html)  | 51| [ner_covid_trials](https://nlp.johnsnowlabs.com/2022/10/19/ner_covid_trials_en.html)  | 52| [ner_crf](https://nlp.johnsnowlabs.com/2020/01/28/ner_crf_en.html)  |\n","| 53| [ner_deid_enriched_langtest](https://nlp.johnsnowlabs.com/2023/10/15/ner_deid_enriched_langtest_en.html)  | 54| [ner_deid_generic_augmented_allUpperCased_langtest](https://nlp.johnsnowlabs.com/2023/10/16/ner_deid_generic_augmented_allUpperCased_langtest_en.html)  | 55| [ner_deid_generic_augmented_langtest](https://nlp.johnsnowlabs.com/2023/09/26/ner_deid_generic_augmented_langtest_en.html)  | 56| [ner_deid_large_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/04/12/ner_deid_large_emb_clinical_large_en.html)  |\n","| 57| [ner_deid_large_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/04/12/ner_deid_large_emb_clinical_medium_en.html)  | 58| [ner_deid_large_langtest](https://nlp.johnsnowlabs.com/2023/10/15/ner_deid_large_langtest_en.html)  | 59| [ner_deid_subentity_augmented](https://nlp.johnsnowlabs.com/2021/09/03/ner_deid_subentity_augmented_en.html)  | 60| [ner_deid_subentity_augmented_langtest](https://nlp.johnsnowlabs.com/2023/09/26/ner_deid_subentity_augmented_langtest_en.html)  |\n","| 61| [ner_demographic_extended_healthcare](https://nlp.johnsnowlabs.com/2023/06/08/ner_demographic_extended_healthcare_en.html)  | 62| [ner_diseases](https://nlp.johnsnowlabs.com/2021/03/31/ner_diseases_en.html)  | 63| [ner_diseases_en](https://nlp.johnsnowlabs.com/2020/03/25/ner_diseases_en.html)  | 64| [ner_diseases_langtest](https://nlp.johnsnowlabs.com/2023/10/15/ner_diseases_langtest_en.html)  |\n","| 65| [ner_diseases_large](https://nlp.johnsnowlabs.com/2021/04/01/ner_diseases_large_en.html)  | 66| [ner_drugprot_clinical](https://nlp.johnsnowlabs.com/2021/12/20/ner_drugprot_clinical_en.html)  | 67| [ner_drugs](https://nlp.johnsnowlabs.com/2021/03/31/ner_drugs_en.html)  | 68| [ner_drugs_en](https://nlp.johnsnowlabs.com/2020/03/25/ner_drugs_en.html)  |\n","| 69| [ner_drugs_greedy](https://nlp.johnsnowlabs.com/2021/03/31/ner_drugs_greedy_en.html)  | 70| [ner_drugs_large](https://nlp.johnsnowlabs.com/2021/03/31/ner_drugs_large_en.html)  | 71| [ner_drugs_large_en](https://nlp.johnsnowlabs.com/2021/01/29/ner_drugs_large_en.html)  | 72| [ner_eu_clinical_case](https://nlp.johnsnowlabs.com/2023/01/25/ner_eu_clinical_case_en.html)  |\n","| 73| [ner_eu_clinical_condition](https://nlp.johnsnowlabs.com/2023/02/06/ner_eu_clinical_condition_en.html)  | 74| [ner_eu_clinical_condition_langtest](https://nlp.johnsnowlabs.com/2023/09/22/ner_eu_clinical_condition_langtest_en.html)  | 75| [ner_events_admission_clinical](https://nlp.johnsnowlabs.com/2021/03/31/ner_events_admission_clinical_en.html)  | 76| [ner_events_clinical](https://nlp.johnsnowlabs.com/2021/03/31/ner_events_clinical_en.html)  |\n","| 77| [ner_events_clinical_langtest](https://nlp.johnsnowlabs.com/2023/08/31/ner_events_clinical_langtest_en.html)  | 78| [ner_events_healthcare](https://nlp.johnsnowlabs.com/2021/04/01/ner_events_healthcare_en.html)  | 79| [ner_financial_contract](https://nlp.johnsnowlabs.com/2021/04/01/ner_financial_contract_en.html)  | 80| [ner_genetic_variants](https://nlp.johnsnowlabs.com/2021/06/25/ner_genetic_variants_en.html)  |\n","| 81| [ner_healthcare](https://nlp.johnsnowlabs.com/2021/04/21/ner_healthcare_en.html)  | 82| [ner_healthcare_en](https://nlp.johnsnowlabs.com/2020/03/26/ner_healthcare_en.html)  | 83| [ner_human_phenotype_gene_clinical](https://nlp.johnsnowlabs.com/2020/09/21/ner_human_phenotype_gene_clinical_en.html)  | 84| [ner_human_phenotype_gene_clinical_langtest](https://nlp.johnsnowlabs.com/2023/11/04/ner_human_phenotype_gene_clinical_langtest_en.html)  |\n","| 85| [ner_human_phenotype_go_clinical](https://nlp.johnsnowlabs.com/2020/09/21/ner_human_phenotype_go_clinical_en.html)  | 86| [ner_human_phenotype_go_clinical_langtest](https://nlp.johnsnowlabs.com/2023/11/04/ner_human_phenotype_go_clinical_langtest_en.html)  | 87| [ner_jsl](https://nlp.johnsnowlabs.com/2022/10/19/ner_jsl_en.html)  | 88| [ner_jsl_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/04/12/ner_jsl_emb_clinical_large_en.html)  |\n","| 89| [ner_jsl_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/04/12/ner_jsl_emb_clinical_medium_en.html)  | 90| [ner_jsl_en](https://nlp.johnsnowlabs.com/2020/04/22/ner_jsl_en.html)  | 91| [ner_jsl_enriched](https://nlp.johnsnowlabs.com/2021/10/22/ner_jsl_enriched_en.html)  | 92| [ner_jsl_enriched_en](https://nlp.johnsnowlabs.com/2020/04/22/ner_jsl_enriched_en.html)  |\n","| 93| [ner_jsl_greedy](https://nlp.johnsnowlabs.com/2021/06/24/ner_jsl_greedy_en.html)  | 94| [ner_jsl_langtest](https://nlp.johnsnowlabs.com/2023/07/31/ner_jsl_langtest_en.html)  | 95| [ner_jsl_limited_80p_for_benchmarks](https://nlp.johnsnowlabs.com/2023/04/02/ner_jsl_limited_80p_for_benchmarks_en.html)  | 96| [ner_jsl_slim](https://nlp.johnsnowlabs.com/2021/08/13/ner_jsl_slim_en.html)  |\n","| 97| [ner_living_species](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_en_3_0.html)  | 98| [ner_living_species_langtest](https://nlp.johnsnowlabs.com/2023/10/10/ner_living_species_langtest_en.html)  | 99| [ner_measurements_clinical](https://nlp.johnsnowlabs.com/2021/04/01/ner_measurements_clinical_en.html)  | 100| [ner_medmentions_coarse](https://nlp.johnsnowlabs.com/2021/04/01/ner_medmentions_coarse_en.html)  |\n","| 101| [ner_nature_nero_clinical](https://nlp.johnsnowlabs.com/2022/02/08/ner_nature_nero_clinical_en.html)  | 102| [ner_nihss](https://nlp.johnsnowlabs.com/2021/11/15/ner_nihss_en.html)  | 103| [ner_oncology](https://nlp.johnsnowlabs.com/2022/10/25/ner_oncology_en.html)  | 104| [ner_oncology_anatomy_general](https://nlp.johnsnowlabs.com/2022/10/25/ner_oncology_anatomy_general_en.html)  |\n","| 105| [ner_oncology_anatomy_general_healthcare](https://nlp.johnsnowlabs.com/2023/01/11/ner_oncology_anatomy_general_healthcare_en.html)  | 106| [ner_oncology_anatomy_general_langtest](https://nlp.johnsnowlabs.com/2023/09/03/ner_oncology_anatomy_general_langtest_en.html)  | 107| [ner_oncology_anatomy_general_wip](https://nlp.johnsnowlabs.com/2022/09/30/ner_oncology_anatomy_general_wip_en.html)  | 108| [ner_oncology_anatomy_granular](https://nlp.johnsnowlabs.com/2022/10/25/ner_oncology_anatomy_granular_en.html)  |\n","| 109| [ner_oncology_anatomy_granular_langtest](https://nlp.johnsnowlabs.com/2023/09/03/ner_oncology_anatomy_granular_langtest_en.html)  | 110| [ner_oncology_anatomy_granular_wip](https://nlp.johnsnowlabs.com/2022/10/01/ner_oncology_anatomy_granular_wip_en.html)  | 111| [ner_oncology_biomarker](https://nlp.johnsnowlabs.com/2022/10/25/ner_oncology_biomarker_en.html)  | 112| [ner_oncology_biomarker_healthcare](https://nlp.johnsnowlabs.com/2023/01/11/ner_oncology_biomarker_healthcare_en.html)  |\n","| 113| [ner_oncology_biomarker_langtest](https://nlp.johnsnowlabs.com/2023/09/22/ner_oncology_biomarker_langtest_en.html)  | 114| [ner_oncology_biomarker_wip](https://nlp.johnsnowlabs.com/2022/10/01/ner_oncology_biomarker_wip_en.html)  | 115| [ner_oncology_demographics](https://nlp.johnsnowlabs.com/2022/10/25/ner_oncology_demographics_en.html)  | 116| [ner_oncology_demographics_langtest](https://nlp.johnsnowlabs.com/2023/09/03/ner_oncology_demographics_langtest_en.html)  |\n","| 117| [ner_oncology_demographics_wip](https://nlp.johnsnowlabs.com/2022/09/30/ner_oncology_demographics_wip_en.html)  | 118| [ner_oncology_diagnosis](https://nlp.johnsnowlabs.com/2022/10/25/ner_oncology_diagnosis_en.html)  | 119| [ner_oncology_diagnosis_langtest](https://nlp.johnsnowlabs.com/2023/09/22/ner_oncology_diagnosis_langtest_en.html)  | 120| [ner_oncology_diagnosis_wip](https://nlp.johnsnowlabs.com/2022/09/30/ner_oncology_diagnosis_wip_en.html)  |\n","| 121| [ner_oncology_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/04/12/ner_oncology_emb_clinical_large_en.html)  | 122| [ner_oncology_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/04/12/ner_oncology_emb_clinical_medium_en.html)  | 123| [ner_oncology_langtest](https://nlp.johnsnowlabs.com/2023/10/15/ner_oncology_langtest_en.html)  | 124| [ner_oncology_limited_80p_for_benchmarks](https://nlp.johnsnowlabs.com/2023/04/03/ner_oncology_limited_80p_for_benchmarks_en.html)  |\n","| 125| [ner_oncology_posology](https://nlp.johnsnowlabs.com/2022/10/25/ner_oncology_posology_en.html)  | 126| [ner_oncology_posology_langtest](https://nlp.johnsnowlabs.com/2023/09/04/ner_oncology_posology_langtest_en.html)  | 127| [ner_oncology_posology_wip](https://nlp.johnsnowlabs.com/2022/10/01/ner_oncology_posology_wip_en.html)  | 128| [ner_oncology_response_to_treatment](https://nlp.johnsnowlabs.com/2022/10/25/ner_oncology_response_to_treatment_en.html)  |\n","| 129| [ner_oncology_response_to_treatment_langtest](https://nlp.johnsnowlabs.com/2023/09/04/ner_oncology_response_to_treatment_langtest_en.html)  | 130| [ner_oncology_response_to_treatment_wip](https://nlp.johnsnowlabs.com/2022/10/01/ner_oncology_response_to_treatment_wip_en.html)  | 131| [ner_oncology_test](https://nlp.johnsnowlabs.com/2022/10/25/ner_oncology_test_en.html)  | 132| [ner_oncology_test_langtest](https://nlp.johnsnowlabs.com/2023/09/22/ner_oncology_test_langtest_en.html)  |\n","| 133| [ner_oncology_test_wip](https://nlp.johnsnowlabs.com/2022/09/30/ner_oncology_test_wip_en.html)  | 134| [ner_oncology_therapy](https://nlp.johnsnowlabs.com/2022/10/25/ner_oncology_therapy_en.html)  | 135| [ner_oncology_therapy_langtest](https://nlp.johnsnowlabs.com/2023/09/22/ner_oncology_therapy_langtest_en.html)  | 136| [ner_oncology_therapy_wip](https://nlp.johnsnowlabs.com/2022/09/30/ner_oncology_therapy_wip_en.html)  |\n","| 137| [ner_oncology_tnm](https://nlp.johnsnowlabs.com/2022/10/25/ner_oncology_tnm_en.html)  | 138| [ner_oncology_tnm_langtest](https://nlp.johnsnowlabs.com/2023/09/22/ner_oncology_tnm_langtest_en.html)  | 139| [ner_oncology_tnm_wip](https://nlp.johnsnowlabs.com/2022/09/30/ner_oncology_tnm_wip_en.html)  | 140| [ner_oncology_unspecific_posology](https://nlp.johnsnowlabs.com/2022/10/25/ner_oncology_unspecific_posology_en.html)  |\n","| 141| [ner_oncology_unspecific_posology_healthcare](https://nlp.johnsnowlabs.com/2023/01/11/ner_oncology_unspecific_posology_healthcare_en.html)  | 142| [ner_oncology_unspecific_posology_langtest](https://nlp.johnsnowlabs.com/2023/09/22/ner_oncology_unspecific_posology_langtest_en.html)  | 143| [ner_oncology_unspecific_posology_wip](https://nlp.johnsnowlabs.com/2022/09/30/ner_oncology_unspecific_posology_wip_en.html)  | 144| [ner_oncology_wip](https://nlp.johnsnowlabs.com/2022/09/30/ner_oncology_wip_en.html)  |\n","| 145| [ner_opioid](https://nlp.johnsnowlabs.com/2024/02/28/ner_opioid_en.html)  | 146| [ner_opioid_small_wip](https://nlp.johnsnowlabs.com/2024/01/11/ner_opioid_small_wip_en.html)  | 147| [ner_pathogen](https://nlp.johnsnowlabs.com/2022/06/28/ner_pathogen_en_3_0.html)  | 148| [ner_posology](https://nlp.johnsnowlabs.com/2021/03/31/ner_posology_en.html)  |\n","| 149| [ner_posology_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/04/12/ner_posology_emb_clinical_large_en.html)  | 150| [ner_posology_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/04/12/ner_posology_emb_clinical_medium_en.html)  | 151| [ner_posology_en](https://nlp.johnsnowlabs.com/2020/04/15/ner_posology_en.html)  | 152| [ner_posology_experimental](https://nlp.johnsnowlabs.com/2021/09/01/ner_posology_experimental_en.html)  |\n","| 153| [ner_posology_greedy](https://nlp.johnsnowlabs.com/2021/03/31/ner_posology_greedy_en.html)  | 154| [ner_posology_healthcare](https://nlp.johnsnowlabs.com/2021/04/01/ner_posology_healthcare_en.html)  | 155| [ner_posology_langtest](https://nlp.johnsnowlabs.com/2023/07/28/ner_posology_langtest_en.html)  | 156| [ner_posology_large](https://nlp.johnsnowlabs.com/2021/03/31/ner_posology_large_en.html)  |\n","| 157| [ner_posology_small](https://nlp.johnsnowlabs.com/2021/03/31/ner_posology_small_en.html)  | 158| [ner_radiology](https://nlp.johnsnowlabs.com/2021/03/31/ner_radiology_en.html)  | 159| [ner_radiology_wip_clinical](https://nlp.johnsnowlabs.com/2021/04/01/ner_radiology_wip_clinical_en.html)  | 160| [ner_risk_factors](https://nlp.johnsnowlabs.com/2021/03/31/ner_risk_factors_en.html)  |\n","| 161| [ner_risk_factors_en](https://nlp.johnsnowlabs.com/2020/04/22/ner_risk_factors_en.html)  | 162| [ner_risk_factors_langtest](https://nlp.johnsnowlabs.com/2023/11/06/ner_risk_factors_langtest_en.html)  | 163| [ner_sdoh](https://nlp.johnsnowlabs.com/2023/06/13/ner_sdoh_en.html)  | 164| [ner_sdoh_access_to_healthcare](https://nlp.johnsnowlabs.com/2023/07/02/ner_sdoh_access_to_healthcare_en.html)  |\n","| 165| [ner_sdoh_access_to_healthcare_wip](https://nlp.johnsnowlabs.com/2023/02/24/ner_sdoh_access_to_healthcare_wip_en.html)  | 166| [ner_sdoh_community_condition](https://nlp.johnsnowlabs.com/2023/07/02/ner_sdoh_community_condition_en.html)  | 167| [ner_sdoh_community_condition_wip](https://nlp.johnsnowlabs.com/2023/02/24/ner_sdoh_community_condition_wip_en.html)  | 168| [ner_sdoh_demographics](https://nlp.johnsnowlabs.com/2023/07/02/ner_sdoh_demographics_en.html)  |\n","| 169| [ner_sdoh_demographics_wip](https://nlp.johnsnowlabs.com/2023/02/10/ner_sdoh_demographics_wip_en.html)  | 170| [ner_sdoh_emb_clinical_large_wip](https://nlp.johnsnowlabs.com/2023/04/17/ner_sdoh_emb_clinical_large_wip_en.html)  | 171| [ner_sdoh_emb_clinical_medium_wip](https://nlp.johnsnowlabs.com/2023/04/27/ner_sdoh_emb_clinical_medium_wip_en.html)  | 172| [ner_sdoh_health_behaviours_problems](https://nlp.johnsnowlabs.com/2023/07/02/ner_sdoh_health_behaviours_problems_en.html)  |\n","| 173| [ner_sdoh_health_behaviours_problems_wip](https://nlp.johnsnowlabs.com/2023/02/24/ner_sdoh_health_behaviours_problems_wip_en.html)  | 174| [ner_sdoh_income_social_status](https://nlp.johnsnowlabs.com/2023/07/02/ner_sdoh_income_social_status_en.html)  | 175| [ner_sdoh_income_social_status_wip](https://nlp.johnsnowlabs.com/2023/02/10/ner_sdoh_income_social_status_wip_en.html)  | 176| [ner_sdoh_langtest](https://nlp.johnsnowlabs.com/2023/07/31/ner_sdoh_langtest_en.html)  |\n","| 177| [ner_sdoh_mentions](https://nlp.johnsnowlabs.com/2022/12/18/ner_sdoh_mentions_en.html)  | 178| [ner_sdoh_slim_wip](https://nlp.johnsnowlabs.com/2022/11/15/ner_sdoh_slim_wip_en.html)  | 179| [ner_sdoh_social_environment](https://nlp.johnsnowlabs.com/2023/07/02/ner_sdoh_social_environment_en.html)  | 180| [ner_sdoh_social_environment_wip](https://nlp.johnsnowlabs.com/2023/02/10/ner_sdoh_social_environment_wip_en.html)  |\n","| 181| [ner_sdoh_substance_usage](https://nlp.johnsnowlabs.com/2023/07/02/ner_sdoh_substance_usage_en.html)  | 182| [ner_sdoh_substance_usage_wip](https://nlp.johnsnowlabs.com/2023/02/23/ner_sdoh_substance_usage_wip_en.html)  | 183| [ner_sdoh_wip](https://nlp.johnsnowlabs.com/2023/02/11/ner_sdoh_wip_en.html)  | 184| [ner_section_header_diagnosis](https://nlp.johnsnowlabs.com/2023/07/26/ner_section_header_diagnosis_en.html)  |\n","| 185| [ner_snomed_term](https://nlp.johnsnowlabs.com/2024/02/13/ner_snomed_term_en.html)  | 186| [ner_supplement_clinical](https://nlp.johnsnowlabs.com/2022/02/01/ner_supplement_clinical_en.html)  | 187| [ner_vop](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_en.html)  | 188| [ner_vop_anatomy](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_anatomy_en.html)  |\n","| 189| [ner_vop_anatomy_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_anatomy_emb_clinical_large_en.html)  | 190| [ner_vop_anatomy_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/06/07/ner_vop_anatomy_emb_clinical_medium_en.html)  | 191| [ner_vop_anatomy_langtest](https://nlp.johnsnowlabs.com/2023/09/21/ner_vop_anatomy_langtest_en.html)  | 192| [ner_vop_anatomy_wip](https://nlp.johnsnowlabs.com/2023/05/19/ner_vop_anatomy_wip_en.html)  |\n","| 193| [ner_vop_clinical_dept](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_clinical_dept_en.html)  | 194| [ner_vop_clinical_dept_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_clinical_dept_emb_clinical_large_en.html)  | 195| [ner_vop_clinical_dept_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_clinical_dept_emb_clinical_medium_en.html)  | 196| [ner_vop_clinical_dept_langtest](https://nlp.johnsnowlabs.com/2023/09/21/ner_vop_clinical_dept_langtest_en.html)  |\n","| 197| [ner_vop_clinical_dept_wip](https://nlp.johnsnowlabs.com/2023/05/19/ner_vop_clinical_dept_wip_en.html)  | 198| [ner_vop_demographic](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_demographic_en.html)  | 199| [ner_vop_demographic_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_demographic_emb_clinical_large_en.html)  | 200| [ner_vop_demographic_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_demographic_emb_clinical_medium_en.html)  |\n","| 201| [ner_vop_demographic_langtest](https://nlp.johnsnowlabs.com/2023/09/21/ner_vop_demographic_langtest_en.html)  | 202| [ner_vop_demographic_wip](https://nlp.johnsnowlabs.com/2023/05/19/ner_vop_demographic_wip_en.html)  | 203| [ner_vop_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_emb_clinical_large_en.html)  | 204| [ner_vop_emb_clinical_large_wip](https://nlp.johnsnowlabs.com/2023/04/12/ner_vop_emb_clinical_large_wip_en.html)  |\n","| 205| [ner_vop_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_emb_clinical_medium_en.html)  | 206| [ner_vop_emb_clinical_medium_wip](https://nlp.johnsnowlabs.com/2023/04/12/ner_vop_emb_clinical_medium_wip_en.html)  | 207| [ner_vop_langtest](https://nlp.johnsnowlabs.com/2023/10/10/ner_vop_langtest_en.html)  | 208| [ner_vop_problem](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_problem_en.html)  |\n","| 209| [ner_vop_problem_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_problem_emb_clinical_large_en.html)  | 210| [ner_vop_problem_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_problem_emb_clinical_medium_en.html)  | 211| [ner_vop_problem_langtest](https://nlp.johnsnowlabs.com/2023/09/21/ner_vop_problem_langtest_en.html)  | 212| [ner_vop_problem_reduced](https://nlp.johnsnowlabs.com/2023/06/07/ner_vop_problem_reduced_en.html)  |\n","| 213| [ner_vop_problem_reduced_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/06/07/ner_vop_problem_reduced_emb_clinical_large_en.html)  | 214| [ner_vop_problem_reduced_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/06/07/ner_vop_problem_reduced_emb_clinical_medium_en.html)  | 215| [ner_vop_problem_reduced_langtest](https://nlp.johnsnowlabs.com/2023/09/22/ner_vop_problem_reduced_langtest_en.html)  | 216| [ner_vop_problem_reduced_wip](https://nlp.johnsnowlabs.com/2023/05/19/ner_vop_problem_reduced_wip_en.html)  |\n","| 217| [ner_vop_problem_wip](https://nlp.johnsnowlabs.com/2023/05/19/ner_vop_problem_wip_en.html)  | 218| [ner_vop_slim_wip](https://nlp.johnsnowlabs.com/2023/02/25/ner_vop_slim_wip_en.html)  | 219| [ner_vop_temporal](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_temporal_en.html)  | 220| [ner_vop_temporal_emb_clinical_large_final](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_temporal_emb_clinical_large_final_en.html)  |\n","| 221| [ner_vop_temporal_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_temporal_emb_clinical_medium_en.html)  | 222| [ner_vop_temporal_langtest](https://nlp.johnsnowlabs.com/2023/09/22/ner_vop_temporal_langtest_en.html)  | 223| [ner_vop_temporal_wip](https://nlp.johnsnowlabs.com/2023/05/19/ner_vop_temporal_wip_en.html)  | 224| [ner_vop_test](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_test_en.html)  |\n","| 225| [ner_vop_test_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_test_emb_clinical_large_en.html)  | 226| [ner_vop_test_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_test_emb_clinical_medium_en.html)  | 227| [ner_vop_test_langtest](https://nlp.johnsnowlabs.com/2023/09/22/ner_vop_test_langtest_en.html)  | 228| [ner_vop_test_wip](https://nlp.johnsnowlabs.com/2023/05/19/ner_vop_test_wip_en.html)  |\n","| 229| [ner_vop_treatment](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_treatment_en.html)  | 230| [ner_vop_treatment_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_treatment_emb_clinical_large_en.html)  | 231| [ner_vop_treatment_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/06/06/ner_vop_treatment_emb_clinical_medium_en.html)  | 232| [ner_vop_treatment_langtest](https://nlp.johnsnowlabs.com/2023/09/22/ner_vop_treatment_langtest_en.html)  |\n","| 233| [ner_vop_treatment_wip](https://nlp.johnsnowlabs.com/2023/05/19/ner_vop_treatment_wip_en.html)  | 234| [ner_vop_wip](https://nlp.johnsnowlabs.com/2023/05/19/ner_vop_wip_en.html)  | 235| [ner_vop_wip_emb_clinical_large](https://nlp.johnsnowlabs.com/2023/05/19/ner_vop_wip_embeddings_clinical_large_en.html)  | 236| [ner_vop_wip_emb_clinical_medium](https://nlp.johnsnowlabs.com/2023/05/19/ner_vop_wip_embeddings_clinical_medium_en.html)  |\n","| 237| [nerdl_tumour_demo](https://nlp.johnsnowlabs.com/2021/04/01/nerdl_tumour_demo_en.html)  |\n","\n","\n","\n","\n","- **Multilingual Clinical NER Models**\n","\n","|index|model|index|model|index|model|index|model|\n","|-----:|:-----|-----:|:-----|-----:|:-----|-----:|:-----|\n","| 1| [ner_clinical](https://nlp.johnsnowlabs.com/2023/09/20/ner_clinical_sv.html)  | 2| [ner_clinical_bert](https://nlp.johnsnowlabs.com/2022/11/22/ner_clinical_bert_ro.html)  | 3| [ner_clinical_trials_abstracts](https://nlp.johnsnowlabs.com/2022/08/12/ner_clinical_trials_abstracts_es_3_0.html)  | 4| [ner_deid_generic](https://nlp.johnsnowlabs.com/2023/05/30/ner_deid_generic_ar.html)  |\n","| 5| [ner_deid_generic_arabert](https://nlp.johnsnowlabs.com/2023/09/16/ner_deid_generic_arabert_ar.html)  | 6| [ner_deid_generic_augmented](https://nlp.johnsnowlabs.com/2022/02/16/ner_deid_generic_augmented_es.html)  | 7| [ner_deid_generic_bert](https://nlp.johnsnowlabs.com/2022/11/22/ner_deid_generic_bert_ro.html)  | 8| [ner_deid_generic_camelbert](https://nlp.johnsnowlabs.com/2023/09/16/ner_deid_generic_camelbert_ar.html)  |\n","| 9| [ner_deid_generic_roberta](https://nlp.johnsnowlabs.com/2022/01/17/ner_deid_generic_roberta_es.html)  | 10| [ner_deid_generic_roberta_augmented](https://nlp.johnsnowlabs.com/2022/02/16/ner_deid_generic_roberta_augmented_es.html)  | 11| [ner_deid_multilingual](https://nlp.johnsnowlabs.com/2024/02/12/ner_deid_multilingual_xx.html)  | 12| [ner_deid_name_multilingual](https://nlp.johnsnowlabs.com/2024/01/17/ner_deid_name_multilingual_xx.html)  |\n","| 13| [ner_deid_name_multilingual_clinical](https://nlp.johnsnowlabs.com/2024/02/12/ner_deid_name_multilingual_clinical_xx.html)  | 14| [ner_deid_subentity](https://nlp.johnsnowlabs.com/2023/05/31/ner_deid_subentity_ar.html)  | 15| [ner_deid_subentity_arabert](https://nlp.johnsnowlabs.com/2023/09/16/ner_deid_subentity_arabert_ar.html)  | 16| [ner_deid_subentity_augmented](https://nlp.johnsnowlabs.com/2022/02/16/ner_deid_subentity_augmented_es.html)  |\n","| 17| [ner_deid_subentity_bert](https://nlp.johnsnowlabs.com/2022/06/27/ner_deid_bert_subentity_ro_3_0.html)  | 18| [ner_deid_subentity_camelbert](https://nlp.johnsnowlabs.com/2023/09/22/ner_deid_subentity_camelbert_ar.html)  | 19| [ner_deid_subentity_roberta](https://nlp.johnsnowlabs.com/2022/01/17/ner_deid_subentity_roberta_es.html)  | 20| [ner_deid_subentity_roberta_augmented](https://nlp.johnsnowlabs.com/2022/02/16/ner_deid_subentity_roberta_augmented_es.html)  |\n","| 21| [ner_diag_proc](https://nlp.johnsnowlabs.com/2021/03/31/ner_diag_proc_es.html)  | 22| [ner_eu_clinical_case](https://nlp.johnsnowlabs.com/2023/02/02/ner_eu_clinical_case_eu.html)  | 23| [ner_eu_clinical_condition](https://nlp.johnsnowlabs.com/2023/02/06/ner_eu_clinical_condition_it.html)  | 24| [ner_living_species](https://nlp.johnsnowlabs.com/2022/06/23/ner_living_species_ca_3_0.html)  |\n","| 25| [ner_living_species_300](https://nlp.johnsnowlabs.com/2022/11/22/ner_living_species_300_es.html)  | 26| [ner_living_species_bert](https://nlp.johnsnowlabs.com/2022/06/23/ner_living_species_bert_ro_3_0.html)  | 27| [ner_living_species_roberta](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_roberta_pt_3_0.html)  | 28| [ner_negation_uncertainty](https://nlp.johnsnowlabs.com/2022/08/13/ner_negation_uncertainty_es_3_0.html)  |\n","| 29| [ner_neoplasms](https://nlp.johnsnowlabs.com/2021/03/31/ner_neoplasms_es.html)  | 30| [ner_pharmacology](https://nlp.johnsnowlabs.com/2022/08/13/ner_pharmacology_es_3_0.html)  | 31| [ner_traffic](https://nlp.johnsnowlabs.com/2021/04/01/ner_traffic_de.html)  | 32| [roberta_ner_diag_proc](https://nlp.johnsnowlabs.com/2021/11/04/roberta_ner_diag_proc_es.html)  |\n","\n","- **BioBert NER Models**\n","\n","|index|model|index|model|index|model|index|model|\n","|-----:|:-----|-----:|:-----|-----:|:-----|-----:|:-----|\n","| 1| [jsl_ner_wip_greedy_biobert](https://nlp.johnsnowlabs.com/2021/07/26/jsl_ner_wip_greedy_biobert_en.html)  | 2| [jsl_rd_ner_wip_greedy_biobert](https://nlp.johnsnowlabs.com/2021/07/26/jsl_rd_ner_wip_greedy_biobert_en.html)  | 3| [ner_ade_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_ade_biobert_en.html)  | 4| [ner_anatomy_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_anatomy_biobert_en.html)  |\n","| 5| [ner_anatomy_coarse_biobert](https://nlp.johnsnowlabs.com/2021/03/31/ner_anatomy_coarse_biobert_en.html)  | 6| [ner_anatomy_coarse_biobert_en](https://nlp.johnsnowlabs.com/2020/11/04/ner_anatomy_coarse_biobert_en.html)  | 7| [ner_bionlp_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_bionlp_biobert_en.html)  | 8| [ner_cellular_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_cellular_biobert_en.html)  |\n","| 9| [ner_chemprot_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_chemprot_biobert_en.html)  | 10| [ner_clinical_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_clinical_biobert_en.html)  | 11| [ner_diseases_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_diseases_biobert_en.html)  | 12| [ner_events_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_events_biobert_en.html)  |\n","| 13| [ner_human_phenotype_gene_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_human_phenotype_gene_biobert_en.html)  | 14| [ner_human_phenotype_go_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_human_phenotype_go_biobert_en.html)  | 15| [ner_jsl_biobert](https://nlp.johnsnowlabs.com/2021/09/05/ner_jsl_biobert_en.html)  | 16| [ner_jsl_enriched_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_jsl_enriched_biobert_en.html)  |\n","| 17| [ner_jsl_greedy_biobert](https://nlp.johnsnowlabs.com/2021/08/13/ner_jsl_greedy_biobert_en.html)  | 18| [ner_living_species_biobert](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_biobert_en_3_0.html)  | 19| [ner_posology_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_posology_biobert_en.html)  | 20| [ner_posology_large_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_posology_large_biobert_en.html)  |\n","| 21| [ner_profiling_biobert](https://nlp.johnsnowlabs.com/2023/06/17/ner_profiling_biobert_en.html)  | 22| [ner_risk_factors_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_risk_factors_biobert_en.html)  |\n","\n","\n","- **BertForTokenClassification Clinical NER models**\n","\n","|index|model|index|model|index|model|index|model|\n","|-----:|:-----|-----:|:-----|-----:|:-----|-----:|:-----|\n","| 1| [bert_token_classifier_ade_tweet_binary](https://nlp.johnsnowlabs.com/2022/07/29/bert_token_classifier_ade_tweet_binary_en_3_0.html)  | 2| [bert_token_classifier_disease_mentions_tweet](https://nlp.johnsnowlabs.com/2022/07/28/bert_token_classifier_disease_mentions_tweet_es_3_0.html)  | 3| [bert_token_classifier_drug_development_trials](https://nlp.johnsnowlabs.com/2022/03/22/bert_token_classifier_drug_development_trials_en_3_0.html)  | 4| [bert_token_classifier_negation_uncertainty](https://nlp.johnsnowlabs.com/2022/08/11/bert_token_classifier_negation_uncertainty_es_3_0.html)  |\n","| 5| [bert_token_classifier_ner_ade](https://nlp.johnsnowlabs.com/2021/09/30/bert_token_classifier_ner_ade_en.html)  | 6| [bert_token_classifier_ner_ade_binary](https://nlp.johnsnowlabs.com/2022/07/27/bert_token_classifier_ner_ade_binary_en_3_0.html)  | 7| [bert_token_classifier_ner_anatem](https://nlp.johnsnowlabs.com/2022/07/25/bert_token_classifier_ner_anatem_en_3_0.html)  | 8| [bert_token_classifier_ner_anatomy](https://nlp.johnsnowlabs.com/2021/09/30/bert_token_classifier_ner_anatomy_en.html)  |\n","| 9| [bert_token_classifier_ner_bacteria](https://nlp.johnsnowlabs.com/2021/09/30/bert_token_classifier_ner_bacteria_en.html)  | 10| [bert_token_classifier_ner_bc2gm_gene](https://nlp.johnsnowlabs.com/2022/07/25/bert_token_classifier_ner_bc2gm_gene_en_3_0.html)  | 11| [bert_token_classifier_ner_bc4chemd_chemicals](https://nlp.johnsnowlabs.com/2022/07/25/bert_token_classifier_ner_bc4chemd_chemicals_en_3_0.html)  | 12| [bert_token_classifier_ner_bc5cdr_chemicals](https://nlp.johnsnowlabs.com/2022/07/25/bert_token_classifier_ner_bc5cdr_chemicals_en_3_0.html)  |\n","| 13| [bert_token_classifier_ner_bc5cdr_disease](https://nlp.johnsnowlabs.com/2022/07/25/bert_token_classifier_ner_bc5cdr_disease_en_3_0.html)  | 14| [bert_token_classifier_ner_bionlp](https://nlp.johnsnowlabs.com/2021/11/03/bert_token_classifier_ner_bionlp_en.html)  | 15| [bert_token_classifier_ner_cellular](https://nlp.johnsnowlabs.com/2021/11/03/bert_token_classifier_ner_cellular_en.html)  | 16| [bert_token_classifier_ner_chemicals](https://nlp.johnsnowlabs.com/2021/10/19/bert_token_classifier_ner_chemicals_en.html)  |\n","| 17| [bert_token_classifier_ner_chemprot](https://nlp.johnsnowlabs.com/2021/10/19/bert_token_classifier_ner_chemprot_en.html)  | 18| [bert_token_classifier_ner_clinical](https://nlp.johnsnowlabs.com/2023/07/05/bert_token_classifier_ner_clinical_nl.html)  | 19| [bert_token_classifier_ner_clinical_trials_abstracts](https://nlp.johnsnowlabs.com/2022/06/29/bert_token_classifier_ner_clinical_trials_abstracts_en_3_0.html)  | 20| [bert_token_classifier_ner_deid](https://nlp.johnsnowlabs.com/2021/09/13/bert_token_classifier_ner_deid_en.html)  |\n","| 21| [bert_token_classifier_ner_drugs](https://nlp.johnsnowlabs.com/2021/09/20/bert_token_classifier_ner_drugs_en.html)  | 22| [bert_token_classifier_ner_jnlpba_cellular](https://nlp.johnsnowlabs.com/2022/07/25/bert_token_classifier_ner_jnlpba_cellular_en_3_0.html)  | 23| [bert_token_classifier_ner_jsl](https://nlp.johnsnowlabs.com/2021/09/16/bert_token_classifier_ner_jsl_en.html)  | 24| [bert_token_classifier_ner_jsl_slim](https://nlp.johnsnowlabs.com/2021/09/24/bert_token_classifier_ner_jsl_slim_en.html)  |\n","| 25| [bert_token_classifier_ner_linnaeus_species](https://nlp.johnsnowlabs.com/2022/07/25/bert_token_classifier_ner_linnaeus_species_en_3_0.html)  | 26| [bert_token_classifier_ner_living_species](https://nlp.johnsnowlabs.com/2022/06/27/bert_token_classifier_ner_living_species_it_3_0.html)  | 27| [bert_token_classifier_ner_ncbi_disease](https://nlp.johnsnowlabs.com/2022/07/25/bert_token_classifier_ner_ncbi_disease_en_3_0.html)  | 28| [bert_token_classifier_ner_pathogen](https://nlp.johnsnowlabs.com/2022/07/28/bert_token_classifier_ner_pathogen_en_3_0.html)  |\n","| 29| [bert_token_classifier_ner_species](https://nlp.johnsnowlabs.com/2022/07/25/bert_token_classifier_ner_species_en_3_0.html)  | 30| [bert_token_classifier_ner_supplement](https://nlp.johnsnowlabs.com/2022/02/09/bert_token_classifier_ner_supplement_en.html)  | 31| [bert_token_classifier_pharmacology](https://nlp.johnsnowlabs.com/2022/08/11/bert_token_classifier_pharmacology_es_3_0.html)  |\n","\n","\n","\n","\n","\n","**You can find all these models and more [NLP Models Hub](https://nlp.johnsnowlabs.com/models?task=Named+Entity+Recognition&edition=Spark+NLP+for+Healthcare)**"]},{"cell_type":"markdown","source":["- **Multilingual Clinical NER Models**\n","\n","|index|model_name|supported_languages||index|model_name|supported_languages|\n","|-----:|:---------|:------------------||-----:|:---------|:------------------|\n","| 1 | [ner_clinical](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html) |[ro](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [de](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [nl](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [pt](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [pl](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [tr](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [es](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [fr](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [it](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [da](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [sv](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [no](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [ja](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [vi](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [ar](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [bg](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [fi](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html), [he](https://nlp.johnsnowlabs.com/2022/07/01/ner_clinical_ro_3_0.html) || 2 | [ner_neoplasms](https://nlp.johnsnowlabs.com/2020/07/08/ner_neoplasms_es.html) |[es](https://nlp.johnsnowlabs.com/2020/07/08/ner_neoplasms_es.html) |\n","| 3 | [ner_healthcare](https://nlp.johnsnowlabs.com/2020/09/28/ner_healthcare_de.html) |[de](https://nlp.johnsnowlabs.com/2020/09/28/ner_healthcare_de.html), [de](https://nlp.johnsnowlabs.com/2020/09/28/ner_healthcare_de.html) || 4 | [ner_healthcare_slim](https://nlp.johnsnowlabs.com/2021/04/01/ner_healthcare_slim_de.html) |[de](https://nlp.johnsnowlabs.com/2021/04/01/ner_healthcare_slim_de.html) |\n","| 5 | [ner_traffic](https://nlp.johnsnowlabs.com/2021/04/01/ner_traffic_de.html) |[de](https://nlp.johnsnowlabs.com/2021/04/01/ner_traffic_de.html) || 6 | [cantemist_scielowiki](https://nlp.johnsnowlabs.com/2021/07/23/cantemist_scielowiki_es.html) |[es](https://nlp.johnsnowlabs.com/2021/07/23/cantemist_scielowiki_es.html) |\n","| 7 | [meddroprof_scielowiki](https://nlp.johnsnowlabs.com/2021/07/26/meddroprof_scielowiki_es.html) |[es](https://nlp.johnsnowlabs.com/2021/07/26/meddroprof_scielowiki_es.html) || 8 | [roberta_ner_diag_proc](https://nlp.johnsnowlabs.com/2021/11/04/roberta_ner_diag_proc_es.html) |[es](https://nlp.johnsnowlabs.com/2021/11/04/roberta_ner_diag_proc_es.html) |\n","| 9 | [ner_living_species_bert](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_bert_es_3_0.html) |[es](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_bert_es_3_0.html), [pt](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_bert_es_3_0.html), [fr](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_bert_es_3_0.html), [it](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_bert_es_3_0.html), [ro](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_bert_es_3_0.html) || 10 | [ner_living_species](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_es_3_0.html) |[es](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_es_3_0.html), [pt](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_es_3_0.html), [ca](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_es_3_0.html), [fr](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_es_3_0.html), [gl](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_es_3_0.html), [it](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_es_3_0.html) |\n","| 11 | [ner_living_species_roberta](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_roberta_es_3_0.html) |[es](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_roberta_es_3_0.html), [pt](https://nlp.johnsnowlabs.com/2022/06/22/ner_living_species_roberta_es_3_0.html) || 12 | [bert_token_classifier_ner_living_species](https://nlp.johnsnowlabs.com/2022/06/27/bert_token_classifier_ner_living_species_es_3_0.html) |[es](https://nlp.johnsnowlabs.com/2022/06/27/bert_token_classifier_ner_living_species_es_3_0.html), [it](https://nlp.johnsnowlabs.com/2022/06/27/bert_token_classifier_ner_living_species_es_3_0.html), [pt](https://nlp.johnsnowlabs.com/2022/06/27/bert_token_classifier_ner_living_species_es_3_0.html) |\n","| 13 | [ner_clinical_bert](https://nlp.johnsnowlabs.com/2022/06/30/ner_clinical_bert_ro_3_0.html) |[ro](https://nlp.johnsnowlabs.com/2022/06/30/ner_clinical_bert_ro_3_0.html) || 14 | [ner_diag_proc](https://nlp.johnsnowlabs.com/2020/07/08/ner_diag_proc_es.html) |[es](https://nlp.johnsnowlabs.com/2020/07/08/ner_diag_proc_es.html) |\n","| 15 | [ner_living_species_300](https://nlp.johnsnowlabs.com/2022/07/26/ner_living_species_300_es_3_0.html) |[es](https://nlp.johnsnowlabs.com/2022/07/26/ner_living_species_300_es_3_0.html) || 16 | [bert_token_classifier_disease_mentions_tweet](https://nlp.johnsnowlabs.com/2022/07/28/bert_token_classifier_disease_mentions_tweet_es_3_0.html) |[es](https://nlp.johnsnowlabs.com/2022/07/28/bert_token_classifier_disease_mentions_tweet_es_3_0.html) |\n","| 17 | [bert_token_classifier_negation_uncertainty](https://nlp.johnsnowlabs.com/2022/08/11/bert_token_classifier_negation_uncertainty_es_3_0.html) |[es](https://nlp.johnsnowlabs.com/2022/08/11/bert_token_classifier_negation_uncertainty_es_3_0.html) || 18 | [bert_token_classifier_ner_clinical_trials_abstracts](https://nlp.johnsnowlabs.com/2022/08/11/bert_token_classifier_ner_clinical_trials_abstracts_es_3_0.html) |[es](https://nlp.johnsnowlabs.com/2022/08/11/bert_token_classifier_ner_clinical_trials_abstracts_es_3_0.html) |\n","| 19 | [bert_token_classifier_pharmacology](https://nlp.johnsnowlabs.com/2022/08/11/bert_token_classifier_pharmacology_es_3_0.html) |[es](https://nlp.johnsnowlabs.com/2022/08/11/bert_token_classifier_pharmacology_es_3_0.html) || 20 | [ner_clinical_trials_abstracts](https://nlp.johnsnowlabs.com/2022/08/12/ner_clinical_trials_abstracts_es_3_0.html) |[es](https://nlp.johnsnowlabs.com/2022/08/12/ner_clinical_trials_abstracts_es_3_0.html) |\n","| 21 | [ner_negation_uncertainty](https://nlp.johnsnowlabs.com/2022/08/13/ner_negation_uncertainty_es_3_0.html) |[es](https://nlp.johnsnowlabs.com/2022/08/13/ner_negation_uncertainty_es_3_0.html) || 22 | [ner_pharmacology](https://nlp.johnsnowlabs.com/2022/08/13/ner_pharmacology_es_3_0.html) |[es](https://nlp.johnsnowlabs.com/2022/08/13/ner_pharmacology_es_3_0.html) |\n","| 23 | [disease_mentions_tweet](https://nlp.johnsnowlabs.com/2022/08/14/disease_mentions_tweet_es_3_0.html) |[es](https://nlp.johnsnowlabs.com/2022/08/14/disease_mentions_tweet_es_3_0.html) || 24 | [ner_eu_clinical_case](https://nlp.johnsnowlabs.com/2023/02/01/ner_eu_clinical_case_es.html) |[es](https://nlp.johnsnowlabs.com/2023/02/01/ner_eu_clinical_case_es.html), [fr](https://nlp.johnsnowlabs.com/2023/02/01/ner_eu_clinical_case_es.html), [eu](https://nlp.johnsnowlabs.com/2023/02/01/ner_eu_clinical_case_es.html) |\n","| 25 | [ner_eu_clinical_condition](https://nlp.johnsnowlabs.com/2023/02/06/ner_eu_clinical_condition_es.html) |[es](https://nlp.johnsnowlabs.com/2023/02/06/ner_eu_clinical_condition_es.html), [eu](https://nlp.johnsnowlabs.com/2023/02/06/ner_eu_clinical_condition_es.html), [fr](https://nlp.johnsnowlabs.com/2023/02/06/ner_eu_clinical_condition_es.html), [it](https://nlp.johnsnowlabs.com/2023/02/06/ner_eu_clinical_condition_es.html) || 26 | [bert_token_classifier_ner_clinical](https://nlp.johnsnowlabs.com/2023/07/05/bert_token_classifier_ner_clinical_nl.html) |[nl](https://nlp.johnsnowlabs.com/2023/07/05/bert_token_classifier_ner_clinical_nl.html) |\n","| 27 | [ner_deid_name_multilingual](https://nlp.johnsnowlabs.com/2024/01/17/ner_deid_name_multilingual_xx.html) |[xx](https://nlp.johnsnowlabs.com/2024/01/17/ner_deid_name_multilingual_xx.html) || 28 | [ner_deid_name_multilingual_clinical](https://nlp.johnsnowlabs.com/2024/02/12/ner_deid_name_multilingual_clinical_xx.html) |[xx](https://nlp.johnsnowlabs.com/2024/02/12/ner_deid_name_multilingual_clinical_xx.html) |\n","| 29 | [ner_deid_name_multilingual_clinical_langtest](https://nlp.johnsnowlabs.com/2024/03/12/ner_deid_name_multilingual_clinical_langtest_xx.html) |[xx](https://nlp.johnsnowlabs.com/2024/03/12/ner_deid_name_multilingual_clinical_langtest_xx.html) |"],"metadata":{"id":"BiptUwfQxxtY"}},{"cell_type":"markdown","metadata":{"id":"BZx93c02IyyP"},"source":["**Also, you can print the list of clinical pretrained models/pipelines and annotators with one-line code:**"]},{"cell_type":"code","execution_count":7,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":15269,"status":"ok","timestamp":1712934915258,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"4l4VvgI1dRb6","outputId":"dbde0936-2a98-48e4-f3cd-a576187504a0"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["[['nerdl_tumour_demo', 'en', '1.7.3'],\n"," ['nerdl_tumour_demo', 'en', '1.8.0'],\n"," ['nerdl_tumour_demo', 'en', '2.0.2'],\n"," ['ner_healthcare', 'en', '2.4.4'],\n"," ['ner_radiology', 'en', '2.7.0'],\n"," ['ner_deid_augmented', 'en', '2.7.1'],\n"," ['ner_deidentify_dl', 'en', '2.7.2'],\n"," ['ner_events_admission_clinical', 'en', '2.7.4'],\n"," ['ner_clinical', 'en', '3.0.0'],\n"," ['ner_radiology', 'en', '3.0.0'],\n"," ['ner_bionlp', 'en', '3.0.0'],\n"," ['ner_posology', 'en', '3.0.0'],\n"," ['ner_healthcare', 'de', '3.0.0'],\n"," ['ner_deid_augmented', 'en', '3.0.0'],\n"," ['ner_anatomy', 'en', '3.0.0'],\n"," ['ner_risk_factors', 'en', '3.0.0'],\n"," ['ner_chemprot_clinical', 'en', '3.0.0'],\n"," ['ner_posology_small', 'en', '3.0.0'],\n"," ['ner_diag_proc', 'es', '3.0.0'],\n"," ['ner_posology_greedy', 'en', '3.0.0'],\n"," ['ner_deid_enriched', 'en', '3.0.0'],\n"," ['ner_drugs_greedy', 'en', '3.0.0'],\n"," ['jsl_ner_wip_clinical', 'en', '3.0.0'],\n"," ['ner_posology_large', 'en', '3.0.0'],\n"," ['jsl_ner_wip_greedy_clinical', 'en', '3.0.0'],\n"," ['ner_clinical_large', 'en', '3.0.0'],\n"," ['ner_diseases', 'en', '3.0.0'],\n"," ['ner_aspect_based_sentiment', 'en', '3.0.0'],\n"," ['ner_cellular', 'en', '3.0.0'],\n"," ['ner_drugs', 'en', '3.0.0'],\n"," ['ner_jsl', 'en', '3.0.0'],\n"," ['ner_anatomy_coarse_biobert', 'en', '3.0.0'],\n"," ['ner_cancer_genetics', 'en', '3.0.0'],\n"," ['ner_drugs_large', 'en', '3.0.0'],\n"," ['ner_events_admission_clinical', 'en', '3.0.0'],\n"," ['ner_deidentify_dl', 'en', '3.0.0'],\n"," ['ner_deid_synthetic', 'en', '3.0.0'],\n"," ['ner_jsl_enriched', 'en', '3.0.0'],\n"," ['ner_deid_large', 'en', '3.0.0'],\n"," ['ner_human_phenotype_gene_clinical', 'en', '3.0.0'],\n"," ['ner_human_phenotype_go_clinical', 'en', '3.0.0'],\n"," ['ner_events_clinical', 'en', '3.0.0'],\n"," ['ner_legal', 'de', '3.0.0'],\n"," ['ner_anatomy_coarse', 'en', '3.0.0'],\n"," ['ner_traffic', 'de', '3.0.0'],\n"," ['ner_bionlp_biobert', 'en', '3.0.0'],\n"," ['ner_deid_sd_large', 'en', '3.0.0'],\n"," ['ner_healthcare_slim', 'de', '3.0.0'],\n"," ['ner_risk_factors_biobert', 'en', '3.0.0'],\n"," ['ner_posology_healthcare', 'en', '3.0.0'],\n"," ['ner_diseases_large', 'en', '3.0.0'],\n"," ['ner_ade_biobert', 'en', '3.0.0'],\n"," ['ner_jsl_enriched_biobert', 'en', '3.0.0'],\n"," ['ner_ade_healthcare', 'en', '3.0.0'],\n"," ['ner_deid_sd', 'en', '3.0.0'],\n"," ['ner_events_healthcare', 'en', '3.0.0'],\n"," ['ner_financial_contract', 'en', '3.0.0'],\n"," ['ner_chemprot_biobert', 'en', '3.0.0'],\n"," ['ner_ade_clinicalbert', 'en', '3.0.0'],\n"," ['ner_posology_large_biobert', 'en', '3.0.0'],\n"," ['ner_radiology_wip_clinical', 'en', '3.0.0'],\n"," ['ner_jsl_biobert', 'en', '3.0.0'],\n"," ['ner_clinical_biobert', 'en', '3.0.0'],\n"," ['ner_deid_enriched_biobert', 'en', '3.0.0'],\n"," ['ner_posology_biobert', 'en', '3.0.0'],\n"," ['ner_cellular_biobert', 'en', '3.0.0'],\n"," ['jsl_ner_wip_modifier_clinical', 'en', '3.0.0'],\n"," ['ner_measurements_clinical', 'en', '3.0.0'],\n"," ['ner_medmentions_coarse', 'en', '3.0.0'],\n"," ['ner_chemicals', 'en', '3.0.0'],\n"," ['ner_events_biobert', 'en', '3.0.0'],\n"," ['ner_bacterial_species', 'en', '3.0.0'],\n"," ['ner_diseases_biobert', 'en', '3.0.0'],\n"," ['ner_human_phenotype_gene_biobert', 'en', '3.0.0'],\n"," ['ner_deid_biobert', 'en', '3.0.0'],\n"," ['nerdl_tumour_demo', 'en', '3.0.0'],\n"," ['ner_human_phenotype_go_biobert', 'en', '3.0.0'],\n"," ['ner_anatomy_biobert', 'en', '3.0.0'],\n"," ['ner_ade_clinical', 'en', '3.0.0'],\n"," ['jsl_rd_ner_wip_greedy_clinical', 'en', '3.0.0'],\n"," ['ner_healthcare', 'en', '3.0.0'],\n"," ['ner_deid_generic_augmented', 'en', '3.0.3'],\n"," ['ner_deid_subentity_augmented', 'en', '3.0.3'],\n"," ['ner_deid_generic_glove', 'en', '3.0.4'],\n"," ['ner_deid_subentity_glove', 'en', '3.0.4'],\n"," ['ner_jsl', 'en', '3.1.0'],\n"," ['ner_genetic_variants', 'en', '3.1.0'],\n"," ['ner_deid_subentity_augmented', 'en', '3.1.0'],\n"," ['ner_deid_generic_augmented', 'en', '3.1.0'],\n"," ['lener_bert_base', 'pt', '3.1.2'],\n"," ['cantemist_scielowiki', 'es', '3.1.2'],\n"," ['jsl_ner_wip_greedy_biobert', 'en', '3.1.3'],\n"," ['jsl_rd_ner_wip_greedy_biobert', 'en', '3.1.3'],\n"," ['lener_bert_large', 'pt', '3.1.3'],\n"," ['lener_bert_base', 'pt', '3.1.3'],\n"," ['meddroprof_scielowiki', 'es', '3.1.3'],\n"," ['ner_jsl_greedy_biobert', 'en', '3.2.0'],\n"," ['ner_jsl_slim', 'en', '3.2.0'],\n"," ['ner_posology_experimental', 'en', '3.1.3'],\n"," ['ner_deid_subentity_augmented', 'en', '3.2.0'],\n"," ['ner_jsl_biobert', 'en', '3.2.0'],\n"," ['ner_chexpert', 'en', '3.3.0'],\n"," ['ner_jsl_enriched', 'en', '3.3.0'],\n"," ['ner_covid_trials', 'en', '3.3.2'],\n"," ['ner_chemd_clinical', 'en', '3.3.0'],\n"," ['roberta_ner_diag_proc', 'es', '3.3.0'],\n"," ['ner_nihss', 'en', '3.3.2'],\n"," ['ner_biomarker', 'en', '3.3.3'],\n"," ['ner_deid_subentity_augmented_i2b2', 'en', '3.3.2'],\n"," ['ner_drugprot_clinical', 'en', '3.3.3'],\n"," ['ner_abbreviation_clinical', 'en', '3.3.4'],\n"," ['ner_deid_generic', 'de', '3.3.4'],\n"," ['ner_deid_subentity', 'de', '3.3.4'],\n"," ['ner_deid_subentity_roberta', 'es', '3.3.4'],\n"," ['ner_deid_generic_roberta', 'es', '3.3.4'],\n"," ['ner_deid_generic', 'es', '3.3.4'],\n"," ['ner_deid_subentity', 'es', '3.3.4'],\n"," ['ner_supplement_clinical', 'en', '3.3.4'],\n"," ['ner_nature_nero_clinical', 'en', '3.3.4'],\n"," ['ner_deid_generic', 'fr', '3.4.1'],\n"," ['ner_deid_subentity', 'fr', '3.4.1'],\n"," ['ner_deid_generic_roberta_augmented', 'es', '3.3.4'],\n"," ['ner_deid_generic_augmented', 'es', '3.3.4'],\n"," ['ner_deid_subentity_augmented', 'es', '3.3.4'],\n"," ['ner_deid_subentity_roberta_augmented', 'es', '3.3.4'],\n"," ['ner_deid_subentity', 'it', '3.4.2'],\n"," ['ner_deid_generic', 'it', '3.4.2'],\n"," ['ner_deid_subentity', 'pt', '3.4.2'],\n"," ['ner_deid_generic', 'pt', '3.4.2'],\n"," ['ner_biomedical_bc2gm', 'en', '3.5.1'],\n"," ['ner_living_species', 'en', '3.5.3'],\n"," ['ner_living_species_roberta', 'pt', '3.5.3'],\n"," ['ner_living_species_bert', 'pt', '3.5.3'],\n"," ['ner_living_species', 'pt', '3.5.3'],\n"," ['ner_living_species', 'gl', '3.5.3'],\n"," ['ner_living_species', 'ca', '3.5.3'],\n"," ['ner_living_species_bert', 'ro', '3.5.3'],\n"," ['ner_living_species', 'fr', '3.5.3'],\n"," ['ner_living_species_bert', 'fr', '3.5.3'],\n"," ['ner_living_species', 'it', '3.5.3'],\n"," ['ner_living_species_bert', 'it', '3.5.3'],\n"," ['ner_living_species', 'es', '3.5.3'],\n"," ['ner_living_species_roberta', 'es', '3.5.3'],\n"," ['ner_living_species_bert', 'es', '3.5.3'],\n"," ['ner_living_species_biobert', 'en', '3.5.3'],\n"," ['ner_clinical_trials_abstracts', 'en', '3.5.3'],\n"," ['ner_deid_subentity_bert', 'ro', '4.0.0'],\n"," ['ner_deid_subentity', 'ro', '4.0.0'],\n"," ['ner_pathogen', 'en', '4.0.0'],\n"," ['ner_clinical_bert', 'ro', '4.0.0'],\n"," ['ner_clinical', 'ro', '4.0.0'],\n"," ['ner_deid_generic_bert', 'ro', '3.5.0'],\n"," ['ner_deid_generic', 'ro', '3.5.0'],\n"," ['ner_oncology_wip', 'en', '3.5.0'],\n"," ['ner_living_species_300', 'es', '3.5.0'],\n"," ['ner_ade_binary', 'en', '4.0.0'],\n"," ['ner_living_species_300', 'es', '4.0.0'],\n"," ['ner_clinical_bert', 'ro', '4.0.2'],\n"," ['ner_clinical_trials_abstracts', 'es', '4.0.2'],\n"," ['ner_pharmacology', 'es', '4.0.2'],\n"," ['ner_negation_uncertainty', 'es', '4.0.2'],\n"," ['disease_mentions_tweet', 'es', '4.0.2'],\n"," ['ner_deid_generic_bert', 'ro', '4.0.2'],\n"," ['ner_oncology_unspecific_posology_wip', 'en', '4.0.0'],\n"," ['ner_oncology_wip', 'en', '4.0.0'],\n"," ['ner_oncology_therapy_wip', 'en', '4.0.0'],\n"," ['ner_oncology_posology_wip', 'en', '4.0.0'],\n"," ['ner_oncology_anatomy_general_wip', 'en', '4.0.0'],\n"," ['ner_oncology_tnm_wip', 'en', '4.0.0'],\n"," ['ner_oncology_demographics_wip', 'en', '4.0.0'],\n"," ['ner_oncology_biomarker_wip', 'en', '4.0.0'],\n"," ['ner_oncology_anatomy_granular_wip', 'en', '4.0.0'],\n"," ['ner_oncology_test_wip', 'en', '4.0.0'],\n"," ['ner_oncology_diagnosis_wip', 'en', '4.0.0'],\n"," ['ner_oncology_response_to_treatment_wip', 'en', '4.0.0'],\n"," ['ner_jsl', 'en', '4.2.0'],\n"," ['ner_covid_trials', 'en', '4.2.0'],\n"," ['ner_oncology_unspecific_posology', 'en', '4.0.0'],\n"," ['ner_oncology', 'en', '4.0.0'],\n"," ['ner_oncology_tnm', 'en', '4.0.0'],\n"," ['ner_oncology_anatomy_general', 'en', '4.0.0'],\n"," ['ner_oncology_therapy', 'en', '4.0.0'],\n"," ['ner_oncology_test', 'en', '4.0.0'],\n"," ['ner_oncology_diagnosis', 'en', '4.0.0'],\n"," ['ner_oncology_demographics', 'en', '4.0.0'],\n"," ['ner_oncology_anatomy_granular', 'en', '4.0.0'],\n"," ['ner_oncology_response_to_treatment', 'en', '4.0.0'],\n"," ['ner_oncology_posology', 'en', '4.0.0'],\n"," ['ner_oncology_biomarker', 'en', '4.0.0'],\n"," ['ner_sdoh_slim_wip', 'en', '4.2.1'],\n"," ['ner_clinical_bert', 'ro', '4.2.2'],\n"," ['ner_living_species_300', 'es', '4.2.2'],\n"," ['ner_deid_generic_bert', 'ro', '4.2.2'],\n"," ['ner_oncology_biomarker', 'en', '4.2.2'],\n"," ['ner_oncology_response_to_treatment', 'en', '4.2.2'],\n"," ['ner_oncology_demographics', 'en', '4.2.2'],\n"," ['ner_oncology_therapy', 'en', '4.2.2'],\n"," ['ner_oncology', 'en', '4.2.2'],\n"," ['ner_oncology_anatomy_granular', 'en', '4.2.2'],\n"," ['ner_oncology_anatomy_general', 'en', '4.2.2'],\n"," ['ner_oncology_diagnosis', 'en', '4.2.2'],\n"," ['ner_oncology_tnm', 'en', '4.2.2'],\n"," ['ner_oncology_posology', 'en', '4.2.2'],\n"," ['ner_oncology_unspecific_posology', 'en', '4.2.2'],\n"," ['ner_oncology_test', 'en', '4.2.2'],\n"," ['meddroprof_scielowiki', 'es', '4.2.2'],\n"," ['ner_sdoh_mentions', 'en', '4.2.2'],\n"," ['ner_sdoh_mentions_test', 'en', '4.2.3'],\n"," ['ner_oncology_anatomy_general_healthcare', 'en', '4.2.4'],\n"," ['ner_oncology_biomarker_healthcare', 'en', '4.2.4'],\n"," ['ner_oncology_unspecific_posology_healthcare', 'en', '4.2.4'],\n"," ['ner_eu_clinical_case', 'en', '4.2.7'],\n"," ['ner_eu_clinical_case', 'eu', '4.2.8'],\n"," ['ner_eu_clinical_case', 'es', '4.2.8'],\n"," ['ner_eu_clinical_condition', 'es', '4.2.8'],\n"," ['ner_eu_clinical_condition', 'eu', '4.2.8'],\n"," ['ner_eu_clinical_condition', 'fr', '4.2.8'],\n"," ['ner_eu_clinical_condition', 'en', '4.2.8'],\n"," ['ner_eu_clinical_case', 'fr', '4.2.8'],\n"," ['ner_eu_clinical_condition', 'it', '4.2.8'],\n"," ['ner_sdoh_demographics_wip', 'en', '4.2.8'],\n"," ['ner_sdoh_social_environment_wip', 'en', '4.2.8'],\n"," ['ner_sdoh_income_social_status_wip', 'en', '4.2.8'],\n"," ['ner_sdoh_wip', 'en', '4.2.8'],\n"," ['ner_sdoh_substance_usage_wip', 'en', '4.3.1'],\n"," ['ner_sdoh_access_to_healthcare_wip', 'en', '4.3.1'],\n"," ['ner_sdoh_community_condition_wip', 'en', '4.3.1'],\n"," ['ner_sdoh_health_behaviours_problems_wip', 'en', '4.3.1'],\n"," ['ner_vop_slim_wip', 'en', '4.2.4'],\n"," ['ner_jsl_limited_80p_for_benchmarks', 'en', '4.3.2'],\n"," ['ner_vop_wip', 'en', '4.4.0'],\n"," ['ner_oncology_limited_80p_for_benchmarks', 'en', '4.3.2'],\n"," ['ner_vop_emb_clinical_medium_wip', 'en', '4.4.0'],\n"," ['ner_vop_emb_clinical_large_wip', 'en', '4.4.0'],\n"," ['ner_oncology_emb_clinical_large', 'en', '4.3.2'],\n"," ['ner_oncology_emb_clinical_medium', 'en', '4.3.2'],\n"," ['ner_deid_large_emb_clinical_medium', 'en', '4.3.2'],\n"," ['ner_posology_emb_clinical_medium', 'en', '4.3.2'],\n"," ['ner_deid_large_emb_clinical_large', 'en', '4.3.2'],\n"," ['ner_posology_emb_clinical_large', 'en', '4.3.2'],\n"," ['ner_jsl_emb_clinical_medium', 'en', '4.3.2'],\n"," ['ner_jsl_emb_clinical_large', 'en', '4.3.2'],\n"," ['ner_sdoh_emb_clinical_large_wip', 'en', '4.3.2'],\n"," ['ner_sdoh_emb_clinical_medium_wip', 'en', '4.3.2'],\n"," ['ner_vop_treatment_wip', 'en', '4.4.0'],\n"," ['ner_vop_problem_wip', 'en', '4.4.0'],\n"," ['ner_vop_problem_reduced_wip', 'en', '4.4.0'],\n"," ['ner_vop_temporal_wip', 'en', '4.4.0'],\n"," ['ner_vop_anatomy_wip', 'en', '4.4.0'],\n"," ['ner_vop_clinical_dept_wip', 'en', '4.4.0'],\n"," ['ner_vop_demographic_wip', 'en', '4.4.0'],\n"," ['ner_vop_test_wip', 'en', '4.4.0'],\n"," ['ner_clinical', 'de', '4.4.0'],\n"," ['ner_abbreviation_emb_clinical_large', 'en', '4.4.1'],\n"," ['ner_abbreviation_emb_clinical_medium', 'en', '4.4.1'],\n"," ['ner_anatomy_emb_clinical_medium', 'en', '4.4.1'],\n"," ['ner_anatomy_emb_clinical_large', 'en', '4.4.2'],\n"," ['ner_cellular_emb_clinical_large', 'en', '4.4.2'],\n"," ['ner_cellular_emb_clinical_medium', 'en', '4.4.2'],\n"," ['ner_ade_emb_clinical_large', 'en', '4.4.2'],\n"," ['ner_ade_emb_clinical_medium', 'en', '4.4.2'],\n"," ['ner_bacterial_species_emb_clinical_large', 'en', '4.4.2'],\n"," ['ner_bacterial_species_emb_clinical_medium', 'en', '4.4.2'],\n"," ['ner_vop_treatment_wip', 'en', '4.4.2'],\n"," ['ner_vop_clinical_dept_wip', 'en', '4.4.2'],\n"," ['ner_vop_wip_emb_clinical_large', 'en', '4.4.2'],\n"," ['ner_vop_wip_emb_clinical_medium', 'en', '4.4.2'],\n"," ['ner_vop_problem_reduced_wip', 'en', '4.4.2'],\n"," ['ner_vop_test_wip', 'en', '4.4.2'],\n"," ['ner_vop_wip', 'en', '4.4.2'],\n"," ['ner_vop_problem_wip', 'en', '4.4.2'],\n"," ['ner_vop_anatomy_wip', 'en', '4.4.2'],\n"," ['ner_vop_demographic_wip', 'en', '4.4.2'],\n"," ['ner_vop_temporal_wip', 'en', '4.4.2'],\n"," ['ner_deid_generic', 'ar', '4.4.2'],\n"," ['ner_deid_subentity', 'ar', '4.4.2'],\n"," ['ner_chemicals_emb_clinical_medium', 'en', '4.4.3'],\n"," ['ner_chemicals_emb_clinical_large', 'en', '4.4.3'],\n"," ['ner_vop_problem_emb_clinical_medium', 'en', '4.4.3'],\n"," ['ner_vop_treatment_emb_clinical_medium', 'en', '4.4.3'],\n"," ['ner_vop_temporal_emb_clinical_large_final', 'en', '4.4.3'],\n"," ['ner_vop_temporal', 'en', '4.4.3'],\n"," ['ner_vop_clinical_dept_emb_clinical_large', 'en', '4.4.3'],\n"," ['ner_vop_emb_clinical_medium', 'en', '4.4.3'],\n"," ['ner_vop_demographic_emb_clinical_large', 'en', '4.4.3'],\n"," ['ner_vop_demographic_emb_clinical_medium', 'en', '4.4.3'],\n"," ['ner_vop_test_emb_clinical_large', 'en', '4.4.3'],\n"," ['ner_vop_anatomy', 'en', '4.4.3'],\n"," ['ner_vop_test_emb_clinical_medium', 'en', '4.4.3'],\n"," ['ner_vop_problem_reduced', 'en', '4.4.3'],\n"," ['ner_vop_test', 'en', '4.4.3'],\n"," ['ner_vop_demographic', 'en', '4.4.3'],\n"," ['ner_vop_treatment_emb_clinical_large', 'en', '4.4.3'],\n"," ['ner_vop_emb_clinical_large', 'en', '4.4.3'],\n"," ['ner_vop_problem_emb_clinical_large', 'en', '4.4.3'],\n"," ['ner_vop_treatment', 'en', '4.4.3'],\n"," ['ner_vop_problem_reduced_emb_clinical_large', 'en', '4.4.3'],\n"," ['ner_vop', 'en', '4.4.3'],\n"," ['ner_vop_anatomy_emb_clinical_medium', 'en', '4.4.3'],\n"," ['ner_vop_problem_reduced_emb_clinical_medium', 'en', '4.4.3'],\n"," ['ner_vop_clinical_dept', 'en', '4.4.3'],\n"," ['ner_vop_problem', 'en', '4.4.3'],\n"," ['ner_vop_clinical_dept_emb_clinical_medium', 'en', '4.4.3'],\n"," ['ner_vop_anatomy_emb_clinical_large', 'en', '4.4.3'],\n"," ['ner_vop_temporal_emb_clinical_medium', 'en', '4.4.3'],\n"," ['ner_demographic_extended_healthcare', 'en', '4.4.3'],\n"," ['ner_sdoh', 'en', '4.4.3'],\n"," ['ner_sdoh_substance_usage', 'en', '4.4.4'],\n"," ['ner_sdoh_health_behaviours_problems', 'en', '4.4.4'],\n"," ['ner_sdoh_demographics', 'en', '4.4.4'],\n"," ['ner_sdoh_income_social_status', 'en', '4.4.4'],\n"," ['ner_sdoh_social_environment', 'en', '4.4.4'],\n"," ['ner_sdoh_access_to_healthcare', 'en', '4.4.4'],\n"," ['ner_sdoh_community_condition', 'en', '4.4.4'],\n"," ['ner_clinical', 'nl', '4.4.4'],\n"," ['ner_ade_clinical_langtest', 'en', '5.0.0'],\n"," ['ner_sdoh_langtest', 'en', '5.0.0'],\n"," ['ner_jsl_langtest', 'en', '5.0.0'],\n"," ['ner_posology_langtest', 'en', '5.0.0'],\n"," ['ner_section_header_diagnosis', 'en', '5.0.0'],\n"," ['ner_clinical', 'pt', '5.0.1'],\n"," ['ner_clinical', 'es', '5.0.1'],\n"," ['ner_clinical', 'it', '5.0.1'],\n"," ['ner_clinical', 'pl', '5.0.2'],\n"," ['ner_clinical', 'tr', '5.0.2'],\n"," ['ner_clinical', 'fr', '5.0.1'],\n"," ['ner_oncology_demographics_langtest', 'en', '5.0.2'],\n"," ['ner_oncology_anatomy_general_langtest', 'en', '5.0.2'],\n"," ['ner_events_clinical_langtest', 'en', '5.0.2'],\n"," ['ner_oncology_anatomy_granular_langtest', 'en', '5.0.2'],\n"," ['ner_oncology_posology_langtest', 'en', '5.0.2'],\n"," ['ner_oncology_response_to_treatment_langtest', 'en', '5.0.2'],\n"," ['ner_deid_camelbert_generic', 'ar', '5.1.0'],\n"," ['ner_deid_generic_camelbert', 'ar', '5.1.0'],\n"," ['ner_deid_subentity_arabert', 'ar', '5.1.0'],\n"," ['ner_deid_generic_arabert', 'ar', '5.1.0'],\n"," ['ner_clinical', 'ja', '5.1.0'],\n"," ['ner_clinical', 'vi', '5.1.0'],\n"," ['ner_clinical', 'no', '5.1.0'],\n"," ['ner_clinical', 'da', '5.1.0'],\n"," ['ner_clinical', 'sv', '5.1.0'],\n"," ['ner_deid_subentity_camelbert', 'ar', '5.1.0'],\n"," ['ner_vop_anatomy_langtest', 'en', '5.1.0'],\n"," ['ner_oncology_biomarker_langtest', 'en', '5.1.0'],\n"," ['ner_vop_demographic_langtest', 'en', '5.1.0'],\n"," ['ner_vop_treatment_langtest', 'en', '5.1.0'],\n"," ['ner_vop_test_langtest', 'en', '5.1.0'],\n"," ['ner_oncology_tnm_langtest', 'en', '5.1.0'],\n"," ['ner_vop_temporal_langtest', 'en', '5.1.0'],\n"," ['ner_oncology_therapy_langtest', 'en', '5.1.0'],\n"," ['ner_oncology_test_langtest', 'en', '5.1.0'],\n"," ['ner_oncology_unspecific_posology_langtest', 'en', '5.1.0'],\n"," ['ner_eu_clinical_condition_langtest', 'en', '5.1.0'],\n"," ['ner_vop_problem_langtest', 'en', '5.1.0'],\n"," ['ner_vop_clinical_dept_langtest', 'en', '5.1.0'],\n"," ['ner_oncology_diagnosis_langtest', 'en', '5.1.0'],\n"," ['ner_vop_problem_reduced_langtest', 'en', '5.1.0'],\n"," ['ner_deid_subentity_augmented_langtest', 'en', '5.1.0'],\n"," ['ner_deid_generic_augmented_langtest', 'en', '5.1.0'],\n"," ['ner_clinical_langtest', 'en', '5.1.0'],\n"," ['ner_clinical', 'ar', '5.1.1'],\n"," ['ner_clinical', 'bg', '5.1.1'],\n"," ['ner_clinical', 'fi', '5.1.1'],\n"," ['ner_biomarker_langtest', 'en', '5.1.1'],\n"," ['ner_bionlp_langtest', 'en', '5.1.1'],\n"," ['ner_diseases_langtest', 'en', '5.1.1'],\n"," ['ner_deid_large_langtest', 'en', '5.1.1'],\n"," ['ner_cellular_langtest', 'en', '5.1.1'],\n"," ['ner_bacterial_species_langtest', 'en', '5.1.1'],\n"," ['ner_chemprot_clinical_langtest', 'en', '5.1.1'],\n"," ['ner_deid_enriched_langtest', 'en', '5.1.1'],\n"," ['ner_deid_generic_augmented_allUpperCased_langtest', 'en', '5.1.1'],\n"," ['ner_clinical_large_langtest', 'en', '5.1.1'],\n"," ['ner_living_species_langtest', 'en', '5.1.1'],\n"," ['ner_vop_langtest', 'en', '5.1.1'],\n"," ['ner_oncology_langtest', 'en', '5.1.1'],\n"," ['ner_clinical', 'he', '5.1.1'],\n"," ['ner_human_phenotype_go_clinical_langtest', 'en', '5.1.1'],\n"," ['ner_risk_factors_langtest', 'en', '5.1.1'],\n"," ['ner_human_phenotype_gene_clinical_langtest', 'en', '5.1.1'],\n"," ['ner_opioid_small_wip', 'en', '5.2.0'],\n"," ['ner_deid_name_multilingual', 'xx', '5.2.1'],\n"," ['ner_clinical_abbreviation_langtest', 'en', '5.2.1'],\n"," ['ner_deid_multilingual', 'xx', '5.2.1'],\n"," ['ner_deid_name_multilingual_clinical', 'xx', '5.2.1'],\n"," ['ner_snomed_term', 'en', '5.2.1'],\n"," ['ner_opioid', 'en', '5.2.1'],\n"," ['ner_deid_name_multilingual_clinical_langtest', 'xx', '5.3.0'],\n"," ['ner_opioid', 'en', '5.3.0'],\n"," ['ner_sdoh_core', 'en', '5.3.1']]"]},"metadata":{},"execution_count":7}],"source":["# pipelines = medical.InternalResourceDownloader.returnPrivatePipelines()\n","ner_models = medical.InternalResourceDownloader.returnPrivateModels(\"MedicalNerModel\")\n","ner_models"]},{"cell_type":"code","execution_count":8,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":423},"executionInfo":{"elapsed":36278,"status":"ok","timestamp":1712934951533,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"SvDbCdj_g1G7","outputId":"520592bd-436d-4dce-a0ef-e4fdd71e2750"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["                                name sparkVersion version language  \\\n","0                  nerdl_tumour_demo            2   1.7.3       en   \n","1                 nercrf_tumour_demo            2   1.7.3       en   \n","2                  nerdl_tumour_demo          2.4   1.8.0       en   \n","3                 nercrf_tumour_demo          2.4   1.8.0       en   \n","4                 nercrf_tumour_demo          2.4   2.0.2       en   \n","...                              ...          ...     ...      ...   \n","1748                      ner_opioid          3.0   5.3.0       en   \n","1749  ner_profiling_deidentification          3.4   5.3.1       en   \n","1750  ner_profiling_deidentification          3.2   5.3.1       en   \n","1751  ner_profiling_deidentification          3.0   5.3.1       en   \n","1752                   ner_sdoh_core          3.0   5.3.1       en   \n","\n","                         date readyToUse  \n","0     2018-12-19T16:52:37.735       true  \n","1     2018-12-19T17:23:53.776       true  \n","2     2018-12-22T04:21:25.574       true  \n","3     2018-12-22T04:46:26.992       true  \n","4     2019-05-21T18:23:13.066       true  \n","...                       ...        ...  \n","1748  2024-03-27T05:10:11.243       true  \n","1749  2024-03-28T13:11:24.226       true  \n","1750  2024-03-28T12:57:39.423       true  \n","1751  2024-03-28T13:39:48.135       true  \n","1752  2024-04-08T04:12:22.813       true  \n","\n","[1753 rows x 6 columns]"],"text/html":["\n","  <div id=\"df-59548a83-b6de-4d81-bfdb-1654e6f2c3ef\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>name</th>\n","      <th>sparkVersion</th>\n","      <th>version</th>\n","      <th>language</th>\n","      <th>date</th>\n","      <th>readyToUse</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>nerdl_tumour_demo</td>\n","      <td>2</td>\n","      <td>1.7.3</td>\n","      <td>en</td>\n","      <td>2018-12-19T16:52:37.735</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>nercrf_tumour_demo</td>\n","      <td>2</td>\n","      <td>1.7.3</td>\n","      <td>en</td>\n","      <td>2018-12-19T17:23:53.776</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>nerdl_tumour_demo</td>\n","      <td>2.4</td>\n","      <td>1.8.0</td>\n","      <td>en</td>\n","      <td>2018-12-22T04:21:25.574</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>nercrf_tumour_demo</td>\n","      <td>2.4</td>\n","      <td>1.8.0</td>\n","      <td>en</td>\n","      <td>2018-12-22T04:46:26.992</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>nercrf_tumour_demo</td>\n","      <td>2.4</td>\n","      <td>2.0.2</td>\n","      <td>en</td>\n","      <td>2019-05-21T18:23:13.066</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>...</th>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","    </tr>\n","    <tr>\n","      <th>1748</th>\n","      <td>ner_opioid</td>\n","      <td>3.0</td>\n","      <td>5.3.0</td>\n","      <td>en</td>\n","      <td>2024-03-27T05:10:11.243</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>1749</th>\n","      <td>ner_profiling_deidentification</td>\n","      <td>3.4</td>\n","      <td>5.3.1</td>\n","      <td>en</td>\n","      <td>2024-03-28T13:11:24.226</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>1750</th>\n","      <td>ner_profiling_deidentification</td>\n","      <td>3.2</td>\n","      <td>5.3.1</td>\n","      <td>en</td>\n","      <td>2024-03-28T12:57:39.423</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>1751</th>\n","      <td>ner_profiling_deidentification</td>\n","      <td>3.0</td>\n","      <td>5.3.1</td>\n","      <td>en</td>\n","      <td>2024-03-28T13:39:48.135</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>1752</th>\n","      <td>ner_sdoh_core</td>\n","      <td>3.0</td>\n","      <td>5.3.1</td>\n","      <td>en</td>\n","      <td>2024-04-08T04:12:22.813</td>\n","      <td>true</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>1753 rows × 6 columns</p>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-59548a83-b6de-4d81-bfdb-1654e6f2c3ef')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-59548a83-b6de-4d81-bfdb-1654e6f2c3ef button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-59548a83-b6de-4d81-bfdb-1654e6f2c3ef');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-ea69df00-9d0d-4c20-b0d1-212939338e31\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-ea69df00-9d0d-4c20-b0d1-212939338e31')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-ea69df00-9d0d-4c20-b0d1-212939338e31 button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","  <div id=\"id_ebb07fc8-6bc4-48a9-a8cf-ddc21b3fe3dd\">\n","    <style>\n","      .colab-df-generate {\n","        background-color: #E8F0FE;\n","        border: none;\n","        border-radius: 50%;\n","        cursor: pointer;\n","        display: none;\n","        fill: #1967D2;\n","        height: 32px;\n","        padding: 0 0 0 0;\n","        width: 32px;\n","      }\n","\n","      .colab-df-generate:hover {\n","        background-color: #E2EBFA;\n","        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","        fill: #174EA6;\n","      }\n","\n","      [theme=dark] .colab-df-generate {\n","        background-color: #3B4455;\n","        fill: #D2E3FC;\n","      }\n","\n","      [theme=dark] .colab-df-generate:hover {\n","        background-color: #434B5C;\n","        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","        fill: #FFFFFF;\n","      }\n","    </style>\n","    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('models_df')\"\n","            title=\"Generate code using this dataframe.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n","  </svg>\n","    </button>\n","    <script>\n","      (() => {\n","      const buttonEl =\n","        document.querySelector('#id_ebb07fc8-6bc4-48a9-a8cf-ddc21b3fe3dd button.colab-df-generate');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      buttonEl.onclick = () => {\n","        google.colab.notebook.generateWithVariable('models_df');\n","      }\n","      })();\n","    </script>\n","  </div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"models_df","summary":"{\n  \"name\": \"models_df\",\n  \"rows\": 1753,\n  \"fields\": [\n    {\n      \"column\": \"name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 551,\n        \"samples\": [\n          \"assertion_opioid_general_symptoms_status_wip\",\n          \"ner_lener\",\n          \"bert_token_classifier_ner_chemprot_pipeline\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"sparkVersion\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 6,\n        \"samples\": [\n          \"2\",\n          \"2.4\",\n          \"3.4\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"version\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 75,\n        \"samples\": [\n          \"2.3.3\",\n          \"4.4.3\",\n          \"2.5.0\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"language\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 23,\n        \"samples\": [\n          \"vi\",\n          \"eu\",\n          \"en\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"date\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1669,\n        \"samples\": [\n          \"2023-06-16T15:55:42.785\",\n          \"2023-06-16T17:40:26.593\",\n          \"2022-07-25T13:15:53.153\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"readyToUse\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"true\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":8}],"source":["# find all NER models\n","\n","compatibility = medical.Compatibility(spark)\n","\n","models = compatibility.findVersion('ner')\n","models_df = pd.DataFrame([dict(x) for x in list(models)])\n","models_df"]},{"cell_type":"code","execution_count":9,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":423},"executionInfo":{"elapsed":3591,"status":"ok","timestamp":1712934955121,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"rGnqNHy9d6sG","outputId":"a7d01bcf-74cc-462a-fe8d-a7e4c8112cdd"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["                                     name sparkVersion version language  \\\n","0                        ner_oncology_wip          3.0   3.5.0       en   \n","1    ner_oncology_unspecific_posology_wip          3.0   4.0.0       en   \n","2                        ner_oncology_wip          3.0   4.0.0       en   \n","3                ner_oncology_therapy_wip          3.0   4.0.0       en   \n","4               ner_oncology_posology_wip          3.0   4.0.0       en   \n","..                                    ...          ...     ...      ...   \n","135       ner_oncology_diagnosis_langtest          3.0   5.1.0       en   \n","136                 ner_oncology_langtest          3.0   5.1.1       en   \n","137                 ner_oncology_pipeline          3.0   5.2.0       en   \n","138                 ner_oncology_pipeline          3.2   5.2.0       en   \n","139                 ner_oncology_pipeline          3.4   5.2.0       en   \n","\n","                        date readyToUse  \n","0    2022-07-25T17:48:26.053       true  \n","1    2022-09-30T18:49:12.336       true  \n","2    2022-09-30T16:54:45.893       true  \n","3    2022-09-30T17:12:16.894       true  \n","4    2022-10-01T04:46:44.423       true  \n","..                       ...        ...  \n","135  2023-09-22T14:43:31.887       true  \n","136  2023-10-15T20:38:50.566       true  \n","137  2024-01-08T14:26:57.619       true  \n","138  2024-01-08T14:42:30.181       true  \n","139  2024-01-08T14:53:55.028       true  \n","\n","[140 rows x 6 columns]"],"text/html":["\n","  <div id=\"df-2f3a7853-e670-47c5-b021-74875097194d\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>name</th>\n","      <th>sparkVersion</th>\n","      <th>version</th>\n","      <th>language</th>\n","      <th>date</th>\n","      <th>readyToUse</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>ner_oncology_wip</td>\n","      <td>3.0</td>\n","      <td>3.5.0</td>\n","      <td>en</td>\n","      <td>2022-07-25T17:48:26.053</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>ner_oncology_unspecific_posology_wip</td>\n","      <td>3.0</td>\n","      <td>4.0.0</td>\n","      <td>en</td>\n","      <td>2022-09-30T18:49:12.336</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>ner_oncology_wip</td>\n","      <td>3.0</td>\n","      <td>4.0.0</td>\n","      <td>en</td>\n","      <td>2022-09-30T16:54:45.893</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>ner_oncology_therapy_wip</td>\n","      <td>3.0</td>\n","      <td>4.0.0</td>\n","      <td>en</td>\n","      <td>2022-09-30T17:12:16.894</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>ner_oncology_posology_wip</td>\n","      <td>3.0</td>\n","      <td>4.0.0</td>\n","      <td>en</td>\n","      <td>2022-10-01T04:46:44.423</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>...</th>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","    </tr>\n","    <tr>\n","      <th>135</th>\n","      <td>ner_oncology_diagnosis_langtest</td>\n","      <td>3.0</td>\n","      <td>5.1.0</td>\n","      <td>en</td>\n","      <td>2023-09-22T14:43:31.887</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>136</th>\n","      <td>ner_oncology_langtest</td>\n","      <td>3.0</td>\n","      <td>5.1.1</td>\n","      <td>en</td>\n","      <td>2023-10-15T20:38:50.566</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>137</th>\n","      <td>ner_oncology_pipeline</td>\n","      <td>3.0</td>\n","      <td>5.2.0</td>\n","      <td>en</td>\n","      <td>2024-01-08T14:26:57.619</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>138</th>\n","      <td>ner_oncology_pipeline</td>\n","      <td>3.2</td>\n","      <td>5.2.0</td>\n","      <td>en</td>\n","      <td>2024-01-08T14:42:30.181</td>\n","      <td>true</td>\n","    </tr>\n","    <tr>\n","      <th>139</th>\n","      <td>ner_oncology_pipeline</td>\n","      <td>3.4</td>\n","      <td>5.2.0</td>\n","      <td>en</td>\n","      <td>2024-01-08T14:53:55.028</td>\n","      <td>true</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>140 rows × 6 columns</p>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-2f3a7853-e670-47c5-b021-74875097194d')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-2f3a7853-e670-47c5-b021-74875097194d button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-2f3a7853-e670-47c5-b021-74875097194d');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-06dd94f0-cb8d-48b4-95dd-4eb045d8fafb\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-06dd94f0-cb8d-48b4-95dd-4eb045d8fafb')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-06dd94f0-cb8d-48b4-95dd-4eb045d8fafb button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","  <div id=\"id_cb34859f-5a17-411b-96fb-25bf0d2fe13f\">\n","    <style>\n","      .colab-df-generate {\n","        background-color: #E8F0FE;\n","        border: none;\n","        border-radius: 50%;\n","        cursor: pointer;\n","        display: none;\n","        fill: #1967D2;\n","        height: 32px;\n","        padding: 0 0 0 0;\n","        width: 32px;\n","      }\n","\n","      .colab-df-generate:hover {\n","        background-color: #E2EBFA;\n","        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","        fill: #174EA6;\n","      }\n","\n","      [theme=dark] .colab-df-generate {\n","        background-color: #3B4455;\n","        fill: #D2E3FC;\n","      }\n","\n","      [theme=dark] .colab-df-generate:hover {\n","        background-color: #434B5C;\n","        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","        fill: #FFFFFF;\n","      }\n","    </style>\n","    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('models_df')\"\n","            title=\"Generate code using this dataframe.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n","  </svg>\n","    </button>\n","    <script>\n","      (() => {\n","      const buttonEl =\n","        document.querySelector('#id_cb34859f-5a17-411b-96fb-25bf0d2fe13f button.colab-df-generate');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      buttonEl.onclick = () => {\n","        google.colab.notebook.generateWithVariable('models_df');\n","      }\n","      })();\n","    </script>\n","  </div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"models_df","summary":"{\n  \"name\": \"models_df\",\n  \"rows\": 140,\n  \"fields\": [\n    {\n      \"column\": \"name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 62,\n        \"samples\": [\n          \"ner_oncology_anatomy_general_langtest_pipeline\",\n          \"ner_oncology_tnm_langtest\",\n          \"ner_oncology_wip\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"sparkVersion\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"3.0\",\n          \"3.2\",\n          \"3.4\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"version\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 11,\n        \"samples\": [\n          \"4.3.2\",\n          \"3.5.0\",\n          \"5.1.1\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"language\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"en\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"date\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"num_unique_values\": 140,\n        \"samples\": [\n          \"2023-06-17T09:37:46.228\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"readyToUse\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"true\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":9}],"source":["# find oncology NER models\n","\n","models = compatibility.findVersion(\"ner_oncology\")\n","\n","models_df = pd.DataFrame([dict(x) for x in list(models)])\n","models_df"]},{"cell_type":"markdown","metadata":{"id":"AMU4sAJQ0Rhs"},"source":["## Clinical NER Pipeline (with pretrained models)"]},{"cell_type":"code","execution_count":10,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":161263,"status":"ok","timestamp":1712935116373,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"DEa5SITBxmY0","outputId":"5e503266-7e0f-4108-89e4-205e9d86c5ef"},"outputs":[{"output_type":"stream","name":"stdout","text":["sentence_detector_dl_healthcare download started this may take some time.\n","Approximate size to download 367.3 KB\n","[OK!]\n","embeddings_clinical download started this may take some time.\n","Approximate size to download 1.6 GB\n","[OK!]\n","ner_clinical_large download started this may take some time.\n","[OK!]\n"]}],"source":["# Annotator that transforms a text column from dataframe into an Annotation ready for NLP\n","documentAssembler = nlp.DocumentAssembler()\\\n","    .setInputCol(\"text\")\\\n","    .setOutputCol(\"document\")\n","\n","sentenceDetector = nlp.SentenceDetectorDLModel.pretrained(\"sentence_detector_dl_healthcare\",\"en\",\"clinical/models\")\\\n","    .setInputCols([\"document\"])\\\n","    .setOutputCol(\"sentence\")\n","\n","# Tokenizer splits words in a relevant format for NLP\n","tokenizer = nlp.Tokenizer()\\\n","    .setInputCols([\"sentence\"])\\\n","    .setOutputCol(\"token\")\n","\n","# Clinical word embeddings trained on PubMED dataset\n","word_embeddings = nlp.WordEmbeddingsModel.pretrained(\"embeddings_clinical\",\"en\",\"clinical/models\")\\\n","    .setInputCols([\"sentence\",\"token\"])\\\n","    .setOutputCol(\"embeddings\")\n","\n","# NER model trained on i2b2 (sampled from MIMIC) dataset\n","clinical_ner = medical.NerModel.pretrained(\"ner_clinical_large\",\"en\",\"clinical/models\")\\\n","    .setInputCols([\"sentence\",\"token\",\"embeddings\"])\\\n","    .setOutputCol(\"ner\")\\\n","    .setLabelCasing(\"upper\") #decide if we want to return the tags in upper or lower case\n","\n","ner_converter = medical.NerConverterInternal()\\\n","    .setInputCols([\"sentence\",\"token\",\"ner\"])\\\n","    .setOutputCol(\"ner_chunk\")\n","\n","nlpPipeline = nlp.Pipeline(stages=[\n","    documentAssembler,\n","    sentenceDetector,\n","    tokenizer,\n","    word_embeddings,\n","    clinical_ner,\n","    ner_converter])\n","\n","\n","empty_data = spark.createDataFrame([[\"\"]]).toDF(\"text\")\n","\n","model = nlpPipeline.fit(empty_data)\n"]},{"cell_type":"code","execution_count":11,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":13,"status":"ok","timestamp":1712935116373,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"eCPdErQTt9WS","outputId":"015c8db5-73da-4576-c49e-37eea2fe0072"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["[DocumentAssembler_b06127093d4f,\n"," SentenceDetectorDLModel_6bafc4746ea5,\n"," REGEX_TOKENIZER_ce8b960ac920,\n"," WORD_EMBEDDINGS_MODEL_9004b1d00302,\n"," MedicalNerModel_1a8637089929,\n"," NER_CONVERTER_9d64f35b1990]"]},"metadata":{},"execution_count":11}],"source":["model.stages"]},{"cell_type":"code","execution_count":12,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":8,"status":"ok","timestamp":1712935116373,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"m2hXcbMzvUzf","outputId":"dbf99a55-d73c-4c85-f1bf-a518cb550b36"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["['O',\n"," 'B-TREATMENT',\n"," 'I-TREATMENT',\n"," 'B-PROBLEM',\n"," 'I-PROBLEM',\n"," 'B-TEST',\n"," 'I-TEST']"]},"metadata":{},"execution_count":12}],"source":["clinical_ner.getClasses()"]},{"cell_type":"code","execution_count":13,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":6,"status":"ok","timestamp":1712935116373,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"rPO8wgprvai_","outputId":"133c2699-291d-46cf-e91d-792b855ceb01"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["{Param(parent='MedicalNerModel_1a8637089929', name='doExceptionHandling', doc='If true, tries to compute batch-wise as usual.\\n    If within a batch an exception occurs, the batch will be processed row-wise and for any exception occurring during row-processing a Error Annotation is emitted.\\n    This means only the bad rows are lost instead of entire batch.\\n    This comes with a performance penalty.'): False,\n"," Param(parent='MedicalNerModel_1a8637089929', name='includeAllConfidenceScores', doc='whether to include all confidence scores in annotation metadata or just the score of the predicted tag'): False,\n"," Param(parent='MedicalNerModel_1a8637089929', name='inferenceBatchSize', doc='number of sentences to process in a single batch during inference'): 1,\n"," Param(parent='MedicalNerModel_1a8637089929', name='labelCasing', doc='Setting all labels of the NER models upper/lower case. values upper|lower'): 'upper',\n"," Param(parent='MedicalNerModel_1a8637089929', name='sentenceTokenIndex', doc='Whether to include the token index for each sentence in annotation metadata.'): False,\n"," Param(parent='MedicalNerModel_1a8637089929', name='lazyAnnotator', doc='Whether this AnnotatorModel acts as lazy in RecursivePipelines'): False,\n"," Param(parent='MedicalNerModel_1a8637089929', name='includeConfidence', doc='whether to include confidence scores in annotation metadata'): True,\n"," Param(parent='MedicalNerModel_1a8637089929', name='batchSize', doc='Size of every batch'): 64,\n"," Param(parent='MedicalNerModel_1a8637089929', name='classes', doc='get the tags used to trained this MedicalNerModel'): ['O',\n","  'B-TREATMENT',\n","  'I-TREATMENT',\n","  'B-PROBLEM',\n","  'I-PROBLEM',\n","  'B-TEST',\n","  'I-TEST'],\n"," Param(parent='MedicalNerModel_1a8637089929', name='inputCols', doc='previous annotations columns, if renamed'): ['sentence',\n","  'token',\n","  'embeddings'],\n"," Param(parent='MedicalNerModel_1a8637089929', name='outputCol', doc='output annotation column. can be left default.'): 'ner',\n"," Param(parent='MedicalNerModel_1a8637089929', name='storageRef', doc='unique reference name for identification'): 'clinical'}"]},"metadata":{},"execution_count":13}],"source":["clinical_ner.extractParamMap()"]},{"cell_type":"code","execution_count":14,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":36},"executionInfo":{"elapsed":6,"status":"ok","timestamp":1712935116374,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"gGjoRR5Bvj-U","outputId":"ebf4f172-4b6c-4c5d-b3a2-abe1463a45f7"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["'clinical'"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"string"}},"metadata":{},"execution_count":14}],"source":["clinical_ner.getStorageRef()"]},{"cell_type":"code","execution_count":15,"metadata":{"id":"v1lmxbPt1dZW","executionInfo":{"status":"ok","timestamp":1712935116978,"user_tz":-180,"elapsed":610,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["# Downloading sample datasets.\n","! wget -q https://raw.githubusercontent.com/JohnSnowLabs/spark-nlp-workshop/master/tutorials/Certification_Trainings/Healthcare/data/mt_samples_10.csv"]},{"cell_type":"code","execution_count":16,"metadata":{"id":"f1S9J6CASrSE","executionInfo":{"status":"ok","timestamp":1712935117479,"user_tz":-180,"elapsed":502,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["mt_samples_df = spark.read.csv(\"mt_samples_10.csv\", header=True, multiLine=True)"]},{"cell_type":"code","execution_count":17,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2,"status":"ok","timestamp":1712935117479,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"Kt_9Qfmw30rA","outputId":"132c14e4-ba72-438c-e532-3640ae45c691"},"outputs":[{"output_type":"stream","name":"stdout","text":["root\n"," |-- index: string (nullable = true)\n"," |-- text: string (nullable = true)\n","\n"]}],"source":["mt_samples_df.printSchema()"]},{"cell_type":"code","execution_count":18,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1004,"status":"ok","timestamp":1712935118481,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"te55tz4dwAB5","outputId":"4f4841e6-f149-492e-b049-1770f765c99b"},"outputs":[{"output_type":"stream","name":"stdout","text":["+-----+--------------------+\n","|index|                text|\n","+-----+--------------------+\n","|    0|Sample Type / Med...|\n","|    1|Sample Type / Med...|\n","|    2|Sample Type / Med...|\n","|    3|Sample Type / Med...|\n","|    4|Sample Type / Med...|\n","|    5|Sample Type / Med...|\n","|    6|Sample Type / Med...|\n","|    7|Sample Type / Med...|\n","|    8|Sample Type / Med...|\n","|    9|Sample Type / Med...|\n","+-----+--------------------+\n","\n"]}],"source":["mt_samples_df.show()"]},{"cell_type":"code","execution_count":19,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":3,"status":"ok","timestamp":1712935118481,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"m5soib4Tw4PK","outputId":"5bb282c8-354f-46b1-f028-f7b020e2118c"},"outputs":[{"output_type":"stream","name":"stdout","text":["Sample Type / Medical Specialty:\n","Hematology - Oncology\n","Sample Name:\n","Discharge Summary - Mesothelioma - 1\n","Description:\n","Mesothelioma, pleural effusion, atrial fibrillation, anemia, ascites, esophageal reflux, and history of deep venous thrombosis.\n","(Medical Transcription Sample Report)\n","PRINCIPAL DIAGNOSIS:\n","Mesothelioma.\n","SECONDARY DIAGNOSES:\n","Pleural effusion, atrial fibrillation, anemia, ascites, esophageal reflux, and history of deep venous thrombosis.\n","PROCEDURES\n","1. On August 24, 2007, decortication of the lung with pleural biopsy and transpleural fluoroscopy.\n","2. On August 20, 2007, thoracentesis.\n","3. On August 31, 2007, Port-A-Cath placement.\n","HISTORY AND PHYSICAL:\n","The patient is a 41-year-old Vietnamese female with a nonproductive cough that started last week. She has had right-sided chest pain radiating to her back with fever starting yesterday. She has a history of pericarditis and pericardectomy in May 2006 and developed cough with right-sided chest pain, and went to an urgent care center. Chest x-ray revealed right-sided pleural effusion.\n","PAST MEDICAL HISTORY\n","1. Pericardectomy.\n","2. Pericarditis.\n","2. Atrial fibrillation.\n","4. RNCA with intracranial thrombolytic treatment.\n","5 PTA of MCA.\n","6. Mesenteric venous thrombosis.\n","7. Pericardial window.\n","8. Cholecystectomy.\n","9. Left thoracentesis.\n","FAMILY HISTORY:\n","No family history of coronary artery disease, CVA, diabetes, CHF or MI. The patient has one family member, a sister, with history of cancer.\n","SOCIAL HISTORY:\n","She is married. Employed with the US Post Office. She is a mother of three. Denies tobacco, alcohol or illicit drug use.\n","MEDICATIONS\n","1. Coumadin 1 mg daily. Last INR was on Tuesday, August 14, 2007, and her INR was 2.3.\n","2. Amiodarone 100 mg p.o. daily.\n","REVIEW OF SYSTEMS:\n","Complete review of systems negative except as in pulmonary as noted above. The patient also reports occasional numbness and tingling of her left arm.\n","PHYSICAL EXAMINATION\n","VITAL SIGNS: Blood pressure 123/95, heart rate 83, respirations 20, temperature 97, and oxygen saturation 97%.\n","GENERAL: Positive nonproductive cough and pain with coughing.\n","HEENT: Pupils are equal and reactive to light and accommodation. Tympanic membranes are clear.\n","NECK: Supple. No lymphadenopathy. No masses.\n","RESPIRATORY: Pleural friction rub is noted.\n","GI: Soft, nondistended, and nontender. Positive bowel sounds. No organomegaly.\n","EXTREMITIES: No edema, no clubbing, no cyanosis, no tenderness. Full range of motion. Normal pulses in all extremities.\n","SKIN: No breakdown or lesions. No ulcers.\n","NEUROLOGIC: Grossly intact. No focal deficits. Awake, alert, and oriented to person, place, and time.\n","LABORATORY DATA:\n","Labs are pending.\n","HOSPITAL COURSE:\n","The patient was admitted for a right-sided pleural effusion for thoracentesis on Monday by Dr. X. Her Coumadin was placed on hold. A repeat echocardiogram was checked. She was started on prophylaxis for DVT with Lovenox 40 mg subcutaneously. Her history dated back to March 2005 when she first sought medical attention for evidence of pericarditis, which was treated with pericardial window in an outside hospital, at that time she was also found to have mesenteric pain and thrombosis, is now anticoagulated. Her pericardial fluid was accumulated and she was seen by Dr. Y. At that time, she was recommended for pericardectomy, which was performed by Dr. Z. Review of her CT scan from March 2006 prior to her pericardectomy, already shows bilateral plural effusions. The patient improved clinically after the pericardectomy with resolution of her symptoms. Recently, she was readmitted to the hospital with chest pain and found to have bilateral pleural effusion, the right greater than the left. CT of the chest also revealed a large mediastinal lymph node. We reviewed the pathology obtained from the pericardectomy in March 2006, which was diagnostic of mesothelioma. At this time, chest tube placement for drainage of the fluid occurred and thoracoscopy with fluid biopsies, which were performed, which revealed epithelioid malignant mesothelioma. The patient was then stained with a PET CT, which showed extensive uptake in the chest, bilateral pleural pericardial effusions, and lymphadenopathy. She also had acidic fluid, pectoral and intramammary lymph nodes and uptake in L4 with SUV of 4. This was consistent with stage III disease. Her repeat echocardiogram showed an ejection fraction of 45% to 49%. She was transferred to Oncology service and started on chemotherapy on September 1, 2007 with cisplatin 75 mg/centimeter squared equaling 109 mg IV piggyback over 2 hours on September 1, 2007, Alimta 500 mg/ centimeter squared equaling 730 mg IV piggyback over 10 minutes. This was all initiated after a Port-A-Cath was placed. The chemotherapy was well tolerated and the patient was discharged the following day after discontinuing IV fluid and IV. Her Port-A-Cath was packed with heparin according to protocol.\n","DISCHARGE MEDICATIONS:\n","Zofran, Phenergan, Coumadin, and Lovenox, and Vicodin\n","DISCHARGE INSTRUCTIONS:\n","She was instructed to followup with Dr. XYZ in the office to check her INR on Tuesday. She was instructed to call if she had any other questions or concerns in the interim.\n","Keywords:\n","hematology - oncology, mesothelioma, pleural effusion, atrial fibrillation, anemia, ascites, esophageal reflux, deep venous thrombosis, port-a-cath placement, port a cath, iv piggyback, venous thrombosis, atrial, thrombosis, pericardial, lymphadenopathy, fluid, pericardectomy, chest, pleural,\n"]}],"source":["print(mt_samples_df.limit(1).collect()[0]['text'])"]},{"cell_type":"code","execution_count":20,"metadata":{"id":"7HdysGLU-xfP","executionInfo":{"status":"ok","timestamp":1712935119113,"user_tz":-180,"elapsed":633,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["result = model.transform(mt_samples_df).cache()"]},{"cell_type":"code","execution_count":21,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":12107,"status":"ok","timestamp":1712935131218,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"DHC3kvfIF2tm","outputId":"c45e2475-9e5a-41ad-b22b-7ebb4f4df671"},"outputs":[{"output_type":"stream","name":"stdout","text":["+-----+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+\n","|index|                text|            document|            sentence|               token|          embeddings|                 ner|           ner_chunk|\n","+-----+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+\n","|    0|Sample Type / Med...|[{document, 0, 54...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 118, 129...|\n","|    1|Sample Type / Med...|[{document, 0, 32...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 68, 92, ...|\n","|    2|Sample Type / Med...|[{document, 0, 42...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 68, 73, ...|\n","|    3|Sample Type / Med...|[{document, 0, 20...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 119, 155...|\n","|    4|Sample Type / Med...|[{document, 0, 34...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 97, 116,...|\n","|    5|Sample Type / Med...|[{document, 0, 15...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 68, 84, ...|\n","|    6|Sample Type / Med...|[{document, 0, 25...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 68, 73, ...|\n","|    7|Sample Type / Med...|[{document, 0, 93...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 68, 92, ...|\n","|    8|Sample Type / Med...|[{document, 0, 20...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 155, 189...|\n","|    9|Sample Type / Med...|[{document, 0, 19...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 68, 95, ...|\n","+-----+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+\n","\n"]}],"source":["result.show()"]},{"cell_type":"code","execution_count":22,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":10,"status":"ok","timestamp":1712935131219,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"OGBHfjMVUoCu","outputId":"a20c1106-e406-453e-cb35-cd69e5e8e945"},"outputs":[{"output_type":"stream","name":"stdout","text":["+----------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+\n","|                                                                                              result|                                                                                              result|\n","+----------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Discharge, Sum...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, B-PROBLEM, O, B-PROBLEM, I-PROBLEM, ...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, BCCa, Excision...|[O, O, O, O, O, O, O, O, O, O, O, O, B-TREATMENT, I-TREATMENT, I-TREATMENT, I-TREATMENT, I-TREATM...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Anemia, -, Con...|[O, O, O, O, O, O, O, O, O, O, O, O, B-PROBLEM, O, O, O, O, B-PROBLEM, I-PROBLEM, O, O, B-PROBLEM...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Intensity-Modu...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, B-TREATMENT, I-TREATMENT, I-TREATMENT, O, O, ...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Neck, Dissecti...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, B-PROBLEM, I-PROBLEM, I-PROBLEM, O, B-PROBLEM, I...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, HDR, Brachythe...|[O, O, O, O, O, O, O, O, O, O, O, O, B-TREATMENT, I-TREATMENT, O, O, B-TREATMENT, I-TREATMENT, O,...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Biopsy, -, Cer...|[O, O, O, O, O, O, O, O, O, O, O, O, B-TEST, O, O, O, O, O, O, B-TEST, I-TEST, I-TEST, I-TEST, I-...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Metastatic, Ov...|[O, O, O, O, O, O, O, O, O, O, O, O, B-PROBLEM, I-PROBLEM, I-PROBLEM, O, O, O, O, O, O, O, O, O, ...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Breast, Radiat...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, B-PROBLEM, I-PROBLEM, I-...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Lymph, Node, E...|[O, O, O, O, O, O, O, O, O, O, O, O, B-TEST, I-TEST, I-TEST, I-TEST, O, O, B-TEST, I-TEST, I-TEST...|\n","+----------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+\n","\n","CPU times: user 7.37 ms, sys: 343 µs, total: 7.71 ms\n","Wall time: 229 ms\n"]}],"source":["%%time\n","\n","result.select('token.result','ner.result').show(truncate=100)"]},{"cell_type":"code","execution_count":23,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":6,"status":"ok","timestamp":1712935131219,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"5uBnSe0--_Tn","outputId":"40eb1bf8-b97e-455d-f2c4-d7ddf7275966"},"outputs":[{"output_type":"stream","name":"stdout","text":["+-------------+---------+----------+\n","|        token|ner_label|confidence|\n","+-------------+---------+----------+\n","|       Sample|        O|    0.9996|\n","|         Type|        O|    0.9926|\n","|            /|        O|    0.9755|\n","|      Medical|        O|    0.9824|\n","|    Specialty|        O|    0.9993|\n","|            :|        O|       1.0|\n","|   Hematology|        O|    0.9971|\n","|            -|        O|    0.9908|\n","|     Oncology|        O|    0.9796|\n","|       Sample|        O|    0.9999|\n","|         Name|        O|    0.9994|\n","|            :|        O|       1.0|\n","|    Discharge|        O|    0.9998|\n","|      Summary|        O|    0.9926|\n","|            -|        O|    0.9959|\n","| Mesothelioma|        O|    0.8117|\n","|            -|        O|    0.9934|\n","|            1|        O|    0.7394|\n","|  Description|        O|    0.9995|\n","|            :|        O|    0.9999|\n","| Mesothelioma|B-PROBLEM|    0.9991|\n","|            ,|        O|    0.9999|\n","|      pleural|B-PROBLEM|    0.9911|\n","|     effusion|I-PROBLEM|    0.9982|\n","|            ,|        O|    0.9999|\n","|       atrial|B-PROBLEM|    0.9976|\n","| fibrillation|I-PROBLEM|    0.9982|\n","|            ,|        O|    0.9999|\n","|       anemia|B-PROBLEM|     0.999|\n","|            ,|        O|    0.9999|\n","|      ascites|B-PROBLEM|    0.9996|\n","|            ,|        O|       1.0|\n","|   esophageal|B-PROBLEM|    0.9982|\n","|       reflux|I-PROBLEM|    0.9976|\n","|            ,|        O|       1.0|\n","|          and|        O|    0.9996|\n","|      history|        O|    0.9892|\n","|           of|        O|    0.9769|\n","|         deep|B-PROBLEM|    0.9421|\n","|       venous|I-PROBLEM|    0.9624|\n","|   thrombosis|I-PROBLEM|    0.9983|\n","|            .|        O|       1.0|\n","|            (|        O|    0.9999|\n","|      Medical|        O|     0.986|\n","|Transcription|        O|    0.9214|\n","|       Sample|        O|     0.993|\n","|       Report|        O|    0.9536|\n","|            )|        O|    0.9802|\n","|    PRINCIPAL|        O|    0.9981|\n","|    DIAGNOSIS|        O|    0.4728|\n","+-------------+---------+----------+\n","only showing top 50 rows\n","\n"]}],"source":["result_df = result.select(F.explode(F.arrays_zip(result.token.result,result.ner.result, result.ner.metadata)).alias(\"cols\"))\\\n","                  .select(F.expr(\"cols['0']\").alias(\"token\"),\n","                          F.expr(\"cols['1']\").alias(\"ner_label\"),\n","                          F.expr(\"cols['2']['confidence']\").alias(\"confidence\"))\n","\n","result_df.show(50, truncate=100)"]},{"cell_type":"code","execution_count":24,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1079,"status":"ok","timestamp":1712935132294,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"Y4eM-zZcErru","outputId":"4c2f4004-bac7-4ff0-cfec-8b278584050a"},"outputs":[{"output_type":"stream","name":"stdout","text":["+-----------+-----+\n","|ner_label  |count|\n","+-----------+-----+\n","|O          |4721 |\n","|I-PROBLEM  |526  |\n","|B-PROBLEM  |399  |\n","|I-TREATMENT|326  |\n","|B-TREATMENT|265  |\n","|I-TEST     |173  |\n","|B-TEST     |142  |\n","+-----------+-----+\n","\n"]}],"source":["result_df.select(\"token\", \"ner_label\")\\\n","         .groupBy('ner_label').count()\\\n","         .orderBy('count', ascending=False)\\\n","         .show(truncate=False)"]},{"cell_type":"code","execution_count":25,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1064,"status":"ok","timestamp":1712935133356,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"uzkwbP21ydAK","outputId":"be50cc2f-5488-4397-e95a-c9e6aaa3bbd6"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["[Row(ner_chunk=[Row(annotatorType='chunk', begin=118, end=129, result='Mesothelioma', metadata={'sentence': '2', 'chunk': '0', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9991'}, embeddings=[]), Row(annotatorType='chunk', begin=132, end=147, result='pleural effusion', metadata={'sentence': '2', 'chunk': '1', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.99465'}, embeddings=[]), Row(annotatorType='chunk', begin=150, end=168, result='atrial fibrillation', metadata={'sentence': '2', 'chunk': '2', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9979'}, embeddings=[]), Row(annotatorType='chunk', begin=171, end=176, result='anemia', metadata={'sentence': '2', 'chunk': '3', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.999'}, embeddings=[]), Row(annotatorType='chunk', begin=179, end=185, result='ascites', metadata={'sentence': '2', 'chunk': '4', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9996'}, embeddings=[]), Row(annotatorType='chunk', begin=188, end=204, result='esophageal reflux', metadata={'sentence': '2', 'chunk': '5', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9979'}, embeddings=[]), Row(annotatorType='chunk', begin=222, end=243, result='deep venous thrombosis', metadata={'sentence': '2', 'chunk': '6', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.96760005'}, embeddings=[]), Row(annotatorType='chunk', begin=305, end=316, result='Mesothelioma', metadata={'sentence': '4', 'chunk': '7', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.987'}, embeddings=[]), Row(annotatorType='chunk', begin=340, end=355, result='Pleural effusion', metadata={'sentence': '5', 'chunk': '8', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.97895'}, embeddings=[]), Row(annotatorType='chunk', begin=358, end=376, result='atrial fibrillation', metadata={'sentence': '5', 'chunk': '9', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.99475'}, embeddings=[]), Row(annotatorType='chunk', begin=379, end=384, result='anemia', metadata={'sentence': '5', 'chunk': '10', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9984'}, embeddings=[]), Row(annotatorType='chunk', begin=387, end=393, result='ascites', metadata={'sentence': '5', 'chunk': '11', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9994'}, embeddings=[]), Row(annotatorType='chunk', begin=396, end=412, result='esophageal reflux', metadata={'sentence': '5', 'chunk': '12', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.99724996'}, embeddings=[]), Row(annotatorType='chunk', begin=430, end=451, result='deep venous thrombosis', metadata={'sentence': '5', 'chunk': '13', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9550001'}, embeddings=[]), Row(annotatorType='chunk', begin=488, end=512, result='decortication of the lung', metadata={'sentence': '8', 'chunk': '14', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.756325'}, embeddings=[]), Row(annotatorType='chunk', begin=519, end=532, result='pleural biopsy', metadata={'sentence': '8', 'chunk': '15', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.87715'}, embeddings=[]), Row(annotatorType='chunk', begin=538, end=561, result='transpleural fluoroscopy', metadata={'sentence': '8', 'chunk': '16', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.9527'}, embeddings=[]), Row(annotatorType='chunk', begin=587, end=599, result='thoracentesis', metadata={'sentence': '9', 'chunk': '17', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9075'}, embeddings=[]), Row(annotatorType='chunk', begin=625, end=645, result='Port-A-Cath placement', metadata={'sentence': '10', 'chunk': '18', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.98075'}, embeddings=[]), Row(annotatorType='chunk', begin=722, end=742, result='a nonproductive cough', metadata={'sentence': '11', 'chunk': '19', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.96173334'}, embeddings=[]), Row(annotatorType='chunk', begin=780, end=801, result='right-sided chest pain', metadata={'sentence': '12', 'chunk': '20', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.97246665'}, embeddings=[]), Row(annotatorType='chunk', begin=830, end=834, result='fever', metadata={'sentence': '12', 'chunk': '21', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9998'}, embeddings=[]), Row(annotatorType='chunk', begin=877, end=888, result='pericarditis', metadata={'sentence': '13', 'chunk': '22', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9228'}, embeddings=[]), Row(annotatorType='chunk', begin=894, end=907, result='pericardectomy', metadata={'sentence': '13', 'chunk': '23', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9991'}, embeddings=[]), Row(annotatorType='chunk', begin=935, end=939, result='cough', metadata={'sentence': '13', 'chunk': '24', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9977'}, embeddings=[]), Row(annotatorType='chunk', begin=946, end=967, result='right-sided chest pain', metadata={'sentence': '13', 'chunk': '25', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9766'}, embeddings=[]), Row(annotatorType='chunk', begin=1005, end=1015, result='Chest x-ray', metadata={'sentence': '14', 'chunk': '26', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.91905'}, embeddings=[]), Row(annotatorType='chunk', begin=1026, end=1053, result='right-sided pleural effusion', metadata={'sentence': '14', 'chunk': '27', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9479'}, embeddings=[]), Row(annotatorType='chunk', begin=1080, end=1093, result='Pericardectomy', metadata={'sentence': '16', 'chunk': '28', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9962'}, embeddings=[]), Row(annotatorType='chunk', begin=1099, end=1110, result='Pericarditis', metadata={'sentence': '17', 'chunk': '29', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9861'}, embeddings=[]), Row(annotatorType='chunk', begin=1116, end=1134, result='Atrial fibrillation', metadata={'sentence': '18', 'chunk': '30', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9803'}, embeddings=[]), Row(annotatorType='chunk', begin=1150, end=1184, result='intracranial thrombolytic treatment', metadata={'sentence': '19', 'chunk': '31', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9476333'}, embeddings=[]), Row(annotatorType='chunk', begin=1204, end=1231, result='Mesenteric venous thrombosis', metadata={'sentence': '21', 'chunk': '32', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9354334'}, embeddings=[]), Row(annotatorType='chunk', begin=1237, end=1254, result='Pericardial window', metadata={'sentence': '22', 'chunk': '33', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.6854'}, embeddings=[]), Row(annotatorType='chunk', begin=1260, end=1274, result='Cholecystectomy', metadata={'sentence': '23', 'chunk': '34', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9991'}, embeddings=[]), Row(annotatorType='chunk', begin=1280, end=1297, result='Left thoracentesis', metadata={'sentence': '24', 'chunk': '35', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.64185'}, embeddings=[]), Row(annotatorType='chunk', begin=1337, end=1359, result='coronary artery disease', metadata={'sentence': '25', 'chunk': '36', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9825001'}, embeddings=[]), Row(annotatorType='chunk', begin=1362, end=1364, result='CVA', metadata={'sentence': '25', 'chunk': '37', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9996'}, embeddings=[]), Row(annotatorType='chunk', begin=1367, end=1374, result='diabetes', metadata={'sentence': '25', 'chunk': '38', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9993'}, embeddings=[]), Row(annotatorType='chunk', begin=1377, end=1379, result='CHF', metadata={'sentence': '25', 'chunk': '39', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9997'}, embeddings=[]), Row(annotatorType='chunk', begin=1384, end=1385, result='MI', metadata={'sentence': '25', 'chunk': '40', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9996'}, embeddings=[]), Row(annotatorType='chunk', begin=1449, end=1454, result='cancer', metadata={'sentence': '26', 'chunk': '41', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9749'}, embeddings=[]), Row(annotatorType='chunk', begin=1609, end=1616, result='Coumadin', metadata={'sentence': '33', 'chunk': '42', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9997'}, embeddings=[]), Row(annotatorType='chunk', begin=1635, end=1637, result='INR', metadata={'sentence': '34', 'chunk': '43', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.7696'}, embeddings=[]), Row(annotatorType='chunk', begin=1676, end=1682, result='her INR', metadata={'sentence': '34', 'chunk': '44', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.79190004'}, embeddings=[]), Row(annotatorType='chunk', begin=1696, end=1705, result='Amiodarone', metadata={'sentence': '34', 'chunk': '45', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9998'}, embeddings=[]), Row(annotatorType='chunk', begin=1764, end=1770, result='systems', metadata={'sentence': '35', 'chunk': '46', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.5396'}, embeddings=[]), Row(annotatorType='chunk', begin=1845, end=1863, result='occasional numbness', metadata={'sentence': '36', 'chunk': '47', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.95554996'}, embeddings=[]), Row(annotatorType='chunk', begin=1869, end=1892, result='tingling of her left arm', metadata={'sentence': '36', 'chunk': '48', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.77458'}, embeddings=[]), Row(annotatorType='chunk', begin=1929, end=1942, result='Blood pressure', metadata={'sentence': '37', 'chunk': '49', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.9727'}, embeddings=[]), Row(annotatorType='chunk', begin=1952, end=1961, result='heart rate', metadata={'sentence': '37', 'chunk': '50', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.98315'}, embeddings=[]), Row(annotatorType='chunk', begin=1967, end=1978, result='respirations', metadata={'sentence': '37', 'chunk': '51', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '1.0'}, embeddings=[]), Row(annotatorType='chunk', begin=1984, end=1994, result='temperature', metadata={'sentence': '37', 'chunk': '52', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.9999'}, embeddings=[]), Row(annotatorType='chunk', begin=2004, end=2020, result='oxygen saturation', metadata={'sentence': '37', 'chunk': '53', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.97650003'}, embeddings=[]), Row(annotatorType='chunk', begin=2045, end=2063, result='nonproductive cough', metadata={'sentence': '38', 'chunk': '54', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.83985'}, embeddings=[]), Row(annotatorType='chunk', begin=2069, end=2072, result='pain', metadata={'sentence': '38', 'chunk': '55', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9951'}, embeddings=[]), Row(annotatorType='chunk', begin=2079, end=2086, result='coughing', metadata={'sentence': '38', 'chunk': '56', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9568'}, embeddings=[]), Row(annotatorType='chunk', begin=2201, end=2215, result='lymphadenopathy', metadata={'sentence': '42', 'chunk': '57', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9988'}, embeddings=[]), Row(annotatorType='chunk', begin=2221, end=2226, result='masses', metadata={'sentence': '43', 'chunk': '58', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9995'}, embeddings=[]), Row(annotatorType='chunk', begin=2242, end=2261, result='Pleural friction rub', metadata={'sentence': '44', 'chunk': '59', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.89523333'}, embeddings=[]), Row(annotatorType='chunk', begin=2283, end=2294, result='nondistended', metadata={'sentence': '45', 'chunk': '60', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9994'}, embeddings=[]), Row(annotatorType='chunk', begin=2301, end=2309, result='nontender', metadata={'sentence': '45', 'chunk': '61', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9991'}, embeddings=[]), Row(annotatorType='chunk', begin=2338, end=2349, result='organomegaly', metadata={'sentence': '47', 'chunk': '62', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9977'}, embeddings=[]), Row(annotatorType='chunk', begin=2368, end=2372, result='edema', metadata={'sentence': '48', 'chunk': '63', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9985'}, embeddings=[]), Row(annotatorType='chunk', begin=2378, end=2385, result='clubbing', metadata={'sentence': '48', 'chunk': '64', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9961'}, embeddings=[]), Row(annotatorType='chunk', begin=2391, end=2398, result='cyanosis', metadata={'sentence': '48', 'chunk': '65', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9984'}, embeddings=[]), Row(annotatorType='chunk', begin=2404, end=2413, result='tenderness', metadata={'sentence': '48', 'chunk': '66', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9995'}, embeddings=[]), Row(annotatorType='chunk', begin=2481, end=2489, result='breakdown', metadata={'sentence': '51', 'chunk': '67', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9982'}, embeddings=[]), Row(annotatorType='chunk', begin=2494, end=2500, result='lesions', metadata={'sentence': '51', 'chunk': '68', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9937'}, embeddings=[]), Row(annotatorType='chunk', begin=2506, end=2511, result='ulcers', metadata={'sentence': '52', 'chunk': '69', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9985'}, embeddings=[]), Row(annotatorType='chunk', begin=2545, end=2558, result='focal deficits', metadata={'sentence': '54', 'chunk': '70', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.93095005'}, embeddings=[]), Row(annotatorType='chunk', begin=2633, end=2636, result='Labs', metadata={'sentence': '56', 'chunk': '71', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.9843'}, embeddings=[]), Row(annotatorType='chunk', begin=2697, end=2726, result='a right-sided pleural effusion', metadata={'sentence': '58', 'chunk': '72', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.715325'}, embeddings=[]), Row(annotatorType='chunk', begin=2732, end=2744, result='thoracentesis', metadata={'sentence': '58', 'chunk': '73', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.8775'}, embeddings=[]), Row(annotatorType='chunk', begin=2766, end=2777, result='Her Coumadin', metadata={'sentence': '58', 'chunk': '74', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.98109996'}, embeddings=[]), Row(annotatorType='chunk', begin=2799, end=2821, result='A repeat echocardiogram', metadata={'sentence': '59', 'chunk': '75', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.8780667'}, embeddings=[]), Row(annotatorType='chunk', begin=2855, end=2865, result='prophylaxis', metadata={'sentence': '60', 'chunk': '76', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9996'}, embeddings=[]), Row(annotatorType='chunk', begin=2871, end=2873, result='DVT', metadata={'sentence': '60', 'chunk': '77', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9983'}, embeddings=[]), Row(annotatorType='chunk', begin=2880, end=2886, result='Lovenox', metadata={'sentence': '60', 'chunk': '78', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9982'}, embeddings=[]), Row(annotatorType='chunk', begin=3003, end=3014, result='pericarditis', metadata={'sentence': '61', 'chunk': '79', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.8911'}, embeddings=[]), Row(annotatorType='chunk', begin=3040, end=3057, result='pericardial window', metadata={'sentence': '61', 'chunk': '80', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.96784997'}, embeddings=[]), Row(annotatorType='chunk', begin=3123, end=3137, result='mesenteric pain', metadata={'sentence': '61', 'chunk': '81', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9893'}, embeddings=[]), Row(annotatorType='chunk', begin=3143, end=3152, result='thrombosis', metadata={'sentence': '61', 'chunk': '82', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9983'}, embeddings=[]), Row(annotatorType='chunk', begin=3162, end=3175, result='anticoagulated', metadata={'sentence': '61', 'chunk': '83', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9508'}, embeddings=[]), Row(annotatorType='chunk', begin=3281, end=3294, result='pericardectomy', metadata={'sentence': '62', 'chunk': '84', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9988'}, embeddings=[]), Row(annotatorType='chunk', begin=3337, end=3347, result='her CT scan', metadata={'sentence': '62', 'chunk': '85', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.9552667'}, embeddings=[]), Row(annotatorType='chunk', begin=3374, end=3391, result='her pericardectomy', metadata={'sentence': '62', 'chunk': '86', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.87944996'}, embeddings=[]), Row(annotatorType='chunk', begin=3408, end=3433, result='bilateral plural effusions', metadata={'sentence': '62', 'chunk': '87', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9345667'}, embeddings=[]), Row(annotatorType='chunk', begin=3474, end=3491, result='the pericardectomy', metadata={'sentence': '63', 'chunk': '88', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.98665'}, embeddings=[]), Row(annotatorType='chunk', begin=3512, end=3523, result='her symptoms', metadata={'sentence': '63', 'chunk': '89', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9017'}, embeddings=[]), Row(annotatorType='chunk', begin=3576, end=3585, result='chest pain', metadata={'sentence': '64', 'chunk': '90', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9956'}, embeddings=[]), Row(annotatorType='chunk', begin=3605, end=3630, result='bilateral pleural effusion', metadata={'sentence': '64', 'chunk': '91', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9514'}, embeddings=[]), Row(annotatorType='chunk', begin=3666, end=3680, result='CT of the chest', metadata={'sentence': '65', 'chunk': '92', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.853225'}, embeddings=[]), Row(annotatorType='chunk', begin=3696, end=3725, result='a large mediastinal lymph node', metadata={'sentence': '65', 'chunk': '93', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.8369201'}, embeddings=[]), Row(annotatorType='chunk', begin=3740, end=3752, result='the pathology', metadata={'sentence': '66', 'chunk': '94', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.44089997'}, embeddings=[]), Row(annotatorType='chunk', begin=3768, end=3785, result='the pericardectomy', metadata={'sentence': '66', 'chunk': '95', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9546'}, embeddings=[]), Row(annotatorType='chunk', begin=3826, end=3837, result='mesothelioma', metadata={'sentence': '66', 'chunk': '96', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.961'}, embeddings=[]), Row(annotatorType='chunk', begin=3854, end=3873, result='chest tube placement', metadata={'sentence': '67', 'chunk': '97', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9526333'}, embeddings=[]), Row(annotatorType='chunk', begin=3879, end=3899, result='drainage of the fluid', metadata={'sentence': '67', 'chunk': '98', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.7372'}, embeddings=[]), Row(annotatorType='chunk', begin=3914, end=3925, result='thoracoscopy', metadata={'sentence': '67', 'chunk': '99', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.6497'}, embeddings=[]), Row(annotatorType='chunk', begin=3932, end=3945, result='fluid biopsies', metadata={'sentence': '67', 'chunk': '100', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.7528'}, embeddings=[]), Row(annotatorType='chunk', begin=3985, end=4018, result='epithelioid malignant mesothelioma', metadata={'sentence': '67', 'chunk': '101', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.97339994'}, embeddings=[]), Row(annotatorType='chunk', begin=4055, end=4062, result='a PET CT', metadata={'sentence': '68', 'chunk': '102', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.94320005'}, embeddings=[]), Row(annotatorType='chunk', begin=4078, end=4106, result='extensive uptake in the chest', metadata={'sentence': '68', 'chunk': '103', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.90375996'}, embeddings=[]), Row(annotatorType='chunk', begin=4109, end=4147, result='bilateral pleural pericardial effusions', metadata={'sentence': '68', 'chunk': '104', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.96755'}, embeddings=[]), Row(annotatorType='chunk', begin=4154, end=4168, result='lymphadenopathy', metadata={'sentence': '68', 'chunk': '105', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9956'}, embeddings=[]), Row(annotatorType='chunk', begin=4184, end=4195, result='acidic fluid', metadata={'sentence': '69', 'chunk': '106', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.96854997'}, embeddings=[]), Row(annotatorType='chunk', begin=4198, end=4234, result='pectoral and intramammary lymph nodes', metadata={'sentence': '69', 'chunk': '107', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.748'}, embeddings=[]), Row(annotatorType='chunk', begin=4240, end=4251, result='uptake in L4', metadata={'sentence': '69', 'chunk': '108', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.8012333'}, embeddings=[]), Row(annotatorType='chunk', begin=4258, end=4260, result='SUV', metadata={'sentence': '69', 'chunk': '109', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.9959'}, embeddings=[]), Row(annotatorType='chunk', begin=4293, end=4309, result='stage III disease', metadata={'sentence': '70', 'chunk': '110', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.8552334'}, embeddings=[]), Row(annotatorType='chunk', begin=4312, end=4336, result='Her repeat echocardiogram', metadata={'sentence': '71', 'chunk': '111', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.8606667'}, embeddings=[]), Row(annotatorType='chunk', begin=4345, end=4364, result='an ejection fraction', metadata={'sentence': '71', 'chunk': '112', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.7115'}, embeddings=[]), Row(annotatorType='chunk', begin=4436, end=4447, result='chemotherapy', metadata={'sentence': '72', 'chunk': '113', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9994'}, embeddings=[]), Row(annotatorType='chunk', begin=4475, end=4483, result='cisplatin', metadata={'sentence': '72', 'chunk': '114', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9996'}, embeddings=[]), Row(annotatorType='chunk', begin=4526, end=4537, result='IV piggyback', metadata={'sentence': '72', 'chunk': '115', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.75020003'}, embeddings=[]), Row(annotatorType='chunk', begin=4574, end=4579, result='Alimta', metadata={'sentence': '72', 'chunk': '116', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9999'}, embeddings=[]), Row(annotatorType='chunk', begin=4624, end=4635, result='IV piggyback', metadata={'sentence': '72', 'chunk': '117', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.7633'}, embeddings=[]), Row(annotatorType='chunk', begin=4683, end=4695, result='a Port-A-Cath', metadata={'sentence': '73', 'chunk': '118', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9683'}, embeddings=[]), Row(annotatorType='chunk', begin=4709, end=4724, result='The chemotherapy', metadata={'sentence': '74', 'chunk': '119', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.94605'}, embeddings=[]), Row(annotatorType='chunk', begin=4814, end=4821, result='IV fluid', metadata={'sentence': '74', 'chunk': '120', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.88874996'}, embeddings=[]), Row(annotatorType='chunk', begin=4827, end=4828, result='IV', metadata={'sentence': '74', 'chunk': '121', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9877'}, embeddings=[]), Row(annotatorType='chunk', begin=4831, end=4845, result='Her Port-A-Cath', metadata={'sentence': '75', 'chunk': '122', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.97214997'}, embeddings=[]), Row(annotatorType='chunk', begin=4863, end=4869, result='heparin', metadata={'sentence': '75', 'chunk': '123', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9973'}, embeddings=[]), Row(annotatorType='chunk', begin=4894, end=4914, result='DISCHARGE MEDICATIONS', metadata={'sentence': '76', 'chunk': '124', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.76094997'}, embeddings=[]), Row(annotatorType='chunk', begin=4917, end=4922, result='Zofran', metadata={'sentence': '76', 'chunk': '125', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9997'}, embeddings=[]), Row(annotatorType='chunk', begin=4925, end=4933, result='Phenergan', metadata={'sentence': '76', 'chunk': '126', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9999'}, embeddings=[]), Row(annotatorType='chunk', begin=4936, end=4943, result='Coumadin', metadata={'sentence': '76', 'chunk': '127', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '1.0'}, embeddings=[]), Row(annotatorType='chunk', begin=4950, end=4956, result='Lovenox', metadata={'sentence': '76', 'chunk': '128', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9998'}, embeddings=[]), Row(annotatorType='chunk', begin=4963, end=4969, result='Vicodin', metadata={'sentence': '76', 'chunk': '129', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9989'}, embeddings=[]), Row(annotatorType='chunk', begin=5062, end=5068, result='her INR', metadata={'sentence': '77', 'chunk': '130', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'confidence': '0.93625'}, embeddings=[]), Row(annotatorType='chunk', begin=5201, end=5212, result='mesothelioma', metadata={'sentence': '79', 'chunk': '131', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9981'}, embeddings=[]), Row(annotatorType='chunk', begin=5215, end=5230, result='pleural effusion', metadata={'sentence': '79', 'chunk': '132', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.98370004'}, embeddings=[]), Row(annotatorType='chunk', begin=5233, end=5251, result='atrial fibrillation', metadata={'sentence': '79', 'chunk': '133', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9953'}, embeddings=[]), Row(annotatorType='chunk', begin=5254, end=5259, result='anemia', metadata={'sentence': '79', 'chunk': '134', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9981'}, embeddings=[]), Row(annotatorType='chunk', begin=5262, end=5268, result='ascites', metadata={'sentence': '79', 'chunk': '135', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.999'}, embeddings=[]), Row(annotatorType='chunk', begin=5271, end=5287, result='esophageal reflux', metadata={'sentence': '79', 'chunk': '136', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9963'}, embeddings=[]), Row(annotatorType='chunk', begin=5290, end=5311, result='deep venous thrombosis', metadata={'sentence': '79', 'chunk': '137', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.98296666'}, embeddings=[]), Row(annotatorType='chunk', begin=5314, end=5334, result='port-a-cath placement', metadata={'sentence': '79', 'chunk': '138', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.918'}, embeddings=[]), Row(annotatorType='chunk', begin=5337, end=5347, result='port a cath', metadata={'sentence': '79', 'chunk': '139', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.69156665'}, embeddings=[]), Row(annotatorType='chunk', begin=5350, end=5361, result='iv piggyback', metadata={'sentence': '79', 'chunk': '140', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9642'}, embeddings=[]), Row(annotatorType='chunk', begin=5364, end=5380, result='venous thrombosis', metadata={'sentence': '79', 'chunk': '141', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.90615'}, embeddings=[]), Row(annotatorType='chunk', begin=5383, end=5388, result='atrial', metadata={'sentence': '79', 'chunk': '142', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.6133'}, embeddings=[]), Row(annotatorType='chunk', begin=5391, end=5400, result='thrombosis', metadata={'sentence': '79', 'chunk': '143', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.9317'}, embeddings=[]), Row(annotatorType='chunk', begin=5403, end=5413, result='pericardial', metadata={'sentence': '79', 'chunk': '144', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.7272'}, embeddings=[]), Row(annotatorType='chunk', begin=5416, end=5430, result='lymphadenopathy', metadata={'sentence': '79', 'chunk': '145', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.8981'}, embeddings=[]), Row(annotatorType='chunk', begin=5433, end=5437, result='fluid', metadata={'sentence': '79', 'chunk': '146', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.6176'}, embeddings=[]), Row(annotatorType='chunk', begin=5440, end=5453, result='pericardectomy', metadata={'sentence': '79', 'chunk': '147', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'confidence': '0.9822'}, embeddings=[]), Row(annotatorType='chunk', begin=5456, end=5469, result='chest, pleural', metadata={'sentence': '79', 'chunk': '148', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'confidence': '0.7570999'}, embeddings=[])])]"]},"metadata":{},"execution_count":25}],"source":["result.select('ner_chunk').take(1)"]},{"cell_type":"code","execution_count":26,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":39,"status":"ok","timestamp":1712935133356,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"kprAk32Oi8Fo","outputId":"3665e55f-b4a3-404d-d0ab-c3af35389bb9"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["[{'annotatorType': 'chunk',\n","  'begin': 118,\n","  'end': 129,\n","  'result': 'Mesothelioma',\n","  'metadata': {'sentence': '2',\n","   'chunk': '0',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9991'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 132,\n","  'end': 147,\n","  'result': 'pleural effusion',\n","  'metadata': {'sentence': '2',\n","   'chunk': '1',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.99465'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 150,\n","  'end': 168,\n","  'result': 'atrial fibrillation',\n","  'metadata': {'sentence': '2',\n","   'chunk': '2',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9979'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 171,\n","  'end': 176,\n","  'result': 'anemia',\n","  'metadata': {'sentence': '2',\n","   'chunk': '3',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.999'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 179,\n","  'end': 185,\n","  'result': 'ascites',\n","  'metadata': {'sentence': '2',\n","   'chunk': '4',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9996'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 188,\n","  'end': 204,\n","  'result': 'esophageal reflux',\n","  'metadata': {'sentence': '2',\n","   'chunk': '5',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9979'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 222,\n","  'end': 243,\n","  'result': 'deep venous thrombosis',\n","  'metadata': {'sentence': '2',\n","   'chunk': '6',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.96760005'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 305,\n","  'end': 316,\n","  'result': 'Mesothelioma',\n","  'metadata': {'sentence': '4',\n","   'chunk': '7',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.987'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 340,\n","  'end': 355,\n","  'result': 'Pleural effusion',\n","  'metadata': {'sentence': '5',\n","   'chunk': '8',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.97895'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 358,\n","  'end': 376,\n","  'result': 'atrial fibrillation',\n","  'metadata': {'sentence': '5',\n","   'chunk': '9',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.99475'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 379,\n","  'end': 384,\n","  'result': 'anemia',\n","  'metadata': {'sentence': '5',\n","   'chunk': '10',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9984'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 387,\n","  'end': 393,\n","  'result': 'ascites',\n","  'metadata': {'sentence': '5',\n","   'chunk': '11',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9994'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 396,\n","  'end': 412,\n","  'result': 'esophageal reflux',\n","  'metadata': {'sentence': '5',\n","   'chunk': '12',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.99724996'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 430,\n","  'end': 451,\n","  'result': 'deep venous thrombosis',\n","  'metadata': {'sentence': '5',\n","   'chunk': '13',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9550001'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 488,\n","  'end': 512,\n","  'result': 'decortication of the lung',\n","  'metadata': {'sentence': '8',\n","   'chunk': '14',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.756325'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 519,\n","  'end': 532,\n","  'result': 'pleural biopsy',\n","  'metadata': {'sentence': '8',\n","   'chunk': '15',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.87715'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 538,\n","  'end': 561,\n","  'result': 'transpleural fluoroscopy',\n","  'metadata': {'sentence': '8',\n","   'chunk': '16',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.9527'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 587,\n","  'end': 599,\n","  'result': 'thoracentesis',\n","  'metadata': {'sentence': '9',\n","   'chunk': '17',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9075'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 625,\n","  'end': 645,\n","  'result': 'Port-A-Cath placement',\n","  'metadata': {'sentence': '10',\n","   'chunk': '18',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.98075'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 722,\n","  'end': 742,\n","  'result': 'a nonproductive cough',\n","  'metadata': {'sentence': '11',\n","   'chunk': '19',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.96173334'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 780,\n","  'end': 801,\n","  'result': 'right-sided chest pain',\n","  'metadata': {'sentence': '12',\n","   'chunk': '20',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.97246665'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 830,\n","  'end': 834,\n","  'result': 'fever',\n","  'metadata': {'sentence': '12',\n","   'chunk': '21',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9998'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 877,\n","  'end': 888,\n","  'result': 'pericarditis',\n","  'metadata': {'sentence': '13',\n","   'chunk': '22',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9228'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 894,\n","  'end': 907,\n","  'result': 'pericardectomy',\n","  'metadata': {'sentence': '13',\n","   'chunk': '23',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9991'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 935,\n","  'end': 939,\n","  'result': 'cough',\n","  'metadata': {'sentence': '13',\n","   'chunk': '24',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9977'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 946,\n","  'end': 967,\n","  'result': 'right-sided chest pain',\n","  'metadata': {'sentence': '13',\n","   'chunk': '25',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9766'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1005,\n","  'end': 1015,\n","  'result': 'Chest x-ray',\n","  'metadata': {'sentence': '14',\n","   'chunk': '26',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.91905'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1026,\n","  'end': 1053,\n","  'result': 'right-sided pleural effusion',\n","  'metadata': {'sentence': '14',\n","   'chunk': '27',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9479'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1080,\n","  'end': 1093,\n","  'result': 'Pericardectomy',\n","  'metadata': {'sentence': '16',\n","   'chunk': '28',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9962'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1099,\n","  'end': 1110,\n","  'result': 'Pericarditis',\n","  'metadata': {'sentence': '17',\n","   'chunk': '29',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9861'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1116,\n","  'end': 1134,\n","  'result': 'Atrial fibrillation',\n","  'metadata': {'sentence': '18',\n","   'chunk': '30',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9803'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1150,\n","  'end': 1184,\n","  'result': 'intracranial thrombolytic treatment',\n","  'metadata': {'sentence': '19',\n","   'chunk': '31',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9476333'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1204,\n","  'end': 1231,\n","  'result': 'Mesenteric venous thrombosis',\n","  'metadata': {'sentence': '21',\n","   'chunk': '32',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9354334'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1237,\n","  'end': 1254,\n","  'result': 'Pericardial window',\n","  'metadata': {'sentence': '22',\n","   'chunk': '33',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.6854'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1260,\n","  'end': 1274,\n","  'result': 'Cholecystectomy',\n","  'metadata': {'sentence': '23',\n","   'chunk': '34',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9991'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1280,\n","  'end': 1297,\n","  'result': 'Left thoracentesis',\n","  'metadata': {'sentence': '24',\n","   'chunk': '35',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.64185'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1337,\n","  'end': 1359,\n","  'result': 'coronary artery disease',\n","  'metadata': {'sentence': '25',\n","   'chunk': '36',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9825001'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1362,\n","  'end': 1364,\n","  'result': 'CVA',\n","  'metadata': {'sentence': '25',\n","   'chunk': '37',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9996'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1367,\n","  'end': 1374,\n","  'result': 'diabetes',\n","  'metadata': {'sentence': '25',\n","   'chunk': '38',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9993'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1377,\n","  'end': 1379,\n","  'result': 'CHF',\n","  'metadata': {'sentence': '25',\n","   'chunk': '39',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9997'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1384,\n","  'end': 1385,\n","  'result': 'MI',\n","  'metadata': {'sentence': '25',\n","   'chunk': '40',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9996'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1449,\n","  'end': 1454,\n","  'result': 'cancer',\n","  'metadata': {'sentence': '26',\n","   'chunk': '41',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9749'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1609,\n","  'end': 1616,\n","  'result': 'Coumadin',\n","  'metadata': {'sentence': '33',\n","   'chunk': '42',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9997'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1635,\n","  'end': 1637,\n","  'result': 'INR',\n","  'metadata': {'sentence': '34',\n","   'chunk': '43',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.7696'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1676,\n","  'end': 1682,\n","  'result': 'her INR',\n","  'metadata': {'sentence': '34',\n","   'chunk': '44',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.79190004'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1696,\n","  'end': 1705,\n","  'result': 'Amiodarone',\n","  'metadata': {'sentence': '34',\n","   'chunk': '45',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9998'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1764,\n","  'end': 1770,\n","  'result': 'systems',\n","  'metadata': {'sentence': '35',\n","   'chunk': '46',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.5396'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1845,\n","  'end': 1863,\n","  'result': 'occasional numbness',\n","  'metadata': {'sentence': '36',\n","   'chunk': '47',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.95554996'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1869,\n","  'end': 1892,\n","  'result': 'tingling of her left arm',\n","  'metadata': {'sentence': '36',\n","   'chunk': '48',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.77458'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1929,\n","  'end': 1942,\n","  'result': 'Blood pressure',\n","  'metadata': {'sentence': '37',\n","   'chunk': '49',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.9727'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1952,\n","  'end': 1961,\n","  'result': 'heart rate',\n","  'metadata': {'sentence': '37',\n","   'chunk': '50',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.98315'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1967,\n","  'end': 1978,\n","  'result': 'respirations',\n","  'metadata': {'sentence': '37',\n","   'chunk': '51',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '1.0'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 1984,\n","  'end': 1994,\n","  'result': 'temperature',\n","  'metadata': {'sentence': '37',\n","   'chunk': '52',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.9999'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2004,\n","  'end': 2020,\n","  'result': 'oxygen saturation',\n","  'metadata': {'sentence': '37',\n","   'chunk': '53',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.97650003'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2045,\n","  'end': 2063,\n","  'result': 'nonproductive cough',\n","  'metadata': {'sentence': '38',\n","   'chunk': '54',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.83985'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2069,\n","  'end': 2072,\n","  'result': 'pain',\n","  'metadata': {'sentence': '38',\n","   'chunk': '55',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9951'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2079,\n","  'end': 2086,\n","  'result': 'coughing',\n","  'metadata': {'sentence': '38',\n","   'chunk': '56',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9568'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2201,\n","  'end': 2215,\n","  'result': 'lymphadenopathy',\n","  'metadata': {'sentence': '42',\n","   'chunk': '57',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9988'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2221,\n","  'end': 2226,\n","  'result': 'masses',\n","  'metadata': {'sentence': '43',\n","   'chunk': '58',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9995'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2242,\n","  'end': 2261,\n","  'result': 'Pleural friction rub',\n","  'metadata': {'sentence': '44',\n","   'chunk': '59',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.89523333'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2283,\n","  'end': 2294,\n","  'result': 'nondistended',\n","  'metadata': {'sentence': '45',\n","   'chunk': '60',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9994'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2301,\n","  'end': 2309,\n","  'result': 'nontender',\n","  'metadata': {'sentence': '45',\n","   'chunk': '61',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9991'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2338,\n","  'end': 2349,\n","  'result': 'organomegaly',\n","  'metadata': {'sentence': '47',\n","   'chunk': '62',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9977'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2368,\n","  'end': 2372,\n","  'result': 'edema',\n","  'metadata': {'sentence': '48',\n","   'chunk': '63',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9985'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2378,\n","  'end': 2385,\n","  'result': 'clubbing',\n","  'metadata': {'sentence': '48',\n","   'chunk': '64',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9961'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2391,\n","  'end': 2398,\n","  'result': 'cyanosis',\n","  'metadata': {'sentence': '48',\n","   'chunk': '65',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9984'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2404,\n","  'end': 2413,\n","  'result': 'tenderness',\n","  'metadata': {'sentence': '48',\n","   'chunk': '66',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9995'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2481,\n","  'end': 2489,\n","  'result': 'breakdown',\n","  'metadata': {'sentence': '51',\n","   'chunk': '67',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9982'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2494,\n","  'end': 2500,\n","  'result': 'lesions',\n","  'metadata': {'sentence': '51',\n","   'chunk': '68',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9937'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2506,\n","  'end': 2511,\n","  'result': 'ulcers',\n","  'metadata': {'sentence': '52',\n","   'chunk': '69',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9985'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2545,\n","  'end': 2558,\n","  'result': 'focal deficits',\n","  'metadata': {'sentence': '54',\n","   'chunk': '70',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.93095005'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2633,\n","  'end': 2636,\n","  'result': 'Labs',\n","  'metadata': {'sentence': '56',\n","   'chunk': '71',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.9843'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2697,\n","  'end': 2726,\n","  'result': 'a right-sided pleural effusion',\n","  'metadata': {'sentence': '58',\n","   'chunk': '72',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.715325'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2732,\n","  'end': 2744,\n","  'result': 'thoracentesis',\n","  'metadata': {'sentence': '58',\n","   'chunk': '73',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.8775'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2766,\n","  'end': 2777,\n","  'result': 'Her Coumadin',\n","  'metadata': {'sentence': '58',\n","   'chunk': '74',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.98109996'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2799,\n","  'end': 2821,\n","  'result': 'A repeat echocardiogram',\n","  'metadata': {'sentence': '59',\n","   'chunk': '75',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.8780667'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2855,\n","  'end': 2865,\n","  'result': 'prophylaxis',\n","  'metadata': {'sentence': '60',\n","   'chunk': '76',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9996'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2871,\n","  'end': 2873,\n","  'result': 'DVT',\n","  'metadata': {'sentence': '60',\n","   'chunk': '77',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9983'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 2880,\n","  'end': 2886,\n","  'result': 'Lovenox',\n","  'metadata': {'sentence': '60',\n","   'chunk': '78',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9982'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3003,\n","  'end': 3014,\n","  'result': 'pericarditis',\n","  'metadata': {'sentence': '61',\n","   'chunk': '79',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.8911'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3040,\n","  'end': 3057,\n","  'result': 'pericardial window',\n","  'metadata': {'sentence': '61',\n","   'chunk': '80',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.96784997'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3123,\n","  'end': 3137,\n","  'result': 'mesenteric pain',\n","  'metadata': {'sentence': '61',\n","   'chunk': '81',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9893'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3143,\n","  'end': 3152,\n","  'result': 'thrombosis',\n","  'metadata': {'sentence': '61',\n","   'chunk': '82',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9983'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3162,\n","  'end': 3175,\n","  'result': 'anticoagulated',\n","  'metadata': {'sentence': '61',\n","   'chunk': '83',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9508'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3281,\n","  'end': 3294,\n","  'result': 'pericardectomy',\n","  'metadata': {'sentence': '62',\n","   'chunk': '84',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9988'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3337,\n","  'end': 3347,\n","  'result': 'her CT scan',\n","  'metadata': {'sentence': '62',\n","   'chunk': '85',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.9552667'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3374,\n","  'end': 3391,\n","  'result': 'her pericardectomy',\n","  'metadata': {'sentence': '62',\n","   'chunk': '86',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.87944996'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3408,\n","  'end': 3433,\n","  'result': 'bilateral plural effusions',\n","  'metadata': {'sentence': '62',\n","   'chunk': '87',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9345667'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3474,\n","  'end': 3491,\n","  'result': 'the pericardectomy',\n","  'metadata': {'sentence': '63',\n","   'chunk': '88',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.98665'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3512,\n","  'end': 3523,\n","  'result': 'her symptoms',\n","  'metadata': {'sentence': '63',\n","   'chunk': '89',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9017'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3576,\n","  'end': 3585,\n","  'result': 'chest pain',\n","  'metadata': {'sentence': '64',\n","   'chunk': '90',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9956'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3605,\n","  'end': 3630,\n","  'result': 'bilateral pleural effusion',\n","  'metadata': {'sentence': '64',\n","   'chunk': '91',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9514'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3666,\n","  'end': 3680,\n","  'result': 'CT of the chest',\n","  'metadata': {'sentence': '65',\n","   'chunk': '92',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.853225'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3696,\n","  'end': 3725,\n","  'result': 'a large mediastinal lymph node',\n","  'metadata': {'sentence': '65',\n","   'chunk': '93',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.8369201'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3740,\n","  'end': 3752,\n","  'result': 'the pathology',\n","  'metadata': {'sentence': '66',\n","   'chunk': '94',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.44089997'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3768,\n","  'end': 3785,\n","  'result': 'the pericardectomy',\n","  'metadata': {'sentence': '66',\n","   'chunk': '95',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9546'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3826,\n","  'end': 3837,\n","  'result': 'mesothelioma',\n","  'metadata': {'sentence': '66',\n","   'chunk': '96',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.961'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3854,\n","  'end': 3873,\n","  'result': 'chest tube placement',\n","  'metadata': {'sentence': '67',\n","   'chunk': '97',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9526333'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3879,\n","  'end': 3899,\n","  'result': 'drainage of the fluid',\n","  'metadata': {'sentence': '67',\n","   'chunk': '98',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.7372'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3914,\n","  'end': 3925,\n","  'result': 'thoracoscopy',\n","  'metadata': {'sentence': '67',\n","   'chunk': '99',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.6497'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3932,\n","  'end': 3945,\n","  'result': 'fluid biopsies',\n","  'metadata': {'sentence': '67',\n","   'chunk': '100',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.7528'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 3985,\n","  'end': 4018,\n","  'result': 'epithelioid malignant mesothelioma',\n","  'metadata': {'sentence': '67',\n","   'chunk': '101',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.97339994'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4055,\n","  'end': 4062,\n","  'result': 'a PET CT',\n","  'metadata': {'sentence': '68',\n","   'chunk': '102',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.94320005'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4078,\n","  'end': 4106,\n","  'result': 'extensive uptake in the chest',\n","  'metadata': {'sentence': '68',\n","   'chunk': '103',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.90375996'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4109,\n","  'end': 4147,\n","  'result': 'bilateral pleural pericardial effusions',\n","  'metadata': {'sentence': '68',\n","   'chunk': '104',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.96755'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4154,\n","  'end': 4168,\n","  'result': 'lymphadenopathy',\n","  'metadata': {'sentence': '68',\n","   'chunk': '105',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9956'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4184,\n","  'end': 4195,\n","  'result': 'acidic fluid',\n","  'metadata': {'sentence': '69',\n","   'chunk': '106',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.96854997'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4198,\n","  'end': 4234,\n","  'result': 'pectoral and intramammary lymph nodes',\n","  'metadata': {'sentence': '69',\n","   'chunk': '107',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.748'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4240,\n","  'end': 4251,\n","  'result': 'uptake in L4',\n","  'metadata': {'sentence': '69',\n","   'chunk': '108',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.8012333'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4258,\n","  'end': 4260,\n","  'result': 'SUV',\n","  'metadata': {'sentence': '69',\n","   'chunk': '109',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.9959'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4293,\n","  'end': 4309,\n","  'result': 'stage III disease',\n","  'metadata': {'sentence': '70',\n","   'chunk': '110',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.8552334'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4312,\n","  'end': 4336,\n","  'result': 'Her repeat echocardiogram',\n","  'metadata': {'sentence': '71',\n","   'chunk': '111',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.8606667'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4345,\n","  'end': 4364,\n","  'result': 'an ejection fraction',\n","  'metadata': {'sentence': '71',\n","   'chunk': '112',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.7115'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4436,\n","  'end': 4447,\n","  'result': 'chemotherapy',\n","  'metadata': {'sentence': '72',\n","   'chunk': '113',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9994'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4475,\n","  'end': 4483,\n","  'result': 'cisplatin',\n","  'metadata': {'sentence': '72',\n","   'chunk': '114',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9996'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4526,\n","  'end': 4537,\n","  'result': 'IV piggyback',\n","  'metadata': {'sentence': '72',\n","   'chunk': '115',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.75020003'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4574,\n","  'end': 4579,\n","  'result': 'Alimta',\n","  'metadata': {'sentence': '72',\n","   'chunk': '116',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9999'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4624,\n","  'end': 4635,\n","  'result': 'IV piggyback',\n","  'metadata': {'sentence': '72',\n","   'chunk': '117',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.7633'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4683,\n","  'end': 4695,\n","  'result': 'a Port-A-Cath',\n","  'metadata': {'sentence': '73',\n","   'chunk': '118',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9683'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4709,\n","  'end': 4724,\n","  'result': 'The chemotherapy',\n","  'metadata': {'sentence': '74',\n","   'chunk': '119',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.94605'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4814,\n","  'end': 4821,\n","  'result': 'IV fluid',\n","  'metadata': {'sentence': '74',\n","   'chunk': '120',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.88874996'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4827,\n","  'end': 4828,\n","  'result': 'IV',\n","  'metadata': {'sentence': '74',\n","   'chunk': '121',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9877'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4831,\n","  'end': 4845,\n","  'result': 'Her Port-A-Cath',\n","  'metadata': {'sentence': '75',\n","   'chunk': '122',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.97214997'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4863,\n","  'end': 4869,\n","  'result': 'heparin',\n","  'metadata': {'sentence': '75',\n","   'chunk': '123',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9973'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4894,\n","  'end': 4914,\n","  'result': 'DISCHARGE MEDICATIONS',\n","  'metadata': {'sentence': '76',\n","   'chunk': '124',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.76094997'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4917,\n","  'end': 4922,\n","  'result': 'Zofran',\n","  'metadata': {'sentence': '76',\n","   'chunk': '125',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9997'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4925,\n","  'end': 4933,\n","  'result': 'Phenergan',\n","  'metadata': {'sentence': '76',\n","   'chunk': '126',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9999'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4936,\n","  'end': 4943,\n","  'result': 'Coumadin',\n","  'metadata': {'sentence': '76',\n","   'chunk': '127',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '1.0'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4950,\n","  'end': 4956,\n","  'result': 'Lovenox',\n","  'metadata': {'sentence': '76',\n","   'chunk': '128',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9998'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 4963,\n","  'end': 4969,\n","  'result': 'Vicodin',\n","  'metadata': {'sentence': '76',\n","   'chunk': '129',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9989'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5062,\n","  'end': 5068,\n","  'result': 'her INR',\n","  'metadata': {'sentence': '77',\n","   'chunk': '130',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TEST',\n","   'confidence': '0.93625'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5201,\n","  'end': 5212,\n","  'result': 'mesothelioma',\n","  'metadata': {'sentence': '79',\n","   'chunk': '131',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9981'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5215,\n","  'end': 5230,\n","  'result': 'pleural effusion',\n","  'metadata': {'sentence': '79',\n","   'chunk': '132',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.98370004'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5233,\n","  'end': 5251,\n","  'result': 'atrial fibrillation',\n","  'metadata': {'sentence': '79',\n","   'chunk': '133',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9953'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5254,\n","  'end': 5259,\n","  'result': 'anemia',\n","  'metadata': {'sentence': '79',\n","   'chunk': '134',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9981'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5262,\n","  'end': 5268,\n","  'result': 'ascites',\n","  'metadata': {'sentence': '79',\n","   'chunk': '135',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.999'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5271,\n","  'end': 5287,\n","  'result': 'esophageal reflux',\n","  'metadata': {'sentence': '79',\n","   'chunk': '136',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9963'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5290,\n","  'end': 5311,\n","  'result': 'deep venous thrombosis',\n","  'metadata': {'sentence': '79',\n","   'chunk': '137',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.98296666'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5314,\n","  'end': 5334,\n","  'result': 'port-a-cath placement',\n","  'metadata': {'sentence': '79',\n","   'chunk': '138',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.918'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5337,\n","  'end': 5347,\n","  'result': 'port a cath',\n","  'metadata': {'sentence': '79',\n","   'chunk': '139',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.69156665'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5350,\n","  'end': 5361,\n","  'result': 'iv piggyback',\n","  'metadata': {'sentence': '79',\n","   'chunk': '140',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9642'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5364,\n","  'end': 5380,\n","  'result': 'venous thrombosis',\n","  'metadata': {'sentence': '79',\n","   'chunk': '141',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.90615'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5383,\n","  'end': 5388,\n","  'result': 'atrial',\n","  'metadata': {'sentence': '79',\n","   'chunk': '142',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.6133'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5391,\n","  'end': 5400,\n","  'result': 'thrombosis',\n","  'metadata': {'sentence': '79',\n","   'chunk': '143',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.9317'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5403,\n","  'end': 5413,\n","  'result': 'pericardial',\n","  'metadata': {'sentence': '79',\n","   'chunk': '144',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.7272'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5416,\n","  'end': 5430,\n","  'result': 'lymphadenopathy',\n","  'metadata': {'sentence': '79',\n","   'chunk': '145',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.8981'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5433,\n","  'end': 5437,\n","  'result': 'fluid',\n","  'metadata': {'sentence': '79',\n","   'chunk': '146',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.6176'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5440,\n","  'end': 5453,\n","  'result': 'pericardectomy',\n","  'metadata': {'sentence': '79',\n","   'chunk': '147',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'TREATMENT',\n","   'confidence': '0.9822'}},\n"," {'annotatorType': 'chunk',\n","  'begin': 5456,\n","  'end': 5469,\n","  'result': 'chest, pleural',\n","  'metadata': {'sentence': '79',\n","   'chunk': '148',\n","   'ner_source': 'ner_chunk',\n","   'entity': 'PROBLEM',\n","   'confidence': '0.7570999'}}]"]},"metadata":{},"execution_count":26}],"source":["row_list = [{'annotatorType': row.annotatorType,\n","             'begin': row.begin,\n","             'end': row.end,\n","             'result': row.result,\n","             'metadata': row.metadata}\n","            for row in result.select('ner_chunk').take(1)[0][0]\n","           ]\n","\n","row_list"]},{"cell_type":"code","execution_count":27,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":26,"status":"ok","timestamp":1712935133356,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"iICfEeNoAklD","outputId":"d20576ad-e225-4243-828d-94ad4996bd68"},"outputs":[{"output_type":"stream","name":"stdout","text":["+-------------------------+---------+----------+\n","|chunk                    |ner_label|confidence|\n","+-------------------------+---------+----------+\n","|Mesothelioma             |PROBLEM  |0.9991    |\n","|pleural effusion         |PROBLEM  |0.99465   |\n","|atrial fibrillation      |PROBLEM  |0.9979    |\n","|anemia                   |PROBLEM  |0.999     |\n","|ascites                  |PROBLEM  |0.9996    |\n","|esophageal reflux        |PROBLEM  |0.9979    |\n","|deep venous thrombosis   |PROBLEM  |0.96760005|\n","|Mesothelioma             |PROBLEM  |0.987     |\n","|Pleural effusion         |PROBLEM  |0.97895   |\n","|atrial fibrillation      |PROBLEM  |0.99475   |\n","|anemia                   |PROBLEM  |0.9984    |\n","|ascites                  |PROBLEM  |0.9994    |\n","|esophageal reflux        |PROBLEM  |0.99724996|\n","|deep venous thrombosis   |PROBLEM  |0.9550001 |\n","|decortication of the lung|TREATMENT|0.756325  |\n","|pleural biopsy           |TEST     |0.87715   |\n","|transpleural fluoroscopy |TEST     |0.9527    |\n","|thoracentesis            |TREATMENT|0.9075    |\n","|Port-A-Cath placement    |TREATMENT|0.98075   |\n","|a nonproductive cough    |PROBLEM  |0.96173334|\n","+-------------------------+---------+----------+\n","only showing top 20 rows\n","\n"]}],"source":["result.select(F.explode(F.arrays_zip(result.ner_chunk.result,\n","                                     result.ner_chunk.metadata)).alias(\"cols\")) \\\n","      .select(F.expr(\"cols['0']\").alias(\"chunk\"),\n","              F.expr(\"cols['1']['entity']\").alias(\"ner_label\"),\n","              F.expr(\"cols['1']['confidence']\").alias(\"confidence\")).show(truncate=False)"]},{"cell_type":"markdown","source":["For the alternative explosion method, please visit [Flattener](https://nlp.johnsnowlabs.com/docs/en/licensed_annotators#flattener)"],"metadata":{"id":"JPYUKF47ROgI"}},{"cell_type":"markdown","metadata":{"id":"loYjPzwKx5zW"},"source":["### LightPipelines\n","\n","LightPipelines are Spark NLP specific Pipelines, equivalent to Spark ML Pipeline, but meant to deal with smaller amounts of data. They’re useful working with small datasets, debugging results, or when running either training or prediction from an API that serves one-off requests.\n","\n","**It is nearly 10x faster than using Spark ML Pipeline**\n","\n","Syntax: `LightPipeline(someTrainedPipeline).annotate(someStringOrArray)`\n","\n","For more details, please check: https://medium.com/spark-nlp/spark-nlp-101-lightpipeline-a544e93f20f1"]},{"cell_type":"code","execution_count":28,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":783},"executionInfo":{"elapsed":1137,"status":"ok","timestamp":1712935134490,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"Z5q0BojoAsu3","outputId":"b674871f-23b1-4042-a1ff-66d8537a51e4"},"outputs":[{"output_type":"stream","name":"stdout","text":["\n","A 28-year-old female with a history of gestational diabetes mellitus diagnosed eight years prior to presentation and subsequent type two diabetes mellitus (T2DM), one prior episode of HTG-induced pancreatitis three years prior to presentation, and associated with an acute hepatitis, presented with a one-week history of polyuria, poor appetite, and vomiting.\n","She was on metformin, glipizide, and dapagliflozin for T2DM and atorvastatin and gemfibrozil for HTG. She had been on dapagliflozin for six months at the time of presentation.\n","Physical examination on presentation was significant for dry oral mucosa ; significantly , her abdominal examination was benign with no tenderness, guarding, or rigidity. Pertinent laboratory findings on admission were: serum glucose 111 mg/dl,  creatinine 0.4 mg/dL, triglycerides 508 mg/dL, total cholesterol 122 mg/dL, and venous pH 7.27.\n","\n"]},{"output_type":"execute_result","data":{"text/plain":["                                   chunks  begin  end sentence_id   entities\n","0           gestational diabetes mellitus     40   68           0    PROBLEM\n","1   subsequent type two diabetes mellitus    118  154           0    PROBLEM\n","2                                    T2DM    157  160           0    PROBLEM\n","3                HTG-induced pancreatitis    185  208           0    PROBLEM\n","4                      an acute hepatitis    265  282           0    PROBLEM\n","5                                polyuria    322  329           0    PROBLEM\n","6                           poor appetite    332  344           0    PROBLEM\n","7                                vomiting    351  358           0    PROBLEM\n","8                               metformin    372  380           1  TREATMENT\n","9                               glipizide    383  391           1  TREATMENT\n","10                          dapagliflozin    398  410           1  TREATMENT\n","11                                   T2DM    416  419           1    PROBLEM\n","12                           atorvastatin    425  436           1  TREATMENT\n","13                            gemfibrozil    442  452           1  TREATMENT\n","14                                    HTG    458  460           1    PROBLEM\n","15                          dapagliflozin    479  491           2  TREATMENT\n","16                   Physical examination    537  556           3       TEST\n","17                        dry oral mucosa    594  608           3    PROBLEM\n","18              her abdominal examination    628  652           3       TEST\n","19                             tenderness    673  682           3    PROBLEM"],"text/html":["\n","  <div id=\"df-7cd22d83-926d-4cba-acb9-8b38679ae0d5\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>chunks</th>\n","      <th>begin</th>\n","      <th>end</th>\n","      <th>sentence_id</th>\n","      <th>entities</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>gestational diabetes mellitus</td>\n","      <td>40</td>\n","      <td>68</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>subsequent type two diabetes mellitus</td>\n","      <td>118</td>\n","      <td>154</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>T2DM</td>\n","      <td>157</td>\n","      <td>160</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>HTG-induced pancreatitis</td>\n","      <td>185</td>\n","      <td>208</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>an acute hepatitis</td>\n","      <td>265</td>\n","      <td>282</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>polyuria</td>\n","      <td>322</td>\n","      <td>329</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>poor appetite</td>\n","      <td>332</td>\n","      <td>344</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>vomiting</td>\n","      <td>351</td>\n","      <td>358</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>metformin</td>\n","      <td>372</td>\n","      <td>380</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>glipizide</td>\n","      <td>383</td>\n","      <td>391</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>dapagliflozin</td>\n","      <td>398</td>\n","      <td>410</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>11</th>\n","      <td>T2DM</td>\n","      <td>416</td>\n","      <td>419</td>\n","      <td>1</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>12</th>\n","      <td>atorvastatin</td>\n","      <td>425</td>\n","      <td>436</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>13</th>\n","      <td>gemfibrozil</td>\n","      <td>442</td>\n","      <td>452</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>14</th>\n","      <td>HTG</td>\n","      <td>458</td>\n","      <td>460</td>\n","      <td>1</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>15</th>\n","      <td>dapagliflozin</td>\n","      <td>479</td>\n","      <td>491</td>\n","      <td>2</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>16</th>\n","      <td>Physical examination</td>\n","      <td>537</td>\n","      <td>556</td>\n","      <td>3</td>\n","      <td>TEST</td>\n","    </tr>\n","    <tr>\n","      <th>17</th>\n","      <td>dry oral mucosa</td>\n","      <td>594</td>\n","      <td>608</td>\n","      <td>3</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>18</th>\n","      <td>her abdominal examination</td>\n","      <td>628</td>\n","      <td>652</td>\n","      <td>3</td>\n","      <td>TEST</td>\n","    </tr>\n","    <tr>\n","      <th>19</th>\n","      <td>tenderness</td>\n","      <td>673</td>\n","      <td>682</td>\n","      <td>3</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7cd22d83-926d-4cba-acb9-8b38679ae0d5')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-7cd22d83-926d-4cba-acb9-8b38679ae0d5 button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-7cd22d83-926d-4cba-acb9-8b38679ae0d5');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-3fcfd7cd-bc8a-4a3b-9429-3d33c284733e\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-3fcfd7cd-bc8a-4a3b-9429-3d33c284733e')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-3fcfd7cd-bc8a-4a3b-9429-3d33c284733e button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"df_clinical","summary":"{\n  \"name\": \"df_clinical\",\n  \"rows\": 27,\n  \"fields\": [\n    {\n      \"column\": \"chunks\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 25,\n        \"samples\": [\n          \"metformin\",\n          \"her abdominal examination\",\n          \"gestational diabetes mellitus\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"begin\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 229,\n        \"min\": 40,\n        \"max\": 863,\n        \"num_unique_values\": 27,\n        \"samples\": [\n          372,\n          442,\n          383\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"end\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 227,\n        \"min\": 68,\n        \"max\": 871,\n        \"num_unique_values\": 27,\n        \"samples\": [\n          380,\n          452,\n          391\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"sentence_id\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"1\",\n          \"4\",\n          \"2\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"PROBLEM\",\n          \"TREATMENT\",\n          \"TEST\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":28}],"source":["# fullAnnotate in LightPipeline\n","\n","text = '''\n","A 28-year-old female with a history of gestational diabetes mellitus diagnosed eight years prior to presentation and subsequent type two diabetes mellitus (T2DM), one prior episode of HTG-induced pancreatitis three years prior to presentation, and associated with an acute hepatitis, presented with a one-week history of polyuria, poor appetite, and vomiting.\n","She was on metformin, glipizide, and dapagliflozin for T2DM and atorvastatin and gemfibrozil for HTG. She had been on dapagliflozin for six months at the time of presentation.\n","Physical examination on presentation was significant for dry oral mucosa ; significantly , her abdominal examination was benign with no tenderness, guarding, or rigidity. Pertinent laboratory findings on admission were: serum glucose 111 mg/dl,  creatinine 0.4 mg/dL, triglycerides 508 mg/dL, total cholesterol 122 mg/dL, and venous pH 7.27.\n","'''\n","\n","print (text)\n","\n","light_model = nlp.LightPipeline(model)\n","\n","light_result = light_model.fullAnnotate(text)\n","\n","\n","chunks = []\n","entities = []\n","sentence= []\n","begin = []\n","end = []\n","\n","for n in light_result[0]['ner_chunk']:\n","\n","    begin.append(n.begin)\n","    end.append(n.end)\n","    chunks.append(n.result)\n","    entities.append(n.metadata['entity'])\n","    sentence.append(n.metadata['sentence'])\n","\n","\n","\n","df_clinical = pd.DataFrame({'chunks':chunks,\n","                            'begin': begin,\n","                            'end':end,\n","                            'sentence_id':sentence,\n","                            'entities':entities})\n","\n","df_clinical.head(20)"]},{"cell_type":"code","execution_count":29,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":8803,"status":"ok","timestamp":1712935143289,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"-r7yfZitmbPT","outputId":"e6d3ac0b-3d19-467f-aa5c-0105a1c7f929"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'document': [Annotation(document, 0, 878, \n","  A 28-year-old female with a history of gestational diabetes mellitus diagnosed eight years prior to presentation and subsequent type two diabetes mellitus (T2DM), one prior episode of HTG-induced pancreatitis three years prior to presentation, and associated with an acute hepatitis, presented with a one-week history of polyuria, poor appetite, and vomiting.\n","  She was on metformin, glipizide, and dapagliflozin for T2DM and atorvastatin and gemfibrozil for HTG. She had been on dapagliflozin for six months at the time of presentation.\n","  Physical examination on presentation was significant for dry oral mucosa ; significantly , her abdominal examination was benign with no tenderness, guarding, or rigidity. Pertinent laboratory findings on admission were: serum glucose 111 mg/dl,  creatinine 0.4 mg/dL, triglycerides 508 mg/dL, total cholesterol 122 mg/dL, and venous pH 7.27.\n","  , {}, [])],\n"," 'ner_chunk': [Annotation(chunk, 40, 68, gestational diabetes mellitus, {'chunk': '0', 'confidence': '0.91976666', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'sentence': '0'}, []),\n","  Annotation(chunk, 118, 154, subsequent type two diabetes mellitus, {'chunk': '1', 'confidence': '0.75924003', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'sentence': '0'}, []),\n","  Annotation(chunk, 157, 160, T2DM, {'chunk': '2', 'confidence': '0.9917', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'sentence': '0'}, []),\n","  Annotation(chunk, 185, 208, HTG-induced pancreatitis, {'chunk': '3', 'confidence': '0.97535', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'sentence': '0'}, []),\n","  Annotation(chunk, 265, 282, an acute hepatitis, {'chunk': '4', 'confidence': '0.9440667', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'sentence': '0'}, []),\n","  Annotation(chunk, 322, 329, polyuria, {'chunk': '5', 'confidence': '0.9728', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'sentence': '0'}, []),\n","  Annotation(chunk, 332, 344, poor appetite, {'chunk': '6', 'confidence': '0.9934', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'sentence': '0'}, []),\n","  Annotation(chunk, 351, 358, vomiting, {'chunk': '7', 'confidence': '0.9854', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'sentence': '0'}, []),\n","  Annotation(chunk, 372, 380, metformin, {'chunk': '8', 'confidence': '0.9998', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'sentence': '1'}, []),\n","  Annotation(chunk, 383, 391, glipizide, {'chunk': '9', 'confidence': '0.9999', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'sentence': '1'}, []),\n","  Annotation(chunk, 398, 410, dapagliflozin, {'chunk': '10', 'confidence': '1.0', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'sentence': '1'}, []),\n","  Annotation(chunk, 416, 419, T2DM, {'chunk': '11', 'confidence': '0.9995', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'sentence': '1'}, []),\n","  Annotation(chunk, 425, 436, atorvastatin, {'chunk': '12', 'confidence': '0.9999', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'sentence': '1'}, []),\n","  Annotation(chunk, 442, 452, gemfibrozil, {'chunk': '13', 'confidence': '1.0', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'sentence': '1'}, []),\n","  Annotation(chunk, 458, 460, HTG, {'chunk': '14', 'confidence': '0.9994', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'sentence': '1'}, []),\n","  Annotation(chunk, 479, 491, dapagliflozin, {'chunk': '15', 'confidence': '0.9999', 'ner_source': 'ner_chunk', 'entity': 'TREATMENT', 'sentence': '2'}, []),\n","  Annotation(chunk, 537, 556, Physical examination, {'chunk': '16', 'confidence': '0.9448', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'sentence': '3'}, []),\n","  Annotation(chunk, 594, 608, dry oral mucosa, {'chunk': '17', 'confidence': '0.7242333', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'sentence': '3'}, []),\n","  Annotation(chunk, 628, 652, her abdominal examination, {'chunk': '18', 'confidence': '0.8749666', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'sentence': '3'}, []),\n","  Annotation(chunk, 673, 682, tenderness, {'chunk': '19', 'confidence': '0.9996', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'sentence': '3'}, []),\n","  Annotation(chunk, 685, 692, guarding, {'chunk': '20', 'confidence': '0.9989', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'sentence': '3'}, []),\n","  Annotation(chunk, 698, 705, rigidity, {'chunk': '21', 'confidence': '0.9986', 'ner_source': 'ner_chunk', 'entity': 'PROBLEM', 'sentence': '3'}, []),\n","  Annotation(chunk, 757, 769, serum glucose, {'chunk': '22', 'confidence': '0.969', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'sentence': '4'}, []),\n","  Annotation(chunk, 783, 792, creatinine, {'chunk': '23', 'confidence': '0.9994', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'sentence': '4'}, []),\n","  Annotation(chunk, 805, 817, triglycerides, {'chunk': '24', 'confidence': '0.9999', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'sentence': '4'}, []),\n","  Annotation(chunk, 830, 846, total cholesterol, {'chunk': '25', 'confidence': '0.9827', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'sentence': '4'}, []),\n","  Annotation(chunk, 863, 871, venous pH, {'chunk': '26', 'confidence': '0.97185004', 'ner_source': 'ner_chunk', 'entity': 'TEST', 'sentence': '4'}, [])],\n"," 'token': [Annotation(token, 1, 1, A, {'sentence': '0'}, []),\n","  Annotation(token, 3, 13, 28-year-old, {'sentence': '0'}, []),\n","  Annotation(token, 15, 20, female, {'sentence': '0'}, []),\n","  Annotation(token, 22, 25, with, {'sentence': '0'}, []),\n","  Annotation(token, 27, 27, a, {'sentence': '0'}, []),\n","  Annotation(token, 29, 35, history, {'sentence': '0'}, []),\n","  Annotation(token, 37, 38, of, {'sentence': '0'}, []),\n","  Annotation(token, 40, 50, gestational, {'sentence': '0'}, []),\n","  Annotation(token, 52, 59, diabetes, {'sentence': '0'}, []),\n","  Annotation(token, 61, 68, mellitus, {'sentence': '0'}, []),\n","  Annotation(token, 70, 78, diagnosed, {'sentence': '0'}, []),\n","  Annotation(token, 80, 84, eight, {'sentence': '0'}, []),\n","  Annotation(token, 86, 90, years, {'sentence': '0'}, []),\n","  Annotation(token, 92, 96, prior, {'sentence': '0'}, []),\n","  Annotation(token, 98, 99, to, {'sentence': '0'}, []),\n","  Annotation(token, 101, 112, presentation, {'sentence': '0'}, []),\n","  Annotation(token, 114, 116, and, {'sentence': '0'}, []),\n","  Annotation(token, 118, 127, subsequent, {'sentence': '0'}, []),\n","  Annotation(token, 129, 132, type, {'sentence': '0'}, []),\n","  Annotation(token, 134, 136, two, {'sentence': '0'}, []),\n","  Annotation(token, 138, 145, diabetes, {'sentence': '0'}, []),\n","  Annotation(token, 147, 154, mellitus, {'sentence': '0'}, []),\n","  Annotation(token, 156, 156, (, {'sentence': '0'}, []),\n","  Annotation(token, 157, 160, T2DM, {'sentence': '0'}, []),\n","  Annotation(token, 161, 162, ),, {'sentence': '0'}, []),\n","  Annotation(token, 164, 166, one, {'sentence': '0'}, []),\n","  Annotation(token, 168, 172, prior, {'sentence': '0'}, []),\n","  Annotation(token, 174, 180, episode, {'sentence': '0'}, []),\n","  Annotation(token, 182, 183, of, {'sentence': '0'}, []),\n","  Annotation(token, 185, 195, HTG-induced, {'sentence': '0'}, []),\n","  Annotation(token, 197, 208, pancreatitis, {'sentence': '0'}, []),\n","  Annotation(token, 210, 214, three, {'sentence': '0'}, []),\n","  Annotation(token, 216, 220, years, {'sentence': '0'}, []),\n","  Annotation(token, 222, 226, prior, {'sentence': '0'}, []),\n","  Annotation(token, 228, 229, to, {'sentence': '0'}, []),\n","  Annotation(token, 231, 242, presentation, {'sentence': '0'}, []),\n","  Annotation(token, 243, 243, ,, {'sentence': '0'}, []),\n","  Annotation(token, 245, 247, and, {'sentence': '0'}, []),\n","  Annotation(token, 249, 258, associated, {'sentence': '0'}, []),\n","  Annotation(token, 260, 263, with, {'sentence': '0'}, []),\n","  Annotation(token, 265, 266, an, {'sentence': '0'}, []),\n","  Annotation(token, 268, 272, acute, {'sentence': '0'}, []),\n","  Annotation(token, 274, 282, hepatitis, {'sentence': '0'}, []),\n","  Annotation(token, 283, 283, ,, {'sentence': '0'}, []),\n","  Annotation(token, 285, 293, presented, {'sentence': '0'}, []),\n","  Annotation(token, 295, 298, with, {'sentence': '0'}, []),\n","  Annotation(token, 300, 300, a, {'sentence': '0'}, []),\n","  Annotation(token, 302, 309, one-week, {'sentence': '0'}, []),\n","  Annotation(token, 311, 317, history, {'sentence': '0'}, []),\n","  Annotation(token, 319, 320, of, {'sentence': '0'}, []),\n","  Annotation(token, 322, 329, polyuria, {'sentence': '0'}, []),\n","  Annotation(token, 330, 330, ,, {'sentence': '0'}, []),\n","  Annotation(token, 332, 335, poor, {'sentence': '0'}, []),\n","  Annotation(token, 337, 344, appetite, {'sentence': '0'}, []),\n","  Annotation(token, 345, 345, ,, {'sentence': '0'}, []),\n","  Annotation(token, 347, 349, and, {'sentence': '0'}, []),\n","  Annotation(token, 351, 358, vomiting, {'sentence': '0'}, []),\n","  Annotation(token, 359, 359, ., {'sentence': '0'}, []),\n","  Annotation(token, 361, 363, She, {'sentence': '1'}, []),\n","  Annotation(token, 365, 367, was, {'sentence': '1'}, []),\n","  Annotation(token, 369, 370, on, {'sentence': '1'}, []),\n","  Annotation(token, 372, 380, metformin, {'sentence': '1'}, []),\n","  Annotation(token, 381, 381, ,, {'sentence': '1'}, []),\n","  Annotation(token, 383, 391, glipizide, {'sentence': '1'}, []),\n","  Annotation(token, 392, 392, ,, {'sentence': '1'}, []),\n","  Annotation(token, 394, 396, and, {'sentence': '1'}, []),\n","  Annotation(token, 398, 410, dapagliflozin, {'sentence': '1'}, []),\n","  Annotation(token, 412, 414, for, {'sentence': '1'}, []),\n","  Annotation(token, 416, 419, T2DM, {'sentence': '1'}, []),\n","  Annotation(token, 421, 423, and, {'sentence': '1'}, []),\n","  Annotation(token, 425, 436, atorvastatin, {'sentence': '1'}, []),\n","  Annotation(token, 438, 440, and, {'sentence': '1'}, []),\n","  Annotation(token, 442, 452, gemfibrozil, {'sentence': '1'}, []),\n","  Annotation(token, 454, 456, for, {'sentence': '1'}, []),\n","  Annotation(token, 458, 460, HTG, {'sentence': '1'}, []),\n","  Annotation(token, 461, 461, ., {'sentence': '1'}, []),\n","  Annotation(token, 463, 465, She, {'sentence': '2'}, []),\n","  Annotation(token, 467, 469, had, {'sentence': '2'}, []),\n","  Annotation(token, 471, 474, been, {'sentence': '2'}, []),\n","  Annotation(token, 476, 477, on, {'sentence': '2'}, []),\n","  Annotation(token, 479, 491, dapagliflozin, {'sentence': '2'}, []),\n","  Annotation(token, 493, 495, for, {'sentence': '2'}, []),\n","  Annotation(token, 497, 499, six, {'sentence': '2'}, []),\n","  Annotation(token, 501, 506, months, {'sentence': '2'}, []),\n","  Annotation(token, 508, 509, at, {'sentence': '2'}, []),\n","  Annotation(token, 511, 513, the, {'sentence': '2'}, []),\n","  Annotation(token, 515, 518, time, {'sentence': '2'}, []),\n","  Annotation(token, 520, 521, of, {'sentence': '2'}, []),\n","  Annotation(token, 523, 534, presentation, {'sentence': '2'}, []),\n","  Annotation(token, 535, 535, ., {'sentence': '2'}, []),\n","  Annotation(token, 537, 544, Physical, {'sentence': '3'}, []),\n","  Annotation(token, 546, 556, examination, {'sentence': '3'}, []),\n","  Annotation(token, 558, 559, on, {'sentence': '3'}, []),\n","  Annotation(token, 561, 572, presentation, {'sentence': '3'}, []),\n","  Annotation(token, 574, 576, was, {'sentence': '3'}, []),\n","  Annotation(token, 578, 588, significant, {'sentence': '3'}, []),\n","  Annotation(token, 590, 592, for, {'sentence': '3'}, []),\n","  Annotation(token, 594, 596, dry, {'sentence': '3'}, []),\n","  Annotation(token, 598, 601, oral, {'sentence': '3'}, []),\n","  Annotation(token, 603, 608, mucosa, {'sentence': '3'}, []),\n","  Annotation(token, 610, 610, ;, {'sentence': '3'}, []),\n","  Annotation(token, 612, 624, significantly, {'sentence': '3'}, []),\n","  Annotation(token, 626, 626, ,, {'sentence': '3'}, []),\n","  Annotation(token, 628, 630, her, {'sentence': '3'}, []),\n","  Annotation(token, 632, 640, abdominal, {'sentence': '3'}, []),\n","  Annotation(token, 642, 652, examination, {'sentence': '3'}, []),\n","  Annotation(token, 654, 656, was, {'sentence': '3'}, []),\n","  Annotation(token, 658, 663, benign, {'sentence': '3'}, []),\n","  Annotation(token, 665, 668, with, {'sentence': '3'}, []),\n","  Annotation(token, 670, 671, no, {'sentence': '3'}, []),\n","  Annotation(token, 673, 682, tenderness, {'sentence': '3'}, []),\n","  Annotation(token, 683, 683, ,, {'sentence': '3'}, []),\n","  Annotation(token, 685, 692, guarding, {'sentence': '3'}, []),\n","  Annotation(token, 693, 693, ,, {'sentence': '3'}, []),\n","  Annotation(token, 695, 696, or, {'sentence': '3'}, []),\n","  Annotation(token, 698, 705, rigidity, {'sentence': '3'}, []),\n","  Annotation(token, 706, 706, ., {'sentence': '3'}, []),\n","  Annotation(token, 708, 716, Pertinent, {'sentence': '4'}, []),\n","  Annotation(token, 718, 727, laboratory, {'sentence': '4'}, []),\n","  Annotation(token, 729, 736, findings, {'sentence': '4'}, []),\n","  Annotation(token, 738, 739, on, {'sentence': '4'}, []),\n","  Annotation(token, 741, 749, admission, {'sentence': '4'}, []),\n","  Annotation(token, 751, 754, were, {'sentence': '4'}, []),\n","  Annotation(token, 755, 755, :, {'sentence': '4'}, []),\n","  Annotation(token, 757, 761, serum, {'sentence': '4'}, []),\n","  Annotation(token, 763, 769, glucose, {'sentence': '4'}, []),\n","  Annotation(token, 771, 773, 111, {'sentence': '4'}, []),\n","  Annotation(token, 775, 779, mg/dl, {'sentence': '4'}, []),\n","  Annotation(token, 780, 780, ,, {'sentence': '4'}, []),\n","  Annotation(token, 783, 792, creatinine, {'sentence': '4'}, []),\n","  Annotation(token, 794, 796, 0.4, {'sentence': '4'}, []),\n","  Annotation(token, 798, 802, mg/dL, {'sentence': '4'}, []),\n","  Annotation(token, 803, 803, ,, {'sentence': '4'}, []),\n","  Annotation(token, 805, 817, triglycerides, {'sentence': '4'}, []),\n","  Annotation(token, 819, 821, 508, {'sentence': '4'}, []),\n","  Annotation(token, 823, 827, mg/dL, {'sentence': '4'}, []),\n","  Annotation(token, 828, 828, ,, {'sentence': '4'}, []),\n","  Annotation(token, 830, 834, total, {'sentence': '4'}, []),\n","  Annotation(token, 836, 846, cholesterol, {'sentence': '4'}, []),\n","  Annotation(token, 848, 850, 122, {'sentence': '4'}, []),\n","  Annotation(token, 852, 856, mg/dL, {'sentence': '4'}, []),\n","  Annotation(token, 857, 857, ,, {'sentence': '4'}, []),\n","  Annotation(token, 859, 861, and, {'sentence': '4'}, []),\n","  Annotation(token, 863, 868, venous, {'sentence': '4'}, []),\n","  Annotation(token, 870, 871, pH, {'sentence': '4'}, []),\n","  Annotation(token, 873, 876, 7.27, {'sentence': '4'}, []),\n","  Annotation(token, 877, 877, ., {'sentence': '4'}, [])],\n"," 'ner': [Annotation(named_entity, 1, 1, O, {'word': 'A', 'confidence': '0.9998', 'sentence': '0'}, []),\n","  Annotation(named_entity, 3, 13, O, {'word': '28-year-old', 'confidence': '0.9991', 'sentence': '0'}, []),\n","  Annotation(named_entity, 15, 20, O, {'word': 'female', 'confidence': '0.996', 'sentence': '0'}, []),\n","  Annotation(named_entity, 22, 25, O, {'word': 'with', 'confidence': '0.9998', 'sentence': '0'}, []),\n","  Annotation(named_entity, 27, 27, O, {'word': 'a', 'confidence': '0.9989', 'sentence': '0'}, []),\n","  Annotation(named_entity, 29, 35, O, {'word': 'history', 'confidence': '0.8505', 'sentence': '0'}, []),\n","  Annotation(named_entity, 37, 38, O, {'word': 'of', 'confidence': '0.9632', 'sentence': '0'}, []),\n","  Annotation(named_entity, 40, 50, B-PROBLEM, {'word': 'gestational', 'confidence': '0.8178', 'sentence': '0'}, []),\n","  Annotation(named_entity, 52, 59, I-PROBLEM, {'word': 'diabetes', 'confidence': '0.9808', 'sentence': '0'}, []),\n","  Annotation(named_entity, 61, 68, I-PROBLEM, {'word': 'mellitus', 'confidence': '0.9607', 'sentence': '0'}, []),\n","  Annotation(named_entity, 70, 78, O, {'word': 'diagnosed', 'confidence': '0.9975', 'sentence': '0'}, []),\n","  Annotation(named_entity, 80, 84, O, {'word': 'eight', 'confidence': '0.9996', 'sentence': '0'}, []),\n","  Annotation(named_entity, 86, 90, O, {'word': 'years', 'confidence': '0.9979', 'sentence': '0'}, []),\n","  Annotation(named_entity, 92, 96, O, {'word': 'prior', 'confidence': '0.9993', 'sentence': '0'}, []),\n","  Annotation(named_entity, 98, 99, O, {'word': 'to', 'confidence': '0.9913', 'sentence': '0'}, []),\n","  Annotation(named_entity, 101, 112, O, {'word': 'presentation', 'confidence': '0.9343', 'sentence': '0'}, []),\n","  Annotation(named_entity, 114, 116, O, {'word': 'and', 'confidence': '0.9998', 'sentence': '0'}, []),\n","  Annotation(named_entity, 118, 127, B-PROBLEM, {'word': 'subsequent', 'confidence': '0.8527', 'sentence': '0'}, []),\n","  Annotation(named_entity, 129, 132, I-PROBLEM, {'word': 'type', 'confidence': '0.471', 'sentence': '0'}, []),\n","  Annotation(named_entity, 134, 136, I-PROBLEM, {'word': 'two', 'confidence': '0.5433', 'sentence': '0'}, []),\n","  Annotation(named_entity, 138, 145, I-PROBLEM, {'word': 'diabetes', 'confidence': '0.9388', 'sentence': '0'}, []),\n","  Annotation(named_entity, 147, 154, I-PROBLEM, {'word': 'mellitus', 'confidence': '0.9904', 'sentence': '0'}, []),\n","  Annotation(named_entity, 156, 156, O, {'word': '(', 'confidence': '0.9939', 'sentence': '0'}, []),\n","  Annotation(named_entity, 157, 160, B-PROBLEM, {'word': 'T2DM', 'confidence': '0.9917', 'sentence': '0'}, []),\n","  Annotation(named_entity, 161, 162, O, {'word': '),', 'confidence': '0.9993', 'sentence': '0'}, []),\n","  Annotation(named_entity, 164, 166, O, {'word': 'one', 'confidence': '0.9956', 'sentence': '0'}, []),\n","  Annotation(named_entity, 168, 172, O, {'word': 'prior', 'confidence': '0.9531', 'sentence': '0'}, []),\n","  Annotation(named_entity, 174, 180, O, {'word': 'episode', 'confidence': '0.8323', 'sentence': '0'}, []),\n","  Annotation(named_entity, 182, 183, O, {'word': 'of', 'confidence': '0.9596', 'sentence': '0'}, []),\n","  Annotation(named_entity, 185, 195, B-PROBLEM, {'word': 'HTG-induced', 'confidence': '0.9555', 'sentence': '0'}, []),\n","  Annotation(named_entity, 197, 208, I-PROBLEM, {'word': 'pancreatitis', 'confidence': '0.9952', 'sentence': '0'}, []),\n","  Annotation(named_entity, 210, 214, O, {'word': 'three', 'confidence': '0.9999', 'sentence': '0'}, []),\n","  Annotation(named_entity, 216, 220, O, {'word': 'years', 'confidence': '0.9938', 'sentence': '0'}, []),\n","  Annotation(named_entity, 222, 226, O, {'word': 'prior', 'confidence': '0.9984', 'sentence': '0'}, []),\n","  Annotation(named_entity, 228, 229, O, {'word': 'to', 'confidence': '0.9836', 'sentence': '0'}, []),\n","  Annotation(named_entity, 231, 242, O, {'word': 'presentation', 'confidence': '0.7049', 'sentence': '0'}, []),\n","  Annotation(named_entity, 243, 243, O, {'word': ',', 'confidence': '0.9999', 'sentence': '0'}, []),\n","  Annotation(named_entity, 245, 247, O, {'word': 'and', 'confidence': '0.9974', 'sentence': '0'}, []),\n","  Annotation(named_entity, 249, 258, O, {'word': 'associated', 'confidence': '0.9614', 'sentence': '0'}, []),\n","  Annotation(named_entity, 260, 263, O, {'word': 'with', 'confidence': '0.9991', 'sentence': '0'}, []),\n","  Annotation(named_entity, 265, 266, B-PROBLEM, {'word': 'an', 'confidence': '0.9067', 'sentence': '0'}, []),\n","  Annotation(named_entity, 268, 272, I-PROBLEM, {'word': 'acute', 'confidence': '0.9299', 'sentence': '0'}, []),\n","  Annotation(named_entity, 274, 282, I-PROBLEM, {'word': 'hepatitis', 'confidence': '0.9956', 'sentence': '0'}, []),\n","  Annotation(named_entity, 283, 283, O, {'word': ',', 'confidence': '1.0', 'sentence': '0'}, []),\n","  Annotation(named_entity, 285, 293, O, {'word': 'presented', 'confidence': '0.9982', 'sentence': '0'}, []),\n","  Annotation(named_entity, 295, 298, O, {'word': 'with', 'confidence': '1.0', 'sentence': '0'}, []),\n","  Annotation(named_entity, 300, 300, O, {'word': 'a', 'confidence': '0.999', 'sentence': '0'}, []),\n","  Annotation(named_entity, 302, 309, O, {'word': 'one-week', 'confidence': '0.9924', 'sentence': '0'}, []),\n","  Annotation(named_entity, 311, 317, O, {'word': 'history', 'confidence': '0.7946', 'sentence': '0'}, []),\n","  Annotation(named_entity, 319, 320, O, {'word': 'of', 'confidence': '0.9651', 'sentence': '0'}, []),\n","  Annotation(named_entity, 322, 329, B-PROBLEM, {'word': 'polyuria', 'confidence': '0.9728', 'sentence': '0'}, []),\n","  Annotation(named_entity, 330, 330, O, {'word': ',', 'confidence': '0.9998', 'sentence': '0'}, []),\n","  Annotation(named_entity, 332, 335, B-PROBLEM, {'word': 'poor', 'confidence': '0.9871', 'sentence': '0'}, []),\n","  Annotation(named_entity, 337, 344, I-PROBLEM, {'word': 'appetite', 'confidence': '0.9997', 'sentence': '0'}, []),\n","  Annotation(named_entity, 345, 345, O, {'word': ',', 'confidence': '0.9997', 'sentence': '0'}, []),\n","  Annotation(named_entity, 347, 349, O, {'word': 'and', 'confidence': '0.9886', 'sentence': '0'}, []),\n","  Annotation(named_entity, 351, 358, B-PROBLEM, {'word': 'vomiting', 'confidence': '0.9854', 'sentence': '0'}, []),\n","  Annotation(named_entity, 359, 359, O, {'word': '.', 'confidence': '1.0', 'sentence': '0'}, []),\n","  Annotation(named_entity, 361, 363, O, {'word': 'She', 'confidence': '0.9999', 'sentence': '1'}, []),\n","  Annotation(named_entity, 365, 367, O, {'word': 'was', 'confidence': '0.9999', 'sentence': '1'}, []),\n","  Annotation(named_entity, 369, 370, O, {'word': 'on', 'confidence': '0.9998', 'sentence': '1'}, []),\n","  Annotation(named_entity, 372, 380, B-TREATMENT, {'word': 'metformin', 'confidence': '0.9998', 'sentence': '1'}, []),\n","  Annotation(named_entity, 381, 381, O, {'word': ',', 'confidence': '1.0', 'sentence': '1'}, []),\n","  Annotation(named_entity, 383, 391, B-TREATMENT, {'word': 'glipizide', 'confidence': '0.9999', 'sentence': '1'}, []),\n","  Annotation(named_entity, 392, 392, O, {'word': ',', 'confidence': '1.0', 'sentence': '1'}, []),\n","  Annotation(named_entity, 394, 396, O, {'word': 'and', 'confidence': '0.9993', 'sentence': '1'}, []),\n","  Annotation(named_entity, 398, 410, B-TREATMENT, {'word': 'dapagliflozin', 'confidence': '1.0', 'sentence': '1'}, []),\n","  Annotation(named_entity, 412, 414, O, {'word': 'for', 'confidence': '0.9999', 'sentence': '1'}, []),\n","  Annotation(named_entity, 416, 419, B-PROBLEM, {'word': 'T2DM', 'confidence': '0.9995', 'sentence': '1'}, []),\n","  Annotation(named_entity, 421, 423, O, {'word': 'and', 'confidence': '1.0', 'sentence': '1'}, []),\n","  Annotation(named_entity, 425, 436, B-TREATMENT, {'word': 'atorvastatin', 'confidence': '0.9999', 'sentence': '1'}, []),\n","  Annotation(named_entity, 438, 440, O, {'word': 'and', 'confidence': '1.0', 'sentence': '1'}, []),\n","  Annotation(named_entity, 442, 452, B-TREATMENT, {'word': 'gemfibrozil', 'confidence': '1.0', 'sentence': '1'}, []),\n","  Annotation(named_entity, 454, 456, O, {'word': 'for', 'confidence': '0.9998', 'sentence': '1'}, []),\n","  Annotation(named_entity, 458, 460, B-PROBLEM, {'word': 'HTG', 'confidence': '0.9994', 'sentence': '1'}, []),\n","  Annotation(named_entity, 461, 461, O, {'word': '.', 'confidence': '1.0', 'sentence': '1'}, []),\n","  Annotation(named_entity, 463, 465, O, {'word': 'She', 'confidence': '0.9999', 'sentence': '2'}, []),\n","  Annotation(named_entity, 467, 469, O, {'word': 'had', 'confidence': '0.9997', 'sentence': '2'}, []),\n","  Annotation(named_entity, 471, 474, O, {'word': 'been', 'confidence': '0.9998', 'sentence': '2'}, []),\n","  Annotation(named_entity, 476, 477, O, {'word': 'on', 'confidence': '0.9983', 'sentence': '2'}, []),\n","  Annotation(named_entity, 479, 491, B-TREATMENT, {'word': 'dapagliflozin', 'confidence': '0.9999', 'sentence': '2'}, []),\n","  Annotation(named_entity, 493, 495, O, {'word': 'for', 'confidence': '1.0', 'sentence': '2'}, []),\n","  Annotation(named_entity, 497, 499, O, {'word': 'six', 'confidence': '0.9871', 'sentence': '2'}, []),\n","  Annotation(named_entity, 501, 506, O, {'word': 'months', 'confidence': '0.8859', 'sentence': '2'}, []),\n","  Annotation(named_entity, 508, 509, O, {'word': 'at', 'confidence': '0.9999', 'sentence': '2'}, []),\n","  Annotation(named_entity, 511, 513, O, {'word': 'the', 'confidence': '0.9987', 'sentence': '2'}, []),\n","  Annotation(named_entity, 515, 518, O, {'word': 'time', 'confidence': '0.9579', 'sentence': '2'}, []),\n","  Annotation(named_entity, 520, 521, O, {'word': 'of', 'confidence': '0.9636', 'sentence': '2'}, []),\n","  Annotation(named_entity, 523, 534, O, {'word': 'presentation', 'confidence': '0.7969', 'sentence': '2'}, []),\n","  Annotation(named_entity, 535, 535, O, {'word': '.', 'confidence': '0.9998', 'sentence': '2'}, []),\n","  Annotation(named_entity, 537, 544, B-TEST, {'word': 'Physical', 'confidence': '0.8948', 'sentence': '3'}, []),\n","  Annotation(named_entity, 546, 556, I-TEST, {'word': 'examination', 'confidence': '0.9948', 'sentence': '3'}, []),\n","  Annotation(named_entity, 558, 559, O, {'word': 'on', 'confidence': '0.9928', 'sentence': '3'}, []),\n","  Annotation(named_entity, 561, 572, O, {'word': 'presentation', 'confidence': '0.9298', 'sentence': '3'}, []),\n","  Annotation(named_entity, 574, 576, O, {'word': 'was', 'confidence': '0.9998', 'sentence': '3'}, []),\n","  Annotation(named_entity, 578, 588, O, {'word': 'significant', 'confidence': '0.973', 'sentence': '3'}, []),\n","  Annotation(named_entity, 590, 592, O, {'word': 'for', 'confidence': '0.6964', 'sentence': '3'}, []),\n","  Annotation(named_entity, 594, 596, B-PROBLEM, {'word': 'dry', 'confidence': '0.466', 'sentence': '3'}, []),\n","  Annotation(named_entity, 598, 601, I-PROBLEM, {'word': 'oral', 'confidence': '0.7135', 'sentence': '3'}, []),\n","  Annotation(named_entity, 603, 608, I-PROBLEM, {'word': 'mucosa', 'confidence': '0.9932', 'sentence': '3'}, []),\n","  Annotation(named_entity, 610, 610, O, {'word': ';', 'confidence': '0.9993', 'sentence': '3'}, []),\n","  Annotation(named_entity, 612, 624, O, {'word': 'significantly', 'confidence': '0.6258', 'sentence': '3'}, []),\n","  Annotation(named_entity, 626, 626, O, {'word': ',', 'confidence': '0.988', 'sentence': '3'}, []),\n","  Annotation(named_entity, 628, 630, B-TEST, {'word': 'her', 'confidence': '0.8397', 'sentence': '3'}, []),\n","  Annotation(named_entity, 632, 640, I-TEST, {'word': 'abdominal', 'confidence': '0.7958', 'sentence': '3'}, []),\n","  Annotation(named_entity, 642, 652, I-TEST, {'word': 'examination', 'confidence': '0.9894', 'sentence': '3'}, []),\n","  Annotation(named_entity, 654, 656, O, {'word': 'was', 'confidence': '0.9999', 'sentence': '3'}, []),\n","  Annotation(named_entity, 658, 663, O, {'word': 'benign', 'confidence': '0.9622', 'sentence': '3'}, []),\n","  Annotation(named_entity, 665, 668, O, {'word': 'with', 'confidence': '0.9982', 'sentence': '3'}, []),\n","  Annotation(named_entity, 670, 671, O, {'word': 'no', 'confidence': '0.9994', 'sentence': '3'}, []),\n","  Annotation(named_entity, 673, 682, B-PROBLEM, {'word': 'tenderness', 'confidence': '0.9996', 'sentence': '3'}, []),\n","  Annotation(named_entity, 683, 683, O, {'word': ',', 'confidence': '0.9996', 'sentence': '3'}, []),\n","  Annotation(named_entity, 685, 692, B-PROBLEM, {'word': 'guarding', 'confidence': '0.9989', 'sentence': '3'}, []),\n","  Annotation(named_entity, 693, 693, O, {'word': ',', 'confidence': '0.9993', 'sentence': '3'}, []),\n","  Annotation(named_entity, 695, 696, O, {'word': 'or', 'confidence': '0.9847', 'sentence': '3'}, []),\n","  Annotation(named_entity, 698, 705, B-PROBLEM, {'word': 'rigidity', 'confidence': '0.9986', 'sentence': '3'}, []),\n","  Annotation(named_entity, 706, 706, O, {'word': '.', 'confidence': '0.9999', 'sentence': '3'}, []),\n","  Annotation(named_entity, 708, 716, O, {'word': 'Pertinent', 'confidence': '0.9717', 'sentence': '4'}, []),\n","  Annotation(named_entity, 718, 727, O, {'word': 'laboratory', 'confidence': '0.5973', 'sentence': '4'}, []),\n","  Annotation(named_entity, 729, 736, O, {'word': 'findings', 'confidence': '0.6255', 'sentence': '4'}, []),\n","  Annotation(named_entity, 738, 739, O, {'word': 'on', 'confidence': '0.9936', 'sentence': '4'}, []),\n","  Annotation(named_entity, 741, 749, O, {'word': 'admission', 'confidence': '0.565', 'sentence': '4'}, []),\n","  Annotation(named_entity, 751, 754, O, {'word': 'were', 'confidence': '0.9999', 'sentence': '4'}, []),\n","  Annotation(named_entity, 755, 755, O, {'word': ':', 'confidence': '1.0', 'sentence': '4'}, []),\n","  Annotation(named_entity, 757, 761, B-TEST, {'word': 'serum', 'confidence': '0.9726', 'sentence': '4'}, []),\n","  Annotation(named_entity, 763, 769, I-TEST, {'word': 'glucose', 'confidence': '0.9654', 'sentence': '4'}, []),\n","  Annotation(named_entity, 771, 773, O, {'word': '111', 'confidence': '0.9993', 'sentence': '4'}, []),\n","  Annotation(named_entity, 775, 779, O, {'word': 'mg/dl', 'confidence': '0.9652', 'sentence': '4'}, []),\n","  Annotation(named_entity, 780, 780, O, {'word': ',', 'confidence': '0.9997', 'sentence': '4'}, []),\n","  Annotation(named_entity, 783, 792, B-TEST, {'word': 'creatinine', 'confidence': '0.9994', 'sentence': '4'}, []),\n","  Annotation(named_entity, 794, 796, O, {'word': '0.4', 'confidence': '0.9997', 'sentence': '4'}, []),\n","  Annotation(named_entity, 798, 802, O, {'word': 'mg/dL', 'confidence': '0.7432', 'sentence': '4'}, []),\n","  Annotation(named_entity, 803, 803, O, {'word': ',', 'confidence': '0.9997', 'sentence': '4'}, []),\n","  Annotation(named_entity, 805, 817, B-TEST, {'word': 'triglycerides', 'confidence': '0.9999', 'sentence': '4'}, []),\n","  Annotation(named_entity, 819, 821, O, {'word': '508', 'confidence': '0.9999', 'sentence': '4'}, []),\n","  Annotation(named_entity, 823, 827, O, {'word': 'mg/dL', 'confidence': '0.7211', 'sentence': '4'}, []),\n","  Annotation(named_entity, 828, 828, O, {'word': ',', 'confidence': '0.9999', 'sentence': '4'}, []),\n","  Annotation(named_entity, 830, 834, B-TEST, {'word': 'total', 'confidence': '0.9963', 'sentence': '4'}, []),\n","  Annotation(named_entity, 836, 846, I-TEST, {'word': 'cholesterol', 'confidence': '0.9691', 'sentence': '4'}, []),\n","  Annotation(named_entity, 848, 850, O, {'word': '122', 'confidence': '0.9997', 'sentence': '4'}, []),\n","  Annotation(named_entity, 852, 856, O, {'word': 'mg/dL', 'confidence': '0.8757', 'sentence': '4'}, []),\n","  Annotation(named_entity, 857, 857, O, {'word': ',', 'confidence': '0.9995', 'sentence': '4'}, []),\n","  Annotation(named_entity, 859, 861, O, {'word': 'and', 'confidence': '0.9984', 'sentence': '4'}, []),\n","  Annotation(named_entity, 863, 868, B-TEST, {'word': 'venous', 'confidence': '0.9917', 'sentence': '4'}, []),\n","  Annotation(named_entity, 870, 871, I-TEST, {'word': 'pH', 'confidence': '0.952', 'sentence': '4'}, []),\n","  Annotation(named_entity, 873, 876, O, {'word': '7.27', 'confidence': '0.9998', 'sentence': '4'}, []),\n","  Annotation(named_entity, 877, 877, O, {'word': '.', 'confidence': '0.9998', 'sentence': '4'}, [])],\n"," 'embeddings': [Annotation(word_embeddings, 1, 1, A, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'A', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 3, 13, 28-year-old, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': '28-year-old', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 15, 20, female, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'female', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 22, 25, with, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'with', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 27, 27, a, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'a', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 29, 35, history, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'history', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 37, 38, of, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'of', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 40, 50, gestational, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'gestational', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 52, 59, diabetes, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'diabetes', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 61, 68, mellitus, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'mellitus', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 70, 78, diagnosed, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'diagnosed', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 80, 84, eight, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'eight', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 86, 90, years, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'years', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 92, 96, prior, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'prior', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 98, 99, to, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'to', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 101, 112, presentation, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'presentation', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 114, 116, and, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'and', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 118, 127, subsequent, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'subsequent', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 129, 132, type, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'type', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 134, 136, two, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'two', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 138, 145, diabetes, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'diabetes', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 147, 154, mellitus, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'mellitus', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 156, 156, (, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': '(', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 157, 160, T2DM, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'T2DM', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 161, 162, ),, {'isOOV': 'true', 'pieceId': '-1', 'isWordStart': 'true', 'token': '),', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 164, 166, one, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'one', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 168, 172, prior, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'prior', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 174, 180, episode, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'episode', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 182, 183, of, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'of', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 185, 195, HTG-induced, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'HTG-induced', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 197, 208, pancreatitis, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'pancreatitis', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 210, 214, three, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'three', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 216, 220, years, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'years', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 222, 226, prior, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'prior', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 228, 229, to, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'to', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 231, 242, presentation, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'presentation', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 243, 243, ,, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ',', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 245, 247, and, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'and', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 249, 258, associated, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'associated', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 260, 263, with, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'with', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 265, 266, an, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'an', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 268, 272, acute, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'acute', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 274, 282, hepatitis, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'hepatitis', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 283, 283, ,, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ',', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 285, 293, presented, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'presented', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 295, 298, with, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'with', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 300, 300, a, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'a', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 302, 309, one-week, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'one-week', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 311, 317, history, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'history', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 319, 320, of, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'of', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 322, 329, polyuria, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'polyuria', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 330, 330, ,, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ',', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 332, 335, poor, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'poor', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 337, 344, appetite, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'appetite', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 345, 345, ,, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ',', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 347, 349, and, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'and', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 351, 358, vomiting, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'vomiting', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 359, 359, ., {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': '.', 'sentence': '0'}, []),\n","  Annotation(word_embeddings, 361, 363, She, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'She', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 365, 367, was, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'was', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 369, 370, on, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'on', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 372, 380, metformin, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'metformin', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 381, 381, ,, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ',', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 383, 391, glipizide, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'glipizide', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 392, 392, ,, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ',', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 394, 396, and, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'and', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 398, 410, dapagliflozin, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'dapagliflozin', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 412, 414, for, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'for', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 416, 419, T2DM, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'T2DM', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 421, 423, and, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'and', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 425, 436, atorvastatin, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'atorvastatin', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 438, 440, and, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'and', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 442, 452, gemfibrozil, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'gemfibrozil', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 454, 456, for, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'for', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 458, 460, HTG, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'HTG', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 461, 461, ., {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': '.', 'sentence': '1'}, []),\n","  Annotation(word_embeddings, 463, 465, She, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'She', 'sentence': '2'}, []),\n","  Annotation(word_embeddings, 467, 469, had, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'had', 'sentence': '2'}, []),\n","  Annotation(word_embeddings, 471, 474, been, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'been', 'sentence': '2'}, []),\n","  Annotation(word_embeddings, 476, 477, on, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'on', 'sentence': '2'}, []),\n","  Annotation(word_embeddings, 479, 491, dapagliflozin, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'dapagliflozin', 'sentence': '2'}, []),\n","  Annotation(word_embeddings, 493, 495, for, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'for', 'sentence': '2'}, []),\n","  Annotation(word_embeddings, 497, 499, six, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'six', 'sentence': '2'}, []),\n","  Annotation(word_embeddings, 501, 506, months, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'months', 'sentence': '2'}, []),\n","  Annotation(word_embeddings, 508, 509, at, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'at', 'sentence': '2'}, []),\n","  Annotation(word_embeddings, 511, 513, the, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'the', 'sentence': '2'}, []),\n","  Annotation(word_embeddings, 515, 518, time, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'time', 'sentence': '2'}, []),\n","  Annotation(word_embeddings, 520, 521, of, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'of', 'sentence': '2'}, []),\n","  Annotation(word_embeddings, 523, 534, presentation, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'presentation', 'sentence': '2'}, []),\n","  Annotation(word_embeddings, 535, 535, ., {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': '.', 'sentence': '2'}, []),\n","  Annotation(word_embeddings, 537, 544, Physical, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'Physical', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 546, 556, examination, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'examination', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 558, 559, on, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'on', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 561, 572, presentation, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'presentation', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 574, 576, was, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'was', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 578, 588, significant, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'significant', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 590, 592, for, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'for', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 594, 596, dry, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'dry', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 598, 601, oral, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'oral', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 603, 608, mucosa, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'mucosa', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 610, 610, ;, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ';', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 612, 624, significantly, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'significantly', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 626, 626, ,, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ',', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 628, 630, her, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'her', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 632, 640, abdominal, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'abdominal', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 642, 652, examination, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'examination', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 654, 656, was, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'was', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 658, 663, benign, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'benign', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 665, 668, with, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'with', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 670, 671, no, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'no', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 673, 682, tenderness, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'tenderness', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 683, 683, ,, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ',', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 685, 692, guarding, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'guarding', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 693, 693, ,, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ',', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 695, 696, or, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'or', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 698, 705, rigidity, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'rigidity', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 706, 706, ., {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': '.', 'sentence': '3'}, []),\n","  Annotation(word_embeddings, 708, 716, Pertinent, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'Pertinent', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 718, 727, laboratory, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'laboratory', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 729, 736, findings, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'findings', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 738, 739, on, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'on', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 741, 749, admission, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'admission', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 751, 754, were, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'were', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 755, 755, :, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ':', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 757, 761, serum, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'serum', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 763, 769, glucose, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'glucose', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 771, 773, 111, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': '111', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 775, 779, mg/dl, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'mg/dl', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 780, 780, ,, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ',', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 783, 792, creatinine, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'creatinine', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 794, 796, 0.4, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': '0.4', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 798, 802, mg/dL, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'mg/dL', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 803, 803, ,, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ',', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 805, 817, triglycerides, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'triglycerides', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 819, 821, 508, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': '508', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 823, 827, mg/dL, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'mg/dL', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 828, 828, ,, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ',', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 830, 834, total, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'total', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 836, 846, cholesterol, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'cholesterol', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 848, 850, 122, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': '122', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 852, 856, mg/dL, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'mg/dL', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 857, 857, ,, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': ',', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 859, 861, and, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'and', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 863, 868, venous, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'venous', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 870, 871, pH, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': 'pH', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 873, 876, 7.27, {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': '7.27', 'sentence': '4'}, []),\n","  Annotation(word_embeddings, 877, 877, ., {'isOOV': 'false', 'pieceId': '-1', 'isWordStart': 'true', 'token': '.', 'sentence': '4'}, [])],\n"," 'sentence': [Annotation(document, 1, 359, A 28-year-old female with a history of gestational diabetes mellitus diagnosed eight years prior to presentation and subsequent type two diabetes mellitus (T2DM), one prior episode of HTG-induced pancreatitis three years prior to presentation, and associated with an acute hepatitis, presented with a one-week history of polyuria, poor appetite, and vomiting., {'sentence': '0'}, []),\n","  Annotation(document, 361, 461, She was on metformin, glipizide, and dapagliflozin for T2DM and atorvastatin and gemfibrozil for HTG., {'sentence': '1'}, []),\n","  Annotation(document, 463, 535, She had been on dapagliflozin for six months at the time of presentation., {'sentence': '2'}, []),\n","  Annotation(document, 537, 706, Physical examination on presentation was significant for dry oral mucosa ; significantly , her abdominal examination was benign with no tenderness, guarding, or rigidity., {'sentence': '3'}, []),\n","  Annotation(document, 708, 877, Pertinent laboratory findings on admission were: serum glucose 111 mg/dl,  creatinine 0.4 mg/dL, triglycerides 508 mg/dL, total cholesterol 122 mg/dL, and venous pH 7.27., {'sentence': '4'}, [])]}"]},"metadata":{},"execution_count":29}],"source":["light_result[0]"]},{"cell_type":"markdown","metadata":{"id":"3mZftbxBjgoH"},"source":["**Getting embeddings with LP**"]},{"cell_type":"code","execution_count":30,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":651,"status":"ok","timestamp":1712935143925,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"kY3J_Mu-jdaS","outputId":"0b6b37f0-fec4-4602-9095-d8dfa345985e"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["[('A',\n","  '0.040217556 0.4003962 0.69088835 -0.44649735 -0.40282863 -0.45621523 -0.20415598 0.13707675 -0.010474405 -0.42899358 -0.2879874 -0.03076839 -0.40367207 -0.32588458 0.23294163 -0.16012932 -0.35798857 0.043061644 -0.4091242 0.101141304 -0.61503464 0.068771854 -0.025053142 0.11466996 -0.57737786 -0.25603554 -0.6950373 0.3328502 -0.100428 0.0191588 -0.61035156 -0.94459176 -0.2473863 0.16416614 -0.27778018 0.40670344 0.57310873 0.10776807 0.13333033 0.1447822 0.18491237 0.2196365 0.16771133 -0.40313882 -0.20199609 0.045175955 -0.2593916 -0.066459864 0.15229034 0.002910823 -0.16859493 0.17066914 -9.03164E-4 0.03531517 -0.33315992 -0.034506235 0.40311962 0.023356766 0.13823766 -0.2634582 0.27318013 -0.15994109 0.40696687 0.19050935 -0.19682823 -0.012536205 0.4694685 -0.14188287 -0.024362855 -0.38291803 0.013956237 0.1830807 0.34642568 -0.23263279 -0.14591216 -0.2337688 0.5834314 -0.03057853 -0.367189 0.56245005 -0.032807935 0.122141086 0.17941982 -0.00749746 -0.18216385 0.17958473 -0.3053265 0.22779782 0.02217061 0.029920649 0.17621519 0.011366806 0.0385044 0.10249041 0.3426693 -0.035018172 0.4671417 0.22892669 -0.30640814 0.10803514 0.3855227 -0.115776226 -0.44415653 -0.11332124 0.044015724 0.61063313 0.29600143 -0.012069431 -0.38224095 -0.13973162 0.39301684 0.23673905 0.057245154 0.18140107 -0.051603805 0.7974051 -0.06663643 -0.5508283 0.28209952 0.18742235 0.21872385 -0.25591275 0.066957995 -0.25298932 0.13821244 0.8307903 0.3973212 -0.17276311 0.12846412 -0.092758246 0.0037952603 -0.41248038 0.65204984 -0.03299473 0.21624717 -0.56830674 0.23451968 -0.05922969 -0.3901237 -0.40775988 0.10331385 -0.0052452036 0.17687175 -0.2470756 0.054437738 0.10611684 -0.546031 0.03243354 -0.10689023 -0.39552742 -0.29755193 -0.12058273 0.047433637 0.16503441 -0.033334065 0.015473544 0.026549816 -0.7305153 -0.09848879 0.11045749 0.41713542 -0.13625173 -0.4153576 0.23202501 0.45888668 -0.0958802 -0.1448712 -0.63727266 0.26895493 -0.2672995 0.08526398 -0.14055037 8.3983614E-4 0.23264983 -0.13427801 0.4620594 0.33161256 0.43790993 0.1952479 -0.27517807 0.22294433 -0.28702205 0.36680275 -0.2888683 -0.10005434 0.029136676 -0.19198024 -0.05411223 0.19856672 0.041492134 0.042284537 0.27730006 0.33647385 -0.25736117 0.10052982 0.0914106 -0.5406019 -0.18598284 -0.54862744 -0.43890363'),\n"," ('28-year-old',\n","  '0.124630086 0.041768234 -0.5851833 0.14018573 -0.2683909 -0.6359739 -0.31614816 -0.71483403 0.5511774 0.4304843 -0.5115899 0.7928865 0.0788141 0.0055778096 -0.10271598 -0.5141132 -0.22763066 0.25965604 0.059446163 0.32050937 -0.16531321 -0.0833686 0.82620156 0.22395885 -0.1965371 -0.11455178 -0.5434175 0.2595536 -0.0153490305 -0.474383 0.14677128 -0.080003664 -0.052671567 -0.04031274 -0.22524078 -0.35215047 0.046719514 -0.21722157 -0.32051668 0.23190966 -0.04614518 1.0149231 0.6315361 -0.47952643 0.44472834 0.053033605 0.31127706 0.08055067 -0.7282213 -0.7802679 0.16257171 0.073258646 0.40050963 0.44751948 0.4896496 -0.17256223 0.1598703 -0.65302587 -0.06466415 -0.21638417 0.3140914 0.4975012 -0.12023235 0.09910327 -0.04403189 -0.8829349 0.6587716 0.17046705 0.1637326 0.28226846 0.19858298 0.37132013 0.6070592 -0.11720922 -0.1534436 0.15268378 -0.055180248 -0.23013055 -0.46350223 0.25226158 -0.4663529 0.19255444 -0.5118875 0.1125828 -0.3138261 0.11908515 -0.17581931 -0.48491156 0.20515475 -0.4273081 -0.6629976 0.31330186 0.3916261 -0.056986686 0.11155176 -0.030073252 0.1524556 -0.36797798 0.6462149 -0.08456914 0.050772805 -0.34014466 0.3223971 0.015675943 0.015383259 0.34324646 0.096488 -0.33835164 0.067233734 0.6365546 0.52120185 0.37197444 -0.027830597 0.4061612 0.028669652 0.27176672 0.07524431 -0.55174196 0.28766853 0.07589323 -0.22518751 -0.44933867 -0.4721748 -0.108820066 0.32532197 0.08391404 -0.45878386 0.26986977 0.1009148 0.6153148 -0.048685577 0.30017054 -0.40371877 0.047533885 0.53543746 -0.27002755 0.25372973 0.10320132 -0.14759818 -0.27956298 0.3565558 -0.1178739 0.75151694 0.39113513 0.17251517 0.20465879 0.11995484 0.86947393 -0.3805215 -0.6808252 0.38752273 -0.35575667 -0.40939957 0.33717453 -0.8888754 0.4364028 0.2899467 -0.66873056 -0.79028213 -0.69079405 0.14101063 -0.0026479138 0.15928632 0.3302679 0.47052 -0.32484296 -0.15391971 -0.5722817 0.1116602 -0.08223172 -0.18719476 -0.5508417 0.19215384 -0.3587382 -0.37864798 0.305579 -0.4447613 -0.054872658 -0.42322004 -0.10340428 -0.014868535 0.5060039 0.28027403 0.086417876 0.5062302 -0.046939876 0.08775024 0.36165246 0.64060605 0.7646089 -0.18056732 0.37293836 0.6689069 0.030002259 0.5213449 0.24203093 -0.892416 -0.3918221 0.3560327 -0.3065561'),\n"," ('female',\n","  '0.16302037 0.017953504 0.005551864 -0.032135632 -0.35178208 -0.019391192 -0.23211344 -0.11785924 0.45375252 0.2833919 -0.2134419 0.14760925 -0.48229983 -0.2431024 0.1263231 -0.30963817 -0.17346981 -0.052810684 0.14164582 0.16676669 -0.38038972 -0.019319143 0.5836717 0.12141203 0.29895684 0.010067581 -0.13612121 -0.02027633 -0.05771621 -0.072261736 0.3775381 -0.13991304 -0.34879878 0.12302238 -0.17791995 -0.11906978 0.38496873 0.043683514 0.2418363 -0.09076467 -0.17712079 0.59563845 0.27979994 -0.18298905 0.26172453 0.045466516 -0.061663765 0.18921505 -0.49157807 -0.056109034 0.11156733 0.037744977 0.054749884 0.22867315 -0.30465952 -0.23039347 -0.14796074 0.1978099 -0.45714766 -0.45331523 0.28797394 -0.16251872 -0.34184834 0.024538815 -0.4446561 -0.077740535 0.33576167 -0.21244533 -0.13956763 0.053867884 -0.23181938 0.07911189 0.26846886 0.4207609 -0.039360207 0.024773559 -0.1577196 -0.28330392 -0.20176809 0.006568704 -0.2934455 0.21242368 -0.53532916 0.3434734 0.40760696 -0.0014423534 -0.13468231 -0.20690395 0.19727087 -0.21072188 0.03501189 -0.16372475 -0.21106745 -0.37498906 0.13524377 -0.17323516 0.65397954 0.03559853 0.08741455 -0.27057537 0.09465006 -0.11844379 0.10847475 0.033930477 0.3338633 0.17999977 -0.17902453 -0.5227448 0.0740943 -0.06692055 0.10037465 0.05307225 0.4593874 0.27882725 -0.23101482 -0.4581898 0.4513508 -0.3275554 0.32179078 0.124493495 0.13102262 -0.59937644 0.42163733 -0.29410633 0.13054127 0.115324564 0.13005938 -0.059543554 0.35335496 0.67045325 0.12695797 -0.052896604 -0.3642225 0.025236143 -0.53711045 -0.20451795 -0.13416019 -0.33450902 0.063854285 -0.16292678 0.6290029 -0.35059434 0.006743933 0.103949085 0.5178944 0.19399825 0.15600705 0.942345 -0.30966017 -0.81486416 0.008710009 -0.04229206 -0.050095085 0.09709972 -0.27325177 0.22799088 0.3801335 -0.25588122 -0.30700856 -0.07751691 -0.32024297 0.208939 -0.28964671 0.040439676 0.11234014 -0.15674657 -0.33957657 -0.18361814 -0.18896136 0.50387585 0.3538876 0.09590049 0.54036593 -0.24115056 -0.090039216 -0.21107678 -0.2976875 -0.016052669 0.08931619 -0.37238902 0.045852017 -0.043787874 0.43762046 0.117538385 -0.27680975 -0.407116 0.17236143 0.03687374 -0.3842153 0.07933943 0.12446819 -0.08409634 0.23553246 -0.21332222 -0.05892028 -0.06332811 -0.6847742 0.1558429 0.10073703 -0.11202118')]"]},"metadata":{},"execution_count":30}],"source":["light_model_with_emb = nlp.LightPipeline(model, parse_embeddings=True)\n","light_result_with_emb = light_model_with_emb.annotate(text)\n","\n","list(zip(light_result_with_emb['token'], light_result_with_emb['embeddings']))[:3]"]},{"cell_type":"markdown","metadata":{"id":"mQyQ751amaov"},"source":["**Prettifying the results with NLU**\n","\n"]},{"cell_type":"markdown","metadata":{"id":"5V_lvnPsKQxz"},"source":["John Snow Labs' NLU is a Python library for applying state-of-the-art text mining, directly on any dataframe, with a single line of code. We will use some functionalities of NLU to prettify our results."]},{"cell_type":"code","execution_count":31,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2052,"status":"ok","timestamp":1712935145970,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"ihg5PWSHX0KN","outputId":"563c86ec-b6b6-47a1-dd53-dd65f9c5394c"},"outputs":[{"output_type":"stream","name":"stdout","text":["Warning::Spark Session already created, some configs may not take.\n","Warning::Spark Session already created, some configs may not take.\n"]},{"output_type":"execute_result","data":{"text/plain":["Index(['document', 'document_begin', 'document_end', 'entities_ner_chunk',\n","       'entities_ner_chunk_begin', 'entities_ner_chunk_class',\n","       'entities_ner_chunk_confidence', 'entities_ner_chunk_end',\n","       'entities_ner_chunk_origin_chunk', 'entities_ner_chunk_origin_sentence',\n","       'ner_iob_ner_begin', 'ner_iob_ner_end', 'sentence_dl',\n","       'sentence_dl_begin', 'sentence_dl_end', 'token_begin', 'token_end',\n","       'word_embedding_embeddings', 'word_embedding_embeddings_begin',\n","       'word_embedding_embeddings_end'],\n","      dtype='object')"]},"metadata":{},"execution_count":31}],"source":["nlp.nlu.to_pretty_df(model, text, positions=True, output_level='chunk').columns"]},{"cell_type":"code","execution_count":32,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":800},"executionInfo":{"elapsed":1086,"status":"ok","timestamp":1712935147052,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"IQnM4kPXYLdB","outputId":"ba075648-0ebb-4170-a310-4b793f1da5bc"},"outputs":[{"output_type":"stream","name":"stdout","text":["Warning::Spark Session already created, some configs may not take.\n","Warning::Spark Session already created, some configs may not take.\n"]},{"output_type":"execute_result","data":{"text/plain":["                      entities_ner_chunk entities_ner_chunk_begin  \\\n","0          gestational diabetes mellitus                       40   \n","0  subsequent type two diabetes mellitus                      118   \n","0                                   T2DM                      157   \n","0               HTG-induced pancreatitis                      185   \n","0                     an acute hepatitis                      265   \n","0                               polyuria                      322   \n","0                          poor appetite                      332   \n","0                               vomiting                      351   \n","0                              metformin                      372   \n","0                              glipizide                      383   \n","0                          dapagliflozin                      398   \n","0                                   T2DM                      416   \n","0                           atorvastatin                      425   \n","0                            gemfibrozil                      442   \n","0                                    HTG                      458   \n","0                          dapagliflozin                      479   \n","0                   Physical examination                      537   \n","0                        dry oral mucosa                      594   \n","0              her abdominal examination                      628   \n","0                             tenderness                      673   \n","\n","  entities_ner_chunk_end entities_ner_chunk_origin_sentence  \\\n","0                     68                                  0   \n","0                    154                                  0   \n","0                    160                                  0   \n","0                    208                                  0   \n","0                    282                                  0   \n","0                    329                                  0   \n","0                    344                                  0   \n","0                    358                                  0   \n","0                    380                                  1   \n","0                    391                                  1   \n","0                    410                                  1   \n","0                    419                                  1   \n","0                    436                                  1   \n","0                    452                                  1   \n","0                    460                                  1   \n","0                    491                                  2   \n","0                    556                                  3   \n","0                    608                                  3   \n","0                    652                                  3   \n","0                    682                                  3   \n","\n","  entities_ner_chunk_class  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                TREATMENT  \n","0                TREATMENT  \n","0                TREATMENT  \n","0                  PROBLEM  \n","0                TREATMENT  \n","0                TREATMENT  \n","0                  PROBLEM  \n","0                TREATMENT  \n","0                     TEST  \n","0                  PROBLEM  \n","0                     TEST  \n","0                  PROBLEM  "],"text/html":["\n","  <div id=\"df-6fd5fbe5-a503-4044-93ef-bf575b27946c\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>entities_ner_chunk</th>\n","      <th>entities_ner_chunk_begin</th>\n","      <th>entities_ner_chunk_end</th>\n","      <th>entities_ner_chunk_origin_sentence</th>\n","      <th>entities_ner_chunk_class</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>gestational diabetes mellitus</td>\n","      <td>40</td>\n","      <td>68</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>subsequent type two diabetes mellitus</td>\n","      <td>118</td>\n","      <td>154</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>T2DM</td>\n","      <td>157</td>\n","      <td>160</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>HTG-induced pancreatitis</td>\n","      <td>185</td>\n","      <td>208</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>an acute hepatitis</td>\n","      <td>265</td>\n","      <td>282</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>polyuria</td>\n","      <td>322</td>\n","      <td>329</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>poor appetite</td>\n","      <td>332</td>\n","      <td>344</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>vomiting</td>\n","      <td>351</td>\n","      <td>358</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>metformin</td>\n","      <td>372</td>\n","      <td>380</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>glipizide</td>\n","      <td>383</td>\n","      <td>391</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>dapagliflozin</td>\n","      <td>398</td>\n","      <td>410</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>T2DM</td>\n","      <td>416</td>\n","      <td>419</td>\n","      <td>1</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>atorvastatin</td>\n","      <td>425</td>\n","      <td>436</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>gemfibrozil</td>\n","      <td>442</td>\n","      <td>452</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>HTG</td>\n","      <td>458</td>\n","      <td>460</td>\n","      <td>1</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>dapagliflozin</td>\n","      <td>479</td>\n","      <td>491</td>\n","      <td>2</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>Physical examination</td>\n","      <td>537</td>\n","      <td>556</td>\n","      <td>3</td>\n","      <td>TEST</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>dry oral mucosa</td>\n","      <td>594</td>\n","      <td>608</td>\n","      <td>3</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>her abdominal examination</td>\n","      <td>628</td>\n","      <td>652</td>\n","      <td>3</td>\n","      <td>TEST</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>tenderness</td>\n","      <td>673</td>\n","      <td>682</td>\n","      <td>3</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6fd5fbe5-a503-4044-93ef-bf575b27946c')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-6fd5fbe5-a503-4044-93ef-bf575b27946c button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-6fd5fbe5-a503-4044-93ef-bf575b27946c');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-3adbd839-dc20-4a0e-a8a4-d970042f0a4d\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-3adbd839-dc20-4a0e-a8a4-d970042f0a4d')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-3adbd839-dc20-4a0e-a8a4-d970042f0a4d button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"df_clinical","summary":"{\n  \"name\": \"df_clinical\",\n  \"rows\": 27,\n  \"fields\": [\n    {\n      \"column\": \"entities_ner_chunk\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 25,\n        \"samples\": [\n          \"metformin\",\n          \"her abdominal examination\",\n          \"gestational diabetes mellitus\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_begin\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 40,\n        \"max\": 863,\n        \"num_unique_values\": 27,\n        \"samples\": [\n          372,\n          442,\n          383\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_end\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 68,\n        \"max\": 871,\n        \"num_unique_values\": 27,\n        \"samples\": [\n          380,\n          452,\n          391\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_origin_sentence\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"1\",\n          \"4\",\n          \"2\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_class\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"PROBLEM\",\n          \"TREATMENT\",\n          \"TEST\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":32}],"source":["#For a given pipeline output level is automatically set to the last anntator's output level by default.\n","#This can be changed by defining the value of \"output_level\" parameter.\n","#Values can be set as; token, sentence, document, chunk and relation .\n","\n","cols = [\n","     'entities_ner_chunk',\n","     'entities_ner_chunk_begin',\n","     'entities_ner_chunk_end',\n","     'entities_ner_chunk_origin_sentence',\n","     'entities_ner_chunk_class',\n","]\n","df_clinical = nlp.nlu.to_pretty_df(model, text, positions=True, output_level='chunk')[cols]\n","df_clinical.head(20)"]},{"cell_type":"markdown","metadata":{"id":"iQ1KUzkryBvz"},"source":["### NER Visualizer\n","\n","For saving the visualization result as html, provide `save_path` parameter in the display function."]},{"cell_type":"code","execution_count":33,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":751},"executionInfo":{"elapsed":8,"status":"ok","timestamp":1712935147052,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"Jh1zoknFybBO","outputId":"b7c3481b-f9ff-4612-f3b3-ac4eebe29716"},"outputs":[{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["\n","<style>\n","    @import url('https://fonts.googleapis.com/css2?family=Montserrat:wght@300;400;500;600;700&display=swap');\n","    @import url('https://fonts.googleapis.com/css2?family=Vistol Regular:wght@300;400;500;600;700&display=swap');\n","    \n","    .spark-nlp-display-scroll-entities {\n","        border: 1px solid #E7EDF0;\n","        border-radius: 3px;\n","        text-align: justify;\n","        \n","    }\n","    .spark-nlp-display-scroll-entities span {  \n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #536B76;\n","        font-family: 'Montserrat', sans-serif !important;\n","    }\n","    \n","    .spark-nlp-display-entity-wrapper{\n","    \n","        display: inline-grid;\n","        text-align: center;\n","        border-radius: 4px;\n","        margin: 0 2px 5px 2px;\n","        padding: 1px\n","    }\n","    .spark-nlp-display-entity-name{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        background: #f1f2f3;\n","        border-width: medium;\n","        text-align: center;\n","        \n","        font-weight: 400;\n","        \n","        border-radius: 5px;\n","        padding: 2px 5px;\n","        display: block;\n","        margin: 3px 2px;\n","    \n","    }\n","    .spark-nlp-display-entity-type{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-entity-resolution{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Vistol Regular', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-others{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        font-weight: 400;\n","    }\n","\n","</style>\n"," <span class=\"spark-nlp-display-others\" style=\"background-color: white\"><br>A 28-year-old female with a history of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">gestational diabetes mellitus </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> diagnosed eight years prior to presentation and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">subsequent type two diabetes mellitus </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> (</span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">T2DM </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">), one prior episode of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">HTG-induced pancreatitis </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> three years prior to presentation, and associated with </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">an acute hepatitis </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, presented with a one-week history of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">polyuria </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">poor appetite </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">vomiting </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">.<br>She was on </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">metformin </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">glipizide </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">dapagliflozin </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">T2DM </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">atorvastatin </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">gemfibrozil </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">HTG </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">. She had been on </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">dapagliflozin </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> for six months at the time of presentation.<br></span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">Physical examination </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> on presentation was significant for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">dry oral mucosa </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> ; significantly , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">her abdominal examination </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> was benign with no </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">tenderness </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">guarding </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, or </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">rigidity </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">. Pertinent laboratory findings on admission were: </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">serum glucose </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 111 mg/dl,  </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">creatinine </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 0.4 mg/dL, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">triglycerides </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 508 mg/dL, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">total cholesterol </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 122 mg/dL, and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">venous pH </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 7.27.<br></span></div>"]},"metadata":{}}],"source":["visualiser = nlp.viz.NerVisualizer()\n","\n","visualiser.display(light_result[0], label_col='ner_chunk', document_col='document', save_path=\"display_result.html\")\n","\n","\n","# Change color of an entity label\n","\n","# visualiser.set_label_colors({'PROBLEM':'#008080', 'TEST':'#800080', 'TREATMENT':'#808080'})\n","# visualiser.display(light_result[0], label_col='ner_chunk')\n","\n","\n","# Set label filter\n","\n","# visualiser.display(light_result[0], label_col='ner_chunk', document_col='document',\n","                   #labels=['PROBLEM','TEST'])"]},{"cell_type":"markdown","metadata":{"id":"XfsS_dlJGT1f"},"source":["### ModelTracer"]},{"cell_type":"markdown","metadata":{"id":"K1xVnq63Gatj"},"source":["`ModelTracer` allows to track the uids and timestamps of each stage of the pipeline.\n","\n","Let's apply it over an empty data with our existing pipeline."]},{"cell_type":"code","execution_count":34,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":3226,"status":"ok","timestamp":1712935150272,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"36LR6KhPKfUT","outputId":"a22b9c9d-bcb9-47b6-de57-a374a96f5235"},"outputs":[{"output_type":"stream","name":"stdout","text":["+----+------------------------------------------+--------+-----+----------+---+---------+----------------------------------------------------------------------+----------------------------------------------------------------------------+--------------------------------------------------------------+--------------------------------------------------------------------------+--------------------------------------------------------------------+-------------------------------------------------------------------------+\n","|text|document                                  |sentence|token|embeddings|ner|ner_chunk|documentassembler_model_uid                                           |sentencedetectordlmodel_model_uid                                           |tokenizer_model_uid                                           |word_embeddings_model_model_uid                                           |medicalnermodel_model_uid                                           |nerconverterinternal_model_uid                                           |\n","+----+------------------------------------------+--------+-----+----------+---+---------+----------------------------------------------------------------------+----------------------------------------------------------------------------+--------------------------------------------------------------+--------------------------------------------------------------------------+--------------------------------------------------------------------+-------------------------------------------------------------------------+\n","|    |[{document, 0, -1, , {sentence -> 0}, []}]|[]      |[]   |[]        |[] |[]       |{uid -> DocumentAssembler_b06127093d4f, timestamp -> 2024-04-12_15:19}|{uid -> SentenceDetectorDLModel_6bafc4746ea5, timestamp -> 2024-04-12_15:19}|{uid -> Tokenizer_b75717cb0f7a, timestamp -> 2024-04-12_15:19}|{uid -> WORD_EMBEDDINGS_MODEL_9004b1d00302, timestamp -> 2024-04-12_15:19}|{uid -> MedicalNerModel_1a8637089929, timestamp -> 2024-04-12_15:19}|{uid -> NerConverterInternal_20542571f649, timestamp -> 2024-04-12_15:19}|\n","+----+------------------------------------------+--------+-----+----------+---+---------+----------------------------------------------------------------------+----------------------------------------------------------------------------+--------------------------------------------------------------+--------------------------------------------------------------------------+--------------------------------------------------------------------+-------------------------------------------------------------------------+\n","\n"]}],"source":["data = spark.createDataFrame([['']]).toDF(\"text\")\n","\n","df = nlpPipeline.fit(data).transform(data)\n","\n","tracer_result = medical.ModelTracer().addUidCols(pipeline = nlpPipeline, df = df)\n","\n","tracer_result.show(truncate=False)"]},{"cell_type":"markdown","metadata":{"id":"zOXLMXIWRs_l"},"source":["## NER JSL\n","Let's show an example of `ner_jsl` model that has about 80 clinical entity labels by changing just only the model name.\n","\n","**Entities**\n","\n","| | | | | |\n","|-|-|-|-|-|\n","|Injury_or_Poisoning|Direction|Test|Admission_Discharge|Death_Entity|\n","|Relationship_Status|Duration|Respiration|Hyperlipidemia|Birth_Entity|\n","|Age|Labour_Delivery|Family_History_Header|BMI|Temperature|\n","|Alcohol|Kidney_Disease|Oncological|Medical_History_Header|Cerebrovascular_Disease|\n","|Oxygen_Therapy|O2_Saturation|Psychological_Condition|Heart_Disease|Employment|\n","|Obesity|Disease_Syndrome_Disorder|Pregnancy|ImagingFindings|Procedure|\n","|Medical_Device|Race_Ethnicity|Section_Header|Symptom|Treatment|\n","|Substance|Route|Drug_Ingredient|Blood_Pressure|Diet|\n","|External_body_part_or_region|LDL|VS_Finding|Allergen|EKG_Findings|\n","|Imaging_Technique|Triglycerides|RelativeTime|Gender|Pulse|\n","|Social_History_Header|Substance_Quantity|Diabetes|Modifier|Internal_organ_or_component|\n","|Clinical_Dept|Form|Drug_BrandName|Strength|Fetus_NewBorn|\n","|RelativeDate|Height|Test_Result|Sexually_Active_or_Sexual_Orientation|Frequency|\n","|Time|Weight|Vaccine|Vital_Signs_Header|Communicable_Disease|\n","|Dosage|Overweight|Hypertension|HDL|Total_Cholesterol|\n","|Smoking|Date||||"]},{"cell_type":"code","execution_count":35,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":13650,"status":"ok","timestamp":1712935163905,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"xzfAWMGjR3ov","outputId":"da40c72f-bf38-433a-e9a5-3443299aa6d1"},"outputs":[{"output_type":"stream","name":"stdout","text":["sentence_detector_dl_healthcare download started this may take some time.\n","Approximate size to download 367.3 KB\n","[OK!]\n","embeddings_clinical download started this may take some time.\n","Approximate size to download 1.6 GB\n","[OK!]\n","ner_jsl download started this may take some time.\n","[OK!]\n"]}],"source":["documentAssembler = nlp.DocumentAssembler()\\\n","    .setInputCol(\"text\")\\\n","    .setOutputCol(\"document\")\n","\n","sentenceDetector = nlp.SentenceDetectorDLModel.pretrained(\"sentence_detector_dl_healthcare\",\"en\",\"clinical/models\") \\\n","    .setInputCols([\"document\"]) \\\n","    .setOutputCol(\"sentence\")\n","\n","tokenizer = nlp.Tokenizer()\\\n","    .setInputCols([\"sentence\"])\\\n","    .setOutputCol(\"token\")\n","\n","word_embeddings = nlp.WordEmbeddingsModel.pretrained(\"embeddings_clinical\", \"en\", \"clinical/models\")\\\n","    .setInputCols([\"sentence\", \"token\"])\\\n","    .setOutputCol(\"embeddings\")\n","\n","jsl_ner = medical.NerModel.pretrained(\"ner_jsl\", \"en\", \"clinical/models\") \\\n","    .setInputCols([\"sentence\", \"token\", \"embeddings\"]) \\\n","    .setOutputCol(\"jsl_ner\")\n","\n","jsl_ner_converter = medical.NerConverterInternal() \\\n","    .setInputCols([\"sentence\", \"token\", \"jsl_ner\"]) \\\n","    .setOutputCol(\"jsl_ner_chunk\")\n","\n","jsl_ner_pipeline = nlp.Pipeline(stages=[\n","    documentAssembler,\n","    sentenceDetector,\n","    tokenizer,\n","    word_embeddings,\n","    jsl_ner,\n","    jsl_ner_converter])\n","\n","empty_data = spark.createDataFrame([[\"\"]]).toDF(\"text\")\n","\n","jsl_ner_model = jsl_ner_pipeline.fit(empty_data)\n"]},{"cell_type":"code","execution_count":36,"metadata":{"id":"GfEYvGsXrouX","executionInfo":{"status":"ok","timestamp":1712935165576,"user_tz":-180,"elapsed":1674,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["jsl_light_model = nlp.LightPipeline(jsl_ner_model)\n","\n","jsl_light_result = jsl_light_model.fullAnnotate(text)"]},{"cell_type":"code","execution_count":37,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":748},"executionInfo":{"elapsed":902,"status":"ok","timestamp":1712935166474,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"87JCd2EGP9FA","outputId":"eefe6b9c-6031-42a5-c1a6-2ba71a55745f"},"outputs":[{"output_type":"stream","name":"stdout","text":["Warning::Spark Session already created, some configs may not take.\n","Warning::Spark Session already created, some configs may not take.\n"]},{"output_type":"execute_result","data":{"text/plain":["          entities_jsl_ner_chunk entities_jsl_ner_chunk_begin  \\\n","0                    28-year-old                            3   \n","0                         female                           15   \n","0  gestational diabetes mellitus                           40   \n","0              eight years prior                           80   \n","0     type two diabetes mellitus                          129   \n","0                           T2DM                          157   \n","0       HTG-induced pancreatitis                          185   \n","0              three years prior                          210   \n","0                          acute                          268   \n","0                      hepatitis                          274   \n","0                       one-week                          302   \n","0                       polyuria                          322   \n","0                  poor appetite                          332   \n","0                       vomiting                          351   \n","0                            She                          361   \n","0                      metformin                          372   \n","0                      glipizide                          383   \n","0                  dapagliflozin                          398   \n","0                           T2DM                          416   \n","0                   atorvastatin                          425   \n","\n","  entities_jsl_ner_chunk_end entities_jsl_ner_chunk_origin_sentence  \\\n","0                         13                                      0   \n","0                         20                                      0   \n","0                         68                                      0   \n","0                         96                                      0   \n","0                        154                                      0   \n","0                        160                                      0   \n","0                        208                                      0   \n","0                        226                                      0   \n","0                        272                                      0   \n","0                        282                                      0   \n","0                        309                                      0   \n","0                        329                                      0   \n","0                        344                                      0   \n","0                        358                                      0   \n","0                        363                                      1   \n","0                        380                                      1   \n","0                        391                                      1   \n","0                        410                                      1   \n","0                        419                                      1   \n","0                        436                                      1   \n","\n","  entities_jsl_ner_chunk_class  \n","0                          Age  \n","0                       Gender  \n","0                     Diabetes  \n","0                 RelativeDate  \n","0                     Diabetes  \n","0                     Diabetes  \n","0    Disease_Syndrome_Disorder  \n","0                 RelativeDate  \n","0                     Modifier  \n","0    Disease_Syndrome_Disorder  \n","0                     Duration  \n","0                      Symptom  \n","0                      Symptom  \n","0                      Symptom  \n","0                       Gender  \n","0              Drug_Ingredient  \n","0              Drug_Ingredient  \n","0              Drug_Ingredient  \n","0                     Diabetes  \n","0              Drug_Ingredient  "],"text/html":["\n","  <div id=\"df-7f21eb27-9896-44e6-98a3-089c0874b8dc\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>entities_jsl_ner_chunk</th>\n","      <th>entities_jsl_ner_chunk_begin</th>\n","      <th>entities_jsl_ner_chunk_end</th>\n","      <th>entities_jsl_ner_chunk_origin_sentence</th>\n","      <th>entities_jsl_ner_chunk_class</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>28-year-old</td>\n","      <td>3</td>\n","      <td>13</td>\n","      <td>0</td>\n","      <td>Age</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>female</td>\n","      <td>15</td>\n","      <td>20</td>\n","      <td>0</td>\n","      <td>Gender</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>gestational diabetes mellitus</td>\n","      <td>40</td>\n","      <td>68</td>\n","      <td>0</td>\n","      <td>Diabetes</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>eight years prior</td>\n","      <td>80</td>\n","      <td>96</td>\n","      <td>0</td>\n","      <td>RelativeDate</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>type two diabetes mellitus</td>\n","      <td>129</td>\n","      <td>154</td>\n","      <td>0</td>\n","      <td>Diabetes</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>T2DM</td>\n","      <td>157</td>\n","      <td>160</td>\n","      <td>0</td>\n","      <td>Diabetes</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>HTG-induced pancreatitis</td>\n","      <td>185</td>\n","      <td>208</td>\n","      <td>0</td>\n","      <td>Disease_Syndrome_Disorder</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>three years prior</td>\n","      <td>210</td>\n","      <td>226</td>\n","      <td>0</td>\n","      <td>RelativeDate</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>acute</td>\n","      <td>268</td>\n","      <td>272</td>\n","      <td>0</td>\n","      <td>Modifier</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>hepatitis</td>\n","      <td>274</td>\n","      <td>282</td>\n","      <td>0</td>\n","      <td>Disease_Syndrome_Disorder</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>one-week</td>\n","      <td>302</td>\n","      <td>309</td>\n","      <td>0</td>\n","      <td>Duration</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>polyuria</td>\n","      <td>322</td>\n","      <td>329</td>\n","      <td>0</td>\n","      <td>Symptom</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>poor appetite</td>\n","      <td>332</td>\n","      <td>344</td>\n","      <td>0</td>\n","      <td>Symptom</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>vomiting</td>\n","      <td>351</td>\n","      <td>358</td>\n","      <td>0</td>\n","      <td>Symptom</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>She</td>\n","      <td>361</td>\n","      <td>363</td>\n","      <td>1</td>\n","      <td>Gender</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>metformin</td>\n","      <td>372</td>\n","      <td>380</td>\n","      <td>1</td>\n","      <td>Drug_Ingredient</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>glipizide</td>\n","      <td>383</td>\n","      <td>391</td>\n","      <td>1</td>\n","      <td>Drug_Ingredient</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>dapagliflozin</td>\n","      <td>398</td>\n","      <td>410</td>\n","      <td>1</td>\n","      <td>Drug_Ingredient</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>T2DM</td>\n","      <td>416</td>\n","      <td>419</td>\n","      <td>1</td>\n","      <td>Diabetes</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>atorvastatin</td>\n","      <td>425</td>\n","      <td>436</td>\n","      <td>1</td>\n","      <td>Drug_Ingredient</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7f21eb27-9896-44e6-98a3-089c0874b8dc')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-7f21eb27-9896-44e6-98a3-089c0874b8dc button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-7f21eb27-9896-44e6-98a3-089c0874b8dc');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-5282b2b9-c15b-423e-ba19-91ca79372edc\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-5282b2b9-c15b-423e-ba19-91ca79372edc')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-5282b2b9-c15b-423e-ba19-91ca79372edc button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"jsl_df","summary":"{\n  \"name\": \"jsl_df\",\n  \"rows\": 41,\n  \"fields\": [\n    {\n      \"column\": \"entities_jsl_ner_chunk\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 38,\n        \"samples\": [\n          \"triglycerides\",\n          \"venous pH\",\n          \"type two diabetes mellitus\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_jsl_ner_chunk_begin\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 3,\n        \"max\": 873,\n        \"num_unique_values\": 41,\n        \"samples\": [\n          493,\n          351,\n          268\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_jsl_ner_chunk_end\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 13,\n        \"max\": 876,\n        \"num_unique_values\": 41,\n        \"samples\": [\n          506,\n          358,\n          272\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_jsl_ner_chunk_origin_sentence\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"1\",\n          \"4\",\n          \"2\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_jsl_ner_chunk_class\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 16,\n        \"samples\": [\n          \"Age\",\n          \"Gender\",\n          \"Modifier\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":37}],"source":["cols = [\n","        'entities_jsl_ner_chunk',\n","        'entities_jsl_ner_chunk_begin',\n","        'entities_jsl_ner_chunk_end',\n","        'entities_jsl_ner_chunk_origin_sentence',\n","        'entities_jsl_ner_chunk_class',\n","       ]\n","\n","jsl_df = nlp.nlu.to_pretty_df(jsl_ner_model, text,positions=True, output_level='chunk')[cols]\n","jsl_df.head(20)"]},{"cell_type":"markdown","metadata":{"id":"Pk9617IMzGMI"},"source":["### Checking `ner_clinical_large` and `ner_jsl` results together\n","\n"]},{"cell_type":"code","execution_count":38,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":933,"resources":{"http://localhost:8080/nbextensions/google.colab/tabbar.css":{"data":"Lmdvb2ctdGFie3Bvc2l0aW9uOnJlbGF0aXZlO3BhZGRpbmc6NHB4IDhweDtjb2xvcjojMDBjO3RleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7Y3Vyc29yOmRlZmF1bHR9Lmdvb2ctdGFiLWJhci10b3AgLmdvb2ctdGFie21hcmdpbjoxcHggNHB4IDAgMDtib3JkZXItYm90dG9tOjA7ZmxvYXQ6bGVmdH0uZ29vZy10YWItYmFyLXRvcDphZnRlciwuZ29vZy10YWItYmFyLWJvdHRvbTphZnRlcntjb250ZW50OiIgIjtkaXNwbGF5OmJsb2NrO2hlaWdodDowO2NsZWFyOmJvdGg7dmlzaWJpbGl0eTpoaWRkZW59Lmdvb2ctdGFiLWJhci1ib3R0b20gLmdvb2ctdGFie21hcmdpbjowIDRweCAxcHggMDtib3JkZXItdG9wOjA7ZmxvYXQ6bGVmdH0uZ29vZy10YWItYmFyLXN0YXJ0IC5nb29nLXRhYnttYXJnaW46MCAwIDRweCAxcHg7Ym9yZGVyLXJpZ2h0OjB9Lmdvb2ctdGFiLWJhci1lbmQgLmdvb2ctdGFie21hcmdpbjowIDFweCA0cHggMDtib3JkZXItbGVmdDowfS5nb29nLXRhYi1ob3ZlcntiYWNrZ3JvdW5kOiNlZWV9Lmdvb2ctdGFiLWRpc2FibGVke2NvbG9yOiM2NjZ9Lmdvb2ctdGFiLXNlbGVjdGVke2NvbG9yOiMwMDA7YmFja2dyb3VuZDojZmZmO3RleHQtZGVjb3JhdGlvbjpub25lO2ZvbnQtd2VpZ2h0OmJvbGQ7Ym9yZGVyOjFweCBzb2xpZCAjNmI5MGRhfS5nb29nLXRhYi1iYXItdG9we3BhZGRpbmctdG9wOjVweCFpbXBvcnRhbnQ7cGFkZGluZy1sZWZ0OjVweCFpbXBvcnRhbnQ7Ym9yZGVyLWJvdHRvbToxcHggc29saWQgIzZiOTBkYSFpbXBvcnRhbnR9Lmdvb2ctdGFiLWJhci10b3AgLmdvb2ctdGFiLXNlbGVjdGVke3RvcDoxcHg7bWFyZ2luLXRvcDowO3BhZGRpbmctYm90dG9tOjVweH0uZ29vZy10YWItYmFyLWJvdHRvbSAuZ29vZy10YWItc2VsZWN0ZWR7dG9wOi0xcHg7bWFyZ2luLWJvdHRvbTowO3BhZGRpbmctdG9wOjVweH0uZ29vZy10YWItYmFyLXN0YXJ0IC5nb29nLXRhYi1zZWxlY3RlZHtsZWZ0OjFweDttYXJnaW4tbGVmdDowO3BhZGRpbmctcmlnaHQ6OXB4fS5nb29nLXRhYi1iYXItZW5kIC5nb29nLXRhYi1zZWxlY3RlZHtsZWZ0Oi0xcHg7bWFyZ2luLXJpZ2h0OjA7cGFkZGluZy1sZWZ0OjlweH0uZ29vZy10YWItYmFye21hcmdpbjowO2JvcmRlcjowO3BhZGRpbmc6MDtsaXN0LXN0eWxlOm5vbmU7Y3Vyc29yOmRlZmF1bHQ7b3V0bGluZTpub25lO2JhY2tncm91bmQ6I2ViZWZmOX0uZ29vZy10YWItYmFyLWNsZWFye2NsZWFyOmJvdGg7aGVpZ2h0OjA7b3ZlcmZsb3c6aGlkZGVufS5nb29nLXRhYi1iYXItc3RhcnR7ZmxvYXQ6bGVmdH0uZ29vZy10YWItYmFyLWVuZHtmbG9hdDpyaWdodH0qIGh0bWwgLmdvb2ctdGFiLWJhci1zdGFydHttYXJnaW4tcmlnaHQ6LTNweH0qIGh0bWwgLmdvb2ctdGFiLWJhci1lbmR7bWFyZ2luLWxlZnQ6LTNweH0=","headers":[["content-type","text/css"]],"ok":true,"status":200,"status_text":""},"http://localhost:8080/nbextensions/google.colab/tabbar_main.min.js":{"data":"Ly8gY2xhbmctZm9ybWF0IG9mZgooZnVuY3Rpb24oKXt2YXIgaCxhYT1mdW5jdGlvbihhKXt2YXIgYj0wO3JldHVybiBmdW5jdGlvbigpe3JldHVybiBiPGEubGVuZ3RoP3tkb25lOiExLHZhbHVlOmFbYisrXX06e2RvbmU6ITB9fX0sYmE9ZnVuY3Rpb24oYSl7dmFyIGI9InVuZGVmaW5lZCIhPXR5cGVvZiBTeW1ib2wmJlN5bWJvbC5pdGVyYXRvciYmYVtTeW1ib2wuaXRlcmF0b3JdO3JldHVybiBiP2IuY2FsbChhKTp7bmV4dDphYShhKX19LGw9dGhpc3x8c2VsZixjYT1mdW5jdGlvbigpe30sZGE9ZnVuY3Rpb24oYSl7YS5CYT12b2lkIDA7YS5SPWZ1bmN0aW9uKCl7cmV0dXJuIGEuQmE/YS5CYTphLkJhPW5ldyBhfX0sZWE9ZnVuY3Rpb24oYSl7dmFyIGI9dHlwZW9mIGE7aWYoIm9iamVjdCI9PWIpaWYoYSl7aWYoYSBpbnN0YW5jZW9mIEFycmF5KXJldHVybiJhcnJheSI7aWYoYSBpbnN0YW5jZW9mIE9iamVjdClyZXR1cm4gYjt2YXIgYz1PYmplY3QucHJvdG90eXBlLnRvU3RyaW5nLmNhbGwoYSk7aWYoIltvYmplY3QgV2luZG93XSI9PWMpcmV0dXJuIm9iamVjdCI7CmlmKCJbb2JqZWN0IEFycmF5XSI9PWN8fCJudW1iZXIiPT10eXBlb2YgYS5sZW5ndGgmJiJ1bmRlZmluZWQiIT10eXBlb2YgYS5zcGxpY2UmJiJ1bmRlZmluZWQiIT10eXBlb2YgYS5wcm9wZXJ0eUlzRW51bWVyYWJsZSYmIWEucHJvcGVydHlJc0VudW1lcmFibGUoInNwbGljZSIpKXJldHVybiJhcnJheSI7aWYoIltvYmplY3QgRnVuY3Rpb25dIj09Y3x8InVuZGVmaW5lZCIhPXR5cGVvZiBhLmNhbGwmJiJ1bmRlZmluZWQiIT10eXBlb2YgYS5wcm9wZXJ0eUlzRW51bWVyYWJsZSYmIWEucHJvcGVydHlJc0VudW1lcmFibGUoImNhbGwiKSlyZXR1cm4iZnVuY3Rpb24ifWVsc2UgcmV0dXJuIm51bGwiO2Vsc2UgaWYoImZ1bmN0aW9uIj09YiYmInVuZGVmaW5lZCI9PXR5cGVvZiBhLmNhbGwpcmV0dXJuIm9iamVjdCI7cmV0dXJuIGJ9LGZhPWZ1bmN0aW9uKGEpe3JldHVybiJhcnJheSI9PWVhKGEpfSxoYT1mdW5jdGlvbihhKXt2YXIgYj1lYShhKTtyZXR1cm4iYXJyYXkiPT1ifHwib2JqZWN0Ij09CmImJiJudW1iZXIiPT10eXBlb2YgYS5sZW5ndGh9LGlhPWZ1bmN0aW9uKGEpe3JldHVybiJmdW5jdGlvbiI9PWVhKGEpfSxtPWZ1bmN0aW9uKGEpe3ZhciBiPXR5cGVvZiBhO3JldHVybiJvYmplY3QiPT1iJiZudWxsIT1hfHwiZnVuY3Rpb24iPT1ifSxqYT0iY2xvc3VyZV91aWRfIisoMUU5Kk1hdGgucmFuZG9tKCk+Pj4wKSxrYT0wLGxhPWZ1bmN0aW9uKGEsYil7dmFyIGM9QXJyYXkucHJvdG90eXBlLnNsaWNlLmNhbGwoYXJndW1lbnRzLDEpO3JldHVybiBmdW5jdGlvbigpe3ZhciBkPWMuc2xpY2UoKTtkLnB1c2guYXBwbHkoZCxhcmd1bWVudHMpO3JldHVybiBhLmFwcGx5KHRoaXMsZCl9fSxuPWZ1bmN0aW9uKGEsYil7ZnVuY3Rpb24gYygpe31jLnByb3RvdHlwZT1iLnByb3RvdHlwZTthLmk9Yi5wcm90b3R5cGU7YS5wcm90b3R5cGU9bmV3IGM7YS5wcm90b3R5cGUuY29uc3RydWN0b3I9YX07dmFyIG1hLG5hPXtlYjoiYWN0aXZlZGVzY2VuZGFudCIsamI6ImF0b21pYyIsa2I6ImF1dG9jb21wbGV0ZSIsbWI6ImJ1c3kiLHBiOiJjaGVja2VkIixxYjoiY29saW5kZXgiLHZiOiJjb250cm9scyIseGI6ImRlc2NyaWJlZGJ5IixBYjoiZGlzYWJsZWQiLENiOiJkcm9wZWZmZWN0IixEYjoiZXhwYW5kZWQiLEViOiJmbG93dG8iLEdiOiJncmFiYmVkIixLYjoiaGFzcG9wdXAiLE1iOiJoaWRkZW4iLE9iOiJpbnZhbGlkIixQYjoibGFiZWwiLFFiOiJsYWJlbGxlZGJ5IixSYjoibGV2ZWwiLFdiOiJsaXZlIixrYzoibXVsdGlsaW5lIixsYzoibXVsdGlzZWxlY3RhYmxlIixwYzoib3JpZW50YXRpb24iLHFjOiJvd25zIixyYzoicG9zaW5zZXQiLHRjOiJwcmVzc2VkIix4YzoicmVhZG9ubHkiLHpjOiJyZWxldmFudCIsQWM6InJlcXVpcmVkIixFYzoicm93aW5kZXgiLEhjOiJzZWxlY3RlZCIsSmM6InNldHNpemUiLExjOiJzb3J0IixaYzoidmFsdWVtYXgiLCRjOiJ2YWx1ZW1pbiIsYWQ6InZhbHVlbm93IiwKYmQ6InZhbHVldGV4dCJ9O3ZhciBvYT1mdW5jdGlvbihhLGIsYyl7Zm9yKHZhciBkIGluIGEpYi5jYWxsKGMsYVtkXSxkLGEpfSxwYT1mdW5jdGlvbihhLGIpe2Zvcih2YXIgYyBpbiBhKWlmKGFbY109PWIpcmV0dXJuITA7cmV0dXJuITF9LHFhPWZ1bmN0aW9uKGEsYixjKXtpZihudWxsIT09YSYmYiBpbiBhKXRocm93IEVycm9yKCdUaGUgb2JqZWN0IGFscmVhZHkgY29udGFpbnMgdGhlIGtleSAiJytiKyciJyk7YVtiXT1jfSxzYT1mdW5jdGlvbihhKXt2YXIgYj17fSxjO2ZvcihjIGluIGEpYlthW2NdXT1jO3JldHVybiBifSx0YT0iY29uc3RydWN0b3IgaGFzT3duUHJvcGVydHkgaXNQcm90b3R5cGVPZiBwcm9wZXJ0eUlzRW51bWVyYWJsZSB0b0xvY2FsZVN0cmluZyB0b1N0cmluZyB2YWx1ZU9mIi5zcGxpdCgiICIpLHVhPWZ1bmN0aW9uKGEsYil7Zm9yKHZhciBjLGQsZT0xO2U8YXJndW1lbnRzLmxlbmd0aDtlKyspe2Q9YXJndW1lbnRzW2VdO2ZvcihjIGluIGQpYVtjXT1kW2NdO2Zvcih2YXIgZj0wO2Y8dGEubGVuZ3RoO2YrKyljPQp0YVtmXSxPYmplY3QucHJvdG90eXBlLmhhc093blByb3BlcnR5LmNhbGwoZCxjKSYmKGFbY109ZFtjXSl9fTt2YXIgdmE9e2ZiOiJhbGVydCIsZ2I6ImFsZXJ0ZGlhbG9nIixoYjoiYXBwbGljYXRpb24iLGliOiJhcnRpY2xlIixsYjoiYmFubmVyIixuYjoiYnV0dG9uIixvYjoiY2hlY2tib3giLHJiOiJjb2x1bW5oZWFkZXIiLHNiOiJjb21ib2JveCIsdGI6ImNvbXBsZW1lbnRhcnkiLHViOiJjb250ZW50aW5mbyIsd2I6ImRlZmluaXRpb24iLHliOiJkaWFsb2ciLHpiOiJkaXJlY3RvcnkiLEJiOiJkb2N1bWVudCIsRmI6ImZvcm0iLEhiOiJncmlkIixJYjoiZ3JpZGNlbGwiLEpiOiJncm91cCIsTGI6ImhlYWRpbmciLE5iOiJpbWciLFNiOiJsaW5rIixUYjoibGlzdCIsVWI6Imxpc3Rib3giLFZiOiJsaXN0aXRlbSIsWGI6ImxvZyIsWWI6Im1haW4iLFpiOiJtYXJxdWVlIiwkYjoibWF0aCIsYWM6Im1lbnUiLGJjOiJtZW51YmFyIixjYzoibWVudWl0ZW0iLGRjOiJtZW51aXRlbWNoZWNrYm94IixlYzoibWVudWl0ZW1yYWRpbyIsbWM6Im5hdmlnYXRpb24iLG5jOiJub3RlIixvYzoib3B0aW9uIiwKc2M6InByZXNlbnRhdGlvbiIsdWM6InByb2dyZXNzYmFyIix2YzoicmFkaW8iLHdjOiJyYWRpb2dyb3VwIix5YzoicmVnaW9uIixCYzoicm93IixDYzoicm93Z3JvdXAiLERjOiJyb3doZWFkZXIiLEZjOiJzY3JvbGxiYXIiLEdjOiJzZWFyY2giLEljOiJzZXBhcmF0b3IiLEtjOiJzbGlkZXIiLE1jOiJzcGluYnV0dG9uIixOYzoic3RhdHVzIixPYzoidGFiIixQYzoidGFibGlzdCIsUWM6InRhYnBhbmVsIixSYzoidGV4dGJveCIsU2M6InRleHRpbmZvIixUYzoidGltZXIiLFVjOiJ0b29sYmFyIixWYzoidG9vbHRpcCIsV2M6InRyZWUiLFhjOiJ0cmVlZ3JpZCIsWWM6InRyZWVpdGVtIn07dmFyIHdhPWZ1bmN0aW9uKGEpe2lmKEVycm9yLmNhcHR1cmVTdGFja1RyYWNlKUVycm9yLmNhcHR1cmVTdGFja1RyYWNlKHRoaXMsd2EpO2Vsc2V7dmFyIGI9RXJyb3IoKS5zdGFjaztiJiYodGhpcy5zdGFjaz1iKX1hJiYodGhpcy5tZXNzYWdlPVN0cmluZyhhKSl9O24od2EsRXJyb3IpO3dhLnByb3RvdHlwZS5uYW1lPSJDdXN0b21FcnJvciI7dmFyIHhhO3ZhciB5YT1mdW5jdGlvbihhLGIpe2E9YS5zcGxpdCgiJXMiKTtmb3IodmFyIGM9IiIsZD1hLmxlbmd0aC0xLGU9MDtlPGQ7ZSsrKWMrPWFbZV0rKGU8Yi5sZW5ndGg/YltlXToiJXMiKTt3YS5jYWxsKHRoaXMsYythW2RdKX07bih5YSx3YSk7eWEucHJvdG90eXBlLm5hbWU9IkFzc2VydGlvbkVycm9yIjsKdmFyIHphPWZ1bmN0aW9uKGEsYixjLGQpe3ZhciBlPSJBc3NlcnRpb24gZmFpbGVkIjtpZihjKXtlKz0iOiAiK2M7dmFyIGY9ZH1lbHNlIGEmJihlKz0iOiAiK2EsZj1iKTt0aHJvdyBuZXcgeWEoIiIrZSxmfHxbXSk7fSxxPWZ1bmN0aW9uKGEsYixjKXthfHx6YSgiIixudWxsLGIsQXJyYXkucHJvdG90eXBlLnNsaWNlLmNhbGwoYXJndW1lbnRzLDIpKTtyZXR1cm4gYX0sQWE9ZnVuY3Rpb24oYSxiLGMpe20oYSkmJjE9PWEubm9kZVR5cGV8fHphKCJFeHBlY3RlZCBFbGVtZW50IGJ1dCBnb3QgJXM6ICVzLiIsW2VhKGEpLGFdLGIsQXJyYXkucHJvdG90eXBlLnNsaWNlLmNhbGwoYXJndW1lbnRzLDIpKX0sQ2E9ZnVuY3Rpb24oYSxiLGMsZCl7YSBpbnN0YW5jZW9mIGJ8fHphKCJFeHBlY3RlZCBpbnN0YW5jZW9mICVzIGJ1dCBnb3QgJXMuIixbQmEoYiksQmEoYSldLGMsQXJyYXkucHJvdG90eXBlLnNsaWNlLmNhbGwoYXJndW1lbnRzLDMpKX0sQmE9ZnVuY3Rpb24oYSl7cmV0dXJuIGEgaW5zdGFuY2VvZgpGdW5jdGlvbj9hLmRpc3BsYXlOYW1lfHxhLm5hbWV8fCJ1bmtub3duIHR5cGUgbmFtZSI6YSBpbnN0YW5jZW9mIE9iamVjdD9hLmNvbnN0cnVjdG9yLmRpc3BsYXlOYW1lfHxhLmNvbnN0cnVjdG9yLm5hbWV8fE9iamVjdC5wcm90b3R5cGUudG9TdHJpbmcuY2FsbChhKTpudWxsPT09YT8ibnVsbCI6dHlwZW9mIGF9O3ZhciBEYT1BcnJheS5wcm90b3R5cGUuaW5kZXhPZj9mdW5jdGlvbihhLGIpe3EobnVsbCE9YS5sZW5ndGgpO3JldHVybiBBcnJheS5wcm90b3R5cGUuaW5kZXhPZi5jYWxsKGEsYix2b2lkIDApfTpmdW5jdGlvbihhLGIpe2lmKCJzdHJpbmciPT09dHlwZW9mIGEpcmV0dXJuInN0cmluZyIhPT10eXBlb2YgYnx8MSE9Yi5sZW5ndGg/LTE6YS5pbmRleE9mKGIsMCk7Zm9yKHZhciBjPTA7YzxhLmxlbmd0aDtjKyspaWYoYyBpbiBhJiZhW2NdPT09YilyZXR1cm4gYztyZXR1cm4tMX0scj1BcnJheS5wcm90b3R5cGUuZm9yRWFjaD9mdW5jdGlvbihhLGIsYyl7cShudWxsIT1hLmxlbmd0aCk7QXJyYXkucHJvdG90eXBlLmZvckVhY2guY2FsbChhLGIsYyl9OmZ1bmN0aW9uKGEsYixjKXtmb3IodmFyIGQ9YS5sZW5ndGgsZT0ic3RyaW5nIj09PXR5cGVvZiBhP2Euc3BsaXQoIiIpOmEsZj0wO2Y8ZDtmKyspZiBpbiBlJiZiLmNhbGwoYyxlW2ZdLGYsYSl9LEVhPUFycmF5LnByb3RvdHlwZS5maWx0ZXI/CmZ1bmN0aW9uKGEsYil7cShudWxsIT1hLmxlbmd0aCk7cmV0dXJuIEFycmF5LnByb3RvdHlwZS5maWx0ZXIuY2FsbChhLGIsdm9pZCAwKX06ZnVuY3Rpb24oYSxiKXtmb3IodmFyIGM9YS5sZW5ndGgsZD1bXSxlPTAsZj0ic3RyaW5nIj09PXR5cGVvZiBhP2Euc3BsaXQoIiIpOmEsZz0wO2c8YztnKyspaWYoZyBpbiBmKXt2YXIgaz1mW2ddO2IuY2FsbCh2b2lkIDAsayxnLGEpJiYoZFtlKytdPWspfXJldHVybiBkfSxGYT1BcnJheS5wcm90b3R5cGUuZXZlcnk/ZnVuY3Rpb24oYSxiKXtxKG51bGwhPWEubGVuZ3RoKTtyZXR1cm4gQXJyYXkucHJvdG90eXBlLmV2ZXJ5LmNhbGwoYSxiLHZvaWQgMCl9OmZ1bmN0aW9uKGEsYil7Zm9yKHZhciBjPWEubGVuZ3RoLGQ9InN0cmluZyI9PT10eXBlb2YgYT9hLnNwbGl0KCIiKTphLGU9MDtlPGM7ZSsrKWlmKGUgaW4gZCYmIWIuY2FsbCh2b2lkIDAsZFtlXSxlLGEpKXJldHVybiExO3JldHVybiEwfSxHYT1mdW5jdGlvbihhLGIpe3JldHVybiAwPD0KRGEoYSxiKX0sSGE9ZnVuY3Rpb24oYSxiKXtiPURhKGEsYik7dmFyIGM7aWYoYz0wPD1iKXEobnVsbCE9YS5sZW5ndGgpLEFycmF5LnByb3RvdHlwZS5zcGxpY2UuY2FsbChhLGIsMSk7cmV0dXJuIGN9LElhPWZ1bmN0aW9uKGEpe3JldHVybiBBcnJheS5wcm90b3R5cGUuY29uY2F0LmFwcGx5KFtdLGFyZ3VtZW50cyl9LEphPWZ1bmN0aW9uKGEpe3ZhciBiPWEubGVuZ3RoO2lmKDA8Yil7Zm9yKHZhciBjPUFycmF5KGIpLGQ9MDtkPGI7ZCsrKWNbZF09YVtkXTtyZXR1cm4gY31yZXR1cm5bXX0sTGE9ZnVuY3Rpb24oYSxiLGMsZCl7cShudWxsIT1hLmxlbmd0aCk7QXJyYXkucHJvdG90eXBlLnNwbGljZS5hcHBseShhLEthKGFyZ3VtZW50cywxKSl9LEthPWZ1bmN0aW9uKGEsYixjKXtxKG51bGwhPWEubGVuZ3RoKTtyZXR1cm4gMj49YXJndW1lbnRzLmxlbmd0aD9BcnJheS5wcm90b3R5cGUuc2xpY2UuY2FsbChhLGIpOkFycmF5LnByb3RvdHlwZS5zbGljZS5jYWxsKGEsYixjKX07dmFyIE1hPVN0cmluZy5wcm90b3R5cGUudHJpbT9mdW5jdGlvbihhKXtyZXR1cm4gYS50cmltKCl9OmZ1bmN0aW9uKGEpe3JldHVybi9eW1xzXHhhMF0qKFtcc1xTXSo/KVtcc1x4YTBdKiQvLmV4ZWMoYSlbMV19LE5hPS8mL2csT2E9LzwvZyxQYT0vPi9nLFFhPS8iL2csUmE9LycvZyxTYT0vXHgwMC9nLFRhPS9bXHgwMCY8PiInXS8sdD1mdW5jdGlvbihhLGIpe3JldHVybi0xIT1hLmluZGV4T2YoYil9LFVhPWZ1bmN0aW9uKGEsYil7cmV0dXJuIGE8Yj8tMTphPmI/MTowfTt2YXIgdTthOnt2YXIgVmE9bC5uYXZpZ2F0b3I7aWYoVmEpe3ZhciBXYT1WYS51c2VyQWdlbnQ7aWYoV2Epe3U9V2E7YnJlYWsgYX19dT0iIn07LyoKCiBDb3B5cmlnaHQgVGhlIENsb3N1cmUgTGlicmFyeSBBdXRob3JzLgogU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEFwYWNoZS0yLjAKKi8KdmFyIFhhPWZ1bmN0aW9uKGEpe1RhLnRlc3QoYSkmJigtMSE9YS5pbmRleE9mKCImIikmJihhPWEucmVwbGFjZShOYSwiJmFtcDsiKSksLTEhPWEuaW5kZXhPZigiPCIpJiYoYT1hLnJlcGxhY2UoT2EsIiZsdDsiKSksLTEhPWEuaW5kZXhPZigiPiIpJiYoYT1hLnJlcGxhY2UoUGEsIiZndDsiKSksLTEhPWEuaW5kZXhPZignIicpJiYoYT1hLnJlcGxhY2UoUWEsIiZxdW90OyIpKSwtMSE9YS5pbmRleE9mKCInIikmJihhPWEucmVwbGFjZShSYSwiJiMzOTsiKSksLTEhPWEuaW5kZXhPZigiXHgwMCIpJiYoYT1hLnJlcGxhY2UoU2EsIiYjMDsiKSkpO3JldHVybiBhfTt2YXIgWWE9ZnVuY3Rpb24oYSl7WWFbIiAiXShhKTtyZXR1cm4gYX07WWFbIiAiXT1jYTt2YXIgJGE9ZnVuY3Rpb24oYSxiKXt2YXIgYz1aYTtyZXR1cm4gT2JqZWN0LnByb3RvdHlwZS5oYXNPd25Qcm9wZXJ0eS5jYWxsKGMsYSk/Y1thXTpjW2FdPWIoYSl9O3ZhciBhYj10KHUsIk9wZXJhIiksdj10KHUsIlRyaWRlbnQiKXx8dCh1LCJNU0lFIiksYmI9dCh1LCJFZGdlIikseD10KHUsIkdlY2tvIikmJiEodCh1LnRvTG93ZXJDYXNlKCksIndlYmtpdCIpJiYhdCh1LCJFZGdlIikpJiYhKHQodSwiVHJpZGVudCIpfHx0KHUsIk1TSUUiKSkmJiF0KHUsIkVkZ2UiKSx5PXQodS50b0xvd2VyQ2FzZSgpLCJ3ZWJraXQiKSYmIXQodSwiRWRnZSIpLHo9dCh1LCJNYWNpbnRvc2giKSxjYj1mdW5jdGlvbigpe3ZhciBhPWwuZG9jdW1lbnQ7cmV0dXJuIGE/YS5kb2N1bWVudE1vZGU6dm9pZCAwfSxkYjsKYTp7dmFyIGViPSIiLGZiPWZ1bmN0aW9uKCl7dmFyIGE9dTtpZih4KXJldHVybi9ydjooW15cKTtdKykoXCl8OykvLmV4ZWMoYSk7aWYoYmIpcmV0dXJuL0VkZ2VcLyhbXGRcLl0rKS8uZXhlYyhhKTtpZih2KXJldHVybi9cYig/Ok1TSUV8cnYpWzogXShbXlwpO10rKShcKXw7KS8uZXhlYyhhKTtpZih5KXJldHVybi9XZWJLaXRcLyhcUyspLy5leGVjKGEpO2lmKGFiKXJldHVybi8oPzpWZXJzaW9uKVsgXC9dPyhcUyspLy5leGVjKGEpfSgpO2ZiJiYoZWI9ZmI/ZmJbMV06IiIpO2lmKHYpe3ZhciBnYj1jYigpO2lmKG51bGwhPWdiJiZnYj5wYXJzZUZsb2F0KGViKSl7ZGI9U3RyaW5nKGdiKTticmVhayBhfX1kYj1lYn0KdmFyIGhiPWRiLFphPXt9LEE9ZnVuY3Rpb24oYSl7cmV0dXJuICRhKGEsZnVuY3Rpb24oKXtmb3IodmFyIGI9MCxjPU1hKFN0cmluZyhoYikpLnNwbGl0KCIuIiksZD1NYShTdHJpbmcoYSkpLnNwbGl0KCIuIiksZT1NYXRoLm1heChjLmxlbmd0aCxkLmxlbmd0aCksZj0wOzA9PWImJmY8ZTtmKyspe3ZhciBnPWNbZl18fCIiLGs9ZFtmXXx8IiI7ZG97Zz0vKFxkKikoXEQqKSguKikvLmV4ZWMoZyl8fFsiIiwiIiwiIiwiIl07az0vKFxkKikoXEQqKSguKikvLmV4ZWMoayl8fFsiIiwiIiwiIiwiIl07aWYoMD09Z1swXS5sZW5ndGgmJjA9PWtbMF0ubGVuZ3RoKWJyZWFrO2I9VWEoMD09Z1sxXS5sZW5ndGg/MDpwYXJzZUludChnWzFdLDEwKSwwPT1rWzFdLmxlbmd0aD8wOnBhcnNlSW50KGtbMV0sMTApKXx8VWEoMD09Z1syXS5sZW5ndGgsMD09a1syXS5sZW5ndGgpfHxVYShnWzJdLGtbMl0pO2c9Z1szXTtrPWtbM119d2hpbGUoMD09Yil9cmV0dXJuIDA8PWJ9KX0saWI7CmliPWwuZG9jdW1lbnQmJnY/Y2IoKTp2b2lkIDA7dmFyIGpiPSF2fHw5PD1OdW1iZXIoaWIpO3ZhciBsYj1mdW5jdGlvbihhLGIpe29hKGIsZnVuY3Rpb24oYyxkKXtjJiYib2JqZWN0Ij09dHlwZW9mIGMmJmMuZGQmJihjPWMuY2QoKSk7InN0eWxlIj09ZD9hLnN0eWxlLmNzc1RleHQ9YzoiY2xhc3MiPT1kP2EuY2xhc3NOYW1lPWM6ImZvciI9PWQ/YS5odG1sRm9yPWM6a2IuaGFzT3duUHJvcGVydHkoZCk/YS5zZXRBdHRyaWJ1dGUoa2JbZF0sYyk6MD09ZC5sYXN0SW5kZXhPZigiYXJpYS0iLDApfHwwPT1kLmxhc3RJbmRleE9mKCJkYXRhLSIsMCk/YS5zZXRBdHRyaWJ1dGUoZCxjKTphW2RdPWN9KX0sa2I9e2NlbGxwYWRkaW5nOiJjZWxsUGFkZGluZyIsY2VsbHNwYWNpbmc6ImNlbGxTcGFjaW5nIixjb2xzcGFuOiJjb2xTcGFuIixmcmFtZWJvcmRlcjoiZnJhbWVCb3JkZXIiLGhlaWdodDoiaGVpZ2h0IixtYXhsZW5ndGg6Im1heExlbmd0aCIsbm9uY2U6Im5vbmNlIixyb2xlOiJyb2xlIixyb3dzcGFuOiJyb3dTcGFuIix0eXBlOiJ0eXBlIix1c2VtYXA6InVzZU1hcCIsdmFsaWduOiJ2QWxpZ24iLAp3aWR0aDoid2lkdGgifSxtYj1mdW5jdGlvbihhLGIsYyl7ZnVuY3Rpb24gZChrKXtrJiZiLmFwcGVuZENoaWxkKCJzdHJpbmciPT09dHlwZW9mIGs/YS5jcmVhdGVUZXh0Tm9kZShrKTprKX1mb3IodmFyIGU9MjtlPGMubGVuZ3RoO2UrKyl7dmFyIGY9Y1tlXTtpZighaGEoZil8fG0oZikmJjA8Zi5ub2RlVHlwZSlkKGYpO2Vsc2V7YTp7aWYoZiYmIm51bWJlciI9PXR5cGVvZiBmLmxlbmd0aCl7aWYobShmKSl7dmFyIGc9ImZ1bmN0aW9uIj09dHlwZW9mIGYuaXRlbXx8InN0cmluZyI9PXR5cGVvZiBmLml0ZW07YnJlYWsgYX1pZihpYShmKSl7Zz0iZnVuY3Rpb24iPT10eXBlb2YgZi5pdGVtO2JyZWFrIGF9fWc9ITF9cihnP0phKGYpOmYsZCl9fX0sbmI9ZnVuY3Rpb24oYSxiKXtiPVN0cmluZyhiKTsiYXBwbGljYXRpb24veGh0bWwreG1sIj09PWEuY29udGVudFR5cGUmJihiPWIudG9Mb3dlckNhc2UoKSk7cmV0dXJuIGEuY3JlYXRlRWxlbWVudChiKX0sb2I9ZnVuY3Rpb24oYSl7YSYmCmEucGFyZW50Tm9kZSYmYS5wYXJlbnROb2RlLnJlbW92ZUNoaWxkKGEpfSxwYj1mdW5jdGlvbihhLGIpe2lmKCFhfHwhYilyZXR1cm4hMTtpZihhLmNvbnRhaW5zJiYxPT1iLm5vZGVUeXBlKXJldHVybiBhPT1ifHxhLmNvbnRhaW5zKGIpO2lmKCJ1bmRlZmluZWQiIT10eXBlb2YgYS5jb21wYXJlRG9jdW1lbnRQb3NpdGlvbilyZXR1cm4gYT09Ynx8ISEoYS5jb21wYXJlRG9jdW1lbnRQb3NpdGlvbihiKSYxNik7Zm9yKDtiJiZhIT1iOyliPWIucGFyZW50Tm9kZTtyZXR1cm4gYj09YX0scWI9ZnVuY3Rpb24oYSl7cShhLCJOb2RlIGNhbm5vdCBiZSBudWxsIG9yIHVuZGVmaW5lZC4iKTtyZXR1cm4gOT09YS5ub2RlVHlwZT9hOmEub3duZXJEb2N1bWVudHx8YS5kb2N1bWVudH0scmI9ZnVuY3Rpb24oYSxiKXtiP2EudGFiSW5kZXg9MDooYS50YWJJbmRleD0tMSxhLnJlbW92ZUF0dHJpYnV0ZSgidGFiSW5kZXgiKSl9LHNiPWZ1bmN0aW9uKGEpe3JldHVybiB2JiYhQSgiOSIpPyhhPWEuZ2V0QXR0cmlidXRlTm9kZSgidGFiaW5kZXgiKSwKbnVsbCE9YSYmYS5zcGVjaWZpZWQpOmEuaGFzQXR0cmlidXRlKCJ0YWJpbmRleCIpfSx0Yj1mdW5jdGlvbihhKXthPWEudGFiSW5kZXg7cmV0dXJuIm51bWJlciI9PT10eXBlb2YgYSYmMDw9YSYmMzI3Njg+YX0sdWI9ZnVuY3Rpb24oYSl7dGhpcy5hPWF8fGwuZG9jdW1lbnR8fGRvY3VtZW50fTt1Yi5wcm90b3R5cGUuZj1mdW5jdGlvbihhKXtyZXR1cm4ic3RyaW5nIj09PXR5cGVvZiBhP3RoaXMuYS5nZXRFbGVtZW50QnlJZChhKTphfTsKdWIucHJvdG90eXBlLmI9ZnVuY3Rpb24oYSxiLGMpe3ZhciBkPXRoaXMuYSxlPWFyZ3VtZW50cyxmPVN0cmluZyhlWzBdKSxnPWVbMV07aWYoIWpiJiZnJiYoZy5uYW1lfHxnLnR5cGUpKXtmPVsiPCIsZl07Zy5uYW1lJiZmLnB1c2goJyBuYW1lPSInLFhhKGcubmFtZSksJyInKTtpZihnLnR5cGUpe2YucHVzaCgnIHR5cGU9IicsWGEoZy50eXBlKSwnIicpO3ZhciBrPXt9O3VhKGssZyk7ZGVsZXRlIGsudHlwZTtnPWt9Zi5wdXNoKCI+Iik7Zj1mLmpvaW4oIiIpfWY9bmIoZCxmKTtnJiYoInN0cmluZyI9PT10eXBlb2YgZz9mLmNsYXNzTmFtZT1nOmZhKGcpP2YuY2xhc3NOYW1lPWcuam9pbigiICIpOmxiKGYsZykpOzI8ZS5sZW5ndGgmJm1iKGQsZixlKTtyZXR1cm4gZn07dmFyIHZiPWZ1bmN0aW9uKGEsYil7Yj8ocShwYSh2YSxiKSwiTm8gc3VjaCBBUklBIHJvbGUgIitiKSxhLnNldEF0dHJpYnV0ZSgicm9sZSIsYikpOmEucmVtb3ZlQXR0cmlidXRlKCJyb2xlIil9LHhiPWZ1bmN0aW9uKGEsYixjKXtmYShjKSYmKGM9Yy5qb2luKCIgIikpO3ZhciBkPXdiKGIpOyIiPT09Y3x8dm9pZCAwPT1jPyhtYXx8KG1hPXthdG9taWM6ITEsYXV0b2NvbXBsZXRlOiJub25lIixkcm9wZWZmZWN0OiJub25lIixoYXNwb3B1cDohMSxsaXZlOiJvZmYiLG11bHRpbGluZTohMSxtdWx0aXNlbGVjdGFibGU6ITEsb3JpZW50YXRpb246InZlcnRpY2FsIixyZWFkb25seTohMSxyZWxldmFudDoiYWRkaXRpb25zIHRleHQiLHJlcXVpcmVkOiExLHNvcnQ6Im5vbmUiLGJ1c3k6ITEsZGlzYWJsZWQ6ITEsaGlkZGVuOiExLGludmFsaWQ6ImZhbHNlIn0pLGM9bWEsYiBpbiBjP2Euc2V0QXR0cmlidXRlKGQsY1tiXSk6YS5yZW1vdmVBdHRyaWJ1dGUoZCkpOmEuc2V0QXR0cmlidXRlKGQsCmMpfSx3Yj1mdW5jdGlvbihhKXtxKGEsIkFSSUEgYXR0cmlidXRlIGNhbm5vdCBiZSBlbXB0eS4iKTtxKHBhKG5hLGEpLCJObyBzdWNoIEFSSUEgYXR0cmlidXRlICIrYSk7cmV0dXJuImFyaWEtIithfTt2YXIgeWI9T2JqZWN0LmZyZWV6ZXx8ZnVuY3Rpb24oYSl7cmV0dXJuIGF9O3ZhciBDPWZ1bmN0aW9uKCl7dGhpcy5VPXRoaXMuVTt0aGlzLko9dGhpcy5KfTtDLnByb3RvdHlwZS5VPSExO0MucHJvdG90eXBlLk49ZnVuY3Rpb24oKXt0aGlzLlV8fCh0aGlzLlU9ITAsdGhpcy51KCkpfTt2YXIgemI9ZnVuY3Rpb24oYSxiKXthLlU/YigpOihhLkp8fChhLko9W10pLGEuSi5wdXNoKGIpKX07Qy5wcm90b3R5cGUudT1mdW5jdGlvbigpe2lmKHRoaXMuSilmb3IoO3RoaXMuSi5sZW5ndGg7KXRoaXMuSi5zaGlmdCgpKCl9O3ZhciBBYj1mdW5jdGlvbihhKXthJiYiZnVuY3Rpb24iPT10eXBlb2YgYS5OJiZhLk4oKX07dmFyIEJiPWZ1bmN0aW9uKGEpe3JldHVybiJzdHJpbmciPT10eXBlb2YgYS5jbGFzc05hbWU/YS5jbGFzc05hbWU6YS5nZXRBdHRyaWJ1dGUmJmEuZ2V0QXR0cmlidXRlKCJjbGFzcyIpfHwiIn0sRD1mdW5jdGlvbihhKXtyZXR1cm4gYS5jbGFzc0xpc3Q/YS5jbGFzc0xpc3Q6QmIoYSkubWF0Y2goL1xTKy9nKXx8W119LENiPWZ1bmN0aW9uKGEsYil7InN0cmluZyI9PXR5cGVvZiBhLmNsYXNzTmFtZT9hLmNsYXNzTmFtZT1iOmEuc2V0QXR0cmlidXRlJiZhLnNldEF0dHJpYnV0ZSgiY2xhc3MiLGIpfSxEYj1mdW5jdGlvbihhLGIpe3JldHVybiBhLmNsYXNzTGlzdD9hLmNsYXNzTGlzdC5jb250YWlucyhiKTpHYShEKGEpLGIpfSxFYj1mdW5jdGlvbihhLGIpe2lmKGEuY2xhc3NMaXN0KWEuY2xhc3NMaXN0LmFkZChiKTtlbHNlIGlmKCFEYihhLGIpKXt2YXIgYz1CYihhKTtDYihhLGMrKDA8Yy5sZW5ndGg/IiAiK2I6YikpfX0sRmI9ZnVuY3Rpb24oYSxiKXtpZihhLmNsYXNzTGlzdClyKGIsCmZ1bmN0aW9uKGUpe0ViKGEsZSl9KTtlbHNle3ZhciBjPXt9O3IoRChhKSxmdW5jdGlvbihlKXtjW2VdPSEwfSk7cihiLGZ1bmN0aW9uKGUpe2NbZV09ITB9KTtiPSIiO2Zvcih2YXIgZCBpbiBjKWIrPTA8Yi5sZW5ndGg/IiAiK2Q6ZDtDYihhLGIpfX0sR2I9ZnVuY3Rpb24oYSxiKXthLmNsYXNzTGlzdD9hLmNsYXNzTGlzdC5yZW1vdmUoYik6RGIoYSxiKSYmQ2IoYSxFYShEKGEpLGZ1bmN0aW9uKGMpe3JldHVybiBjIT1ifSkuam9pbigiICIpKX0sSGI9ZnVuY3Rpb24oYSxiKXthLmNsYXNzTGlzdD9yKGIsZnVuY3Rpb24oYyl7R2IoYSxjKX0pOkNiKGEsRWEoRChhKSxmdW5jdGlvbihjKXtyZXR1cm4hR2EoYixjKX0pLmpvaW4oIiAiKSl9O3ZhciBJYj0hdnx8OTw9TnVtYmVyKGliKSxKYj0hdnx8OTw9TnVtYmVyKGliKSxLYj12JiYhQSgiOSIpLExiPWZ1bmN0aW9uKCl7aWYoIWwuYWRkRXZlbnRMaXN0ZW5lcnx8IU9iamVjdC5kZWZpbmVQcm9wZXJ0eSlyZXR1cm4hMTt2YXIgYT0hMSxiPU9iamVjdC5kZWZpbmVQcm9wZXJ0eSh7fSwicGFzc2l2ZSIse2dldDpmdW5jdGlvbigpe2E9ITB9fSk7dHJ5e2wuYWRkRXZlbnRMaXN0ZW5lcigidGVzdCIsY2EsYiksbC5yZW1vdmVFdmVudExpc3RlbmVyKCJ0ZXN0IixjYSxiKX1jYXRjaChjKXt9cmV0dXJuIGF9KCk7dmFyIEU9ZnVuY3Rpb24oYSxiKXt0aGlzLnR5cGU9YTt0aGlzLmE9dGhpcy50YXJnZXQ9Yjt0aGlzLmg9ITE7dGhpcy5LYT0hMH07RS5wcm90b3R5cGUuaj1mdW5jdGlvbigpe3RoaXMuaD0hMH07RS5wcm90b3R5cGUuZz1mdW5jdGlvbigpe3RoaXMuS2E9ITF9O3ZhciBGPXtMOiJtb3VzZWRvd24iLE06Im1vdXNldXAiLGFhOiJtb3VzZWNhbmNlbCIsaGM6Im1vdXNlbW92ZSIsamM6Im1vdXNlb3ZlciIsaWM6Im1vdXNlb3V0IixmYzoibW91c2VlbnRlciIsZ2M6Im1vdXNlbGVhdmUifTt2YXIgRz1mdW5jdGlvbihhLGIpe0UuY2FsbCh0aGlzLGE/YS50eXBlOiIiKTt0aGlzLnJlbGF0ZWRUYXJnZXQ9dGhpcy5hPXRoaXMudGFyZ2V0PW51bGw7dGhpcy5idXR0b249dGhpcy5zY3JlZW5ZPXRoaXMuc2NyZWVuWD10aGlzLmNsaWVudFk9dGhpcy5jbGllbnRYPTA7dGhpcy5rZXk9IiI7dGhpcy5jPTA7dGhpcy5BPXRoaXMubWV0YUtleT10aGlzLnNoaWZ0S2V5PXRoaXMuYWx0S2V5PXRoaXMuY3RybEtleT0hMTt0aGlzLnBvaW50ZXJJZD0wO3RoaXMucG9pbnRlclR5cGU9IiI7dGhpcy5iPW51bGw7aWYoYSl7dmFyIGM9dGhpcy50eXBlPWEudHlwZSxkPWEuY2hhbmdlZFRvdWNoZXMmJmEuY2hhbmdlZFRvdWNoZXMubGVuZ3RoP2EuY2hhbmdlZFRvdWNoZXNbMF06bnVsbDt0aGlzLnRhcmdldD1hLnRhcmdldHx8YS5zcmNFbGVtZW50O3RoaXMuYT1iO2lmKGI9YS5yZWxhdGVkVGFyZ2V0KXtpZih4KXthOnt0cnl7WWEoYi5ub2RlTmFtZSk7dmFyIGU9ITA7YnJlYWsgYX1jYXRjaChmKXt9ZT0KITF9ZXx8KGI9bnVsbCl9fWVsc2UibW91c2VvdmVyIj09Yz9iPWEuZnJvbUVsZW1lbnQ6Im1vdXNlb3V0Ij09YyYmKGI9YS50b0VsZW1lbnQpO3RoaXMucmVsYXRlZFRhcmdldD1iO2Q/KHRoaXMuY2xpZW50WD12b2lkIDAhPT1kLmNsaWVudFg/ZC5jbGllbnRYOmQucGFnZVgsdGhpcy5jbGllbnRZPXZvaWQgMCE9PWQuY2xpZW50WT9kLmNsaWVudFk6ZC5wYWdlWSx0aGlzLnNjcmVlblg9ZC5zY3JlZW5YfHwwLHRoaXMuc2NyZWVuWT1kLnNjcmVlbll8fDApOih0aGlzLmNsaWVudFg9dm9pZCAwIT09YS5jbGllbnRYP2EuY2xpZW50WDphLnBhZ2VYLHRoaXMuY2xpZW50WT12b2lkIDAhPT1hLmNsaWVudFk/YS5jbGllbnRZOmEucGFnZVksdGhpcy5zY3JlZW5YPWEuc2NyZWVuWHx8MCx0aGlzLnNjcmVlblk9YS5zY3JlZW5ZfHwwKTt0aGlzLmJ1dHRvbj1hLmJ1dHRvbjt0aGlzLmM9YS5rZXlDb2RlfHwwO3RoaXMua2V5PWEua2V5fHwiIjt0aGlzLmN0cmxLZXk9YS5jdHJsS2V5O3RoaXMuYWx0S2V5PQphLmFsdEtleTt0aGlzLnNoaWZ0S2V5PWEuc2hpZnRLZXk7dGhpcy5tZXRhS2V5PWEubWV0YUtleTt0aGlzLkE9ej9hLm1ldGFLZXk6YS5jdHJsS2V5O3RoaXMucG9pbnRlcklkPWEucG9pbnRlcklkfHwwO3RoaXMucG9pbnRlclR5cGU9InN0cmluZyI9PT10eXBlb2YgYS5wb2ludGVyVHlwZT9hLnBvaW50ZXJUeXBlOk1iW2EucG9pbnRlclR5cGVdfHwiIjt0aGlzLmI9YTthLmRlZmF1bHRQcmV2ZW50ZWQmJnRoaXMuZygpfX07bihHLEUpO3ZhciBOYj15YihbMSw0LDJdKSxNYj15Yih7MjoidG91Y2giLDM6InBlbiIsNDoibW91c2UifSksT2I9ZnVuY3Rpb24oYSl7cmV0dXJuIEliPzA9PWEuYi5idXR0b246ImNsaWNrIj09YS50eXBlPyEwOiEhKGEuYi5idXR0b24mTmJbMF0pfTtHLnByb3RvdHlwZS5qPWZ1bmN0aW9uKCl7Ry5pLmouY2FsbCh0aGlzKTt0aGlzLmIuc3RvcFByb3BhZ2F0aW9uP3RoaXMuYi5zdG9wUHJvcGFnYXRpb24oKTp0aGlzLmIuY2FuY2VsQnViYmxlPSEwfTsKRy5wcm90b3R5cGUuZz1mdW5jdGlvbigpe0cuaS5nLmNhbGwodGhpcyk7dmFyIGE9dGhpcy5iO2lmKGEucHJldmVudERlZmF1bHQpYS5wcmV2ZW50RGVmYXVsdCgpO2Vsc2UgaWYoYS5yZXR1cm5WYWx1ZT0hMSxLYil0cnl7aWYoYS5jdHJsS2V5fHwxMTI8PWEua2V5Q29kZSYmMTIzPj1hLmtleUNvZGUpYS5rZXlDb2RlPS0xfWNhdGNoKGIpe319O3ZhciBQYj0iY2xvc3VyZV9saXN0ZW5hYmxlXyIrKDFFNipNYXRoLnJhbmRvbSgpfDApLFFiPWZ1bmN0aW9uKGEpe3JldHVybiEoIWF8fCFhW1BiXSl9LFJiPTA7dmFyIFNiPWZ1bmN0aW9uKGEsYixjLGQsZSl7dGhpcy5saXN0ZW5lcj1hO3RoaXMuYT1udWxsO3RoaXMuc3JjPWI7dGhpcy50eXBlPWM7dGhpcy5jYXB0dXJlPSEhZDt0aGlzLm1hPWU7dGhpcy5rZXk9KytSYjt0aGlzLlg9dGhpcy5oYT0hMX0sVGI9ZnVuY3Rpb24oYSl7YS5YPSEwO2EubGlzdGVuZXI9bnVsbDthLmE9bnVsbDthLnNyYz1udWxsO2EubWE9bnVsbH07dmFyIFViPWZ1bmN0aW9uKGEpe3RoaXMuc3JjPWE7dGhpcy5hPXt9O3RoaXMuYj0wfTtVYi5wcm90b3R5cGUuYWRkPWZ1bmN0aW9uKGEsYixjLGQsZSl7dmFyIGY9YS50b1N0cmluZygpO2E9dGhpcy5hW2ZdO2F8fChhPXRoaXMuYVtmXT1bXSx0aGlzLmIrKyk7dmFyIGc9VmIoYSxiLGQsZSk7LTE8Zz8oYj1hW2ddLGN8fChiLmhhPSExKSk6KGI9bmV3IFNiKGIsdGhpcy5zcmMsZiwhIWQsZSksYi5oYT1jLGEucHVzaChiKSk7cmV0dXJuIGJ9Owp2YXIgV2I9ZnVuY3Rpb24oYSxiKXt2YXIgYz1iLnR5cGU7YyBpbiBhLmEmJkhhKGEuYVtjXSxiKSYmKFRiKGIpLDA9PWEuYVtjXS5sZW5ndGgmJihkZWxldGUgYS5hW2NdLGEuYi0tKSl9LFhiPWZ1bmN0aW9uKGEsYixjLGQsZSl7YT1hLmFbYi50b1N0cmluZygpXTtiPS0xO2EmJihiPVZiKGEsYyxkLGUpKTtyZXR1cm4tMTxiP2FbYl06bnVsbH0sVmI9ZnVuY3Rpb24oYSxiLGMsZCl7Zm9yKHZhciBlPTA7ZTxhLmxlbmd0aDsrK2Upe3ZhciBmPWFbZV07aWYoIWYuWCYmZi5saXN0ZW5lcj09YiYmZi5jYXB0dXJlPT0hIWMmJmYubWE9PWQpcmV0dXJuIGV9cmV0dXJuLTF9O3ZhciBZYj0iY2xvc3VyZV9sbV8iKygxRTYqTWF0aC5yYW5kb20oKXwwKSxaYj17fSwkYj0wLGJjPWZ1bmN0aW9uKGEsYixjLGQsZSl7aWYoZCYmZC5vbmNlKXJldHVybiBhYyhhLGIsYyxkLGUpO2lmKGZhKGIpKXtmb3IodmFyIGY9MDtmPGIubGVuZ3RoO2YrKyliYyhhLGJbZl0sYyxkLGUpO3JldHVybiBudWxsfWM9Y2MoYyk7cmV0dXJuIFFiKGEpP2RjKGEsYixjLG0oZCk/ISFkLmNhcHR1cmU6ISFkLGUpOmVjKGEsYixjLCExLGQsZSl9LGVjPWZ1bmN0aW9uKGEsYixjLGQsZSxmKXtpZighYil0aHJvdyBFcnJvcigiSW52YWxpZCBldmVudCB0eXBlIik7dmFyIGc9bShlKT8hIWUuY2FwdHVyZTohIWUsaz1mYyhhKTtrfHwoYVtZYl09az1uZXcgVWIoYSkpO2M9ay5hZGQoYixjLGQsZyxmKTtpZihjLmEpcmV0dXJuIGM7ZD1nYygpO2MuYT1kO2Quc3JjPWE7ZC5saXN0ZW5lcj1jO2lmKGEuYWRkRXZlbnRMaXN0ZW5lcilMYnx8KGU9Zyksdm9pZCAwPT09ZSYmKGU9ITEpLGEuYWRkRXZlbnRMaXN0ZW5lcihiLnRvU3RyaW5nKCksCmQsZSk7ZWxzZSBpZihhLmF0dGFjaEV2ZW50KWEuYXR0YWNoRXZlbnQoaGMoYi50b1N0cmluZygpKSxkKTtlbHNlIGlmKGEuYWRkTGlzdGVuZXImJmEucmVtb3ZlTGlzdGVuZXIpcSgiY2hhbmdlIj09PWIsIk1lZGlhUXVlcnlMaXN0IG9ubHkgaGFzIGEgY2hhbmdlIGV2ZW50IiksYS5hZGRMaXN0ZW5lcihkKTtlbHNlIHRocm93IEVycm9yKCJhZGRFdmVudExpc3RlbmVyIGFuZCBhdHRhY2hFdmVudCBhcmUgdW5hdmFpbGFibGUuIik7JGIrKztyZXR1cm4gY30sZ2M9ZnVuY3Rpb24oKXt2YXIgYT1pYyxiPUpiP2Z1bmN0aW9uKGMpe3JldHVybiBhLmNhbGwoYi5zcmMsYi5saXN0ZW5lcixjKX06ZnVuY3Rpb24oYyl7Yz1hLmNhbGwoYi5zcmMsYi5saXN0ZW5lcixjKTtpZighYylyZXR1cm4gY307cmV0dXJuIGJ9LGFjPWZ1bmN0aW9uKGEsYixjLGQsZSl7aWYoZmEoYikpe2Zvcih2YXIgZj0wO2Y8Yi5sZW5ndGg7ZisrKWFjKGEsYltmXSxjLGQsZSk7cmV0dXJuIG51bGx9Yz1jYyhjKTtyZXR1cm4gUWIoYSk/CmEuaC5hZGQoU3RyaW5nKGIpLGMsITAsbShkKT8hIWQuY2FwdHVyZTohIWQsZSk6ZWMoYSxiLGMsITAsZCxlKX0samM9ZnVuY3Rpb24oYSxiLGMsZCxlKXtpZihmYShiKSlmb3IodmFyIGY9MDtmPGIubGVuZ3RoO2YrKylqYyhhLGJbZl0sYyxkLGUpO2Vsc2UgZD1tKGQpPyEhZC5jYXB0dXJlOiEhZCxjPWNjKGMpLFFiKGEpPyhhPWEuaCxiPVN0cmluZyhiKS50b1N0cmluZygpLGIgaW4gYS5hJiYoZj1hLmFbYl0sYz1WYihmLGMsZCxlKSwtMTxjJiYoVGIoZltjXSkscShudWxsIT1mLmxlbmd0aCksQXJyYXkucHJvdG90eXBlLnNwbGljZS5jYWxsKGYsYywxKSwwPT1mLmxlbmd0aCYmKGRlbGV0ZSBhLmFbYl0sYS5iLS0pKSkpOmEmJihhPWZjKGEpKSYmKGM9WGIoYSxiLGMsZCxlKSkmJmtjKGMpfSxrYz1mdW5jdGlvbihhKXtpZigibnVtYmVyIiE9PXR5cGVvZiBhJiZhJiYhYS5YKXt2YXIgYj1hLnNyYztpZihRYihiKSlXYihiLmgsYSk7ZWxzZXt2YXIgYz1hLnR5cGUsZD1hLmE7Yi5yZW1vdmVFdmVudExpc3RlbmVyPwpiLnJlbW92ZUV2ZW50TGlzdGVuZXIoYyxkLGEuY2FwdHVyZSk6Yi5kZXRhY2hFdmVudD9iLmRldGFjaEV2ZW50KGhjKGMpLGQpOmIuYWRkTGlzdGVuZXImJmIucmVtb3ZlTGlzdGVuZXImJmIucmVtb3ZlTGlzdGVuZXIoZCk7JGItLTsoYz1mYyhiKSk/KFdiKGMsYSksMD09Yy5iJiYoYy5zcmM9bnVsbCxiW1liXT1udWxsKSk6VGIoYSl9fX0saGM9ZnVuY3Rpb24oYSl7cmV0dXJuIGEgaW4gWmI/WmJbYV06WmJbYV09Im9uIithfSxtYz1mdW5jdGlvbihhLGIsYyxkKXt2YXIgZT0hMDtpZihhPWZjKGEpKWlmKGI9YS5hW2IudG9TdHJpbmcoKV0pZm9yKGI9Yi5jb25jYXQoKSxhPTA7YTxiLmxlbmd0aDthKyspe3ZhciBmPWJbYV07ZiYmZi5jYXB0dXJlPT1jJiYhZi5YJiYoZj1sYyhmLGQpLGU9ZSYmITEhPT1mKX1yZXR1cm4gZX0sbGM9ZnVuY3Rpb24oYSxiKXt2YXIgYz1hLmxpc3RlbmVyLGQ9YS5tYXx8YS5zcmM7YS5oYSYma2MoYSk7cmV0dXJuIGMuY2FsbChkLGIpfSxpYz1mdW5jdGlvbihhLApiKXtpZihhLlgpcmV0dXJuITA7aWYoIUpiKXtpZighYilhOntiPVsid2luZG93IiwiZXZlbnQiXTtmb3IodmFyIGM9bCxkPTA7ZDxiLmxlbmd0aDtkKyspaWYoYz1jW2JbZF1dLG51bGw9PWMpe2I9bnVsbDticmVhayBhfWI9Y31kPWI7Yj1uZXcgRyhkLHRoaXMpO2M9ITA7aWYoISgwPmQua2V5Q29kZXx8dm9pZCAwIT1kLnJldHVyblZhbHVlKSl7YTp7dmFyIGU9ITE7aWYoMD09ZC5rZXlDb2RlKXRyeXtkLmtleUNvZGU9LTE7YnJlYWsgYX1jYXRjaChnKXtlPSEwfWlmKGV8fHZvaWQgMD09ZC5yZXR1cm5WYWx1ZSlkLnJldHVyblZhbHVlPSEwfWQ9W107Zm9yKGU9Yi5hO2U7ZT1lLnBhcmVudE5vZGUpZC5wdXNoKGUpO2E9YS50eXBlO2ZvcihlPWQubGVuZ3RoLTE7IWIuaCYmMDw9ZTtlLS0pe2IuYT1kW2VdO3ZhciBmPW1jKGRbZV0sYSwhMCxiKTtjPWMmJmZ9Zm9yKGU9MDshYi5oJiZlPGQubGVuZ3RoO2UrKyliLmE9ZFtlXSxmPW1jKGRbZV0sYSwhMSxiKSxjPWMmJmZ9cmV0dXJuIGN9cmV0dXJuIGxjKGEsCm5ldyBHKGIsdGhpcykpfSxmYz1mdW5jdGlvbihhKXthPWFbWWJdO3JldHVybiBhIGluc3RhbmNlb2YgVWI/YTpudWxsfSxuYz0iX19jbG9zdXJlX2V2ZW50c19mbl8iKygxRTkqTWF0aC5yYW5kb20oKT4+PjApLGNjPWZ1bmN0aW9uKGEpe3EoYSwiTGlzdGVuZXIgY2FuIG5vdCBiZSBudWxsLiIpO2lmKGlhKGEpKXJldHVybiBhO3EoYS5oYW5kbGVFdmVudCwiQW4gb2JqZWN0IGxpc3RlbmVyIG11c3QgaGF2ZSBoYW5kbGVFdmVudCBtZXRob2QuIik7YVtuY118fChhW25jXT1mdW5jdGlvbihiKXtyZXR1cm4gYS5oYW5kbGVFdmVudChiKX0pO3JldHVybiBhW25jXX07dmFyIG9jPWZ1bmN0aW9uKGEpe0MuY2FsbCh0aGlzKTt0aGlzLmI9YTt0aGlzLmE9e319O24ob2MsQyk7CnZhciBwYz1bXSxIPWZ1bmN0aW9uKGEsYixjLGQpe2ZhKGMpfHwoYyYmKHBjWzBdPWMudG9TdHJpbmcoKSksYz1wYyk7Zm9yKHZhciBlPTA7ZTxjLmxlbmd0aDtlKyspe3ZhciBmPWJjKGIsY1tlXSxkfHxhLmhhbmRsZUV2ZW50LCExLGEuYnx8YSk7aWYoIWYpYnJlYWs7YS5hW2Yua2V5XT1mfXJldHVybiBhfSxxYz1mdW5jdGlvbihhLGIsYyxkLGUsZil7aWYoZmEoYykpZm9yKHZhciBnPTA7ZzxjLmxlbmd0aDtnKyspcWMoYSxiLGNbZ10sZCxlLGYpO2Vsc2UgZD1kfHxhLmhhbmRsZUV2ZW50LGU9bShlKT8hIWUuY2FwdHVyZTohIWUsZj1mfHxhLmJ8fGEsZD1jYyhkKSxlPSEhZSxjPVFiKGIpP1hiKGIuaCxTdHJpbmcoYyksZCxlLGYpOmI/KGI9ZmMoYikpP1hiKGIsYyxkLGUsZik6bnVsbDpudWxsLGMmJihrYyhjKSxkZWxldGUgYS5hW2Mua2V5XSk7cmV0dXJuIGF9LHJjPWZ1bmN0aW9uKGEpe29hKGEuYSxmdW5jdGlvbihiLGMpe3RoaXMuYS5oYXNPd25Qcm9wZXJ0eShjKSYma2MoYil9LAphKTthLmE9e319O29jLnByb3RvdHlwZS51PWZ1bmN0aW9uKCl7b2MuaS51LmNhbGwodGhpcyk7cmModGhpcyl9O29jLnByb3RvdHlwZS5oYW5kbGVFdmVudD1mdW5jdGlvbigpe3Rocm93IEVycm9yKCJFdmVudEhhbmRsZXIuaGFuZGxlRXZlbnQgbm90IGltcGxlbWVudGVkIik7fTt2YXIgST1mdW5jdGlvbigpe0MuY2FsbCh0aGlzKTt0aGlzLmg9bmV3IFViKHRoaXMpO3RoaXMuTWE9dGhpczt0aGlzLnBhPW51bGx9O24oSSxDKTtJLnByb3RvdHlwZVtQYl09ITA7SS5wcm90b3R5cGUucmE9ZnVuY3Rpb24oYSl7dGhpcy5wYT1hfTtJLnByb3RvdHlwZS5yZW1vdmVFdmVudExpc3RlbmVyPWZ1bmN0aW9uKGEsYixjLGQpe2pjKHRoaXMsYSxiLGMsZCl9Owp2YXIgdWM9ZnVuY3Rpb24oYSxiKXtzYyhhKTt2YXIgYz1hLnBhO2lmKGMpe3ZhciBkPVtdO2Zvcih2YXIgZT0xO2M7Yz1jLnBhKWQucHVzaChjKSxxKDFFMz4rK2UsImluZmluaXRlIGxvb3AiKX1hPWEuTWE7Yz1iLnR5cGV8fGI7InN0cmluZyI9PT10eXBlb2YgYj9iPW5ldyBFKGIsYSk6YiBpbnN0YW5jZW9mIEU/Yi50YXJnZXQ9Yi50YXJnZXR8fGE6KGU9YixiPW5ldyBFKGMsYSksdWEoYixlKSk7ZT0hMDtpZihkKWZvcih2YXIgZj1kLmxlbmd0aC0xOyFiLmgmJjA8PWY7Zi0tKXt2YXIgZz1iLmE9ZFtmXTtlPXRjKGcsYywhMCxiKSYmZX1iLmh8fChnPWIuYT1hLGU9dGMoZyxjLCEwLGIpJiZlLGIuaHx8KGU9dGMoZyxjLCExLGIpJiZlKSk7aWYoZClmb3IoZj0wOyFiLmgmJmY8ZC5sZW5ndGg7ZisrKWc9Yi5hPWRbZl0sZT10YyhnLGMsITEsYikmJmU7cmV0dXJuIGV9OwpJLnByb3RvdHlwZS51PWZ1bmN0aW9uKCl7SS5pLnUuY2FsbCh0aGlzKTtpZih0aGlzLmgpe3ZhciBhPXRoaXMuaCxiPTAsYztmb3IoYyBpbiBhLmEpe2Zvcih2YXIgZD1hLmFbY10sZT0wO2U8ZC5sZW5ndGg7ZSsrKSsrYixUYihkW2VdKTtkZWxldGUgYS5hW2NdO2EuYi0tfX10aGlzLnBhPW51bGx9O3ZhciBkYz1mdW5jdGlvbihhLGIsYyxkLGUpe3NjKGEpO3JldHVybiBhLmguYWRkKFN0cmluZyhiKSxjLCExLGQsZSl9LHRjPWZ1bmN0aW9uKGEsYixjLGQpe2I9YS5oLmFbU3RyaW5nKGIpXTtpZighYilyZXR1cm4hMDtiPWIuY29uY2F0KCk7Zm9yKHZhciBlPSEwLGY9MDtmPGIubGVuZ3RoOysrZil7dmFyIGc9YltmXTtpZihnJiYhZy5YJiZnLmNhcHR1cmU9PWMpe3ZhciBrPWcubGlzdGVuZXIscD1nLm1hfHxnLnNyYztnLmhhJiZXYihhLmgsZyk7ZT0hMSE9PWsuY2FsbChwLGQpJiZlfX1yZXR1cm4gZSYmMCE9ZC5LYX0sc2M9ZnVuY3Rpb24oYSl7cShhLmgsIkV2ZW50IHRhcmdldCBpcyBub3QgaW5pdGlhbGl6ZWQuIERpZCB5b3UgY2FsbCB0aGUgc3VwZXJjbGFzcyAoZ29vZy5ldmVudHMuRXZlbnRUYXJnZXQpIGNvbnN0cnVjdG9yPyIpfTt2YXIgeGM9ZnVuY3Rpb24oYSxiLGMsZCxlLGYpe2lmKHkmJiFBKCI1MjUiKSlyZXR1cm4hMDtpZih6JiZlKXJldHVybiB2YyhhKTtpZihlJiYhZClyZXR1cm4hMTtpZigheCl7Im51bWJlciI9PT10eXBlb2YgYiYmKGI9d2MoYikpO3ZhciBnPTE3PT1ifHwxOD09Ynx8eiYmOTE9PWI7aWYoKCFjfHx6KSYmZ3x8eiYmMTY9PWImJihkfHxmKSlyZXR1cm4hMX1pZigoeXx8YmIpJiZkJiZjKXN3aXRjaChhKXtjYXNlIDIyMDpjYXNlIDIxOTpjYXNlIDIyMTpjYXNlIDE5MjpjYXNlIDE4NjpjYXNlIDE4OTpjYXNlIDE4NzpjYXNlIDE4ODpjYXNlIDE5MDpjYXNlIDE5MTpjYXNlIDE5MjpjYXNlIDIyMjpyZXR1cm4hMX1pZih2JiZkJiZiPT1hKXJldHVybiExO3N3aXRjaChhKXtjYXNlIDEzOnJldHVybiB4P2Z8fGU/ITE6IShjJiZkKTohMDtjYXNlIDI3OnJldHVybiEoeXx8YmJ8fHgpfXJldHVybiB4JiYoZHx8ZXx8Zik/ITE6dmMoYSl9LHZjPWZ1bmN0aW9uKGEpe2lmKDQ4PD1hJiY1Nz49YXx8Cjk2PD1hJiYxMDY+PWF8fDY1PD1hJiY5MD49YXx8KHl8fGJiKSYmMD09YSlyZXR1cm4hMDtzd2l0Y2goYSl7Y2FzZSAzMjpjYXNlIDQzOmNhc2UgNjM6Y2FzZSA2NDpjYXNlIDEwNzpjYXNlIDEwOTpjYXNlIDExMDpjYXNlIDExMTpjYXNlIDE4NjpjYXNlIDU5OmNhc2UgMTg5OmNhc2UgMTg3OmNhc2UgNjE6Y2FzZSAxODg6Y2FzZSAxOTA6Y2FzZSAxOTE6Y2FzZSAxOTI6Y2FzZSAyMjI6Y2FzZSAyMTk6Y2FzZSAyMjA6Y2FzZSAyMjE6Y2FzZSAxNjM6Y2FzZSA1ODpyZXR1cm4hMDtjYXNlIDE3MzpyZXR1cm4geDtkZWZhdWx0OnJldHVybiExfX0sd2M9ZnVuY3Rpb24oYSl7aWYoeClhPXljKGEpO2Vsc2UgaWYoeiYmeSlzd2l0Y2goYSl7Y2FzZSA5MzphPTkxfXJldHVybiBhfSx5Yz1mdW5jdGlvbihhKXtzd2l0Y2goYSl7Y2FzZSA2MTpyZXR1cm4gMTg3O2Nhc2UgNTk6cmV0dXJuIDE4NjtjYXNlIDE3MzpyZXR1cm4gMTg5O2Nhc2UgMjI0OnJldHVybiA5MTtjYXNlIDA6cmV0dXJuIDIyNDsKZGVmYXVsdDpyZXR1cm4gYX19O3ZhciBKPWZ1bmN0aW9uKGEsYil7SS5jYWxsKHRoaXMpO2EmJnpjKHRoaXMsYSxiKX07bihKLEkpO2g9Si5wcm90b3R5cGU7aC5TPW51bGw7aC5uYT1udWxsO2guQ2E9bnVsbDtoLm9hPW51bGw7aC53PS0xO2guRj0tMTtoLnVhPSExOwp2YXIgQWM9ezM6MTMsMTI6MTQ0LDYzMjMyOjM4LDYzMjMzOjQwLDYzMjM0OjM3LDYzMjM1OjM5LDYzMjM2OjExMiw2MzIzNzoxMTMsNjMyMzg6MTE0LDYzMjM5OjExNSw2MzI0MDoxMTYsNjMyNDE6MTE3LDYzMjQyOjExOCw2MzI0MzoxMTksNjMyNDQ6MTIwLDYzMjQ1OjEyMSw2MzI0NjoxMjIsNjMyNDc6MTIzLDYzMjQ4OjQ0LDYzMjcyOjQ2LDYzMjczOjM2LDYzMjc1OjM1LDYzMjc2OjMzLDYzMjc3OjM0LDYzMjg5OjE0NCw2MzMwMjo0NX0sQmM9e1VwOjM4LERvd246NDAsTGVmdDozNyxSaWdodDozOSxFbnRlcjoxMyxGMToxMTIsRjI6MTEzLEYzOjExNCxGNDoxMTUsRjU6MTE2LEY2OjExNyxGNzoxMTgsRjg6MTE5LEY5OjEyMCxGMTA6MTIxLEYxMToxMjIsRjEyOjEyMywiVSswMDdGIjo0NixIb21lOjM2LEVuZDozNSxQYWdlVXA6MzMsUGFnZURvd246MzQsSW5zZXJ0OjQ1fSxDYz0heXx8QSgiNTI1IiksRGM9eiYmeDsKSi5wcm90b3R5cGUuYT1mdW5jdGlvbihhKXtpZih5fHxiYilpZigxNz09dGhpcy53JiYhYS5jdHJsS2V5fHwxOD09dGhpcy53JiYhYS5hbHRLZXl8fHomJjkxPT10aGlzLncmJiFhLm1ldGFLZXkpdGhpcy5GPXRoaXMudz0tMTstMT09dGhpcy53JiYoYS5jdHJsS2V5JiYxNyE9YS5jP3RoaXMudz0xNzphLmFsdEtleSYmMTghPWEuYz90aGlzLnc9MTg6YS5tZXRhS2V5JiY5MSE9YS5jJiYodGhpcy53PTkxKSk7Q2MmJiF4YyhhLmMsdGhpcy53LGEuc2hpZnRLZXksYS5jdHJsS2V5LGEuYWx0S2V5LGEubWV0YUtleSk/dGhpcy5oYW5kbGVFdmVudChhKToodGhpcy5GPXdjKGEuYyksRGMmJih0aGlzLnVhPWEuYWx0S2V5KSl9O0oucHJvdG90eXBlLmI9ZnVuY3Rpb24oYSl7dGhpcy5GPXRoaXMudz0tMTt0aGlzLnVhPWEuYWx0S2V5fTsKSi5wcm90b3R5cGUuaGFuZGxlRXZlbnQ9ZnVuY3Rpb24oYSl7dmFyIGI9YS5iLGM9Yi5hbHRLZXk7aWYodiYmImtleXByZXNzIj09YS50eXBlKXt2YXIgZD10aGlzLkY7dmFyIGU9MTMhPWQmJjI3IT1kP2Iua2V5Q29kZTowfWVsc2UoeXx8YmIpJiYia2V5cHJlc3MiPT1hLnR5cGU/KGQ9dGhpcy5GLGU9MDw9Yi5jaGFyQ29kZSYmNjMyMzI+Yi5jaGFyQ29kZSYmdmMoZCk/Yi5jaGFyQ29kZTowKTphYiYmIXk/KGQ9dGhpcy5GLGU9dmMoZCk/Yi5rZXlDb2RlOjApOigia2V5cHJlc3MiPT1hLnR5cGU/KERjJiYoYz10aGlzLnVhKSxiLmtleUNvZGU9PWIuY2hhckNvZGU/MzI+Yi5rZXlDb2RlPyhkPWIua2V5Q29kZSxlPTApOihkPXRoaXMuRixlPWIuY2hhckNvZGUpOihkPWIua2V5Q29kZXx8dGhpcy5GLGU9Yi5jaGFyQ29kZXx8MCkpOihkPWIua2V5Q29kZXx8dGhpcy5GLGU9Yi5jaGFyQ29kZXx8MCkseiYmNjM9PWUmJjIyND09ZCYmKGQ9MTkxKSk7dmFyIGY9ZD13YyhkKTtkPzYzMjMyPD0KZCYmZCBpbiBBYz9mPUFjW2RdOjI1PT1kJiZhLnNoaWZ0S2V5JiYoZj05KTpiLmtleUlkZW50aWZpZXImJmIua2V5SWRlbnRpZmllciBpbiBCYyYmKGY9QmNbYi5rZXlJZGVudGlmaWVyXSk7eCYmQ2MmJiJrZXlwcmVzcyI9PWEudHlwZSYmIXhjKGYsdGhpcy53LGEuc2hpZnRLZXksYS5jdHJsS2V5LGMsYS5tZXRhS2V5KXx8KGE9Zj09dGhpcy53LHRoaXMudz1mLGI9bmV3IEVjKGYsZSxhLGIpLGIuYWx0S2V5PWMsdWModGhpcyxiKSl9O0oucHJvdG90eXBlLmY9ZnVuY3Rpb24oKXtyZXR1cm4gdGhpcy5TfTsKdmFyIHpjPWZ1bmN0aW9uKGEsYixjKXthLm9hJiZGYyhhKTthLlM9YjthLm5hPWJjKGEuUywia2V5cHJlc3MiLGEsYyk7YS5DYT1iYyhhLlMsImtleWRvd24iLGEuYSxjLGEpO2Eub2E9YmMoYS5TLCJrZXl1cCIsYS5iLGMsYSl9LEZjPWZ1bmN0aW9uKGEpe2EubmEmJihrYyhhLm5hKSxrYyhhLkNhKSxrYyhhLm9hKSxhLm5hPW51bGwsYS5DYT1udWxsLGEub2E9bnVsbCk7YS5TPW51bGw7YS53PS0xO2EuRj0tMX07Si5wcm90b3R5cGUudT1mdW5jdGlvbigpe0ouaS51LmNhbGwodGhpcyk7RmModGhpcyl9O3ZhciBFYz1mdW5jdGlvbihhLGIsYyxkKXtHLmNhbGwodGhpcyxkKTt0aGlzLnR5cGU9ImtleSI7dGhpcy5jPWE7dGhpcy5yZXBlYXQ9Y307bihFYyxHKTt2YXIgR2M9eD8iTW96VXNlclNlbGVjdCI6eXx8YmI/IldlYmtpdFVzZXJTZWxlY3QiOm51bGwsSGM9ZnVuY3Rpb24oYSxiKXtiPWI/bnVsbDphLmdldEVsZW1lbnRzQnlUYWdOYW1lKCIqIik7aWYoR2Mpe3ZhciBjPSJub25lIjthLnN0eWxlJiYoYS5zdHlsZVtHY109Yyk7aWYoYil7YT0wO2Zvcih2YXIgZDtkPWJbYV07YSsrKWQuc3R5bGUmJihkLnN0eWxlW0djXT1jKX19ZWxzZSBpZih2fHxhYilpZihjPSJvbiIsYS5zZXRBdHRyaWJ1dGUoInVuc2VsZWN0YWJsZSIsYyksYilmb3IoYT0wO2Q9YlthXTthKyspZC5zZXRBdHRyaWJ1dGUoInVuc2VsZWN0YWJsZSIsYyl9O3ZhciBJYz1mdW5jdGlvbigpe307ZGEoSWMpO0ljLnByb3RvdHlwZS5hPTA7dmFyIEs9ZnVuY3Rpb24oYSl7SS5jYWxsKHRoaXMpO3RoaXMuaj1hfHx4YXx8KHhhPW5ldyB1Yik7dGhpcy5xYT1KYzt0aGlzLlk9bnVsbDt0aGlzLm09ITE7dGhpcy5hPW51bGw7dGhpcy5BPXZvaWQgMDt0aGlzLmc9dGhpcy5jPXRoaXMuYj1udWxsO3RoaXMuR2E9ITF9O24oSyxJKTtLLnByb3RvdHlwZS5PYT1JYy5SKCk7CnZhciBKYz1udWxsLEtjPWZ1bmN0aW9uKGEsYil7c3dpdGNoKGEpe2Nhc2UgMTpyZXR1cm4gYj8iZGlzYWJsZSI6ImVuYWJsZSI7Y2FzZSAyOnJldHVybiBiPyJoaWdobGlnaHQiOiJ1bmhpZ2hsaWdodCI7Y2FzZSA0OnJldHVybiBiPyJhY3RpdmF0ZSI6ImRlYWN0aXZhdGUiO2Nhc2UgODpyZXR1cm4gYj8ic2VsZWN0IjoidW5zZWxlY3QiO2Nhc2UgMTY6cmV0dXJuIGI/ImNoZWNrIjoidW5jaGVjayI7Y2FzZSAzMjpyZXR1cm4gYj8iZm9jdXMiOiJibHVyIjtjYXNlIDY0OnJldHVybiBiPyJvcGVuIjoiY2xvc2UifXRocm93IEVycm9yKCJJbnZhbGlkIGNvbXBvbmVudCBzdGF0ZSIpO30sTGM9ZnVuY3Rpb24oYSl7cmV0dXJuIGEuWXx8KGEuWT0iOiIrKGEuT2EuYSsrKS50b1N0cmluZygzNikpfSxNYz1mdW5jdGlvbihhLGIpe2lmKGEuYiYmYS5iLmcpe3ZhciBjPWEuYi5nLGQ9YS5ZO2QgaW4gYyYmZGVsZXRlIGNbZF07cWEoYS5iLmcsYixhKX1hLlk9Yn07Sy5wcm90b3R5cGUuZj1mdW5jdGlvbigpe3JldHVybiB0aGlzLmF9Owp2YXIgTmM9ZnVuY3Rpb24oYSl7YT1hLmE7cShhLCJDYW4gbm90IGNhbGwgZ2V0RWxlbWVudFN0cmljdCBiZWZvcmUgcmVuZGVyaW5nL2RlY29yYXRpbmcuIik7cmV0dXJuIGF9LE9jPWZ1bmN0aW9uKGEpe2EuQXx8KGEuQT1uZXcgb2MoYSkpO3JldHVybiBxKGEuQSl9O0sucHJvdG90eXBlLnJhPWZ1bmN0aW9uKGEpe2lmKHRoaXMuYiYmdGhpcy5iIT1hKXRocm93IEVycm9yKCJNZXRob2Qgbm90IHN1cHBvcnRlZCIpO0suaS5yYS5jYWxsKHRoaXMsYSl9O0sucHJvdG90eXBlLmlhPWZ1bmN0aW9uKCl7dGhpcy5hPW5iKHRoaXMuai5hLCJESVYiKX07CnZhciBQYz1mdW5jdGlvbihhLGIpe2lmKGEubSl0aHJvdyBFcnJvcigiQ29tcG9uZW50IGFscmVhZHkgcmVuZGVyZWQiKTtpZihiJiZhLnhhKGIpKXthLkdhPSEwO3ZhciBjPXFiKGIpO2EuaiYmYS5qLmE9PWN8fChhLmo9Yj9uZXcgdWIocWIoYikpOnhhfHwoeGE9bmV3IHViKSk7YS52YShiKTthLkQoKX1lbHNlIHRocm93IEVycm9yKCJJbnZhbGlkIGVsZW1lbnQgdG8gZGVjb3JhdGUiKTt9O2g9Sy5wcm90b3R5cGU7aC54YT1mdW5jdGlvbigpe3JldHVybiEwfTtoLnZhPWZ1bmN0aW9uKGEpe3RoaXMuYT1hfTtoLkQ9ZnVuY3Rpb24oKXt0aGlzLm09ITA7UWModGhpcyxmdW5jdGlvbihhKXshYS5tJiZhLmYoKSYmYS5EKCl9KX07aC5QPWZ1bmN0aW9uKCl7UWModGhpcyxmdW5jdGlvbihhKXthLm0mJmEuUCgpfSk7dGhpcy5BJiZyYyh0aGlzLkEpO3RoaXMubT0hMX07CmgudT1mdW5jdGlvbigpe3RoaXMubSYmdGhpcy5QKCk7dGhpcy5BJiYodGhpcy5BLk4oKSxkZWxldGUgdGhpcy5BKTtRYyh0aGlzLGZ1bmN0aW9uKGEpe2EuTigpfSk7IXRoaXMuR2EmJnRoaXMuYSYmb2IodGhpcy5hKTt0aGlzLmI9dGhpcy5hPXRoaXMuZz10aGlzLmM9bnVsbDtLLmkudS5jYWxsKHRoaXMpfTtoLnNhPWZ1bmN0aW9uKGEsYil7dGhpcy50YShhLFJjKHRoaXMpLGIpfTsKaC50YT1mdW5jdGlvbihhLGIsYyl7cSghIWEsIlByb3ZpZGVkIGVsZW1lbnQgbXVzdCBub3QgYmUgbnVsbC4iKTtpZihhLm0mJihjfHwhdGhpcy5tKSl0aHJvdyBFcnJvcigiQ29tcG9uZW50IGFscmVhZHkgcmVuZGVyZWQiKTtpZigwPmJ8fGI+UmModGhpcykpdGhyb3cgRXJyb3IoIkNoaWxkIGNvbXBvbmVudCBpbmRleCBvdXQgb2YgYm91bmRzIik7dGhpcy5nJiZ0aGlzLmN8fCh0aGlzLmc9e30sdGhpcy5jPVtdKTtpZihhLmI9PXRoaXMpe3ZhciBkPUxjKGEpO3RoaXMuZ1tkXT1hO0hhKHRoaXMuYyxhKX1lbHNlIHFhKHRoaXMuZyxMYyhhKSxhKTtpZihhPT10aGlzKXRocm93IEVycm9yKCJVbmFibGUgdG8gc2V0IHBhcmVudCBjb21wb25lbnQiKTtpZihkPXRoaXMmJmEuYiYmYS5ZKXt2YXIgZT1hLmI7ZD1hLlk7ZS5nJiZkPyhlPWUuZyxkPShudWxsIT09ZSYmZCBpbiBlP2VbZF06dm9pZCAwKXx8bnVsbCk6ZD1udWxsfWlmKGQmJmEuYiE9dGhpcyl0aHJvdyBFcnJvcigiVW5hYmxlIHRvIHNldCBwYXJlbnQgY29tcG9uZW50Iik7CmEuYj10aGlzO0suaS5yYS5jYWxsKGEsdGhpcyk7TGEodGhpcy5jLGIsMCxhKTtpZihhLm0mJnRoaXMubSYmYS5iPT10aGlzKWM9dGhpcy5qYSgpLGI9Yy5jaGlsZE5vZGVzW2JdfHxudWxsLGIhPWEuZigpJiZjLmluc2VydEJlZm9yZShhLmYoKSxiKTtlbHNlIGlmKGMpe3RoaXMuYXx8dGhpcy5pYSgpO2M9TCh0aGlzLGIrMSk7Yj10aGlzLmphKCk7Yz1jP2MuYTpudWxsO2lmKGEubSl0aHJvdyBFcnJvcigiQ29tcG9uZW50IGFscmVhZHkgcmVuZGVyZWQiKTthLmF8fGEuaWEoKTtiP2IuaW5zZXJ0QmVmb3JlKGEuYSxjfHxudWxsKTphLmouYS5ib2R5LmFwcGVuZENoaWxkKGEuYSk7YS5iJiYhYS5iLm18fGEuRCgpfWVsc2UgdGhpcy5tJiYhYS5tJiZhLmEmJmEuYS5wYXJlbnROb2RlJiYxPT1hLmEucGFyZW50Tm9kZS5ub2RlVHlwZSYmYS5EKCl9O2guamE9ZnVuY3Rpb24oKXtyZXR1cm4gdGhpcy5hfTsKdmFyIFNjPWZ1bmN0aW9uKGEpe2lmKG51bGw9PWEucWEpe3ZhciBiPWEubT9hLmE6YS5qLmEuYm9keTthOnt2YXIgYz1xYihiKTtpZihjLmRlZmF1bHRWaWV3JiZjLmRlZmF1bHRWaWV3LmdldENvbXB1dGVkU3R5bGUmJihjPWMuZGVmYXVsdFZpZXcuZ2V0Q29tcHV0ZWRTdHlsZShiLG51bGwpKSl7Yz1jLmRpcmVjdGlvbnx8Yy5nZXRQcm9wZXJ0eVZhbHVlKCJkaXJlY3Rpb24iKXx8IiI7YnJlYWsgYX1jPSIifWEucWE9InJ0bCI9PShjfHwoYi5jdXJyZW50U3R5bGU/Yi5jdXJyZW50U3R5bGUuZGlyZWN0aW9uOm51bGwpfHxiLnN0eWxlJiZiLnN0eWxlLmRpcmVjdGlvbil9cmV0dXJuIGEucWF9LFJjPWZ1bmN0aW9uKGEpe3JldHVybiBhLmM/YS5jLmxlbmd0aDowfSxMPWZ1bmN0aW9uKGEsYil7cmV0dXJuIGEuYz9hLmNbYl18fG51bGw6bnVsbH0sUWM9ZnVuY3Rpb24oYSxiLGMpe2EuYyYmcihhLmMsYixjKX0sVGM9ZnVuY3Rpb24oYSxiKXtyZXR1cm4gYS5jJiZiP0RhKGEuYyxiKTotMX07dmFyIFZjPWZ1bmN0aW9uKGEsYil7aWYoIWEpdGhyb3cgRXJyb3IoIkludmFsaWQgY2xhc3MgbmFtZSAiK2EpO2lmKCFpYShiKSl0aHJvdyBFcnJvcigiSW52YWxpZCBkZWNvcmF0b3IgZnVuY3Rpb24gIitiKTtVY1thXT1ifSxXYz17fSxVYz17fTt2YXIgWGM9ZnVuY3Rpb24oYSl7dGhpcy5oPWF9O2RhKFhjKTt2YXIgWWM9ZnVuY3Rpb24oYSxiKXthJiYoYS50YWJJbmRleD1iPzA6LTEpfSwkYz1mdW5jdGlvbihhLGIsYyl7Yy5pZCYmTWMoYixjLmlkKTt2YXIgZD1hLmIoKSxlPSExLGY9RChjKTtmJiZyKGYsZnVuY3Rpb24oZyl7Zz09ZD9lPSEwOmcmJnRoaXMuZyhiLGcsZCl9LGEpO2V8fEViKGMsZCk7WmMoYixjKTtyZXR1cm4gY307WGMucHJvdG90eXBlLmc9ZnVuY3Rpb24oYSxiLGMpe2I9PWMrIi1kaXNhYmxlZCI/YS5aKCExKTpiPT1jKyItaG9yaXpvbnRhbCI/YWQoYSwiaG9yaXpvbnRhbCIpOmI9PWMrIi12ZXJ0aWNhbCImJmFkKGEsInZlcnRpY2FsIil9Owp2YXIgWmM9ZnVuY3Rpb24oYSxiKXtpZihiKWZvcih2YXIgYz1iLmZpcnN0Q2hpbGQsZDtjJiZjLnBhcmVudE5vZGU9PWI7KXtkPWMubmV4dFNpYmxpbmc7aWYoMT09Yy5ub2RlVHlwZSl7YTp7dmFyIGU9YztxKGUpO2U9RChlKTtmb3IodmFyIGY9MCxnPWUubGVuZ3RoO2Y8ZztmKyspe3ZhciBrPWVbZl07aWYoaz1rIGluIFVjP1VjW2tdKCk6bnVsbCl7ZT1rO2JyZWFrIGF9fWU9bnVsbH1lJiYoZS5hPWMsYS5pc0VuYWJsZWQoKXx8ZS5aKCExKSxhLnNhKGUpLFBjKGUsYykpfWVsc2UgYy5ub2RlVmFsdWUmJiIiIT1NYShjLm5vZGVWYWx1ZSl8fGIucmVtb3ZlQ2hpbGQoYyk7Yz1kfX0sYmQ9ZnVuY3Rpb24oYSxiKXtiPWIuZigpO3EoYiwiVGhlIGNvbnRhaW5lciBET00gZWxlbWVudCBjYW5ub3QgYmUgbnVsbC4iKTtIYyhiLHgpO3YmJihiLmhpZGVGb2N1cz0hMCk7KGE9YS5oKSYmdmIoYixhKX07WGMucHJvdG90eXBlLmI9ZnVuY3Rpb24oKXtyZXR1cm4iZ29vZy1jb250YWluZXIifTsKWGMucHJvdG90eXBlLmM9ZnVuY3Rpb24oYSl7dmFyIGI9dGhpcy5iKCksYz1bYiwiaG9yaXpvbnRhbCI9PWEuSD9iKyItaG9yaXpvbnRhbCI6YisiLXZlcnRpY2FsIl07YS5pc0VuYWJsZWQoKXx8Yy5wdXNoKGIrIi1kaXNhYmxlZCIpO3JldHVybiBjfTt2YXIgTT1mdW5jdGlvbigpe30sY2Q7ZGEoTSk7dmFyIGRkPXtidXR0b246InByZXNzZWQiLGNoZWNrYm94OiJjaGVja2VkIixtZW51aXRlbToic2VsZWN0ZWQiLG1lbnVpdGVtY2hlY2tib3g6ImNoZWNrZWQiLG1lbnVpdGVtcmFkaW86ImNoZWNrZWQiLHJhZGlvOiJjaGVja2VkIix0YWI6InNlbGVjdGVkIix0cmVlaXRlbToic2VsZWN0ZWQifTtNLnByb3RvdHlwZS5oPWZ1bmN0aW9uKCl7fTtNLnByb3RvdHlwZS5jPWZ1bmN0aW9uKGEpe3JldHVybiBhLmouYigiRElWIixlZCh0aGlzLGEpLmpvaW4oIiAiKSxhLmJhKX07dmFyIGdkPWZ1bmN0aW9uKGEsYixjKXtpZihhPWEuZj9hLmYoKTphKXt2YXIgZD1bYl07diYmIUEoIjciKSYmKGQ9ZmQoRChhKSxiKSxkLnB1c2goYikpOyhjP0ZiOkhiKShhLGQpfX07Ck0ucHJvdG90eXBlLmc9ZnVuY3Rpb24oYSxiKXtiLmlkJiZNYyhhLGIuaWQpO2ImJmIuZmlyc3RDaGlsZD9oZChhLGIuZmlyc3RDaGlsZC5uZXh0U2libGluZz9KYShiLmNoaWxkTm9kZXMpOmIuZmlyc3RDaGlsZCk6YS5iYT1udWxsO3ZhciBjPTAsZD10aGlzLmEoKSxlPXRoaXMuYSgpLGY9ITEsZz0hMSxrPSExLHA9SmEoRChiKSk7cihwLGZ1bmN0aW9uKEIpe2Z8fEIhPWQ/Z3x8QiE9ZT9jfD1pZCh0aGlzLEIpOmc9ITA6KGY9ITAsZT09ZCYmKGc9ITApKTsxPT1pZCh0aGlzLEIpJiYoQWEoYiksc2IoYikmJnRiKGIpJiZyYihiLCExKSl9LHRoaXMpO2Eubz1jO2Z8fChwLnB1c2goZCksZT09ZCYmKGc9ITApKTtnfHxwLnB1c2goZSk7KGE9YS53YSkmJnAucHVzaC5hcHBseShwLGEpO2lmKHYmJiFBKCI3Iikpe3ZhciB3PWZkKHApOzA8dy5sZW5ndGgmJihwLnB1c2guYXBwbHkocCx3KSxrPSEwKX1mJiZnJiYhYSYmIWt8fENiKGIscC5qb2luKCIgIikpO3JldHVybiBifTsKdmFyIGpkPWZ1bmN0aW9uKGEsYil7aWYoYT1hLmgoKSl7cShiLCJUaGUgZWxlbWVudCBwYXNzZWQgYXMgYSBmaXJzdCBwYXJhbWV0ZXIgY2Fubm90IGJlIG51bGwuIik7dmFyIGM9Yi5nZXRBdHRyaWJ1dGUoInJvbGUiKXx8bnVsbDthIT1jJiZ2YihiLGEpfX0sa2Q9ZnVuY3Rpb24oYSxiKXt2YXIgYztpZihhLnYmMzImJihjPWEuZigpKSl7aWYoIWImJmEubyYzMil7dHJ5e2MuYmx1cigpfWNhdGNoKGQpe31hLm8mMzImJmEuRmEobnVsbCl9KHNiKGMpJiZ0YihjKSkhPWImJnJiKGMsYil9fSxsZD1mdW5jdGlvbihhLGIsYyl7Y2R8fChjZD17MToiZGlzYWJsZWQiLDg6InNlbGVjdGVkIiwxNjoiY2hlY2tlZCIsNjQ6ImV4cGFuZGVkIn0pO3EoYSwiVGhlIGVsZW1lbnQgcGFzc2VkIGFzIGEgZmlyc3QgcGFyYW1ldGVyIGNhbm5vdCBiZSBudWxsLiIpO2I9Y2RbYl07dmFyIGQ9YS5nZXRBdHRyaWJ1dGUoInJvbGUiKXx8bnVsbDtkJiYoZD1kZFtkXXx8YixiPSJjaGVja2VkIj09Ynx8InNlbGVjdGVkIj09CmI/ZDpiKTtiJiZ4YihhLGIsYyl9O00ucHJvdG90eXBlLmE9ZnVuY3Rpb24oKXtyZXR1cm4iZ29vZy1jb250cm9sIn07CnZhciBlZD1mdW5jdGlvbihhLGIpe3ZhciBjPWEuYSgpLGQ9W2NdLGU9YS5hKCk7ZSE9YyYmZC5wdXNoKGUpO2M9Yi5vO2ZvcihlPVtdO2M7KXt2YXIgZj1jJi1jO2UucHVzaChtZChhLGYpKTtjJj1+Zn1kLnB1c2guYXBwbHkoZCxlKTsoYT1iLndhKSYmZC5wdXNoLmFwcGx5KGQsYSk7diYmIUEoIjciKSYmZC5wdXNoLmFwcGx5KGQsZmQoZCkpO3JldHVybiBkfSxmZD1mdW5jdGlvbihhLGIpe3ZhciBjPVtdO2ImJihhPUlhKGEsW2JdKSk7cihbXSxmdW5jdGlvbihkKXshRmEoZCxsYShHYSxhKSl8fGImJiFHYShkLGIpfHxjLnB1c2goZC5qb2luKCJfIikpfSk7cmV0dXJuIGN9LG1kPWZ1bmN0aW9uKGEsYil7YS5ifHxuZChhKTtyZXR1cm4gYS5iW2JdfSxpZD1mdW5jdGlvbihhLGIpe2Euanx8KGEuYnx8bmQoYSksYS5qPXNhKGEuYikpO2E9cGFyc2VJbnQoYS5qW2JdLDEwKTtyZXR1cm4gaXNOYU4oYSk/MDphfSxuZD1mdW5jdGlvbihhKXt2YXIgYj1hLmEoKSxjPSF0KGIucmVwbGFjZSgvXHhhMHxccy9nLAoiICIpLCIgIik7cShjLCJDb250cm9sUmVuZGVyZXIgaGFzIGFuIGludmFsaWQgY3NzIGNsYXNzOiAnIitiKyInIik7YS5iPXsxOmIrIi1kaXNhYmxlZCIsMjpiKyItaG92ZXIiLDQ6YisiLWFjdGl2ZSIsODpiKyItc2VsZWN0ZWQiLDE2OmIrIi1jaGVja2VkIiwzMjpiKyItZm9jdXNlZCIsNjQ6YisiLW9wZW4ifX07dmFyIE49ZnVuY3Rpb24oYSxiLGMpe0suY2FsbCh0aGlzLGMpO2lmKCFiKXtiPXRoaXMuY29uc3RydWN0b3I7Zm9yKHZhciBkO2I7KXtkPWJbamFdfHwoYltqYV09KytrYSk7aWYoZD1XY1tkXSlicmVhaztiPWIuaT9iLmkuY29uc3RydWN0b3I6bnVsbH1iPWQ/aWEoZC5SKT9kLlIoKTpuZXcgZDpudWxsfXRoaXMuQj1iO3RoaXMuYmE9dm9pZCAwIT09YT9hOm51bGx9O24oTixLKTtoPU4ucHJvdG90eXBlO2guYmE9bnVsbDtoLm89MDtoLnY9Mzk7aC5LPTA7aC5JYT0hMDtoLndhPW51bGw7aC55YT0hMDtoLmlhPWZ1bmN0aW9uKCl7dmFyIGE9dGhpcy5CLmModGhpcyk7dGhpcy5hPWE7amQodGhpcy5CLGEpO0hjKGEsIXYmJiFhYik7dGhpcy5pc1Zpc2libGUoKXx8KGEuc3R5bGUuZGlzcGxheT0ibm9uZSIsYSYmeGIoYSwiaGlkZGVuIiwhMCkpfTtoLmphPWZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMuZigpfTtoLnhhPWZ1bmN0aW9uKCl7cmV0dXJuITB9OwpoLnZhPWZ1bmN0aW9uKGEpe3RoaXMuYT1hPXRoaXMuQi5nKHRoaXMsYSk7amQodGhpcy5CLGEpO0hjKGEsIXYmJiFhYik7dGhpcy5JYT0ibm9uZSIhPWEuc3R5bGUuZGlzcGxheX07CmguRD1mdW5jdGlvbigpe04uaS5ELmNhbGwodGhpcyk7dmFyIGE9TmModGhpcyk7cSh0aGlzKTtxKGEpO3RoaXMuaXNWaXNpYmxlKCl8fHhiKGEsImhpZGRlbiIsIXRoaXMuaXNWaXNpYmxlKCkpO3RoaXMuaXNFbmFibGVkKCl8fGxkKGEsMSwhdGhpcy5pc0VuYWJsZWQoKSk7dGhpcy52JjgmJmxkKGEsOCwhISh0aGlzLm8mOCkpO3RoaXMudiYxNiYmbGQoYSwxNiwhISh0aGlzLm8mMTYpKTt0aGlzLnYmNjQmJmxkKGEsNjQsISEodGhpcy5vJjY0KSk7YT10aGlzLkI7U2ModGhpcykmJmdkKHRoaXMuZigpLGEuYSgpKyItcnRsIiwhMCk7dGhpcy5pc0VuYWJsZWQoKSYma2QodGhpcyx0aGlzLmlzVmlzaWJsZSgpKTtpZih0aGlzLnYmLTImJih0aGlzLnlhJiZvZCh0aGlzLCEwKSx0aGlzLnYmMzImJihhPXRoaXMuZigpKSkpe3ZhciBiPXRoaXMuV3x8KHRoaXMuVz1uZXcgSik7emMoYixhKTtIKEgoSChPYyh0aGlzKSxiLCJrZXkiLHRoaXMuZmEpLGEsImZvY3VzIix0aGlzLk5hKSxhLCJibHVyIiwKdGhpcy5GYSl9fTt2YXIgb2Q9ZnVuY3Rpb24oYSxiKXt2YXIgYz1PYyhhKSxkPWEuZigpO2I/KEgoSChIKEgoYyxkLEYuTCxhLmthKSxkLFtGLk0sRi5hYV0sYS5sYSksZCwibW91c2VvdmVyIixhLkFhKSxkLCJtb3VzZW91dCIsYS56YSksYS5lYSE9Y2EmJkgoYyxkLCJjb250ZXh0bWVudSIsYS5lYSksdiYmKEEoOSl8fEgoYyxkLCJkYmxjbGljayIsYS5KYSksYS5nYXx8KGEuZ2E9bmV3IHBkKGEpLHpiKGEsbGEoQWIsYS5nYSkpKSkpOihxYyhxYyhxYyhxYyhjLGQsRi5MLGEua2EpLGQsW0YuTSxGLmFhXSxhLmxhKSxkLCJtb3VzZW92ZXIiLGEuQWEpLGQsIm1vdXNlb3V0IixhLnphKSxhLmVhIT1jYSYmcWMoYyxkLCJjb250ZXh0bWVudSIsYS5lYSksdiYmKEEoOSl8fHFjKGMsZCwiZGJsY2xpY2siLGEuSmEpLEFiKGEuZ2EpLGEuZ2E9bnVsbCkpfTsKTi5wcm90b3R5cGUuUD1mdW5jdGlvbigpe04uaS5QLmNhbGwodGhpcyk7dGhpcy5XJiZGYyh0aGlzLlcpO3RoaXMuaXNWaXNpYmxlKCkmJnRoaXMuaXNFbmFibGVkKCkmJmtkKHRoaXMsITEpfTtOLnByb3RvdHlwZS51PWZ1bmN0aW9uKCl7Ti5pLnUuY2FsbCh0aGlzKTt0aGlzLlcmJih0aGlzLlcuTigpLGRlbGV0ZSB0aGlzLlcpO2RlbGV0ZSB0aGlzLkI7dGhpcy5nYT10aGlzLndhPXRoaXMuYmE9bnVsbH07dmFyIGhkPWZ1bmN0aW9uKGEsYil7YS5iYT1ifTtOLnByb3RvdHlwZS5pc1Zpc2libGU9ZnVuY3Rpb24oKXtyZXR1cm4gdGhpcy5JYX07Ti5wcm90b3R5cGUuaXNFbmFibGVkPWZ1bmN0aW9uKCl7cmV0dXJuISh0aGlzLm8mMSl9OwpOLnByb3RvdHlwZS5aPWZ1bmN0aW9uKGEpe3ZhciBiPXRoaXMuYjtiJiYiZnVuY3Rpb24iPT10eXBlb2YgYi5pc0VuYWJsZWQmJiFiLmlzRW5hYmxlZCgpfHwhTyh0aGlzLDEsIWEpfHwoYXx8KHFkKHRoaXMsITEpLFAodGhpcywhMSkpLHRoaXMuaXNWaXNpYmxlKCkmJmtkKHRoaXMsYSksUSh0aGlzLDEsIWEsITApKX07CnZhciBQPWZ1bmN0aW9uKGEsYil7TyhhLDIsYikmJlEoYSwyLGIpfSxxZD1mdW5jdGlvbihhLGIpe08oYSw0LGIpJiZRKGEsNCxiKX0scmQ9ZnVuY3Rpb24oYSxiKXtPKGEsOCxiKSYmUShhLDgsYil9LHNkPWZ1bmN0aW9uKGEsYil7TyhhLDY0LGIpJiZRKGEsNjQsYil9LFE9ZnVuY3Rpb24oYSxiLGMsZCl7aWYoIWQmJjE9PWIpYS5aKCFjKTtlbHNlIGlmKGEudiZiJiZjIT0hIShhLm8mYikpe3ZhciBlPWEuQjtpZihkPWEuZigpKShlPW1kKGUsYikpJiZnZChhLGUsYyksbGQoZCxiLGMpO2Eubz1jP2Eub3xiOmEubyZ+Yn19LHRkPWZ1bmN0aW9uKGEsYixjKXtpZihhLm0mJmEubyZiJiYhYyl0aHJvdyBFcnJvcigiQ29tcG9uZW50IGFscmVhZHkgcmVuZGVyZWQiKTshYyYmYS5vJmImJlEoYSxiLCExKTthLnY9Yz9hLnZ8YjphLnYmfmJ9LFI9ZnVuY3Rpb24oYSxiKXtyZXR1cm4hISgyNTUmYikmJiEhKGEudiZiKX0sTz1mdW5jdGlvbihhLGIsYyl7cmV0dXJuISEoYS52JmIpJiYhIShhLm8mCmIpIT1jJiYoIShhLksmYil8fHVjKGEsS2MoYixjKSkpJiYhYS5VfTtoPU4ucHJvdG90eXBlO2guQWE9ZnVuY3Rpb24oYSl7KCFhLnJlbGF0ZWRUYXJnZXR8fCFwYih0aGlzLmYoKSxhLnJlbGF0ZWRUYXJnZXQpKSYmdWModGhpcywiZW50ZXIiKSYmdGhpcy5pc0VuYWJsZWQoKSYmUih0aGlzLDIpJiZQKHRoaXMsITApfTtoLnphPWZ1bmN0aW9uKGEpe2EucmVsYXRlZFRhcmdldCYmcGIodGhpcy5mKCksYS5yZWxhdGVkVGFyZ2V0KXx8IXVjKHRoaXMsImxlYXZlIil8fChSKHRoaXMsNCkmJnFkKHRoaXMsITEpLFIodGhpcywyKSYmUCh0aGlzLCExKSl9O2guZWE9Y2E7Cmgua2E9ZnVuY3Rpb24oYSl7aWYodGhpcy5pc0VuYWJsZWQoKSYmKFIodGhpcywyKSYmUCh0aGlzLCEwKSxPYihhKSYmISh5JiZ6JiZhLmN0cmxLZXkpKSl7Uih0aGlzLDQpJiZxZCh0aGlzLCEwKTt2YXIgYjtpZihiPXRoaXMuQil7dmFyIGM7Yj10aGlzLnYmMzImJihjPXRoaXMuZigpKT9zYihjKSYmdGIoYyk6ITF9YiYmdGhpcy5mKCkuZm9jdXMoKX0hT2IoYSl8fHkmJnomJmEuY3RybEtleXx8YS5nKCl9O2gubGE9ZnVuY3Rpb24oYSl7dGhpcy5pc0VuYWJsZWQoKSYmKFIodGhpcywyKSYmUCh0aGlzLCEwKSx0aGlzLm8mNCYmdWQodGhpcyxhKSYmUih0aGlzLDQpJiZxZCh0aGlzLCExKSl9O2guSmE9ZnVuY3Rpb24oYSl7dGhpcy5pc0VuYWJsZWQoKSYmdWQodGhpcyxhKX07CnZhciB1ZD1mdW5jdGlvbihhLGIpe2lmKFIoYSwxNikpe3ZhciBjPSEoYS5vJjE2KTtPKGEsMTYsYykmJlEoYSwxNixjKX1SKGEsOCkmJnJkKGEsITApO1IoYSw2NCkmJnNkKGEsIShhLm8mNjQpKTtjPW5ldyBFKCJhY3Rpb24iLGEpO2ImJihjLmFsdEtleT1iLmFsdEtleSxjLmN0cmxLZXk9Yi5jdHJsS2V5LGMubWV0YUtleT1iLm1ldGFLZXksYy5zaGlmdEtleT1iLnNoaWZ0S2V5LGMuQT1iLkEpO3JldHVybiB1YyhhLGMpfTtOLnByb3RvdHlwZS5OYT1mdW5jdGlvbigpe1IodGhpcywzMikmJk8odGhpcywzMiwhMCkmJlEodGhpcywzMiwhMCl9O04ucHJvdG90eXBlLkZhPWZ1bmN0aW9uKCl7Uih0aGlzLDQpJiZxZCh0aGlzLCExKTtSKHRoaXMsMzIpJiZPKHRoaXMsMzIsITEpJiZRKHRoaXMsMzIsITEpfTsKTi5wcm90b3R5cGUuZmE9ZnVuY3Rpb24oYSl7cmV0dXJuIHRoaXMuaXNWaXNpYmxlKCkmJnRoaXMuaXNFbmFibGVkKCkmJjEzPT1hLmMmJnVkKHRoaXMsYSk/KGEuZygpLGEuaigpLCEwKTohMX07aWYoIWlhKE4pKXRocm93IEVycm9yKCJJbnZhbGlkIGNvbXBvbmVudCBjbGFzcyAiK04pO2lmKCFpYShNKSl0aHJvdyBFcnJvcigiSW52YWxpZCByZW5kZXJlciBjbGFzcyAiK00pO3ZhciB2ZD1OW2phXXx8KE5bamFdPSsra2EpO1djW3ZkXT1NO1ZjKCJnb29nLWNvbnRyb2wiLGZ1bmN0aW9uKCl7cmV0dXJuIG5ldyBOKG51bGwpfSk7dmFyIHBkPWZ1bmN0aW9uKGEpe0MuY2FsbCh0aGlzKTt0aGlzLmI9YTt0aGlzLmE9ITE7dGhpcy5jPW5ldyBvYyh0aGlzKTt6Yih0aGlzLGxhKEFiLHRoaXMuYykpO2E9TmModGhpcy5iKTtIKEgoSCh0aGlzLmMsYSxGLkwsdGhpcy5oKSxhLEYuTSx0aGlzLmopLGEsImNsaWNrIix0aGlzLmcpfTtuKHBkLEMpO3ZhciB3ZD0hdnx8OTw9TnVtYmVyKGliKTsKcGQucHJvdG90eXBlLmg9ZnVuY3Rpb24oKXt0aGlzLmE9ITF9O3BkLnByb3RvdHlwZS5qPWZ1bmN0aW9uKCl7dGhpcy5hPSEwfTt2YXIgeGQ9ZnVuY3Rpb24oYSxiKXtpZighd2QpcmV0dXJuIGEuYnV0dG9uPTAsYS50eXBlPWIsYTt2YXIgYz1kb2N1bWVudC5jcmVhdGVFdmVudCgiTW91c2VFdmVudHMiKTtjLmluaXRNb3VzZUV2ZW50KGIsYS5idWJibGVzLGEuY2FuY2VsYWJsZSxhLnZpZXd8fG51bGwsYS5kZXRhaWwsYS5zY3JlZW5YLGEuc2NyZWVuWSxhLmNsaWVudFgsYS5jbGllbnRZLGEuY3RybEtleSxhLmFsdEtleSxhLnNoaWZ0S2V5LGEubWV0YUtleSwwLGEucmVsYXRlZFRhcmdldHx8bnVsbCk7cmV0dXJuIGN9OwpwZC5wcm90b3R5cGUuZz1mdW5jdGlvbihhKXtpZih0aGlzLmEpdGhpcy5hPSExO2Vsc2V7dmFyIGI9YS5iLGM9Yi5idXR0b24sZD1iLnR5cGUsZT14ZChiLCJtb3VzZWRvd24iKTt0aGlzLmIua2EobmV3IEcoZSxhLmEpKTtlPXhkKGIsIm1vdXNldXAiKTt0aGlzLmIubGEobmV3IEcoZSxhLmEpKTt3ZHx8KGIuYnV0dG9uPWMsYi50eXBlPWQpfX07cGQucHJvdG90eXBlLnU9ZnVuY3Rpb24oKXt0aGlzLmI9bnVsbDtwZC5pLnUuY2FsbCh0aGlzKX07dmFyIFM9ZnVuY3Rpb24oYSxiLGMpe0suY2FsbCh0aGlzLGMpO3RoaXMuZGE9Ynx8WGMuUigpO3RoaXMuSD1hfHwidmVydGljYWwifTtuKFMsSyk7aD1TLnByb3RvdHlwZTtoLkRhPW51bGw7aC5jYT1udWxsO2guZGE9bnVsbDtoLkg9bnVsbDtoLlQ9ITA7aC5PPSEwO2gubD0tMTtoLnM9bnVsbDtoLlY9ITE7aC5JPW51bGw7dmFyIHlkPWZ1bmN0aW9uKGEpe3JldHVybiBhLkRhfHxhLmYoKX07aD1TLnByb3RvdHlwZTtoLmlhPWZ1bmN0aW9uKCl7dGhpcy5hPXRoaXMuai5iKCJESVYiLHRoaXMuZGEuYyh0aGlzKS5qb2luKCIgIikpfTtoLmphPWZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMuZigpfTtoLnhhPWZ1bmN0aW9uKGEpe3JldHVybiJESVYiPT1hLnRhZ05hbWV9O2gudmE9ZnVuY3Rpb24oYSl7dGhpcy5hPSRjKHRoaXMuZGEsdGhpcyxhKTsibm9uZSI9PWEuc3R5bGUuZGlzcGxheSYmKHRoaXMuVD0hMSl9OwpoLkQ9ZnVuY3Rpb24oKXtTLmkuRC5jYWxsKHRoaXMpO1FjKHRoaXMsZnVuY3Rpb24oYil7Yi5tJiZ6ZCh0aGlzLGIpfSx0aGlzKTt2YXIgYT10aGlzLmYoKTtiZCh0aGlzLmRhLHRoaXMpO0FkKHRoaXMsdGhpcy5UKTtIKEgoSChIKEgoSChIKEgoT2ModGhpcyksdGhpcywiZW50ZXIiLHRoaXMuV2EpLHRoaXMsImhpZ2hsaWdodCIsdGhpcy5YYSksdGhpcywidW5oaWdobGlnaHQiLHRoaXMuY2IpLHRoaXMsIm9wZW4iLHRoaXMuWWEpLHRoaXMsImNsb3NlIix0aGlzLlVhKSxhLEYuTCx0aGlzLlNhKSxxYihhKSxbRi5NLEYuYWFdLHRoaXMuVmEpLGEsW0YuTCxGLk0sRi5hYSwibW91c2VvdmVyIiwibW91c2VvdXQiLCJjb250ZXh0bWVudSJdLHRoaXMuVGEpO0JkKHRoaXMpfTt2YXIgQmQ9ZnVuY3Rpb24oYSl7dmFyIGI9T2MoYSksYz15ZChhKTtIKEgoSChiLGMsImZvY3VzIixhLkhhKSxjLCJibHVyIixhLlFhKSxhLmNhfHwoYS5jYT1uZXcgSih5ZChhKSkpLCJrZXkiLGEuUmEpfTtoPVMucHJvdG90eXBlOwpoLlA9ZnVuY3Rpb24oKXtDZCh0aGlzLC0xKTt0aGlzLnMmJnNkKHRoaXMucywhMSk7dGhpcy5WPSExO1MuaS5QLmNhbGwodGhpcyl9O2gudT1mdW5jdGlvbigpe1MuaS51LmNhbGwodGhpcyk7dGhpcy5jYSYmKHRoaXMuY2EuTigpLHRoaXMuY2E9bnVsbCk7dGhpcy5kYT10aGlzLnM9dGhpcy5JPXRoaXMuRGE9bnVsbH07aC5XYT1mdW5jdGlvbigpe3JldHVybiEwfTsKaC5YYT1mdW5jdGlvbihhKXt2YXIgYj1UYyh0aGlzLGEudGFyZ2V0KTtpZigtMTxiJiZiIT10aGlzLmwpe3ZhciBjPUwodGhpcyx0aGlzLmwpO2MmJlAoYywhMSk7dGhpcy5sPWI7Yz1MKHRoaXMsdGhpcy5sKTt0aGlzLlYmJnFkKGMsITApO3RoaXMucyYmYyE9dGhpcy5zJiYoYy52JjY0P3NkKGMsITApOnNkKHRoaXMucywhMSkpfWI9dGhpcy5mKCk7cShiLCJUaGUgRE9NIGVsZW1lbnQgZm9yIHRoZSBjb250YWluZXIgY2Fubm90IGJlIG51bGwuIik7bnVsbCE9YS50YXJnZXQuZigpJiZ4YihiLCJhY3RpdmVkZXNjZW5kYW50IixhLnRhcmdldC5mKCkuaWQpfTtoLmNiPWZ1bmN0aW9uKGEpe2EudGFyZ2V0PT1MKHRoaXMsdGhpcy5sKSYmKHRoaXMubD0tMSk7YT10aGlzLmYoKTtxKGEsIlRoZSBET00gZWxlbWVudCBmb3IgdGhlIGNvbnRhaW5lciBjYW5ub3QgYmUgbnVsbC4iKTthLnJlbW92ZUF0dHJpYnV0ZSh3YigiYWN0aXZlZGVzY2VuZGFudCIpKX07CmguWWE9ZnVuY3Rpb24oYSl7KGE9YS50YXJnZXQpJiZhIT10aGlzLnMmJmEuYj09dGhpcyYmKHRoaXMucyYmc2QodGhpcy5zLCExKSx0aGlzLnM9YSl9O2guVWE9ZnVuY3Rpb24oYSl7YS50YXJnZXQ9PXRoaXMucyYmKHRoaXMucz1udWxsKTt2YXIgYj10aGlzLmYoKSxjPWEudGFyZ2V0LmYoKTtiJiZhLnRhcmdldC5vJjImJmMmJihhPSIiLGMmJihhPWMuaWQscShhLCJUaGUgYWN0aXZlIGVsZW1lbnQgc2hvdWxkIGhhdmUgYW4gaWQuIikpLHhiKGIsImFjdGl2ZWRlc2NlbmRhbnQiLGEpKX07aC5TYT1mdW5jdGlvbihhKXt0aGlzLk8mJih0aGlzLlY9ITApO3ZhciBiPXlkKHRoaXMpO2ImJnNiKGIpJiZ0YihiKT9iLmZvY3VzKCk6YS5nKCl9O2guVmE9ZnVuY3Rpb24oKXt0aGlzLlY9ITF9OwpoLlRhPWZ1bmN0aW9uKGEpe2E6e3ZhciBiPWEudGFyZ2V0O2lmKHRoaXMuSSlmb3IodmFyIGM9dGhpcy5mKCk7YiYmYiE9PWM7KXt2YXIgZD1iLmlkO2lmKGQgaW4gdGhpcy5JKXtiPXRoaXMuSVtkXTticmVhayBhfWI9Yi5wYXJlbnROb2RlfWI9bnVsbH1pZihiKXN3aXRjaChhLnR5cGUpe2Nhc2UgRi5MOmIua2EoYSk7YnJlYWs7Y2FzZSBGLk06Y2FzZSBGLmFhOmIubGEoYSk7YnJlYWs7Y2FzZSAibW91c2VvdmVyIjpiLkFhKGEpO2JyZWFrO2Nhc2UgIm1vdXNlb3V0IjpiLnphKGEpO2JyZWFrO2Nhc2UgImNvbnRleHRtZW51IjpiLmVhKGEpfX07aC5IYT1mdW5jdGlvbigpe307aC5RYT1mdW5jdGlvbigpe0NkKHRoaXMsLTEpO3RoaXMuVj0hMTt0aGlzLnMmJnNkKHRoaXMucywhMSl9OwpoLlJhPWZ1bmN0aW9uKGEpe3JldHVybiB0aGlzLmlzRW5hYmxlZCgpJiZ0aGlzLmlzVmlzaWJsZSgpJiYoMCE9UmModGhpcyl8fHRoaXMuRGEpJiZEZCh0aGlzLGEpPyhhLmcoKSxhLmooKSwhMCk6ITF9Owp2YXIgRGQ9ZnVuY3Rpb24oYSxiKXt2YXIgYz1MKGEsYS5sKTtpZihjJiYiZnVuY3Rpb24iPT10eXBlb2YgYy5mYSYmYy5mYShiKXx8YS5zJiZhLnMhPWMmJiJmdW5jdGlvbiI9PXR5cGVvZiBhLnMuZmEmJmEucy5mYShiKSlyZXR1cm4hMDtpZihiLnNoaWZ0S2V5fHxiLmN0cmxLZXl8fGIubWV0YUtleXx8Yi5hbHRLZXkpcmV0dXJuITE7c3dpdGNoKGIuYyl7Y2FzZSAyNzp5ZChhKS5ibHVyKCk7YnJlYWs7Y2FzZSAzNjpFZChhKTticmVhaztjYXNlIDM1OkZkKGEpO2JyZWFrO2Nhc2UgMzg6aWYoInZlcnRpY2FsIj09YS5IKUdkKGEpO2Vsc2UgcmV0dXJuITE7YnJlYWs7Y2FzZSAzNzppZigiaG9yaXpvbnRhbCI9PWEuSClTYyhhKT9IZChhKTpHZChhKTtlbHNlIHJldHVybiExO2JyZWFrO2Nhc2UgNDA6aWYoInZlcnRpY2FsIj09YS5IKUhkKGEpO2Vsc2UgcmV0dXJuITE7YnJlYWs7Y2FzZSAzOTppZigiaG9yaXpvbnRhbCI9PWEuSClTYyhhKT9HZChhKTpIZChhKTtlbHNlIHJldHVybiExOwpicmVhaztkZWZhdWx0OnJldHVybiExfXJldHVybiEwfSx6ZD1mdW5jdGlvbihhLGIpe3ZhciBjPWIuZigpO2M9Yy5pZHx8KGMuaWQ9TGMoYikpO2EuSXx8KGEuST17fSk7YS5JW2NdPWJ9O1MucHJvdG90eXBlLnNhPWZ1bmN0aW9uKGEsYil7Q2EoYSxOLCJUaGUgY2hpbGQgb2YgYSBjb250YWluZXIgbXVzdCBiZSBhIGNvbnRyb2wiKTtTLmkuc2EuY2FsbCh0aGlzLGEsYil9O1MucHJvdG90eXBlLnRhPWZ1bmN0aW9uKGEsYixjKXtDYShhLE4pO2EuS3w9MjthLkt8PTY0O3RkKGEsMzIsITEpO2EubSYmMCE9YS55YSYmb2QoYSwhMSk7YS55YT0hMTt2YXIgZD1hLmI9PXRoaXM/VGModGhpcyxhKTotMTtTLmkudGEuY2FsbCh0aGlzLGEsYixjKTthLm0mJnRoaXMubSYmemQodGhpcyxhKTthPWQ7LTE9PWEmJihhPVJjKHRoaXMpKTthPT10aGlzLmw/dGhpcy5sPU1hdGgubWluKFJjKHRoaXMpLTEsYik6YT50aGlzLmwmJmI8PXRoaXMubD90aGlzLmwrKzphPHRoaXMubCYmYj50aGlzLmwmJnRoaXMubC0tfTsKdmFyIGFkPWZ1bmN0aW9uKGEsYil7aWYoYS5mKCkpdGhyb3cgRXJyb3IoIkNvbXBvbmVudCBhbHJlYWR5IHJlbmRlcmVkIik7YS5IPWJ9O1MucHJvdG90eXBlLmlzVmlzaWJsZT1mdW5jdGlvbigpe3JldHVybiB0aGlzLlR9O3ZhciBBZD1mdW5jdGlvbihhLGIpe2EuVD1iO3ZhciBjPWEuZigpO2MmJihjLnN0eWxlLmRpc3BsYXk9Yj8iIjoibm9uZSIsWWMoeWQoYSksYS5PJiZhLlQpKX07Uy5wcm90b3R5cGUuaXNFbmFibGVkPWZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMuT307Uy5wcm90b3R5cGUuWj1mdW5jdGlvbihhKXt0aGlzLk8hPWEmJnVjKHRoaXMsYT8iZW5hYmxlIjoiZGlzYWJsZSIpJiYoYT8odGhpcy5PPSEwLFFjKHRoaXMsZnVuY3Rpb24oYil7Yi5MYT9kZWxldGUgYi5MYTpiLlooITApfSkpOihRYyh0aGlzLGZ1bmN0aW9uKGIpe2IuaXNFbmFibGVkKCk/Yi5aKCExKTpiLkxhPSEwfSksdGhpcy5WPXRoaXMuTz0hMSksWWMoeWQodGhpcyksYSYmdGhpcy5UKSl9Owp2YXIgQ2Q9ZnVuY3Rpb24oYSxiKXsoYj1MKGEsYikpP1AoYiwhMCk6LTE8YS5sJiZQKEwoYSxhLmwpLCExKX0sRWQ9ZnVuY3Rpb24oYSl7SWQoYSxmdW5jdGlvbihiLGMpe3JldHVybihiKzEpJWN9LFJjKGEpLTEpfSxGZD1mdW5jdGlvbihhKXtJZChhLGZ1bmN0aW9uKGIsYyl7Yi0tO3JldHVybiAwPmI/Yy0xOmJ9LDApfSxIZD1mdW5jdGlvbihhKXtJZChhLGZ1bmN0aW9uKGIsYyl7cmV0dXJuKGIrMSklY30sYS5sKX0sR2Q9ZnVuY3Rpb24oYSl7SWQoYSxmdW5jdGlvbihiLGMpe2ItLTtyZXR1cm4gMD5iP2MtMTpifSxhLmwpfSxJZD1mdW5jdGlvbihhLGIsYyl7Yz0wPmM/VGMoYSxhLnMpOmM7dmFyIGQ9UmMoYSk7Yz1iLmNhbGwoYSxjLGQpO2Zvcih2YXIgZT0wO2U8PWQ7KXt2YXIgZj1MKGEsYyksZztpZihnPWYpZz1mLmlzVmlzaWJsZSgpJiZmLmlzRW5hYmxlZCgpJiYhIShmLnYmMik7aWYoZyl7YS5FYShjKTticmVha31lKys7Yz1iLmNhbGwoYSxjLGQpfX07ClMucHJvdG90eXBlLkVhPWZ1bmN0aW9uKGEpe0NkKHRoaXMsYSl9O3ZhciBUPWZ1bmN0aW9uKCl7fTtuKFQsTSk7ZGEoVCk7VC5wcm90b3R5cGUuYT1mdW5jdGlvbigpe3JldHVybiJnb29nLXRhYiJ9O1QucHJvdG90eXBlLmg9ZnVuY3Rpb24oKXtyZXR1cm4idGFiIn07VC5wcm90b3R5cGUuYz1mdW5jdGlvbihhKXt2YXIgYj1ULmkuYy5jYWxsKHRoaXMsYSk7KGE9YS5QYSkmJmImJihiLnRpdGxlPWF8fCIiKTtyZXR1cm4gYn07VC5wcm90b3R5cGUuZz1mdW5jdGlvbihhLGIpe2I9VC5pLmcuY2FsbCh0aGlzLGEsYik7dmFyIGM9Yi50aXRsZXx8IiI7YyYmKGEuUGE9Yyk7YS5vJjgmJihjPWEuYikmJmlhKGMuRykmJihRKGEsOCwhMSksYy5HKGEpKTtyZXR1cm4gYn07dmFyIEpkPWZ1bmN0aW9uKGEsYixjKXtOLmNhbGwodGhpcyxhLGJ8fFQuUigpLGMpO3RkKHRoaXMsOCwhMCk7dGhpcy5LfD05fTtuKEpkLE4pO1ZjKCJnb29nLXRhYiIsZnVuY3Rpb24oKXtyZXR1cm4gbmV3IEpkKG51bGwpfSk7dmFyIFU9ZnVuY3Rpb24oKXt0aGlzLmg9InRhYmxpc3QifTtuKFUsWGMpO2RhKFUpO1UucHJvdG90eXBlLmI9ZnVuY3Rpb24oKXtyZXR1cm4iZ29vZy10YWItYmFyIn07VS5wcm90b3R5cGUuZz1mdW5jdGlvbihhLGIsYyl7dGhpcy5qfHwodGhpcy5hfHxLZCh0aGlzKSx0aGlzLmo9c2EodGhpcy5hKSk7dmFyIGQ9dGhpcy5qW2JdO2Q/KGFkKGEsTGQoZCkpLGEuQj1kKTpVLmkuZy5jYWxsKHRoaXMsYSxiLGMpfTtVLnByb3RvdHlwZS5jPWZ1bmN0aW9uKGEpe3ZhciBiPVUuaS5jLmNhbGwodGhpcyxhKTt0aGlzLmF8fEtkKHRoaXMpO2IucHVzaCh0aGlzLmFbYS5CXSk7cmV0dXJuIGJ9O3ZhciBLZD1mdW5jdGlvbihhKXt2YXIgYj1hLmIoKTthLmE9e3RvcDpiKyItdG9wIixib3R0b206YisiLWJvdHRvbSIsc3RhcnQ6YisiLXN0YXJ0IixlbmQ6YisiLWVuZCJ9fTt2YXIgVj1mdW5jdGlvbihhLGIsYyl7YT1hfHwidG9wIjthZCh0aGlzLExkKGEpKTt0aGlzLkI9YTtTLmNhbGwodGhpcyx0aGlzLkgsYnx8VS5SKCksYyk7TWQodGhpcyl9O24oVixTKTtoPVYucHJvdG90eXBlO2guQz1udWxsO2guRD1mdW5jdGlvbigpe1YuaS5ELmNhbGwodGhpcyk7TWQodGhpcyl9O2gudT1mdW5jdGlvbigpe1YuaS51LmNhbGwodGhpcyk7dGhpcy5DPW51bGx9O2guRWE9ZnVuY3Rpb24oYSl7Vi5pLkVhLmNhbGwodGhpcyxhKTt0aGlzLkcoTCh0aGlzLGEpKX07aC5HPWZ1bmN0aW9uKGEpe2E/cmQoYSwhMCk6dGhpcy5DJiZyZCh0aGlzLkMsITEpfTsKdmFyIE5kPWZ1bmN0aW9uKGEsYil7aWYoYiYmYj09YS5DKXtmb3IodmFyIGM9VGMoYSxiKSxkPWMtMTtiPUwoYSxkKTtkLS0pe3ZhciBlPWI7aWYoZS5pc1Zpc2libGUoKSYmZS5pc0VuYWJsZWQoKSl7YS5HKGIpO3JldHVybn19Zm9yKGMrPTE7Yj1MKGEsYyk7YysrKWlmKGQ9YixkLmlzVmlzaWJsZSgpJiZkLmlzRW5hYmxlZCgpKXthLkcoYik7cmV0dXJufWEuRyhudWxsKX19O2g9Vi5wcm90b3R5cGU7aC5hYj1mdW5jdGlvbihhKXt0aGlzLkMmJnRoaXMuQyE9YS50YXJnZXQmJnJkKHRoaXMuQywhMSk7dGhpcy5DPWEudGFyZ2V0fTtoLmJiPWZ1bmN0aW9uKGEpe2EudGFyZ2V0PT10aGlzLkMmJih0aGlzLkM9bnVsbCl9O2guWmE9ZnVuY3Rpb24oYSl7TmQodGhpcyxhLnRhcmdldCl9O2guJGE9ZnVuY3Rpb24oYSl7TmQodGhpcyxhLnRhcmdldCl9O2guSGE9ZnVuY3Rpb24oKXtMKHRoaXMsdGhpcy5sKXx8Q2QodGhpcyxUYyh0aGlzLHRoaXMuQ3x8TCh0aGlzLDApKSl9Owp2YXIgTWQ9ZnVuY3Rpb24oYSl7SChIKEgoSChPYyhhKSxhLCJzZWxlY3QiLGEuYWIpLGEsInVuc2VsZWN0IixhLmJiKSxhLCJkaXNhYmxlIixhLlphKSxhLCJoaWRlIixhLiRhKX0sTGQ9ZnVuY3Rpb24oYSl7cmV0dXJuInN0YXJ0Ij09YXx8ImVuZCI9PWE/InZlcnRpY2FsIjoiaG9yaXpvbnRhbCJ9O1ZjKCJnb29nLXRhYi1iYXIiLGZ1bmN0aW9uKCl7cmV0dXJuIG5ldyBWfSk7ZnVuY3Rpb24gT2QoYSl7dmFyIGI9e3RvcDoiYm90dG9tIixib3R0b206InRvcCIsc3RhcnQ6InJpZ2h0IixlbmQ6ImxlZnQifVthLmxvY2F0aW9uXSxjPWEuZWxlbWVudElkLGQ9ZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgic3R5bGUiKTtkLnRleHRDb250ZW50PSJcbiAgICBmaWVsZHNldCB7XG4gICAgICBwYWRkaW5nOiAxMHB4O1xuICAgICAgYm9yZGVyOiAxcHggc29saWQgIzM2OTtcbiAgICB9XG5cbiAgICAjIitjKyIgLmdvb2ctdGFiLWNvbnRlbnQge1xuICAgICAgbWluLWhlaWdodDogM2VtO1xuICAgICAgbWFyZ2luOiAwO1xuICAgICAgYm9yZGVyOiAiK2EuYm9yZGVyKyIgc29saWQgIithLmJvcmRlckNvbG9yKyI7XG4gICAgICBib3JkZXItdG9wOiAwO1xuICAgICAgaGVpZ2h0OiAiK2EuY29udGVudEhlaWdodCsiO1xuICAgICAgcGFkZGluZzogNHB4IDhweDtcbiAgICAgIG1hcmdpbi1yaWdodDogNHB4O1xuICAgICAgYmFja2dyb3VuZDogdmFyKC0tY29sYWItcHJpbWFyeS1zdXJmYWNlLWNvbG9yKTtcbiAgICAgIG92ZXJmbG93OiBhdXRvO1xuICAgIH1cblxuICAgICMiKwpjKyIgLmdvb2ctdGFiLWJhci0iK2EubG9jYXRpb24rIiAuZ29vZy10YWItc2VsZWN0ZWQge1xuICAgICAgYmFja2dyb3VuZC1jb2xvcjogdmFyKC0tY29sYWItcHJpbWFyeS1zdXJmYWNlLWNvbG9yKTtcbiAgICAgIGJvcmRlcjogMXB4IHNvbGlkICIrYS5ib3JkZXJDb2xvcisiO1xuICAgICAgYm9yZGVyLSIrYisiOiAwcHg7XG4gICAgfVxuXG4gICAgIyIrYysiIC5nb29nLXRhYi1iYXItIithLmxvY2F0aW9uKyIge1xuICAgICAgcGFkZGluZy0iK2EubG9jYXRpb24rIjogNXB4ICFpbXBvcnRhbnQ7XG4gICAgICBib3JkZXItIitiKyI6IDFweCBzb2xpZCAiK2EuYm9yZGVyQ29sb3IrIiAhaW1wb3J0YW50O1xuICAgICAgYmFja2dyb3VuZDogdmFyKC0tY29sYWItcHJpbWFyeS1zdXJmYWNlLWNvbG9yKTtcbiAgICB9XG5cbiAgICAjIitjKyIgLmdvb2ctdGFiLWJhciB7XG4gICAgICAgbWFyZ2luOiAwO1xuICAgICAgIGJvcmRlcjogMDtcbiAgICAgICBwYWRkaW5nOiAwO1xuICAgICAgIGxpc3Qtc3R5bGU6IG5vbmU7XG4gICAgICAgY3Vyc29yOiBkZWZhdWx0O1xuICAgICAgIG91dGxpbmU6IG5vbmU7XG4gICAgICAgYmFja2dyb3VuZDogdmFyKC0tY29sYWItcHJpbWFyeS1zdXJmYWNlLWNvbG9yKTtcbiAgICAgICBtYXJnaW4tcmlnaHQ6IDRweDtcbiAgICAgIH1cblxuICAgICAjIisKYysiIC5nb29nLXRhYiB7XG4gICAgICAgcG9zaXRpb246IHJlbGF0aXZlO1xuICAgICAgIHBhZGRpbmc6IDRweCA4cHg7XG4gICAgICAgY29sb3I6IHZhcigtLWNvbGFiLXByaW1hcnktdGV4dC1jb2xvcik7XG4gICAgICAgdGV4dC1kZWNvcmF0aW9uOiBpbml0aWFsO1xuICAgICAgIGN1cnNvcjogZGVmYXVsdDtcbiAgICAgIH1cblxuICAgICAgIyIrYysiIC5nb29nLXRhYi1ob3ZlciB7XG4gICAgICAgIGJhY2tncm91bmQtY29sb3I6IHZhcigtLWNvbGFiLWhpZ2hsaWdodGVkLXN1cmZhY2UtY29sb3IpO1xuICAgICAgfVxuICAgICAgIjtyZXR1cm4gZH0KdmFyIFBkPWZ1bmN0aW9uKGEpe3ZhciBiPWEuZWxlbWVudElkLGM9YS50YWJOYW1lcyxkPWEuc2VsZWN0ZWRJbmRleDsiY29udGVudEJvcmRlciJpbiBhfHwoYS5jb250ZW50Qm9yZGVyPSIwcHgiKTsiY29udGVudEhlaWdodCJpbiBhfHwoYS5jb250ZW50SGVpZ2h0PSJpbml0aWFsIik7ImJvcmRlckNvbG9yImluIGF8fChhLmJvcmRlckNvbG9yPSJ2YXIoLS1jb2xhYi1ib3JkZXItY29sb3IpIik7YS5sb2NhdGlvbnx8KGEubG9jYXRpb249InRvcCIpO3ZhciBlPWRvY3VtZW50LnF1ZXJ5U2VsZWN0b3IoIiMiK2IpLGY9ZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgiZGl2Iik7Zi5jbGFzc0xpc3QuYWRkKCJnb29nLXRhYi1iYXIiKTt2YXIgZz1hLmxvY2F0aW9uO2YuY2xhc3NMaXN0LmFkZCgiZ29vZy10YWItYmFyLSIrZyk7Zm9yKHZhciBrPVtdLHA9YmEoYyksdz1wLm5leHQoKTshdy5kb25lO3c9cC5uZXh0KCkpe3c9dy52YWx1ZTt2YXIgQj1kb2N1bWVudC5jcmVhdGVFbGVtZW50KCJkaXYiKTsKQi5jbGFzc0xpc3QuYWRkKCJnb29nLXRhYiIpO0IudGV4dENvbnRlbnQ9dztmLmFwcGVuZENoaWxkKEIpO2sucHVzaChCKX0iYm90dG9tIiE9ZyYmZS5hcHBlbmRDaGlsZChmKTtwPW51bGw7aWYoInRvcCI9PWd8fCJib3R0b20iPT1nKXA9ZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgiZGl2IikscC5jbGFzc0xpc3QuYWRkKCJnb29nLXRhYi1iYXItY2xlYXIiKTsidG9wIj09ZyYmcCYmZS5hcHBlbmRDaGlsZChwKTtCPWRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoImRpdiIpO0IuY2xhc3NMaXN0LmFkZCgiZ29vZy10YWItY29udGVudCIpO3ZhciByYT1bXTtjPWJhKGMpO2Zvcih3PWMubmV4dCgpOyF3LmRvbmU7dz1jLm5leHQoKSl3PWRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoImRpdiIpLHcuaWQ9ZS5pZCsiX2NvbnRlbnRfIityYS5sZW5ndGgsdy5zdHlsZS5kaXNwbGF5PSJub25lIixCLmFwcGVuZENoaWxkKHcpLHJhLnB1c2godyk7ZS5hcHBlbmRDaGlsZChCKTsiYm90dG9tIj09ZyYmKHAmJgplLmFwcGVuZENoaWxkKHApLGUuYXBwZW5kQ2hpbGQoZikpO3ZhciBYPW5ldyBWKGcpO1BjKFgsZik7dmFyIFk9LTE7ZGMoWCwic2VsZWN0IixmdW5jdGlvbihaKXtaPWsuaW5kZXhPZihaLnRhcmdldC5mKCkpO1ohPVkmJigwPD1ZJiZZPHJhLmxlbmd0aCYmKHJhW1ldLnN0eWxlLmRpc3BsYXk9Im5vbmUiLFk9LTEpLDA8PVomJlo8cmEubGVuZ3RoJiYoWT1aLHJhW1ldLnN0eWxlLmRpc3BsYXk9ImlubGluZSIsd2luZG93LmRpc3BhdGNoRXZlbnQobmV3IEV2ZW50KCJyZXNpemUiKSksZ29vZ2xlLmNvbGFiLm91dHB1dC5yZXNpemVJZnJhbWVUb0NvbnRlbnQoKSksWC5HKEwoWCxZKSkpfSk7WC5HKEwoWCxkKSk7d2luZG93W2JdPXtzZXRTZWxlY3RlZFRhYkluZGV4OmZ1bmN0aW9uKFope1guRyhMKFgsWikpfX07ZG9jdW1lbnQuaGVhZC5hcHBlbmRDaGlsZChPZChhKSl9LFFkPVsiY29sYWJfbGliIiwiY3JlYXRlVGFiQmFyIl0sVz1sOwpRZFswXWluIFd8fCJ1bmRlZmluZWQiPT10eXBlb2YgVy5leGVjU2NyaXB0fHxXLmV4ZWNTY3JpcHQoInZhciAiK1FkWzBdKTtmb3IodmFyIFJkO1FkLmxlbmd0aCYmKFJkPVFkLnNoaWZ0KCkpOylRZC5sZW5ndGh8fHZvaWQgMD09PVBkP1dbUmRdJiZXW1JkXSE9PU9iamVjdC5wcm90b3R5cGVbUmRdP1c9V1tSZF06Vz1XW1JkXT17fTpXW1JkXT1QZDt9KS5jYWxsKHRoaXMpOwovLyBjbGFuZy1mb3JtYXQgb24K","headers":[["content-type","application/javascript"]],"ok":true,"status":200,"status_text":""}}},"executionInfo":{"elapsed":505,"status":"ok","timestamp":1712935166968,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"pKz-3HfVrN30","outputId":"fdce00ad-8873-4ece-8b06-c6ac54a65fa2"},"outputs":[{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["<link rel=stylesheet type=text/css href='/nbextensions/google.colab/tabbar.css'></link>"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["<script src='/nbextensions/google.colab/tabbar_main.min.js'></script>"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["<div id=\"id1\"></div>"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c47a0b0-f8e0-11ee-89e5-0242ac1c000c\"] = colab_lib.createTabBar({\"location\": \"top\", \"elementId\": \"id1\", \"tabNames\": [\"ner_clinical_large\", \"ner_jsl\", \"viz_clinical\", \"viz_jsl\"], \"initialSelection\": 0, \"contentBorder\": [\"0px\"], \"contentHeight\": [\"initial\"], \"borderColor\": [\"#a7a7a7\"]});\n","//# sourceURL=js_e7c781701d"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c47fd6c-f8e0-11ee-89e5-0242ac1c000c\"] = window[\"id1\"].setSelectedTabIndex(0);\n","//# sourceURL=js_8af8a230c9"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c4a0634-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.getActiveOutputArea();\n","//# sourceURL=js_24ba615a00"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c4a661a-f8e0-11ee-89e5-0242ac1c000c\"] = document.querySelector(\"#id1_content_0\");\n","//# sourceURL=js_c6d9791dc8"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c4ac4f2-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"0c4a661a-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_dd94df35a1"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c4b2bae-f8e0-11ee-89e5-0242ac1c000c\"] = window[\"id1\"].setSelectedTabIndex(0);\n","//# sourceURL=js_30a73b97cd"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["                      entities_ner_chunk entities_ner_chunk_begin  \\\n","0          gestational diabetes mellitus                       40   \n","0  subsequent type two diabetes mellitus                      118   \n","0                                   T2DM                      157   \n","0               HTG-induced pancreatitis                      185   \n","0                     an acute hepatitis                      265   \n","0                               polyuria                      322   \n","0                          poor appetite                      332   \n","0                               vomiting                      351   \n","0                              metformin                      372   \n","0                              glipizide                      383   \n","0                          dapagliflozin                      398   \n","0                                   T2DM                      416   \n","0                           atorvastatin                      425   \n","0                            gemfibrozil                      442   \n","0                                    HTG                      458   \n","0                          dapagliflozin                      479   \n","0                   Physical examination                      537   \n","0                        dry oral mucosa                      594   \n","0              her abdominal examination                      628   \n","0                             tenderness                      673   \n","0                               guarding                      685   \n","0                               rigidity                      698   \n","0                          serum glucose                      757   \n","0                             creatinine                      783   \n","0                          triglycerides                      805   \n","0                      total cholesterol                      830   \n","0                              venous pH                      863   \n","\n","  entities_ner_chunk_end entities_ner_chunk_origin_sentence  \\\n","0                     68                                  0   \n","0                    154                                  0   \n","0                    160                                  0   \n","0                    208                                  0   \n","0                    282                                  0   \n","0                    329                                  0   \n","0                    344                                  0   \n","0                    358                                  0   \n","0                    380                                  1   \n","0                    391                                  1   \n","0                    410                                  1   \n","0                    419                                  1   \n","0                    436                                  1   \n","0                    452                                  1   \n","0                    460                                  1   \n","0                    491                                  2   \n","0                    556                                  3   \n","0                    608                                  3   \n","0                    652                                  3   \n","0                    682                                  3   \n","0                    692                                  3   \n","0                    705                                  3   \n","0                    769                                  4   \n","0                    792                                  4   \n","0                    817                                  4   \n","0                    846                                  4   \n","0                    871                                  4   \n","\n","  entities_ner_chunk_class  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                TREATMENT  \n","0                TREATMENT  \n","0                TREATMENT  \n","0                  PROBLEM  \n","0                TREATMENT  \n","0                TREATMENT  \n","0                  PROBLEM  \n","0                TREATMENT  \n","0                     TEST  \n","0                  PROBLEM  \n","0                     TEST  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                  PROBLEM  \n","0                     TEST  \n","0                     TEST  \n","0                     TEST  \n","0                     TEST  \n","0                     TEST  "],"text/html":["\n","  <div id=\"df-946451a5-0dd6-4c37-92d8-f2f1f0d331ff\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>entities_ner_chunk</th>\n","      <th>entities_ner_chunk_begin</th>\n","      <th>entities_ner_chunk_end</th>\n","      <th>entities_ner_chunk_origin_sentence</th>\n","      <th>entities_ner_chunk_class</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>gestational diabetes mellitus</td>\n","      <td>40</td>\n","      <td>68</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>subsequent type two diabetes mellitus</td>\n","      <td>118</td>\n","      <td>154</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>T2DM</td>\n","      <td>157</td>\n","      <td>160</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>HTG-induced pancreatitis</td>\n","      <td>185</td>\n","      <td>208</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>an acute hepatitis</td>\n","      <td>265</td>\n","      <td>282</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>polyuria</td>\n","      <td>322</td>\n","      <td>329</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>poor appetite</td>\n","      <td>332</td>\n","      <td>344</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>vomiting</td>\n","      <td>351</td>\n","      <td>358</td>\n","      <td>0</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>metformin</td>\n","      <td>372</td>\n","      <td>380</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>glipizide</td>\n","      <td>383</td>\n","      <td>391</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>dapagliflozin</td>\n","      <td>398</td>\n","      <td>410</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>T2DM</td>\n","      <td>416</td>\n","      <td>419</td>\n","      <td>1</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>atorvastatin</td>\n","      <td>425</td>\n","      <td>436</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>gemfibrozil</td>\n","      <td>442</td>\n","      <td>452</td>\n","      <td>1</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>HTG</td>\n","      <td>458</td>\n","      <td>460</td>\n","      <td>1</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>dapagliflozin</td>\n","      <td>479</td>\n","      <td>491</td>\n","      <td>2</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>Physical examination</td>\n","      <td>537</td>\n","      <td>556</td>\n","      <td>3</td>\n","      <td>TEST</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>dry oral mucosa</td>\n","      <td>594</td>\n","      <td>608</td>\n","      <td>3</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>her abdominal examination</td>\n","      <td>628</td>\n","      <td>652</td>\n","      <td>3</td>\n","      <td>TEST</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>tenderness</td>\n","      <td>673</td>\n","      <td>682</td>\n","      <td>3</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>guarding</td>\n","      <td>685</td>\n","      <td>692</td>\n","      <td>3</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>rigidity</td>\n","      <td>698</td>\n","      <td>705</td>\n","      <td>3</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>serum glucose</td>\n","      <td>757</td>\n","      <td>769</td>\n","      <td>4</td>\n","      <td>TEST</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>creatinine</td>\n","      <td>783</td>\n","      <td>792</td>\n","      <td>4</td>\n","      <td>TEST</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>triglycerides</td>\n","      <td>805</td>\n","      <td>817</td>\n","      <td>4</td>\n","      <td>TEST</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>total cholesterol</td>\n","      <td>830</td>\n","      <td>846</td>\n","      <td>4</td>\n","      <td>TEST</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>venous pH</td>\n","      <td>863</td>\n","      <td>871</td>\n","      <td>4</td>\n","      <td>TEST</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-946451a5-0dd6-4c37-92d8-f2f1f0d331ff')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-946451a5-0dd6-4c37-92d8-f2f1f0d331ff button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-946451a5-0dd6-4c37-92d8-f2f1f0d331ff');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-b8dfe0ee-c07e-4381-9910-5d34705c54ba\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b8dfe0ee-c07e-4381-9910-5d34705c54ba')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-b8dfe0ee-c07e-4381-9910-5d34705c54ba button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","  <div id=\"id_bf25bb2b-371d-4e52-b093-e0c6b26ef475\">\n","    <style>\n","      .colab-df-generate {\n","        background-color: #E8F0FE;\n","        border: none;\n","        border-radius: 50%;\n","        cursor: pointer;\n","        display: none;\n","        fill: #1967D2;\n","        height: 32px;\n","        padding: 0 0 0 0;\n","        width: 32px;\n","      }\n","\n","      .colab-df-generate:hover {\n","        background-color: #E2EBFA;\n","        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","        fill: #174EA6;\n","      }\n","\n","      [theme=dark] .colab-df-generate {\n","        background-color: #3B4455;\n","        fill: #D2E3FC;\n","      }\n","\n","      [theme=dark] .colab-df-generate:hover {\n","        background-color: #434B5C;\n","        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","        fill: #FFFFFF;\n","      }\n","    </style>\n","    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('df_clinical')\"\n","            title=\"Generate code using this dataframe.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n","  </svg>\n","    </button>\n","    <script>\n","      (() => {\n","      const buttonEl =\n","        document.querySelector('#id_bf25bb2b-371d-4e52-b093-e0c6b26ef475 button.colab-df-generate');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      buttonEl.onclick = () => {\n","        google.colab.notebook.generateWithVariable('df_clinical');\n","      }\n","      })();\n","    </script>\n","  </div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"df_clinical","summary":"{\n  \"name\": \"df_clinical\",\n  \"rows\": 27,\n  \"fields\": [\n    {\n      \"column\": \"entities_ner_chunk\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 25,\n        \"samples\": [\n          \"metformin\",\n          \"her abdominal examination\",\n          \"gestational diabetes mellitus\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_begin\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 40,\n        \"max\": 863,\n        \"num_unique_values\": 27,\n        \"samples\": [\n          372,\n          442,\n          383\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_end\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 68,\n        \"max\": 871,\n        \"num_unique_values\": 27,\n        \"samples\": [\n          380,\n          452,\n          391\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_origin_sentence\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"1\",\n          \"4\",\n          \"2\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_class\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"PROBLEM\",\n          \"TREATMENT\",\n          \"TEST\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c4f84c4-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"0c4a0634-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_c3cd8520ac"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c52245e-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.getActiveOutputArea();\n","//# sourceURL=js_0e91a6ef1a"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c529286-f8e0-11ee-89e5-0242ac1c000c\"] = document.querySelector(\"#id1_content_1\");\n","//# sourceURL=js_5c5c90c3fd"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c52fc3a-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"0c529286-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_3d23a5a85b"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c53621a-f8e0-11ee-89e5-0242ac1c000c\"] = window[\"id1\"].setSelectedTabIndex(1);\n","//# sourceURL=js_91391aaeac"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["          entities_jsl_ner_chunk entities_jsl_ner_chunk_begin  \\\n","0                    28-year-old                            3   \n","0                         female                           15   \n","0  gestational diabetes mellitus                           40   \n","0              eight years prior                           80   \n","0     type two diabetes mellitus                          129   \n","0                           T2DM                          157   \n","0       HTG-induced pancreatitis                          185   \n","0              three years prior                          210   \n","0                          acute                          268   \n","0                      hepatitis                          274   \n","0                       one-week                          302   \n","0                       polyuria                          322   \n","0                  poor appetite                          332   \n","0                       vomiting                          351   \n","0                            She                          361   \n","0                      metformin                          372   \n","0                      glipizide                          383   \n","0                  dapagliflozin                          398   \n","0                           T2DM                          416   \n","0                   atorvastatin                          425   \n","0                    gemfibrozil                          442   \n","0                            HTG                          458   \n","0                            She                          463   \n","0                  dapagliflozin                          479   \n","0                 for six months                          493   \n","0                dry oral mucosa                          594   \n","0                            her                          628   \n","0                      abdominal                          632   \n","0                     tenderness                          673   \n","0                       guarding                          685   \n","0                       rigidity                          698   \n","0                      admission                          741   \n","0                  serum glucose                          757   \n","0                      111 mg/dl                          771   \n","0                     creatinine                          783   \n","0                      0.4 mg/dL                          794   \n","0                  triglycerides                          805   \n","0                      508 mg/dL                          819   \n","0    total cholesterol 122 mg/dL                          830   \n","0                      venous pH                          863   \n","0                           7.27                          873   \n","\n","  entities_jsl_ner_chunk_end entities_jsl_ner_chunk_origin_sentence  \\\n","0                         13                                      0   \n","0                         20                                      0   \n","0                         68                                      0   \n","0                         96                                      0   \n","0                        154                                      0   \n","0                        160                                      0   \n","0                        208                                      0   \n","0                        226                                      0   \n","0                        272                                      0   \n","0                        282                                      0   \n","0                        309                                      0   \n","0                        329                                      0   \n","0                        344                                      0   \n","0                        358                                      0   \n","0                        363                                      1   \n","0                        380                                      1   \n","0                        391                                      1   \n","0                        410                                      1   \n","0                        419                                      1   \n","0                        436                                      1   \n","0                        452                                      1   \n","0                        460                                      1   \n","0                        465                                      2   \n","0                        491                                      2   \n","0                        506                                      2   \n","0                        608                                      3   \n","0                        630                                      3   \n","0                        640                                      3   \n","0                        682                                      3   \n","0                        692                                      3   \n","0                        705                                      3   \n","0                        749                                      4   \n","0                        769                                      4   \n","0                        779                                      4   \n","0                        792                                      4   \n","0                        802                                      4   \n","0                        817                                      4   \n","0                        827                                      4   \n","0                        856                                      4   \n","0                        871                                      4   \n","0                        876                                      4   \n","\n","   entities_jsl_ner_chunk_class  \n","0                           Age  \n","0                        Gender  \n","0                      Diabetes  \n","0                  RelativeDate  \n","0                      Diabetes  \n","0                      Diabetes  \n","0     Disease_Syndrome_Disorder  \n","0                  RelativeDate  \n","0                      Modifier  \n","0     Disease_Syndrome_Disorder  \n","0                      Duration  \n","0                       Symptom  \n","0                       Symptom  \n","0                       Symptom  \n","0                        Gender  \n","0               Drug_Ingredient  \n","0               Drug_Ingredient  \n","0               Drug_Ingredient  \n","0                      Diabetes  \n","0               Drug_Ingredient  \n","0               Drug_Ingredient  \n","0                Hyperlipidemia  \n","0                        Gender  \n","0               Drug_Ingredient  \n","0                      Duration  \n","0                       Symptom  \n","0                        Gender  \n","0  External_body_part_or_region  \n","0                       Symptom  \n","0                       Symptom  \n","0                       Symptom  \n","0           Admission_Discharge  \n","0                          Test  \n","0                   Test_Result  \n","0                          Test  \n","0                   Test_Result  \n","0                 Triglycerides  \n","0                   Test_Result  \n","0             Total_Cholesterol  \n","0                          Test  \n","0                   Test_Result  "],"text/html":["\n","  <div id=\"df-3d120bc6-0e48-4ae1-8b95-4755c886f3ad\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>entities_jsl_ner_chunk</th>\n","      <th>entities_jsl_ner_chunk_begin</th>\n","      <th>entities_jsl_ner_chunk_end</th>\n","      <th>entities_jsl_ner_chunk_origin_sentence</th>\n","      <th>entities_jsl_ner_chunk_class</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>28-year-old</td>\n","      <td>3</td>\n","      <td>13</td>\n","      <td>0</td>\n","      <td>Age</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>female</td>\n","      <td>15</td>\n","      <td>20</td>\n","      <td>0</td>\n","      <td>Gender</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>gestational diabetes mellitus</td>\n","      <td>40</td>\n","      <td>68</td>\n","      <td>0</td>\n","      <td>Diabetes</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>eight years prior</td>\n","      <td>80</td>\n","      <td>96</td>\n","      <td>0</td>\n","      <td>RelativeDate</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>type two diabetes mellitus</td>\n","      <td>129</td>\n","      <td>154</td>\n","      <td>0</td>\n","      <td>Diabetes</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>T2DM</td>\n","      <td>157</td>\n","      <td>160</td>\n","      <td>0</td>\n","      <td>Diabetes</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>HTG-induced pancreatitis</td>\n","      <td>185</td>\n","      <td>208</td>\n","      <td>0</td>\n","      <td>Disease_Syndrome_Disorder</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>three years prior</td>\n","      <td>210</td>\n","      <td>226</td>\n","      <td>0</td>\n","      <td>RelativeDate</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>acute</td>\n","      <td>268</td>\n","      <td>272</td>\n","      <td>0</td>\n","      <td>Modifier</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>hepatitis</td>\n","      <td>274</td>\n","      <td>282</td>\n","      <td>0</td>\n","      <td>Disease_Syndrome_Disorder</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>one-week</td>\n","      <td>302</td>\n","      <td>309</td>\n","      <td>0</td>\n","      <td>Duration</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>polyuria</td>\n","      <td>322</td>\n","      <td>329</td>\n","      <td>0</td>\n","      <td>Symptom</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>poor appetite</td>\n","      <td>332</td>\n","      <td>344</td>\n","      <td>0</td>\n","      <td>Symptom</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>vomiting</td>\n","      <td>351</td>\n","      <td>358</td>\n","      <td>0</td>\n","      <td>Symptom</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>She</td>\n","      <td>361</td>\n","      <td>363</td>\n","      <td>1</td>\n","      <td>Gender</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>metformin</td>\n","      <td>372</td>\n","      <td>380</td>\n","      <td>1</td>\n","      <td>Drug_Ingredient</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>glipizide</td>\n","      <td>383</td>\n","      <td>391</td>\n","      <td>1</td>\n","      <td>Drug_Ingredient</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>dapagliflozin</td>\n","      <td>398</td>\n","      <td>410</td>\n","      <td>1</td>\n","      <td>Drug_Ingredient</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>T2DM</td>\n","      <td>416</td>\n","      <td>419</td>\n","      <td>1</td>\n","      <td>Diabetes</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>atorvastatin</td>\n","      <td>425</td>\n","      <td>436</td>\n","      <td>1</td>\n","      <td>Drug_Ingredient</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>gemfibrozil</td>\n","      <td>442</td>\n","      <td>452</td>\n","      <td>1</td>\n","      <td>Drug_Ingredient</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>HTG</td>\n","      <td>458</td>\n","      <td>460</td>\n","      <td>1</td>\n","      <td>Hyperlipidemia</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>She</td>\n","      <td>463</td>\n","      <td>465</td>\n","      <td>2</td>\n","      <td>Gender</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>dapagliflozin</td>\n","      <td>479</td>\n","      <td>491</td>\n","      <td>2</td>\n","      <td>Drug_Ingredient</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>for six months</td>\n","      <td>493</td>\n","      <td>506</td>\n","      <td>2</td>\n","      <td>Duration</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>dry oral mucosa</td>\n","      <td>594</td>\n","      <td>608</td>\n","      <td>3</td>\n","      <td>Symptom</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>her</td>\n","      <td>628</td>\n","      <td>630</td>\n","      <td>3</td>\n","      <td>Gender</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>abdominal</td>\n","      <td>632</td>\n","      <td>640</td>\n","      <td>3</td>\n","      <td>External_body_part_or_region</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>tenderness</td>\n","      <td>673</td>\n","      <td>682</td>\n","      <td>3</td>\n","      <td>Symptom</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>guarding</td>\n","      <td>685</td>\n","      <td>692</td>\n","      <td>3</td>\n","      <td>Symptom</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>rigidity</td>\n","      <td>698</td>\n","      <td>705</td>\n","      <td>3</td>\n","      <td>Symptom</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>admission</td>\n","      <td>741</td>\n","      <td>749</td>\n","      <td>4</td>\n","      <td>Admission_Discharge</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>serum glucose</td>\n","      <td>757</td>\n","      <td>769</td>\n","      <td>4</td>\n","      <td>Test</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>111 mg/dl</td>\n","      <td>771</td>\n","      <td>779</td>\n","      <td>4</td>\n","      <td>Test_Result</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>creatinine</td>\n","      <td>783</td>\n","      <td>792</td>\n","      <td>4</td>\n","      <td>Test</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>0.4 mg/dL</td>\n","      <td>794</td>\n","      <td>802</td>\n","      <td>4</td>\n","      <td>Test_Result</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>triglycerides</td>\n","      <td>805</td>\n","      <td>817</td>\n","      <td>4</td>\n","      <td>Triglycerides</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>508 mg/dL</td>\n","      <td>819</td>\n","      <td>827</td>\n","      <td>4</td>\n","      <td>Test_Result</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>total cholesterol 122 mg/dL</td>\n","      <td>830</td>\n","      <td>856</td>\n","      <td>4</td>\n","      <td>Total_Cholesterol</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>venous pH</td>\n","      <td>863</td>\n","      <td>871</td>\n","      <td>4</td>\n","      <td>Test</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>7.27</td>\n","      <td>873</td>\n","      <td>876</td>\n","      <td>4</td>\n","      <td>Test_Result</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-3d120bc6-0e48-4ae1-8b95-4755c886f3ad')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-3d120bc6-0e48-4ae1-8b95-4755c886f3ad button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-3d120bc6-0e48-4ae1-8b95-4755c886f3ad');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-bbbf9c6c-ed42-4085-9555-1df460d083df\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-bbbf9c6c-ed42-4085-9555-1df460d083df')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-bbbf9c6c-ed42-4085-9555-1df460d083df button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","  <div id=\"id_39067355-105b-4bf4-873c-c314e374db2c\">\n","    <style>\n","      .colab-df-generate {\n","        background-color: #E8F0FE;\n","        border: none;\n","        border-radius: 50%;\n","        cursor: pointer;\n","        display: none;\n","        fill: #1967D2;\n","        height: 32px;\n","        padding: 0 0 0 0;\n","        width: 32px;\n","      }\n","\n","      .colab-df-generate:hover {\n","        background-color: #E2EBFA;\n","        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","        fill: #174EA6;\n","      }\n","\n","      [theme=dark] .colab-df-generate {\n","        background-color: #3B4455;\n","        fill: #D2E3FC;\n","      }\n","\n","      [theme=dark] .colab-df-generate:hover {\n","        background-color: #434B5C;\n","        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","        fill: #FFFFFF;\n","      }\n","    </style>\n","    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('jsl_df')\"\n","            title=\"Generate code using this dataframe.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n","  </svg>\n","    </button>\n","    <script>\n","      (() => {\n","      const buttonEl =\n","        document.querySelector('#id_39067355-105b-4bf4-873c-c314e374db2c button.colab-df-generate');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      buttonEl.onclick = () => {\n","        google.colab.notebook.generateWithVariable('jsl_df');\n","      }\n","      })();\n","    </script>\n","  </div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"jsl_df","summary":"{\n  \"name\": \"jsl_df\",\n  \"rows\": 41,\n  \"fields\": [\n    {\n      \"column\": \"entities_jsl_ner_chunk\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 38,\n        \"samples\": [\n          \"triglycerides\",\n          \"venous pH\",\n          \"type two diabetes mellitus\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_jsl_ner_chunk_begin\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 3,\n        \"max\": 873,\n        \"num_unique_values\": 41,\n        \"samples\": [\n          493,\n          351,\n          268\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_jsl_ner_chunk_end\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 13,\n        \"max\": 876,\n        \"num_unique_values\": 41,\n        \"samples\": [\n          506,\n          358,\n          272\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_jsl_ner_chunk_origin_sentence\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"1\",\n          \"4\",\n          \"2\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_jsl_ner_chunk_class\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 16,\n        \"samples\": [\n          \"Age\",\n          \"Gender\",\n          \"Modifier\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c57ffaa-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"0c52245e-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_cdb8d107a4"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c5b3292-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.getActiveOutputArea();\n","//# sourceURL=js_2b66f69b40"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c5b95a2-f8e0-11ee-89e5-0242ac1c000c\"] = document.querySelector(\"#id1_content_2\");\n","//# sourceURL=js_c529181d1c"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c5bea98-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"0c5b95a2-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_458def8ecd"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c5c41e6-f8e0-11ee-89e5-0242ac1c000c\"] = window[\"id1\"].setSelectedTabIndex(2);\n","//# sourceURL=js_5ff8a565ea"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["\n","<style>\n","    @import url('https://fonts.googleapis.com/css2?family=Montserrat:wght@300;400;500;600;700&display=swap');\n","    @import url('https://fonts.googleapis.com/css2?family=Vistol Regular:wght@300;400;500;600;700&display=swap');\n","    \n","    .spark-nlp-display-scroll-entities {\n","        border: 1px solid #E7EDF0;\n","        border-radius: 3px;\n","        text-align: justify;\n","        \n","    }\n","    .spark-nlp-display-scroll-entities span {  \n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #536B76;\n","        font-family: 'Montserrat', sans-serif !important;\n","    }\n","    \n","    .spark-nlp-display-entity-wrapper{\n","    \n","        display: inline-grid;\n","        text-align: center;\n","        border-radius: 4px;\n","        margin: 0 2px 5px 2px;\n","        padding: 1px\n","    }\n","    .spark-nlp-display-entity-name{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        background: #f1f2f3;\n","        border-width: medium;\n","        text-align: center;\n","        \n","        font-weight: 400;\n","        \n","        border-radius: 5px;\n","        padding: 2px 5px;\n","        display: block;\n","        margin: 3px 2px;\n","    \n","    }\n","    .spark-nlp-display-entity-type{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-entity-resolution{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Vistol Regular', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-others{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        font-weight: 400;\n","    }\n","\n","</style>\n"," <span class=\"spark-nlp-display-others\" style=\"background-color: white\"><br>A 28-year-old female with a history of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">gestational diabetes mellitus </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> diagnosed eight years prior to presentation and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">subsequent type two diabetes mellitus </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> (</span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">T2DM </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">), one prior episode of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">HTG-induced pancreatitis </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> three years prior to presentation, and associated with </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">an acute hepatitis </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, presented with a one-week history of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">polyuria </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">poor appetite </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">vomiting </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">.<br>She was on </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">metformin </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">glipizide </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">dapagliflozin </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">T2DM </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">atorvastatin </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">gemfibrozil </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">HTG </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">. She had been on </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">dapagliflozin </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> for six months at the time of presentation.<br></span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">Physical examination </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> on presentation was significant for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">dry oral mucosa </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> ; significantly , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">her abdominal examination </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> was benign with no </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">tenderness </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">guarding </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, or </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">rigidity </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">. Pertinent laboratory findings on admission were: </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">serum glucose </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 111 mg/dl,  </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">creatinine </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 0.4 mg/dL, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">triglycerides </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 508 mg/dL, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">total cholesterol </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 122 mg/dL, and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">venous pH </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 7.27.<br></span></div>"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c5f7c62-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"0c5b3292-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_41d8e55075"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c621616-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.getActiveOutputArea();\n","//# sourceURL=js_a32fcad31d"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c6285f6-f8e0-11ee-89e5-0242ac1c000c\"] = document.querySelector(\"#id1_content_3\");\n","//# sourceURL=js_cdfad54050"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c6301c0-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"0c6285f6-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_1698dd1450"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c636f34-f8e0-11ee-89e5-0242ac1c000c\"] = window[\"id1\"].setSelectedTabIndex(3);\n","//# sourceURL=js_a6a8a26909"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["\n","<style>\n","    @import url('https://fonts.googleapis.com/css2?family=Montserrat:wght@300;400;500;600;700&display=swap');\n","    @import url('https://fonts.googleapis.com/css2?family=Vistol Regular:wght@300;400;500;600;700&display=swap');\n","    \n","    .spark-nlp-display-scroll-entities {\n","        border: 1px solid #E7EDF0;\n","        border-radius: 3px;\n","        text-align: justify;\n","        \n","    }\n","    .spark-nlp-display-scroll-entities span {  \n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #536B76;\n","        font-family: 'Montserrat', sans-serif !important;\n","    }\n","    \n","    .spark-nlp-display-entity-wrapper{\n","    \n","        display: inline-grid;\n","        text-align: center;\n","        border-radius: 4px;\n","        margin: 0 2px 5px 2px;\n","        padding: 1px\n","    }\n","    .spark-nlp-display-entity-name{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        background: #f1f2f3;\n","        border-width: medium;\n","        text-align: center;\n","        \n","        font-weight: 400;\n","        \n","        border-radius: 5px;\n","        padding: 2px 5px;\n","        display: block;\n","        margin: 3px 2px;\n","    \n","    }\n","    .spark-nlp-display-entity-type{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-entity-resolution{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Vistol Regular', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-others{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        font-weight: 400;\n","    }\n","\n","</style>\n"," <span class=\"spark-nlp-display-others\" style=\"background-color: white\"><br>A </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #ffe0ac\"><span class=\"spark-nlp-display-entity-name\">28-year-old </span><span class=\"spark-nlp-display-entity-type\">Age</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #ffacb7\"><span class=\"spark-nlp-display-entity-name\">female </span><span class=\"spark-nlp-display-entity-type\">Gender</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> with a history of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #5BB49A\"><span class=\"spark-nlp-display-entity-name\">gestational diabetes mellitus </span><span class=\"spark-nlp-display-entity-type\">Diabetes</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> diagnosed </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #654BAE\"><span class=\"spark-nlp-display-entity-name\">eight years prior </span><span class=\"spark-nlp-display-entity-type\">RelativeDate</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> to presentation and subsequent </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #5BB49A\"><span class=\"spark-nlp-display-entity-name\">type two diabetes mellitus </span><span class=\"spark-nlp-display-entity-type\">Diabetes</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> (</span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #5BB49A\"><span class=\"spark-nlp-display-entity-name\">T2DM </span><span class=\"spark-nlp-display-entity-type\">Diabetes</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">), one prior episode of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #BA22AE\"><span class=\"spark-nlp-display-entity-name\">HTG-induced pancreatitis </span><span class=\"spark-nlp-display-entity-type\">Disease_Syndrome_Disorder</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #654BAE\"><span class=\"spark-nlp-display-entity-name\">three years prior </span><span class=\"spark-nlp-display-entity-type\">RelativeDate</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> to presentation, and associated with an </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8B475D\"><span class=\"spark-nlp-display-entity-name\">acute </span><span class=\"spark-nlp-display-entity-type\">Modifier</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #BA22AE\"><span class=\"spark-nlp-display-entity-name\">hepatitis </span><span class=\"spark-nlp-display-entity-type\">Disease_Syndrome_Disorder</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, presented with a </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #21A828\"><span class=\"spark-nlp-display-entity-name\">one-week </span><span class=\"spark-nlp-display-entity-type\">Duration</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> history of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #C1279C\"><span class=\"spark-nlp-display-entity-name\">polyuria </span><span class=\"spark-nlp-display-entity-type\">Symptom</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #C1279C\"><span class=\"spark-nlp-display-entity-name\">poor appetite </span><span class=\"spark-nlp-display-entity-type\">Symptom</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #C1279C\"><span class=\"spark-nlp-display-entity-name\">vomiting </span><span class=\"spark-nlp-display-entity-type\">Symptom</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">.<br></span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #ffacb7\"><span class=\"spark-nlp-display-entity-name\">She </span><span class=\"spark-nlp-display-entity-type\">Gender</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> was on </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #2FA559\"><span class=\"spark-nlp-display-entity-name\">metformin </span><span class=\"spark-nlp-display-entity-type\">Drug_Ingredient</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #2FA559\"><span class=\"spark-nlp-display-entity-name\">glipizide </span><span class=\"spark-nlp-display-entity-type\">Drug_Ingredient</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #2FA559\"><span class=\"spark-nlp-display-entity-name\">dapagliflozin </span><span class=\"spark-nlp-display-entity-type\">Drug_Ingredient</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #5BB49A\"><span class=\"spark-nlp-display-entity-name\">T2DM </span><span class=\"spark-nlp-display-entity-type\">Diabetes</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #2FA559\"><span class=\"spark-nlp-display-entity-name\">atorvastatin </span><span class=\"spark-nlp-display-entity-type\">Drug_Ingredient</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #2FA559\"><span class=\"spark-nlp-display-entity-name\">gemfibrozil </span><span class=\"spark-nlp-display-entity-type\">Drug_Ingredient</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #1406B8\"><span class=\"spark-nlp-display-entity-name\">HTG </span><span class=\"spark-nlp-display-entity-type\">Hyperlipidemia</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">. </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #ffacb7\"><span class=\"spark-nlp-display-entity-name\">She </span><span class=\"spark-nlp-display-entity-type\">Gender</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> had been on </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #2FA559\"><span class=\"spark-nlp-display-entity-name\">dapagliflozin </span><span class=\"spark-nlp-display-entity-type\">Drug_Ingredient</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #21A828\"><span class=\"spark-nlp-display-entity-name\">for six months </span><span class=\"spark-nlp-display-entity-type\">Duration</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> at the time of presentation.<br>Physical examination on presentation was significant for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #C1279C\"><span class=\"spark-nlp-display-entity-name\">dry oral mucosa </span><span class=\"spark-nlp-display-entity-type\">Symptom</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> ; significantly , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #ffacb7\"><span class=\"spark-nlp-display-entity-name\">her </span><span class=\"spark-nlp-display-entity-type\">Gender</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #0E8F8B\"><span class=\"spark-nlp-display-entity-name\">abdominal </span><span class=\"spark-nlp-display-entity-type\">External_body_part_or_region</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> examination was benign with no </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #C1279C\"><span class=\"spark-nlp-display-entity-name\">tenderness </span><span class=\"spark-nlp-display-entity-type\">Symptom</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #C1279C\"><span class=\"spark-nlp-display-entity-name\">guarding </span><span class=\"spark-nlp-display-entity-type\">Symptom</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, or </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #C1279C\"><span class=\"spark-nlp-display-entity-name\">rigidity </span><span class=\"spark-nlp-display-entity-type\">Symptom</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">. Pertinent laboratory findings on </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #A46E8B\"><span class=\"spark-nlp-display-entity-name\">admission </span><span class=\"spark-nlp-display-entity-type\">Admission_Discharge</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> were: </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">serum glucose </span><span class=\"spark-nlp-display-entity-type\">Test</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #833B52\"><span class=\"spark-nlp-display-entity-name\">111 mg/dl </span><span class=\"spark-nlp-display-entity-type\">Test_Result</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">,  </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">creatinine </span><span class=\"spark-nlp-display-entity-type\">Test</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #833B52\"><span class=\"spark-nlp-display-entity-name\">0.4 mg/dL </span><span class=\"spark-nlp-display-entity-type\">Test_Result</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #5CB17E\"><span class=\"spark-nlp-display-entity-name\">triglycerides </span><span class=\"spark-nlp-display-entity-type\">Triglycerides</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #833B52\"><span class=\"spark-nlp-display-entity-name\">508 mg/dL </span><span class=\"spark-nlp-display-entity-type\">Test_Result</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #7B1A73\"><span class=\"spark-nlp-display-entity-name\">total cholesterol 122 mg/dL </span><span class=\"spark-nlp-display-entity-type\">Total_Cholesterol</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">venous pH </span><span class=\"spark-nlp-display-entity-type\">Test</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #833B52\"><span class=\"spark-nlp-display-entity-name\">7.27 </span><span class=\"spark-nlp-display-entity-type\">Test_Result</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">.<br></span></div>"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"0c676be8-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"0c621616-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_c9ec185208"]},"metadata":{}}],"source":["from google.colab import widgets\n","\n","t = widgets.TabBar([\"ner_clinical_large\", \"ner_jsl\", \"viz_clinical\", \"viz_jsl\"])\n","\n","with t.output_to(0):\n","    display(df_clinical)\n","\n","with t.output_to(1):\n","    display(jsl_df)\n","\n","with t.output_to(2):\n","    visualiser.display(light_result[0], label_col='ner_chunk', document_col='document')\n","\n","with t.output_to(3):\n","    visualiser.display(jsl_light_result[0], label_col='jsl_ner_chunk', document_col='document')"]},{"cell_type":"markdown","metadata":{"id":"NZ_3PK8iyPIV"},"source":["## Posology NER\n","**Entities**\n","\n","- DOSAGE\n","- DRUG\n","- DURATION\n","- FORM\n","- FREQUENCY\n","- ROUTE\n","- STRENGTH"]},{"cell_type":"code","execution_count":39,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":22722,"status":"ok","timestamp":1712935189683,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"qDJDk1o41jfH","outputId":"19fae6a6-d766-4df4-b25c-e4db6471746d"},"outputs":[{"output_type":"stream","name":"stdout","text":["ner_posology download started this may take some time.\n","[OK!]\n","ner_posology_greedy download started this may take some time.\n","[OK!]\n"]}],"source":["# NER model trained on i2b2 (sampled from MIMIC) dataset\n","posology_ner = medical.NerModel.pretrained(\"ner_posology\", \"en\", \"clinical/models\") \\\n","    .setInputCols([\"sentence\", \"token\", \"embeddings\"]) \\\n","    .setOutputCol(\"ner\")\n","\n","posology_ner_converter = medical.NerConverterInternal()\\\n","    .setInputCols([\"sentence\",\"token\",\"ner\"])\\\n","    .setOutputCol(\"ner_chunk\")\n","\n","# greedy model\n","posology_ner_greedy = medical.NerModel.pretrained(\"ner_posology_greedy\", \"en\", \"clinical/models\") \\\n","    .setInputCols([\"sentence\", \"token\", \"embeddings\"]) \\\n","    .setOutputCol(\"ner_greedy\")\n","\n","ner_converter_greedy = medical.NerConverterInternal()\\\n","    .setInputCols([\"sentence\",\"token\",\"ner_greedy\"])\\\n","    .setOutputCol(\"ner_chunk_greedy\")\n","\n","nlpPipeline = nlp.Pipeline(stages=[\n","    documentAssembler,\n","    sentenceDetector,\n","    tokenizer,\n","    word_embeddings,\n","    posology_ner,\n","    posology_ner_converter,\n","    posology_ner_greedy,\n","    ner_converter_greedy])\n","\n","empty_data = spark.createDataFrame([[\"\"]]).toDF(\"text\")\n","\n","posology_model = nlpPipeline.fit(empty_data)\n"]},{"cell_type":"code","execution_count":40,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":8,"status":"ok","timestamp":1712935189683,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"tGlKLjiJ1pk3","outputId":"97cf837e-691c-4000-f785-3b659b24511e"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["['O',\n"," 'B-DOSAGE',\n"," 'B-STRENGTH',\n"," 'I-STRENGTH',\n"," 'B-ROUTE',\n"," 'B-FREQUENCY',\n"," 'I-FREQUENCY',\n"," 'B-DRUG',\n"," 'I-DRUG',\n"," 'B-FORM',\n"," 'I-DOSAGE',\n"," 'B-DURATION',\n"," 'I-DURATION',\n"," 'I-FORM',\n"," 'I-ROUTE']"]},"metadata":{},"execution_count":40}],"source":["posology_ner.getClasses()"]},{"cell_type":"code","execution_count":41,"metadata":{"id":"dLi2DHEpyepZ","executionInfo":{"status":"ok","timestamp":1712935189684,"user_tz":-180,"elapsed":4,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["posology_result = posology_model.transform(mt_samples_df).cache()"]},{"cell_type":"code","execution_count":42,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":14557,"status":"ok","timestamp":1712935204237,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"OjeG2WLly_ym","outputId":"b311f465-9b2c-4f57-d89e-0e483fe3d36e"},"outputs":[{"output_type":"stream","name":"stdout","text":["+-----+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+\n","|index|                text|            document|            sentence|               token|          embeddings|                 ner|           ner_chunk|          ner_greedy|    ner_chunk_greedy|\n","+-----+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+\n","|    0|Sample Type / Med...|[{document, 0, 54...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 1609, 16...|[{named_entity, 0...|[{chunk, 1609, 16...|\n","|    1|Sample Type / Med...|[{document, 0, 32...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 742, 750...|[{named_entity, 0...|[{chunk, 742, 753...|\n","|    2|Sample Type / Med...|[{document, 0, 42...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 230, 240...|[{named_entity, 0...|[{chunk, 369, 385...|\n","|    3|Sample Type / Med...|[{document, 0, 20...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|                  []|[{named_entity, 0...|                  []|\n","|    4|Sample Type / Med...|[{document, 0, 34...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 1102, 11...|[{named_entity, 0...|[{chunk, 1102, 11...|\n","|    5|Sample Type / Med...|[{document, 0, 15...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|                  []|[{named_entity, 0...|[{chunk, 68, 84, ...|\n","|    6|Sample Type / Med...|[{document, 0, 25...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 1242, 12...|[{named_entity, 0...|[{chunk, 389, 398...|\n","|    7|Sample Type / Med...|[{document, 0, 93...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 667, 677...|[{named_entity, 0...|[{chunk, 667, 677...|\n","|    8|Sample Type / Med...|[{document, 0, 20...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 495, 505...|[{named_entity, 0...|[{chunk, 495, 513...|\n","|    9|Sample Type / Med...|[{document, 0, 19...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 524, 526...|[{named_entity, 0...|[{chunk, 1234, 12...|\n","+-----+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+\n","\n"]}],"source":["posology_result.show(10)"]},{"cell_type":"code","execution_count":43,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1337,"status":"ok","timestamp":1712935205572,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"pL_TMXSv1paw","outputId":"4067a022-69c7-4fd1-fdcb-d1f5ecda1996"},"outputs":[{"output_type":"stream","name":"stdout","text":["+-----+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+---+\n","|index|                text|            document|            sentence|               token|          embeddings|                 ner|           ner_chunk|          ner_greedy|    ner_chunk_greedy| id|\n","+-----+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+---+\n","|    0|Sample Type / Med...|[{document, 0, 54...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 1609, 16...|[{named_entity, 0...|[{chunk, 1609, 16...|  0|\n","|    1|Sample Type / Med...|[{document, 0, 32...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 742, 750...|[{named_entity, 0...|[{chunk, 742, 753...|  1|\n","|    2|Sample Type / Med...|[{document, 0, 42...|[{document, 0, 53...|[{token, 0, 5, Sa...|[{word_embeddings...|[{named_entity, 0...|[{chunk, 230, 240...|[{named_entity, 0...|[{chunk, 369, 385...|  2|\n","+-----+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+---+\n","only showing top 3 rows\n","\n"]}],"source":["# This will return a new DF with all the columns + id\n","posology_result = posology_result.withColumn(\"id\", F.monotonically_increasing_id())\n","\n","posology_result.show(3)"]},{"cell_type":"code","execution_count":44,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":5,"status":"ok","timestamp":1712935205572,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"YK0prud8UoDh","outputId":"f7e04106-559c-40ea-d1e9-157c493a14d1"},"outputs":[{"output_type":"stream","name":"stdout","text":["+----------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+\n","|                                                                                              result|                                                                                              result|\n","+----------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Discharge, Sum...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, ...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, BCCa, Excision...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, ...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Anemia, -, Con...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, ...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Intensity-Modu...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, ...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Neck, Dissecti...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, ...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, HDR, Brachythe...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, ...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Biopsy, -, Cer...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, ...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Metastatic, Ov...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, ...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Breast, Radiat...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, ...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Name, :, Lymph, Node, E...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, ...|\n","+----------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+\n","\n"]}],"source":["posology_result.select('token.result','ner.result').show(truncate=100)"]},{"cell_type":"code","execution_count":45,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":4,"status":"ok","timestamp":1712935205572,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"DHV5kfhOClV8","outputId":"3d468e3e-8766-41f0-d75c-bda04cc5c7ea"},"outputs":[{"output_type":"stream","name":"stdout","text":["+--------------------------------------------------------------------------------+--------------------------------------------------------------------------------+\n","|                                                                          result|                                                                          result|\n","+--------------------------------------------------------------------------------+--------------------------------------------------------------------------------+\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Nam...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Nam...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Nam...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Nam...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O...|\n","|[Sample, Type, /, Medical, Specialty, :, Hematology, -, Oncology, Sample, Nam...|[O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O, O...|\n","+--------------------------------------------------------------------------------+--------------------------------------------------------------------------------+\n","only showing top 5 rows\n","\n"]}],"source":["posology_result.select('token.result','ner.result').show(5, truncate=80)"]},{"cell_type":"code","execution_count":46,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":4,"status":"ok","timestamp":1712935205573,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"3aEoR8Y5y8kE","outputId":"7c3fbe42-d197-4bd8-e559-2d9ab924c85e"},"outputs":[{"output_type":"stream","name":"stdout","text":["+--------------+-----------+----------+\n","|         token|  ner_label|confidence|\n","+--------------+-----------+----------+\n","|      Coumadin|     B-DRUG|    0.9989|\n","|             1| B-STRENGTH|    0.8618|\n","|            mg| I-STRENGTH|    0.5132|\n","|         daily|B-FREQUENCY|    0.9999|\n","|    Amiodarone|     B-DRUG|    0.9993|\n","|           100| B-STRENGTH|    0.8538|\n","|            mg| I-STRENGTH|    0.5216|\n","|           p.o|    B-ROUTE|    0.9992|\n","|         daily|B-FREQUENCY|    0.9928|\n","|      Coumadin|     B-DRUG|    0.9915|\n","|       Lovenox|     B-DRUG|    0.8314|\n","|            40| B-STRENGTH|    0.5199|\n","|            mg| I-STRENGTH|    0.5121|\n","|subcutaneously|    B-ROUTE|    0.9887|\n","|  chemotherapy|     B-DRUG|    0.9984|\n","|     cisplatin|     B-DRUG|    0.9257|\n","|            75| B-STRENGTH|    0.9574|\n","| mg/centimeter| I-STRENGTH|    0.5491|\n","|           109| B-STRENGTH|    0.5699|\n","|            mg| I-STRENGTH|    0.6355|\n","+--------------+-----------+----------+\n","only showing top 20 rows\n","\n"]}],"source":["from pyspark.sql import functions as F\n","\n","posology_result.select(F.explode(F.arrays_zip(posology_result.token.result,\n","                                              posology_result.ner.result,\n","                                              posology_result.ner.metadata)).alias(\"cols\")) \\\n","               .select(F.expr(\"cols['0']\").alias(\"token\"),\n","                       F.expr(\"cols['1']\").alias(\"ner_label\"),\n","                       F.expr(\"cols['2']['confidence']\").alias(\"confidence\"))\\\n","               .filter(\"ner_label!='O'\")\\\n","               .show(20, truncate=100)"]},{"cell_type":"code","execution_count":47,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":639,"status":"ok","timestamp":1712935206209,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"T3pH3xulzHfr","outputId":"fcec0484-af87-4aa8-cef1-6bebaaeb3792"},"outputs":[{"output_type":"stream","name":"stdout","text":["+---+-----------+----------------+-----+----+---------+----------+\n","|id |sentence_id|chunk           |begin|end |ner_label|confidence|\n","+---+-----------+----------------+-----+----+---------+----------+\n","|0  |33         |Coumadin        |1609 |1616|DRUG     |0.9989    |\n","|0  |33         |1 mg            |1618 |1621|STRENGTH |0.6875    |\n","|0  |33         |daily           |1623 |1627|FREQUENCY|0.9999    |\n","|0  |34         |Amiodarone      |1696 |1705|DRUG     |0.9993    |\n","|0  |34         |100 mg          |1707 |1712|STRENGTH |0.68770003|\n","|0  |34         |p.o             |1714 |1716|ROUTE    |0.9992    |\n","|0  |34         |daily           |1719 |1723|FREQUENCY|0.9928    |\n","|0  |58         |Coumadin        |2770 |2777|DRUG     |0.9915    |\n","|0  |60         |Lovenox         |2880 |2886|DRUG     |0.8314    |\n","|0  |60         |40 mg           |2888 |2892|STRENGTH |0.51600003|\n","|0  |60         |subcutaneously  |2894 |2907|ROUTE    |0.9887    |\n","|0  |72         |chemotherapy    |4436 |4447|DRUG     |0.9984    |\n","|0  |72         |cisplatin       |4475 |4483|DRUG     |0.9257    |\n","|0  |72         |75 mg/centimeter|4485 |4500|STRENGTH |0.75325   |\n","|0  |72         |109 mg          |4519 |4524|STRENGTH |0.6027    |\n","|0  |72         |IV              |4526 |4527|ROUTE    |0.9993    |\n","|0  |72         |piggyback       |4529 |4537|DRUG     |0.578     |\n","|0  |72         |over 2 hours    |4539 |4550|DURATION |0.7061    |\n","|0  |72         |September       |4555 |4563|DRUG     |0.9277    |\n","|0  |72         |Alimta          |4574 |4579|DRUG     |0.8845    |\n","+---+-----------+----------------+-----+----+---------+----------+\n","only showing top 20 rows\n","\n"]}],"source":["posology_result.select('id',F.explode(F.arrays_zip(posology_result.ner_chunk.result,\n","                                                   posology_result.ner_chunk.begin,\n","                                                   posology_result.ner_chunk.end,\n","                                                   posology_result.ner_chunk.metadata)).alias(\"cols\")) \\\n","               .select('id', F.expr(\"cols['3']['sentence']\").alias(\"sentence_id\"),\n","                             F.expr(\"cols['0']\").alias(\"chunk\"),\n","                             F.expr(\"cols['1']\").alias(\"begin\"),\n","                             F.expr(\"cols['2']\").alias(\"end\"),\n","                             F.expr(\"cols['3']['entity']\").alias(\"ner_label\"),\n","                             F.expr(\"cols['3']['confidence']\").alias(\"confidence\"))\\\n","                    .filter(\"ner_label!='O'\")\\\n","                    .show(truncate=False)"]},{"cell_type":"code","execution_count":48,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":3,"status":"ok","timestamp":1712935206209,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"rMtx9HaN61Oy","outputId":"8e17f1b5-cfbe-4112-d020-ff9d293d4c1a"},"outputs":[{"output_type":"stream","name":"stdout","text":["+---+-----------+--------------------------------------------------------------+-----+----+---------+----------+\n","|id |sentence_id|chunk                                                         |begin|end |ner_label|confidence|\n","+---+-----------+--------------------------------------------------------------+-----+----+---------+----------+\n","|0  |33         |Coumadin 1 mg                                                 |1609 |1621|DRUG     |0.6126    |\n","|0  |33         |daily                                                         |1623 |1627|FREQUENCY|0.9962    |\n","|0  |34         |Amiodarone 100 mg p.o                                         |1696 |1716|DRUG     |0.64115   |\n","|0  |34         |daily                                                         |1719 |1723|FREQUENCY|0.9035    |\n","|0  |58         |Coumadin                                                      |2770 |2777|DRUG     |0.99      |\n","|0  |72         |chemotherapy                                                  |4436 |4447|DRUG     |0.9973    |\n","|0  |72         |cisplatin 75 mg/centimeter                                    |4475 |4500|DRUG     |0.6081667 |\n","|0  |72         |109 mg IV piggyback                                           |4519 |4537|DRUG     |0.57665   |\n","|0  |72         |over 2 hours                                                  |4539 |4550|DURATION |0.5085333 |\n","|0  |72         |Alimta 500 mg/ centimeter squared equaling 730 mg IV piggyback|4574 |4635|DRUG     |0.58996   |\n","|0  |74         |chemotherapy                                                  |4713 |4724|DRUG     |0.9905    |\n","|0  |74         |IV fluid and IV                                               |4814 |4828|DRUG     |0.7181    |\n","|0  |75         |heparin                                                       |4863 |4869|DRUG     |0.6508    |\n","|0  |76         |Zofran                                                        |4917 |4922|DRUG     |0.9919    |\n","|0  |76         |Phenergan                                                     |4925 |4933|DRUG     |0.9978    |\n","|0  |76         |Coumadin                                                      |4936 |4943|DRUG     |0.9989    |\n","|0  |76         |Lovenox                                                       |4950 |4956|DRUG     |0.9775    |\n","|0  |76         |Vicodin                                                       |4963 |4969|DRUG     |0.9946    |\n","|1  |11         |Xylocaine 2%                                                  |742  |753 |DRUG     |0.8614    |\n","|1  |11         |epinephrine was injected                                      |760  |783 |DRUG     |0.8004667 |\n","+---+-----------+--------------------------------------------------------------+-----+----+---------+----------+\n","only showing top 20 rows\n","\n"]}],"source":["posology_result.select('id',F.explode(F.arrays_zip(posology_result.ner_chunk_greedy.result,\n","                                                   posology_result.ner_chunk_greedy.begin,\n","                                                   posology_result.ner_chunk_greedy.end,\n","                                                   posology_result.ner_chunk_greedy.metadata)).alias(\"cols\")) \\\n","               .select('id', F.expr(\"cols['3']['sentence']\").alias(\"sentence_id\"),\n","                             F.expr(\"cols['0']\").alias(\"chunk\"),\n","                             F.expr(\"cols['1']\").alias(\"begin\"),\n","                             F.expr(\"cols['2']\").alias(\"end\"),\n","                             F.expr(\"cols['3']['entity']\").alias(\"ner_label\"),\n","                             F.expr(\"cols['3']['confidence']\").alias(\"confidence\"))\\\n","                    .filter(\"ner_label!='O'\")\\\n","                    .show(truncate=False)"]},{"cell_type":"code","execution_count":49,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":36},"executionInfo":{"elapsed":723,"status":"ok","timestamp":1712935206930,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"CATF6C96D4Fy","outputId":"a9de5475-146a-4a66-cab4-c276833867f4"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["'Xylocaine'"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"string"}},"metadata":{},"execution_count":49}],"source":["posology_result.select('ner_chunk').take(2)[1][0][0].result"]},{"cell_type":"code","execution_count":50,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":9,"status":"ok","timestamp":1712935206931,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"ZX3_nnmKEOam","outputId":"fdd11cce-02d0-4bd9-b126-43a341c8242e"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'sentence': '11',\n"," 'chunk': '0',\n"," 'ner_source': 'ner_chunk',\n"," 'entity': 'DRUG',\n"," 'confidence': '0.9906'}"]},"metadata":{},"execution_count":50}],"source":["posology_result.select('ner_chunk').take(2)[1][0][0].metadata"]},{"cell_type":"code","execution_count":51,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":7,"status":"ok","timestamp":1712935206931,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"iZEj6SNg62Im","outputId":"20e21e15-4e84-43e7-ad91-7486b4668185"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["[('The', 'O'),\n"," ('patient', 'O'),\n"," ('was', 'O'),\n"," ('prescribed', 'O'),\n"," ('1', 'B-DOSAGE'),\n"," ('capsule', 'B-FORM'),\n"," ('of', 'O'),\n"," ('Advil', 'B-DRUG'),\n"," ('for', 'B-DURATION'),\n"," ('5', 'I-DURATION'),\n"," ('days', 'I-DURATION'),\n"," ('.', 'O'),\n"," ('He', 'O'),\n"," ('was', 'O'),\n"," ('seen', 'O'),\n"," ('by', 'O'),\n"," ('the', 'O'),\n"," ('endocrinology', 'O'),\n"," ('service', 'O'),\n"," ('and', 'O'),\n"," ('she', 'O'),\n"," ('was', 'O'),\n"," ('discharged', 'O'),\n"," ('on', 'O'),\n"," ('40', 'B-DOSAGE'),\n"," ('units', 'I-DOSAGE'),\n"," ('of', 'O'),\n"," ('insulin', 'B-DRUG'),\n"," ('glargine', 'I-DRUG'),\n"," ('at', 'B-FREQUENCY'),\n"," ('night', 'I-FREQUENCY'),\n"," (',', 'O'),\n"," ('12', 'B-DOSAGE'),\n"," ('units', 'I-DOSAGE'),\n"," ('of', 'O'),\n"," ('insulin', 'B-DRUG'),\n"," ('lispro', 'I-DRUG'),\n"," ('with', 'B-FREQUENCY'),\n"," ('meals', 'I-FREQUENCY'),\n"," (',', 'O'),\n"," ('and', 'O'),\n"," ('metformin', 'B-DRUG'),\n"," ('1000', 'B-STRENGTH'),\n"," ('mg', 'I-STRENGTH'),\n"," ('two', 'B-FREQUENCY'),\n"," ('times', 'I-FREQUENCY'),\n"," ('a', 'I-FREQUENCY'),\n"," ('day', 'I-FREQUENCY'),\n"," ('.', 'O'),\n"," ('It', 'O'),\n"," ('was', 'O'),\n"," ('determined', 'O'),\n"," ('that', 'O'),\n"," ('all', 'O'),\n"," ('SGLT2', 'B-DRUG'),\n"," ('inhibitors', 'I-DRUG'),\n"," ('should', 'O'),\n"," ('be', 'O'),\n"," ('discontinued', 'O'),\n"," ('indefinitely', 'O'),\n"," ('for', 'B-DURATION'),\n"," ('3', 'I-DURATION'),\n"," ('months', 'I-DURATION'),\n"," ('.', 'O')]"]},"metadata":{},"execution_count":51}],"source":["posology_light_model = nlp.LightPipeline(posology_model)\n","\n","text ='The patient was prescribed 1 capsule of Advil for 5 days . He was seen by the endocrinology service and she was discharged on 40 units of insulin glargine at night , 12 units of insulin lispro with meals , and metformin 1000 mg two times a day . It was determined that all SGLT2 inhibitors should be discontinued indefinitely for 3 months .'\n","\n","posology_light_result = posology_light_model.annotate(text)\n","\n","list(zip(posology_light_result['token'], posology_light_result['ner']))"]},{"cell_type":"code","execution_count":52,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":6,"status":"ok","timestamp":1712935206932,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"UwGbpOlEYQIQ","outputId":"5f2393d8-9073-4393-9198-0c31413fcc9b"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["[('The', 'O'),\n"," ('patient', 'O'),\n"," ('was', 'O'),\n"," ('prescribed', 'O'),\n"," ('1', 'B-DRUG'),\n"," ('capsule', 'I-DRUG'),\n"," ('of', 'I-DRUG'),\n"," ('Advil', 'I-DRUG'),\n"," ('for', 'B-DURATION'),\n"," ('5', 'I-DURATION'),\n"," ('days', 'I-DURATION'),\n"," ('.', 'O'),\n"," ('He', 'O'),\n"," ('was', 'O'),\n"," ('seen', 'O'),\n"," ('by', 'O'),\n"," ('the', 'O'),\n"," ('endocrinology', 'O'),\n"," ('service', 'O'),\n"," ('and', 'O'),\n"," ('she', 'O'),\n"," ('was', 'O'),\n"," ('discharged', 'O'),\n"," ('on', 'O'),\n"," ('40', 'B-DRUG'),\n"," ('units', 'I-DRUG'),\n"," ('of', 'I-DRUG'),\n"," ('insulin', 'I-DRUG'),\n"," ('glargine', 'I-DRUG'),\n"," ('at', 'B-FREQUENCY'),\n"," ('night', 'I-FREQUENCY'),\n"," (',', 'O'),\n"," ('12', 'B-DRUG'),\n"," ('units', 'I-DRUG'),\n"," ('of', 'I-DRUG'),\n"," ('insulin', 'I-DRUG'),\n"," ('lispro', 'I-DRUG'),\n"," ('with', 'B-FREQUENCY'),\n"," ('meals', 'I-FREQUENCY'),\n"," (',', 'O'),\n"," ('and', 'O'),\n"," ('metformin', 'B-DRUG'),\n"," ('1000', 'I-DRUG'),\n"," ('mg', 'I-DRUG'),\n"," ('two', 'B-FREQUENCY'),\n"," ('times', 'I-FREQUENCY'),\n"," ('a', 'I-FREQUENCY'),\n"," ('day', 'I-FREQUENCY'),\n"," ('.', 'O'),\n"," ('It', 'O'),\n"," ('was', 'O'),\n"," ('determined', 'O'),\n"," ('that', 'O'),\n"," ('all', 'O'),\n"," ('SGLT2', 'B-DRUG'),\n"," ('inhibitors', 'I-DRUG'),\n"," ('should', 'O'),\n"," ('be', 'O'),\n"," ('discontinued', 'O'),\n"," ('indefinitely', 'O'),\n"," ('for', 'B-DURATION'),\n"," ('3', 'I-DURATION'),\n"," ('months', 'I-DURATION'),\n"," ('.', 'O')]"]},"metadata":{},"execution_count":52}],"source":["list(zip(posology_light_result['token'], posology_light_result['ner_greedy']))"]},{"cell_type":"code","execution_count":53,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":554},"executionInfo":{"elapsed":1736,"status":"ok","timestamp":1712935208663,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"rPcavWpP-Fu9","outputId":"2bfd3096-fc6c-4374-e33d-0a2b6c710667"},"outputs":[{"output_type":"stream","name":"stdout","text":["Warning::Spark Session already created, some configs may not take.\n","Warning::Spark Session already created, some configs may not take.\n"]},{"output_type":"execute_result","data":{"text/plain":["  entities_ner_chunk entities_ner_chunk_class entities_ner_chunk_begin  \\\n","0                  1                   DOSAGE                       27   \n","0            capsule                     FORM                       29   \n","0              Advil                     DRUG                       40   \n","0         for 5 days                 DURATION                       46   \n","0           40 units                   DOSAGE                      126   \n","0   insulin glargine                     DRUG                      138   \n","0           at night                FREQUENCY                      155   \n","0           12 units                   DOSAGE                      166   \n","0     insulin lispro                     DRUG                      178   \n","0         with meals                FREQUENCY                      193   \n","0          metformin                     DRUG                      210   \n","0            1000 mg                 STRENGTH                      220   \n","0    two times a day                FREQUENCY                      228   \n","0   SGLT2 inhibitors                     DRUG                      273   \n","0       for 3 months                 DURATION                      326   \n","\n","  entities_ner_chunk_end  \n","0                     27  \n","0                     35  \n","0                     44  \n","0                     55  \n","0                    133  \n","0                    153  \n","0                    162  \n","0                    173  \n","0                    191  \n","0                    202  \n","0                    218  \n","0                    226  \n","0                    242  \n","0                    288  \n","0                    337  "],"text/html":["\n","  <div id=\"df-45e9bcea-3357-43c9-8b24-af98486c5fee\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>entities_ner_chunk</th>\n","      <th>entities_ner_chunk_class</th>\n","      <th>entities_ner_chunk_begin</th>\n","      <th>entities_ner_chunk_end</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>DOSAGE</td>\n","      <td>27</td>\n","      <td>27</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>capsule</td>\n","      <td>FORM</td>\n","      <td>29</td>\n","      <td>35</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>Advil</td>\n","      <td>DRUG</td>\n","      <td>40</td>\n","      <td>44</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>for 5 days</td>\n","      <td>DURATION</td>\n","      <td>46</td>\n","      <td>55</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>40 units</td>\n","      <td>DOSAGE</td>\n","      <td>126</td>\n","      <td>133</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>insulin glargine</td>\n","      <td>DRUG</td>\n","      <td>138</td>\n","      <td>153</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>at night</td>\n","      <td>FREQUENCY</td>\n","      <td>155</td>\n","      <td>162</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>12 units</td>\n","      <td>DOSAGE</td>\n","      <td>166</td>\n","      <td>173</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>insulin lispro</td>\n","      <td>DRUG</td>\n","      <td>178</td>\n","      <td>191</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>with meals</td>\n","      <td>FREQUENCY</td>\n","      <td>193</td>\n","      <td>202</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>metformin</td>\n","      <td>DRUG</td>\n","      <td>210</td>\n","      <td>218</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>1000 mg</td>\n","      <td>STRENGTH</td>\n","      <td>220</td>\n","      <td>226</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>two times a day</td>\n","      <td>FREQUENCY</td>\n","      <td>228</td>\n","      <td>242</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>SGLT2 inhibitors</td>\n","      <td>DRUG</td>\n","      <td>273</td>\n","      <td>288</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>for 3 months</td>\n","      <td>DURATION</td>\n","      <td>326</td>\n","      <td>337</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-45e9bcea-3357-43c9-8b24-af98486c5fee')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-45e9bcea-3357-43c9-8b24-af98486c5fee button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-45e9bcea-3357-43c9-8b24-af98486c5fee');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-e037377e-467d-4009-af83-13bb20602c99\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e037377e-467d-4009-af83-13bb20602c99')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-e037377e-467d-4009-af83-13bb20602c99 button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"posology_result_df","summary":"{\n  \"name\": \"posology_result_df\",\n  \"rows\": 15,\n  \"fields\": [\n    {\n      \"column\": \"entities_ner_chunk\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 15,\n        \"samples\": [\n          \"with meals\",\n          \"1000 mg\",\n          \"1\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_class\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 6,\n        \"samples\": [\n          \"DOSAGE\",\n          \"FORM\",\n          \"STRENGTH\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_begin\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 27,\n        \"max\": 326,\n        \"num_unique_values\": 15,\n        \"samples\": [\n          193,\n          220,\n          27\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_end\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 27,\n        \"max\": 337,\n        \"num_unique_values\": 15,\n        \"samples\": [\n          202,\n          226,\n          27\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":53}],"source":["cols = [\n","        'entities_ner_chunk',\n","        'entities_ner_chunk_class',\n","        'entities_ner_chunk_begin',\n","        'entities_ner_chunk_end',\n","       ]\n","\n","posology_result_df = nlp.nlu.to_pretty_df(posology_light_model, text, positions=True, output_level='chunk')[cols]\n","posology_result_df.head(20)"]},{"cell_type":"code","execution_count":54,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":417},"executionInfo":{"elapsed":1106,"status":"ok","timestamp":1712935209765,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"EozCGm7gq4vF","outputId":"cafb0d7c-4b72-4fc1-fcc6-7af7e9bf8983"},"outputs":[{"output_type":"stream","name":"stdout","text":["Warning::Spark Session already created, some configs may not take.\n","Warning::Spark Session already created, some configs may not take.\n"]},{"output_type":"execute_result","data":{"text/plain":["      entities_ner_chunk_greedy entities_ner_chunk_greedy_class  \\\n","0            1 capsule of Advil                            DRUG   \n","0                    for 5 days                        DURATION   \n","0  40 units of insulin glargine                            DRUG   \n","0                      at night                       FREQUENCY   \n","0    12 units of insulin lispro                            DRUG   \n","0                    with meals                       FREQUENCY   \n","0             metformin 1000 mg                            DRUG   \n","0               two times a day                       FREQUENCY   \n","0              SGLT2 inhibitors                            DRUG   \n","0                  for 3 months                        DURATION   \n","\n","  entities_ner_chunk_greedy_begin entities_ner_chunk_greedy_end  \n","0                              27                            44  \n","0                              46                            55  \n","0                             126                           153  \n","0                             155                           162  \n","0                             166                           191  \n","0                             193                           202  \n","0                             210                           226  \n","0                             228                           242  \n","0                             273                           288  \n","0                             326                           337  "],"text/html":["\n","  <div id=\"df-dd26fb15-cf17-4ad2-9fa5-80e592195a9a\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>entities_ner_chunk_greedy</th>\n","      <th>entities_ner_chunk_greedy_class</th>\n","      <th>entities_ner_chunk_greedy_begin</th>\n","      <th>entities_ner_chunk_greedy_end</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>1 capsule of Advil</td>\n","      <td>DRUG</td>\n","      <td>27</td>\n","      <td>44</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>for 5 days</td>\n","      <td>DURATION</td>\n","      <td>46</td>\n","      <td>55</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>40 units of insulin glargine</td>\n","      <td>DRUG</td>\n","      <td>126</td>\n","      <td>153</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>at night</td>\n","      <td>FREQUENCY</td>\n","      <td>155</td>\n","      <td>162</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>12 units of insulin lispro</td>\n","      <td>DRUG</td>\n","      <td>166</td>\n","      <td>191</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>with meals</td>\n","      <td>FREQUENCY</td>\n","      <td>193</td>\n","      <td>202</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>metformin 1000 mg</td>\n","      <td>DRUG</td>\n","      <td>210</td>\n","      <td>226</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>two times a day</td>\n","      <td>FREQUENCY</td>\n","      <td>228</td>\n","      <td>242</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>SGLT2 inhibitors</td>\n","      <td>DRUG</td>\n","      <td>273</td>\n","      <td>288</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>for 3 months</td>\n","      <td>DURATION</td>\n","      <td>326</td>\n","      <td>337</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-dd26fb15-cf17-4ad2-9fa5-80e592195a9a')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-dd26fb15-cf17-4ad2-9fa5-80e592195a9a button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-dd26fb15-cf17-4ad2-9fa5-80e592195a9a');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-3aa3b584-717f-432a-9a9b-6ede99ec3369\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-3aa3b584-717f-432a-9a9b-6ede99ec3369')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-3aa3b584-717f-432a-9a9b-6ede99ec3369 button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"posology_result_greedy_df","summary":"{\n  \"name\": \"posology_result_greedy_df\",\n  \"rows\": 15,\n  \"fields\": [\n    {\n      \"column\": \"entities_ner_chunk_greedy\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 10,\n        \"samples\": [\n          \"SGLT2 inhibitors\",\n          \"for 5 days\",\n          \"with meals\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_greedy_class\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"DRUG\",\n          \"DURATION\",\n          \"FREQUENCY\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_greedy_begin\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 27,\n        \"max\": 326,\n        \"num_unique_values\": 10,\n        \"samples\": [\n          273,\n          46,\n          193\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_greedy_end\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 44,\n        \"max\": 337,\n        \"num_unique_values\": 10,\n        \"samples\": [\n          288,\n          55,\n          202\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":54}],"source":["cols = [\n","        'entities_ner_chunk_greedy',\n","        'entities_ner_chunk_greedy_class',\n","        'entities_ner_chunk_greedy_begin',\n","        'entities_ner_chunk_greedy_end',\n","       ]\n","\n","posology_result_greedy_df = nlp.nlu.to_pretty_df(posology_light_model, text, positions=True, output_level='chunk')[cols]\n","posology_result_greedy_df.head(10)"]},{"cell_type":"markdown","metadata":{"id":"OVzH2UP2zkc5"},"source":["### Comparison of `ner_posology` and `ner_posology_greedy` results"]},{"cell_type":"code","execution_count":55,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":341,"resources":{"http://localhost:8080/nbextensions/google.colab/tabbar.css":{"data":"Lmdvb2ctdGFie3Bvc2l0aW9uOnJlbGF0aXZlO3BhZGRpbmc6NHB4IDhweDtjb2xvcjojMDBjO3RleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7Y3Vyc29yOmRlZmF1bHR9Lmdvb2ctdGFiLWJhci10b3AgLmdvb2ctdGFie21hcmdpbjoxcHggNHB4IDAgMDtib3JkZXItYm90dG9tOjA7ZmxvYXQ6bGVmdH0uZ29vZy10YWItYmFyLXRvcDphZnRlciwuZ29vZy10YWItYmFyLWJvdHRvbTphZnRlcntjb250ZW50OiIgIjtkaXNwbGF5OmJsb2NrO2hlaWdodDowO2NsZWFyOmJvdGg7dmlzaWJpbGl0eTpoaWRkZW59Lmdvb2ctdGFiLWJhci1ib3R0b20gLmdvb2ctdGFie21hcmdpbjowIDRweCAxcHggMDtib3JkZXItdG9wOjA7ZmxvYXQ6bGVmdH0uZ29vZy10YWItYmFyLXN0YXJ0IC5nb29nLXRhYnttYXJnaW46MCAwIDRweCAxcHg7Ym9yZGVyLXJpZ2h0OjB9Lmdvb2ctdGFiLWJhci1lbmQgLmdvb2ctdGFie21hcmdpbjowIDFweCA0cHggMDtib3JkZXItbGVmdDowfS5nb29nLXRhYi1ob3ZlcntiYWNrZ3JvdW5kOiNlZWV9Lmdvb2ctdGFiLWRpc2FibGVke2NvbG9yOiM2NjZ9Lmdvb2ctdGFiLXNlbGVjdGVke2NvbG9yOiMwMDA7YmFja2dyb3VuZDojZmZmO3RleHQtZGVjb3JhdGlvbjpub25lO2ZvbnQtd2VpZ2h0OmJvbGQ7Ym9yZGVyOjFweCBzb2xpZCAjNmI5MGRhfS5nb29nLXRhYi1iYXItdG9we3BhZGRpbmctdG9wOjVweCFpbXBvcnRhbnQ7cGFkZGluZy1sZWZ0OjVweCFpbXBvcnRhbnQ7Ym9yZGVyLWJvdHRvbToxcHggc29saWQgIzZiOTBkYSFpbXBvcnRhbnR9Lmdvb2ctdGFiLWJhci10b3AgLmdvb2ctdGFiLXNlbGVjdGVke3RvcDoxcHg7bWFyZ2luLXRvcDowO3BhZGRpbmctYm90dG9tOjVweH0uZ29vZy10YWItYmFyLWJvdHRvbSAuZ29vZy10YWItc2VsZWN0ZWR7dG9wOi0xcHg7bWFyZ2luLWJvdHRvbTowO3BhZGRpbmctdG9wOjVweH0uZ29vZy10YWItYmFyLXN0YXJ0IC5nb29nLXRhYi1zZWxlY3RlZHtsZWZ0OjFweDttYXJnaW4tbGVmdDowO3BhZGRpbmctcmlnaHQ6OXB4fS5nb29nLXRhYi1iYXItZW5kIC5nb29nLXRhYi1zZWxlY3RlZHtsZWZ0Oi0xcHg7bWFyZ2luLXJpZ2h0OjA7cGFkZGluZy1sZWZ0OjlweH0uZ29vZy10YWItYmFye21hcmdpbjowO2JvcmRlcjowO3BhZGRpbmc6MDtsaXN0LXN0eWxlOm5vbmU7Y3Vyc29yOmRlZmF1bHQ7b3V0bGluZTpub25lO2JhY2tncm91bmQ6I2ViZWZmOX0uZ29vZy10YWItYmFyLWNsZWFye2NsZWFyOmJvdGg7aGVpZ2h0OjA7b3ZlcmZsb3c6aGlkZGVufS5nb29nLXRhYi1iYXItc3RhcnR7ZmxvYXQ6bGVmdH0uZ29vZy10YWItYmFyLWVuZHtmbG9hdDpyaWdodH0qIGh0bWwgLmdvb2ctdGFiLWJhci1zdGFydHttYXJnaW4tcmlnaHQ6LTNweH0qIGh0bWwgLmdvb2ctdGFiLWJhci1lbmR7bWFyZ2luLWxlZnQ6LTNweH0=","headers":[["content-type","text/css"]],"ok":true,"status":200,"status_text":""},"http://localhost:8080/nbextensions/google.colab/tabbar_main.min.js":{"data":"Ly8gY2xhbmctZm9ybWF0IG9mZgooZnVuY3Rpb24oKXt2YXIgaCxhYT1mdW5jdGlvbihhKXt2YXIgYj0wO3JldHVybiBmdW5jdGlvbigpe3JldHVybiBiPGEubGVuZ3RoP3tkb25lOiExLHZhbHVlOmFbYisrXX06e2RvbmU6ITB9fX0sYmE9ZnVuY3Rpb24oYSl7dmFyIGI9InVuZGVmaW5lZCIhPXR5cGVvZiBTeW1ib2wmJlN5bWJvbC5pdGVyYXRvciYmYVtTeW1ib2wuaXRlcmF0b3JdO3JldHVybiBiP2IuY2FsbChhKTp7bmV4dDphYShhKX19LGw9dGhpc3x8c2VsZixjYT1mdW5jdGlvbigpe30sZGE9ZnVuY3Rpb24oYSl7YS5CYT12b2lkIDA7YS5SPWZ1bmN0aW9uKCl7cmV0dXJuIGEuQmE/YS5CYTphLkJhPW5ldyBhfX0sZWE9ZnVuY3Rpb24oYSl7dmFyIGI9dHlwZW9mIGE7aWYoIm9iamVjdCI9PWIpaWYoYSl7aWYoYSBpbnN0YW5jZW9mIEFycmF5KXJldHVybiJhcnJheSI7aWYoYSBpbnN0YW5jZW9mIE9iamVjdClyZXR1cm4gYjt2YXIgYz1PYmplY3QucHJvdG90eXBlLnRvU3RyaW5nLmNhbGwoYSk7aWYoIltvYmplY3QgV2luZG93XSI9PWMpcmV0dXJuIm9iamVjdCI7CmlmKCJbb2JqZWN0IEFycmF5XSI9PWN8fCJudW1iZXIiPT10eXBlb2YgYS5sZW5ndGgmJiJ1bmRlZmluZWQiIT10eXBlb2YgYS5zcGxpY2UmJiJ1bmRlZmluZWQiIT10eXBlb2YgYS5wcm9wZXJ0eUlzRW51bWVyYWJsZSYmIWEucHJvcGVydHlJc0VudW1lcmFibGUoInNwbGljZSIpKXJldHVybiJhcnJheSI7aWYoIltvYmplY3QgRnVuY3Rpb25dIj09Y3x8InVuZGVmaW5lZCIhPXR5cGVvZiBhLmNhbGwmJiJ1bmRlZmluZWQiIT10eXBlb2YgYS5wcm9wZXJ0eUlzRW51bWVyYWJsZSYmIWEucHJvcGVydHlJc0VudW1lcmFibGUoImNhbGwiKSlyZXR1cm4iZnVuY3Rpb24ifWVsc2UgcmV0dXJuIm51bGwiO2Vsc2UgaWYoImZ1bmN0aW9uIj09YiYmInVuZGVmaW5lZCI9PXR5cGVvZiBhLmNhbGwpcmV0dXJuIm9iamVjdCI7cmV0dXJuIGJ9LGZhPWZ1bmN0aW9uKGEpe3JldHVybiJhcnJheSI9PWVhKGEpfSxoYT1mdW5jdGlvbihhKXt2YXIgYj1lYShhKTtyZXR1cm4iYXJyYXkiPT1ifHwib2JqZWN0Ij09CmImJiJudW1iZXIiPT10eXBlb2YgYS5sZW5ndGh9LGlhPWZ1bmN0aW9uKGEpe3JldHVybiJmdW5jdGlvbiI9PWVhKGEpfSxtPWZ1bmN0aW9uKGEpe3ZhciBiPXR5cGVvZiBhO3JldHVybiJvYmplY3QiPT1iJiZudWxsIT1hfHwiZnVuY3Rpb24iPT1ifSxqYT0iY2xvc3VyZV91aWRfIisoMUU5Kk1hdGgucmFuZG9tKCk+Pj4wKSxrYT0wLGxhPWZ1bmN0aW9uKGEsYil7dmFyIGM9QXJyYXkucHJvdG90eXBlLnNsaWNlLmNhbGwoYXJndW1lbnRzLDEpO3JldHVybiBmdW5jdGlvbigpe3ZhciBkPWMuc2xpY2UoKTtkLnB1c2guYXBwbHkoZCxhcmd1bWVudHMpO3JldHVybiBhLmFwcGx5KHRoaXMsZCl9fSxuPWZ1bmN0aW9uKGEsYil7ZnVuY3Rpb24gYygpe31jLnByb3RvdHlwZT1iLnByb3RvdHlwZTthLmk9Yi5wcm90b3R5cGU7YS5wcm90b3R5cGU9bmV3IGM7YS5wcm90b3R5cGUuY29uc3RydWN0b3I9YX07dmFyIG1hLG5hPXtlYjoiYWN0aXZlZGVzY2VuZGFudCIsamI6ImF0b21pYyIsa2I6ImF1dG9jb21wbGV0ZSIsbWI6ImJ1c3kiLHBiOiJjaGVja2VkIixxYjoiY29saW5kZXgiLHZiOiJjb250cm9scyIseGI6ImRlc2NyaWJlZGJ5IixBYjoiZGlzYWJsZWQiLENiOiJkcm9wZWZmZWN0IixEYjoiZXhwYW5kZWQiLEViOiJmbG93dG8iLEdiOiJncmFiYmVkIixLYjoiaGFzcG9wdXAiLE1iOiJoaWRkZW4iLE9iOiJpbnZhbGlkIixQYjoibGFiZWwiLFFiOiJsYWJlbGxlZGJ5IixSYjoibGV2ZWwiLFdiOiJsaXZlIixrYzoibXVsdGlsaW5lIixsYzoibXVsdGlzZWxlY3RhYmxlIixwYzoib3JpZW50YXRpb24iLHFjOiJvd25zIixyYzoicG9zaW5zZXQiLHRjOiJwcmVzc2VkIix4YzoicmVhZG9ubHkiLHpjOiJyZWxldmFudCIsQWM6InJlcXVpcmVkIixFYzoicm93aW5kZXgiLEhjOiJzZWxlY3RlZCIsSmM6InNldHNpemUiLExjOiJzb3J0IixaYzoidmFsdWVtYXgiLCRjOiJ2YWx1ZW1pbiIsYWQ6InZhbHVlbm93IiwKYmQ6InZhbHVldGV4dCJ9O3ZhciBvYT1mdW5jdGlvbihhLGIsYyl7Zm9yKHZhciBkIGluIGEpYi5jYWxsKGMsYVtkXSxkLGEpfSxwYT1mdW5jdGlvbihhLGIpe2Zvcih2YXIgYyBpbiBhKWlmKGFbY109PWIpcmV0dXJuITA7cmV0dXJuITF9LHFhPWZ1bmN0aW9uKGEsYixjKXtpZihudWxsIT09YSYmYiBpbiBhKXRocm93IEVycm9yKCdUaGUgb2JqZWN0IGFscmVhZHkgY29udGFpbnMgdGhlIGtleSAiJytiKyciJyk7YVtiXT1jfSxzYT1mdW5jdGlvbihhKXt2YXIgYj17fSxjO2ZvcihjIGluIGEpYlthW2NdXT1jO3JldHVybiBifSx0YT0iY29uc3RydWN0b3IgaGFzT3duUHJvcGVydHkgaXNQcm90b3R5cGVPZiBwcm9wZXJ0eUlzRW51bWVyYWJsZSB0b0xvY2FsZVN0cmluZyB0b1N0cmluZyB2YWx1ZU9mIi5zcGxpdCgiICIpLHVhPWZ1bmN0aW9uKGEsYil7Zm9yKHZhciBjLGQsZT0xO2U8YXJndW1lbnRzLmxlbmd0aDtlKyspe2Q9YXJndW1lbnRzW2VdO2ZvcihjIGluIGQpYVtjXT1kW2NdO2Zvcih2YXIgZj0wO2Y8dGEubGVuZ3RoO2YrKyljPQp0YVtmXSxPYmplY3QucHJvdG90eXBlLmhhc093blByb3BlcnR5LmNhbGwoZCxjKSYmKGFbY109ZFtjXSl9fTt2YXIgdmE9e2ZiOiJhbGVydCIsZ2I6ImFsZXJ0ZGlhbG9nIixoYjoiYXBwbGljYXRpb24iLGliOiJhcnRpY2xlIixsYjoiYmFubmVyIixuYjoiYnV0dG9uIixvYjoiY2hlY2tib3giLHJiOiJjb2x1bW5oZWFkZXIiLHNiOiJjb21ib2JveCIsdGI6ImNvbXBsZW1lbnRhcnkiLHViOiJjb250ZW50aW5mbyIsd2I6ImRlZmluaXRpb24iLHliOiJkaWFsb2ciLHpiOiJkaXJlY3RvcnkiLEJiOiJkb2N1bWVudCIsRmI6ImZvcm0iLEhiOiJncmlkIixJYjoiZ3JpZGNlbGwiLEpiOiJncm91cCIsTGI6ImhlYWRpbmciLE5iOiJpbWciLFNiOiJsaW5rIixUYjoibGlzdCIsVWI6Imxpc3Rib3giLFZiOiJsaXN0aXRlbSIsWGI6ImxvZyIsWWI6Im1haW4iLFpiOiJtYXJxdWVlIiwkYjoibWF0aCIsYWM6Im1lbnUiLGJjOiJtZW51YmFyIixjYzoibWVudWl0ZW0iLGRjOiJtZW51aXRlbWNoZWNrYm94IixlYzoibWVudWl0ZW1yYWRpbyIsbWM6Im5hdmlnYXRpb24iLG5jOiJub3RlIixvYzoib3B0aW9uIiwKc2M6InByZXNlbnRhdGlvbiIsdWM6InByb2dyZXNzYmFyIix2YzoicmFkaW8iLHdjOiJyYWRpb2dyb3VwIix5YzoicmVnaW9uIixCYzoicm93IixDYzoicm93Z3JvdXAiLERjOiJyb3doZWFkZXIiLEZjOiJzY3JvbGxiYXIiLEdjOiJzZWFyY2giLEljOiJzZXBhcmF0b3IiLEtjOiJzbGlkZXIiLE1jOiJzcGluYnV0dG9uIixOYzoic3RhdHVzIixPYzoidGFiIixQYzoidGFibGlzdCIsUWM6InRhYnBhbmVsIixSYzoidGV4dGJveCIsU2M6InRleHRpbmZvIixUYzoidGltZXIiLFVjOiJ0b29sYmFyIixWYzoidG9vbHRpcCIsV2M6InRyZWUiLFhjOiJ0cmVlZ3JpZCIsWWM6InRyZWVpdGVtIn07dmFyIHdhPWZ1bmN0aW9uKGEpe2lmKEVycm9yLmNhcHR1cmVTdGFja1RyYWNlKUVycm9yLmNhcHR1cmVTdGFja1RyYWNlKHRoaXMsd2EpO2Vsc2V7dmFyIGI9RXJyb3IoKS5zdGFjaztiJiYodGhpcy5zdGFjaz1iKX1hJiYodGhpcy5tZXNzYWdlPVN0cmluZyhhKSl9O24od2EsRXJyb3IpO3dhLnByb3RvdHlwZS5uYW1lPSJDdXN0b21FcnJvciI7dmFyIHhhO3ZhciB5YT1mdW5jdGlvbihhLGIpe2E9YS5zcGxpdCgiJXMiKTtmb3IodmFyIGM9IiIsZD1hLmxlbmd0aC0xLGU9MDtlPGQ7ZSsrKWMrPWFbZV0rKGU8Yi5sZW5ndGg/YltlXToiJXMiKTt3YS5jYWxsKHRoaXMsYythW2RdKX07bih5YSx3YSk7eWEucHJvdG90eXBlLm5hbWU9IkFzc2VydGlvbkVycm9yIjsKdmFyIHphPWZ1bmN0aW9uKGEsYixjLGQpe3ZhciBlPSJBc3NlcnRpb24gZmFpbGVkIjtpZihjKXtlKz0iOiAiK2M7dmFyIGY9ZH1lbHNlIGEmJihlKz0iOiAiK2EsZj1iKTt0aHJvdyBuZXcgeWEoIiIrZSxmfHxbXSk7fSxxPWZ1bmN0aW9uKGEsYixjKXthfHx6YSgiIixudWxsLGIsQXJyYXkucHJvdG90eXBlLnNsaWNlLmNhbGwoYXJndW1lbnRzLDIpKTtyZXR1cm4gYX0sQWE9ZnVuY3Rpb24oYSxiLGMpe20oYSkmJjE9PWEubm9kZVR5cGV8fHphKCJFeHBlY3RlZCBFbGVtZW50IGJ1dCBnb3QgJXM6ICVzLiIsW2VhKGEpLGFdLGIsQXJyYXkucHJvdG90eXBlLnNsaWNlLmNhbGwoYXJndW1lbnRzLDIpKX0sQ2E9ZnVuY3Rpb24oYSxiLGMsZCl7YSBpbnN0YW5jZW9mIGJ8fHphKCJFeHBlY3RlZCBpbnN0YW5jZW9mICVzIGJ1dCBnb3QgJXMuIixbQmEoYiksQmEoYSldLGMsQXJyYXkucHJvdG90eXBlLnNsaWNlLmNhbGwoYXJndW1lbnRzLDMpKX0sQmE9ZnVuY3Rpb24oYSl7cmV0dXJuIGEgaW5zdGFuY2VvZgpGdW5jdGlvbj9hLmRpc3BsYXlOYW1lfHxhLm5hbWV8fCJ1bmtub3duIHR5cGUgbmFtZSI6YSBpbnN0YW5jZW9mIE9iamVjdD9hLmNvbnN0cnVjdG9yLmRpc3BsYXlOYW1lfHxhLmNvbnN0cnVjdG9yLm5hbWV8fE9iamVjdC5wcm90b3R5cGUudG9TdHJpbmcuY2FsbChhKTpudWxsPT09YT8ibnVsbCI6dHlwZW9mIGF9O3ZhciBEYT1BcnJheS5wcm90b3R5cGUuaW5kZXhPZj9mdW5jdGlvbihhLGIpe3EobnVsbCE9YS5sZW5ndGgpO3JldHVybiBBcnJheS5wcm90b3R5cGUuaW5kZXhPZi5jYWxsKGEsYix2b2lkIDApfTpmdW5jdGlvbihhLGIpe2lmKCJzdHJpbmciPT09dHlwZW9mIGEpcmV0dXJuInN0cmluZyIhPT10eXBlb2YgYnx8MSE9Yi5sZW5ndGg/LTE6YS5pbmRleE9mKGIsMCk7Zm9yKHZhciBjPTA7YzxhLmxlbmd0aDtjKyspaWYoYyBpbiBhJiZhW2NdPT09YilyZXR1cm4gYztyZXR1cm4tMX0scj1BcnJheS5wcm90b3R5cGUuZm9yRWFjaD9mdW5jdGlvbihhLGIsYyl7cShudWxsIT1hLmxlbmd0aCk7QXJyYXkucHJvdG90eXBlLmZvckVhY2guY2FsbChhLGIsYyl9OmZ1bmN0aW9uKGEsYixjKXtmb3IodmFyIGQ9YS5sZW5ndGgsZT0ic3RyaW5nIj09PXR5cGVvZiBhP2Euc3BsaXQoIiIpOmEsZj0wO2Y8ZDtmKyspZiBpbiBlJiZiLmNhbGwoYyxlW2ZdLGYsYSl9LEVhPUFycmF5LnByb3RvdHlwZS5maWx0ZXI/CmZ1bmN0aW9uKGEsYil7cShudWxsIT1hLmxlbmd0aCk7cmV0dXJuIEFycmF5LnByb3RvdHlwZS5maWx0ZXIuY2FsbChhLGIsdm9pZCAwKX06ZnVuY3Rpb24oYSxiKXtmb3IodmFyIGM9YS5sZW5ndGgsZD1bXSxlPTAsZj0ic3RyaW5nIj09PXR5cGVvZiBhP2Euc3BsaXQoIiIpOmEsZz0wO2c8YztnKyspaWYoZyBpbiBmKXt2YXIgaz1mW2ddO2IuY2FsbCh2b2lkIDAsayxnLGEpJiYoZFtlKytdPWspfXJldHVybiBkfSxGYT1BcnJheS5wcm90b3R5cGUuZXZlcnk/ZnVuY3Rpb24oYSxiKXtxKG51bGwhPWEubGVuZ3RoKTtyZXR1cm4gQXJyYXkucHJvdG90eXBlLmV2ZXJ5LmNhbGwoYSxiLHZvaWQgMCl9OmZ1bmN0aW9uKGEsYil7Zm9yKHZhciBjPWEubGVuZ3RoLGQ9InN0cmluZyI9PT10eXBlb2YgYT9hLnNwbGl0KCIiKTphLGU9MDtlPGM7ZSsrKWlmKGUgaW4gZCYmIWIuY2FsbCh2b2lkIDAsZFtlXSxlLGEpKXJldHVybiExO3JldHVybiEwfSxHYT1mdW5jdGlvbihhLGIpe3JldHVybiAwPD0KRGEoYSxiKX0sSGE9ZnVuY3Rpb24oYSxiKXtiPURhKGEsYik7dmFyIGM7aWYoYz0wPD1iKXEobnVsbCE9YS5sZW5ndGgpLEFycmF5LnByb3RvdHlwZS5zcGxpY2UuY2FsbChhLGIsMSk7cmV0dXJuIGN9LElhPWZ1bmN0aW9uKGEpe3JldHVybiBBcnJheS5wcm90b3R5cGUuY29uY2F0LmFwcGx5KFtdLGFyZ3VtZW50cyl9LEphPWZ1bmN0aW9uKGEpe3ZhciBiPWEubGVuZ3RoO2lmKDA8Yil7Zm9yKHZhciBjPUFycmF5KGIpLGQ9MDtkPGI7ZCsrKWNbZF09YVtkXTtyZXR1cm4gY31yZXR1cm5bXX0sTGE9ZnVuY3Rpb24oYSxiLGMsZCl7cShudWxsIT1hLmxlbmd0aCk7QXJyYXkucHJvdG90eXBlLnNwbGljZS5hcHBseShhLEthKGFyZ3VtZW50cywxKSl9LEthPWZ1bmN0aW9uKGEsYixjKXtxKG51bGwhPWEubGVuZ3RoKTtyZXR1cm4gMj49YXJndW1lbnRzLmxlbmd0aD9BcnJheS5wcm90b3R5cGUuc2xpY2UuY2FsbChhLGIpOkFycmF5LnByb3RvdHlwZS5zbGljZS5jYWxsKGEsYixjKX07dmFyIE1hPVN0cmluZy5wcm90b3R5cGUudHJpbT9mdW5jdGlvbihhKXtyZXR1cm4gYS50cmltKCl9OmZ1bmN0aW9uKGEpe3JldHVybi9eW1xzXHhhMF0qKFtcc1xTXSo/KVtcc1x4YTBdKiQvLmV4ZWMoYSlbMV19LE5hPS8mL2csT2E9LzwvZyxQYT0vPi9nLFFhPS8iL2csUmE9LycvZyxTYT0vXHgwMC9nLFRhPS9bXHgwMCY8PiInXS8sdD1mdW5jdGlvbihhLGIpe3JldHVybi0xIT1hLmluZGV4T2YoYil9LFVhPWZ1bmN0aW9uKGEsYil7cmV0dXJuIGE8Yj8tMTphPmI/MTowfTt2YXIgdTthOnt2YXIgVmE9bC5uYXZpZ2F0b3I7aWYoVmEpe3ZhciBXYT1WYS51c2VyQWdlbnQ7aWYoV2Epe3U9V2E7YnJlYWsgYX19dT0iIn07LyoKCiBDb3B5cmlnaHQgVGhlIENsb3N1cmUgTGlicmFyeSBBdXRob3JzLgogU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEFwYWNoZS0yLjAKKi8KdmFyIFhhPWZ1bmN0aW9uKGEpe1RhLnRlc3QoYSkmJigtMSE9YS5pbmRleE9mKCImIikmJihhPWEucmVwbGFjZShOYSwiJmFtcDsiKSksLTEhPWEuaW5kZXhPZigiPCIpJiYoYT1hLnJlcGxhY2UoT2EsIiZsdDsiKSksLTEhPWEuaW5kZXhPZigiPiIpJiYoYT1hLnJlcGxhY2UoUGEsIiZndDsiKSksLTEhPWEuaW5kZXhPZignIicpJiYoYT1hLnJlcGxhY2UoUWEsIiZxdW90OyIpKSwtMSE9YS5pbmRleE9mKCInIikmJihhPWEucmVwbGFjZShSYSwiJiMzOTsiKSksLTEhPWEuaW5kZXhPZigiXHgwMCIpJiYoYT1hLnJlcGxhY2UoU2EsIiYjMDsiKSkpO3JldHVybiBhfTt2YXIgWWE9ZnVuY3Rpb24oYSl7WWFbIiAiXShhKTtyZXR1cm4gYX07WWFbIiAiXT1jYTt2YXIgJGE9ZnVuY3Rpb24oYSxiKXt2YXIgYz1aYTtyZXR1cm4gT2JqZWN0LnByb3RvdHlwZS5oYXNPd25Qcm9wZXJ0eS5jYWxsKGMsYSk/Y1thXTpjW2FdPWIoYSl9O3ZhciBhYj10KHUsIk9wZXJhIiksdj10KHUsIlRyaWRlbnQiKXx8dCh1LCJNU0lFIiksYmI9dCh1LCJFZGdlIikseD10KHUsIkdlY2tvIikmJiEodCh1LnRvTG93ZXJDYXNlKCksIndlYmtpdCIpJiYhdCh1LCJFZGdlIikpJiYhKHQodSwiVHJpZGVudCIpfHx0KHUsIk1TSUUiKSkmJiF0KHUsIkVkZ2UiKSx5PXQodS50b0xvd2VyQ2FzZSgpLCJ3ZWJraXQiKSYmIXQodSwiRWRnZSIpLHo9dCh1LCJNYWNpbnRvc2giKSxjYj1mdW5jdGlvbigpe3ZhciBhPWwuZG9jdW1lbnQ7cmV0dXJuIGE/YS5kb2N1bWVudE1vZGU6dm9pZCAwfSxkYjsKYTp7dmFyIGViPSIiLGZiPWZ1bmN0aW9uKCl7dmFyIGE9dTtpZih4KXJldHVybi9ydjooW15cKTtdKykoXCl8OykvLmV4ZWMoYSk7aWYoYmIpcmV0dXJuL0VkZ2VcLyhbXGRcLl0rKS8uZXhlYyhhKTtpZih2KXJldHVybi9cYig/Ok1TSUV8cnYpWzogXShbXlwpO10rKShcKXw7KS8uZXhlYyhhKTtpZih5KXJldHVybi9XZWJLaXRcLyhcUyspLy5leGVjKGEpO2lmKGFiKXJldHVybi8oPzpWZXJzaW9uKVsgXC9dPyhcUyspLy5leGVjKGEpfSgpO2ZiJiYoZWI9ZmI/ZmJbMV06IiIpO2lmKHYpe3ZhciBnYj1jYigpO2lmKG51bGwhPWdiJiZnYj5wYXJzZUZsb2F0KGViKSl7ZGI9U3RyaW5nKGdiKTticmVhayBhfX1kYj1lYn0KdmFyIGhiPWRiLFphPXt9LEE9ZnVuY3Rpb24oYSl7cmV0dXJuICRhKGEsZnVuY3Rpb24oKXtmb3IodmFyIGI9MCxjPU1hKFN0cmluZyhoYikpLnNwbGl0KCIuIiksZD1NYShTdHJpbmcoYSkpLnNwbGl0KCIuIiksZT1NYXRoLm1heChjLmxlbmd0aCxkLmxlbmd0aCksZj0wOzA9PWImJmY8ZTtmKyspe3ZhciBnPWNbZl18fCIiLGs9ZFtmXXx8IiI7ZG97Zz0vKFxkKikoXEQqKSguKikvLmV4ZWMoZyl8fFsiIiwiIiwiIiwiIl07az0vKFxkKikoXEQqKSguKikvLmV4ZWMoayl8fFsiIiwiIiwiIiwiIl07aWYoMD09Z1swXS5sZW5ndGgmJjA9PWtbMF0ubGVuZ3RoKWJyZWFrO2I9VWEoMD09Z1sxXS5sZW5ndGg/MDpwYXJzZUludChnWzFdLDEwKSwwPT1rWzFdLmxlbmd0aD8wOnBhcnNlSW50KGtbMV0sMTApKXx8VWEoMD09Z1syXS5sZW5ndGgsMD09a1syXS5sZW5ndGgpfHxVYShnWzJdLGtbMl0pO2c9Z1szXTtrPWtbM119d2hpbGUoMD09Yil9cmV0dXJuIDA8PWJ9KX0saWI7CmliPWwuZG9jdW1lbnQmJnY/Y2IoKTp2b2lkIDA7dmFyIGpiPSF2fHw5PD1OdW1iZXIoaWIpO3ZhciBsYj1mdW5jdGlvbihhLGIpe29hKGIsZnVuY3Rpb24oYyxkKXtjJiYib2JqZWN0Ij09dHlwZW9mIGMmJmMuZGQmJihjPWMuY2QoKSk7InN0eWxlIj09ZD9hLnN0eWxlLmNzc1RleHQ9YzoiY2xhc3MiPT1kP2EuY2xhc3NOYW1lPWM6ImZvciI9PWQ/YS5odG1sRm9yPWM6a2IuaGFzT3duUHJvcGVydHkoZCk/YS5zZXRBdHRyaWJ1dGUoa2JbZF0sYyk6MD09ZC5sYXN0SW5kZXhPZigiYXJpYS0iLDApfHwwPT1kLmxhc3RJbmRleE9mKCJkYXRhLSIsMCk/YS5zZXRBdHRyaWJ1dGUoZCxjKTphW2RdPWN9KX0sa2I9e2NlbGxwYWRkaW5nOiJjZWxsUGFkZGluZyIsY2VsbHNwYWNpbmc6ImNlbGxTcGFjaW5nIixjb2xzcGFuOiJjb2xTcGFuIixmcmFtZWJvcmRlcjoiZnJhbWVCb3JkZXIiLGhlaWdodDoiaGVpZ2h0IixtYXhsZW5ndGg6Im1heExlbmd0aCIsbm9uY2U6Im5vbmNlIixyb2xlOiJyb2xlIixyb3dzcGFuOiJyb3dTcGFuIix0eXBlOiJ0eXBlIix1c2VtYXA6InVzZU1hcCIsdmFsaWduOiJ2QWxpZ24iLAp3aWR0aDoid2lkdGgifSxtYj1mdW5jdGlvbihhLGIsYyl7ZnVuY3Rpb24gZChrKXtrJiZiLmFwcGVuZENoaWxkKCJzdHJpbmciPT09dHlwZW9mIGs/YS5jcmVhdGVUZXh0Tm9kZShrKTprKX1mb3IodmFyIGU9MjtlPGMubGVuZ3RoO2UrKyl7dmFyIGY9Y1tlXTtpZighaGEoZil8fG0oZikmJjA8Zi5ub2RlVHlwZSlkKGYpO2Vsc2V7YTp7aWYoZiYmIm51bWJlciI9PXR5cGVvZiBmLmxlbmd0aCl7aWYobShmKSl7dmFyIGc9ImZ1bmN0aW9uIj09dHlwZW9mIGYuaXRlbXx8InN0cmluZyI9PXR5cGVvZiBmLml0ZW07YnJlYWsgYX1pZihpYShmKSl7Zz0iZnVuY3Rpb24iPT10eXBlb2YgZi5pdGVtO2JyZWFrIGF9fWc9ITF9cihnP0phKGYpOmYsZCl9fX0sbmI9ZnVuY3Rpb24oYSxiKXtiPVN0cmluZyhiKTsiYXBwbGljYXRpb24veGh0bWwreG1sIj09PWEuY29udGVudFR5cGUmJihiPWIudG9Mb3dlckNhc2UoKSk7cmV0dXJuIGEuY3JlYXRlRWxlbWVudChiKX0sb2I9ZnVuY3Rpb24oYSl7YSYmCmEucGFyZW50Tm9kZSYmYS5wYXJlbnROb2RlLnJlbW92ZUNoaWxkKGEpfSxwYj1mdW5jdGlvbihhLGIpe2lmKCFhfHwhYilyZXR1cm4hMTtpZihhLmNvbnRhaW5zJiYxPT1iLm5vZGVUeXBlKXJldHVybiBhPT1ifHxhLmNvbnRhaW5zKGIpO2lmKCJ1bmRlZmluZWQiIT10eXBlb2YgYS5jb21wYXJlRG9jdW1lbnRQb3NpdGlvbilyZXR1cm4gYT09Ynx8ISEoYS5jb21wYXJlRG9jdW1lbnRQb3NpdGlvbihiKSYxNik7Zm9yKDtiJiZhIT1iOyliPWIucGFyZW50Tm9kZTtyZXR1cm4gYj09YX0scWI9ZnVuY3Rpb24oYSl7cShhLCJOb2RlIGNhbm5vdCBiZSBudWxsIG9yIHVuZGVmaW5lZC4iKTtyZXR1cm4gOT09YS5ub2RlVHlwZT9hOmEub3duZXJEb2N1bWVudHx8YS5kb2N1bWVudH0scmI9ZnVuY3Rpb24oYSxiKXtiP2EudGFiSW5kZXg9MDooYS50YWJJbmRleD0tMSxhLnJlbW92ZUF0dHJpYnV0ZSgidGFiSW5kZXgiKSl9LHNiPWZ1bmN0aW9uKGEpe3JldHVybiB2JiYhQSgiOSIpPyhhPWEuZ2V0QXR0cmlidXRlTm9kZSgidGFiaW5kZXgiKSwKbnVsbCE9YSYmYS5zcGVjaWZpZWQpOmEuaGFzQXR0cmlidXRlKCJ0YWJpbmRleCIpfSx0Yj1mdW5jdGlvbihhKXthPWEudGFiSW5kZXg7cmV0dXJuIm51bWJlciI9PT10eXBlb2YgYSYmMDw9YSYmMzI3Njg+YX0sdWI9ZnVuY3Rpb24oYSl7dGhpcy5hPWF8fGwuZG9jdW1lbnR8fGRvY3VtZW50fTt1Yi5wcm90b3R5cGUuZj1mdW5jdGlvbihhKXtyZXR1cm4ic3RyaW5nIj09PXR5cGVvZiBhP3RoaXMuYS5nZXRFbGVtZW50QnlJZChhKTphfTsKdWIucHJvdG90eXBlLmI9ZnVuY3Rpb24oYSxiLGMpe3ZhciBkPXRoaXMuYSxlPWFyZ3VtZW50cyxmPVN0cmluZyhlWzBdKSxnPWVbMV07aWYoIWpiJiZnJiYoZy5uYW1lfHxnLnR5cGUpKXtmPVsiPCIsZl07Zy5uYW1lJiZmLnB1c2goJyBuYW1lPSInLFhhKGcubmFtZSksJyInKTtpZihnLnR5cGUpe2YucHVzaCgnIHR5cGU9IicsWGEoZy50eXBlKSwnIicpO3ZhciBrPXt9O3VhKGssZyk7ZGVsZXRlIGsudHlwZTtnPWt9Zi5wdXNoKCI+Iik7Zj1mLmpvaW4oIiIpfWY9bmIoZCxmKTtnJiYoInN0cmluZyI9PT10eXBlb2YgZz9mLmNsYXNzTmFtZT1nOmZhKGcpP2YuY2xhc3NOYW1lPWcuam9pbigiICIpOmxiKGYsZykpOzI8ZS5sZW5ndGgmJm1iKGQsZixlKTtyZXR1cm4gZn07dmFyIHZiPWZ1bmN0aW9uKGEsYil7Yj8ocShwYSh2YSxiKSwiTm8gc3VjaCBBUklBIHJvbGUgIitiKSxhLnNldEF0dHJpYnV0ZSgicm9sZSIsYikpOmEucmVtb3ZlQXR0cmlidXRlKCJyb2xlIil9LHhiPWZ1bmN0aW9uKGEsYixjKXtmYShjKSYmKGM9Yy5qb2luKCIgIikpO3ZhciBkPXdiKGIpOyIiPT09Y3x8dm9pZCAwPT1jPyhtYXx8KG1hPXthdG9taWM6ITEsYXV0b2NvbXBsZXRlOiJub25lIixkcm9wZWZmZWN0OiJub25lIixoYXNwb3B1cDohMSxsaXZlOiJvZmYiLG11bHRpbGluZTohMSxtdWx0aXNlbGVjdGFibGU6ITEsb3JpZW50YXRpb246InZlcnRpY2FsIixyZWFkb25seTohMSxyZWxldmFudDoiYWRkaXRpb25zIHRleHQiLHJlcXVpcmVkOiExLHNvcnQ6Im5vbmUiLGJ1c3k6ITEsZGlzYWJsZWQ6ITEsaGlkZGVuOiExLGludmFsaWQ6ImZhbHNlIn0pLGM9bWEsYiBpbiBjP2Euc2V0QXR0cmlidXRlKGQsY1tiXSk6YS5yZW1vdmVBdHRyaWJ1dGUoZCkpOmEuc2V0QXR0cmlidXRlKGQsCmMpfSx3Yj1mdW5jdGlvbihhKXtxKGEsIkFSSUEgYXR0cmlidXRlIGNhbm5vdCBiZSBlbXB0eS4iKTtxKHBhKG5hLGEpLCJObyBzdWNoIEFSSUEgYXR0cmlidXRlICIrYSk7cmV0dXJuImFyaWEtIithfTt2YXIgeWI9T2JqZWN0LmZyZWV6ZXx8ZnVuY3Rpb24oYSl7cmV0dXJuIGF9O3ZhciBDPWZ1bmN0aW9uKCl7dGhpcy5VPXRoaXMuVTt0aGlzLko9dGhpcy5KfTtDLnByb3RvdHlwZS5VPSExO0MucHJvdG90eXBlLk49ZnVuY3Rpb24oKXt0aGlzLlV8fCh0aGlzLlU9ITAsdGhpcy51KCkpfTt2YXIgemI9ZnVuY3Rpb24oYSxiKXthLlU/YigpOihhLkp8fChhLko9W10pLGEuSi5wdXNoKGIpKX07Qy5wcm90b3R5cGUudT1mdW5jdGlvbigpe2lmKHRoaXMuSilmb3IoO3RoaXMuSi5sZW5ndGg7KXRoaXMuSi5zaGlmdCgpKCl9O3ZhciBBYj1mdW5jdGlvbihhKXthJiYiZnVuY3Rpb24iPT10eXBlb2YgYS5OJiZhLk4oKX07dmFyIEJiPWZ1bmN0aW9uKGEpe3JldHVybiJzdHJpbmciPT10eXBlb2YgYS5jbGFzc05hbWU/YS5jbGFzc05hbWU6YS5nZXRBdHRyaWJ1dGUmJmEuZ2V0QXR0cmlidXRlKCJjbGFzcyIpfHwiIn0sRD1mdW5jdGlvbihhKXtyZXR1cm4gYS5jbGFzc0xpc3Q/YS5jbGFzc0xpc3Q6QmIoYSkubWF0Y2goL1xTKy9nKXx8W119LENiPWZ1bmN0aW9uKGEsYil7InN0cmluZyI9PXR5cGVvZiBhLmNsYXNzTmFtZT9hLmNsYXNzTmFtZT1iOmEuc2V0QXR0cmlidXRlJiZhLnNldEF0dHJpYnV0ZSgiY2xhc3MiLGIpfSxEYj1mdW5jdGlvbihhLGIpe3JldHVybiBhLmNsYXNzTGlzdD9hLmNsYXNzTGlzdC5jb250YWlucyhiKTpHYShEKGEpLGIpfSxFYj1mdW5jdGlvbihhLGIpe2lmKGEuY2xhc3NMaXN0KWEuY2xhc3NMaXN0LmFkZChiKTtlbHNlIGlmKCFEYihhLGIpKXt2YXIgYz1CYihhKTtDYihhLGMrKDA8Yy5sZW5ndGg/IiAiK2I6YikpfX0sRmI9ZnVuY3Rpb24oYSxiKXtpZihhLmNsYXNzTGlzdClyKGIsCmZ1bmN0aW9uKGUpe0ViKGEsZSl9KTtlbHNle3ZhciBjPXt9O3IoRChhKSxmdW5jdGlvbihlKXtjW2VdPSEwfSk7cihiLGZ1bmN0aW9uKGUpe2NbZV09ITB9KTtiPSIiO2Zvcih2YXIgZCBpbiBjKWIrPTA8Yi5sZW5ndGg/IiAiK2Q6ZDtDYihhLGIpfX0sR2I9ZnVuY3Rpb24oYSxiKXthLmNsYXNzTGlzdD9hLmNsYXNzTGlzdC5yZW1vdmUoYik6RGIoYSxiKSYmQ2IoYSxFYShEKGEpLGZ1bmN0aW9uKGMpe3JldHVybiBjIT1ifSkuam9pbigiICIpKX0sSGI9ZnVuY3Rpb24oYSxiKXthLmNsYXNzTGlzdD9yKGIsZnVuY3Rpb24oYyl7R2IoYSxjKX0pOkNiKGEsRWEoRChhKSxmdW5jdGlvbihjKXtyZXR1cm4hR2EoYixjKX0pLmpvaW4oIiAiKSl9O3ZhciBJYj0hdnx8OTw9TnVtYmVyKGliKSxKYj0hdnx8OTw9TnVtYmVyKGliKSxLYj12JiYhQSgiOSIpLExiPWZ1bmN0aW9uKCl7aWYoIWwuYWRkRXZlbnRMaXN0ZW5lcnx8IU9iamVjdC5kZWZpbmVQcm9wZXJ0eSlyZXR1cm4hMTt2YXIgYT0hMSxiPU9iamVjdC5kZWZpbmVQcm9wZXJ0eSh7fSwicGFzc2l2ZSIse2dldDpmdW5jdGlvbigpe2E9ITB9fSk7dHJ5e2wuYWRkRXZlbnRMaXN0ZW5lcigidGVzdCIsY2EsYiksbC5yZW1vdmVFdmVudExpc3RlbmVyKCJ0ZXN0IixjYSxiKX1jYXRjaChjKXt9cmV0dXJuIGF9KCk7dmFyIEU9ZnVuY3Rpb24oYSxiKXt0aGlzLnR5cGU9YTt0aGlzLmE9dGhpcy50YXJnZXQ9Yjt0aGlzLmg9ITE7dGhpcy5LYT0hMH07RS5wcm90b3R5cGUuaj1mdW5jdGlvbigpe3RoaXMuaD0hMH07RS5wcm90b3R5cGUuZz1mdW5jdGlvbigpe3RoaXMuS2E9ITF9O3ZhciBGPXtMOiJtb3VzZWRvd24iLE06Im1vdXNldXAiLGFhOiJtb3VzZWNhbmNlbCIsaGM6Im1vdXNlbW92ZSIsamM6Im1vdXNlb3ZlciIsaWM6Im1vdXNlb3V0IixmYzoibW91c2VlbnRlciIsZ2M6Im1vdXNlbGVhdmUifTt2YXIgRz1mdW5jdGlvbihhLGIpe0UuY2FsbCh0aGlzLGE/YS50eXBlOiIiKTt0aGlzLnJlbGF0ZWRUYXJnZXQ9dGhpcy5hPXRoaXMudGFyZ2V0PW51bGw7dGhpcy5idXR0b249dGhpcy5zY3JlZW5ZPXRoaXMuc2NyZWVuWD10aGlzLmNsaWVudFk9dGhpcy5jbGllbnRYPTA7dGhpcy5rZXk9IiI7dGhpcy5jPTA7dGhpcy5BPXRoaXMubWV0YUtleT10aGlzLnNoaWZ0S2V5PXRoaXMuYWx0S2V5PXRoaXMuY3RybEtleT0hMTt0aGlzLnBvaW50ZXJJZD0wO3RoaXMucG9pbnRlclR5cGU9IiI7dGhpcy5iPW51bGw7aWYoYSl7dmFyIGM9dGhpcy50eXBlPWEudHlwZSxkPWEuY2hhbmdlZFRvdWNoZXMmJmEuY2hhbmdlZFRvdWNoZXMubGVuZ3RoP2EuY2hhbmdlZFRvdWNoZXNbMF06bnVsbDt0aGlzLnRhcmdldD1hLnRhcmdldHx8YS5zcmNFbGVtZW50O3RoaXMuYT1iO2lmKGI9YS5yZWxhdGVkVGFyZ2V0KXtpZih4KXthOnt0cnl7WWEoYi5ub2RlTmFtZSk7dmFyIGU9ITA7YnJlYWsgYX1jYXRjaChmKXt9ZT0KITF9ZXx8KGI9bnVsbCl9fWVsc2UibW91c2VvdmVyIj09Yz9iPWEuZnJvbUVsZW1lbnQ6Im1vdXNlb3V0Ij09YyYmKGI9YS50b0VsZW1lbnQpO3RoaXMucmVsYXRlZFRhcmdldD1iO2Q/KHRoaXMuY2xpZW50WD12b2lkIDAhPT1kLmNsaWVudFg/ZC5jbGllbnRYOmQucGFnZVgsdGhpcy5jbGllbnRZPXZvaWQgMCE9PWQuY2xpZW50WT9kLmNsaWVudFk6ZC5wYWdlWSx0aGlzLnNjcmVlblg9ZC5zY3JlZW5YfHwwLHRoaXMuc2NyZWVuWT1kLnNjcmVlbll8fDApOih0aGlzLmNsaWVudFg9dm9pZCAwIT09YS5jbGllbnRYP2EuY2xpZW50WDphLnBhZ2VYLHRoaXMuY2xpZW50WT12b2lkIDAhPT1hLmNsaWVudFk/YS5jbGllbnRZOmEucGFnZVksdGhpcy5zY3JlZW5YPWEuc2NyZWVuWHx8MCx0aGlzLnNjcmVlblk9YS5zY3JlZW5ZfHwwKTt0aGlzLmJ1dHRvbj1hLmJ1dHRvbjt0aGlzLmM9YS5rZXlDb2RlfHwwO3RoaXMua2V5PWEua2V5fHwiIjt0aGlzLmN0cmxLZXk9YS5jdHJsS2V5O3RoaXMuYWx0S2V5PQphLmFsdEtleTt0aGlzLnNoaWZ0S2V5PWEuc2hpZnRLZXk7dGhpcy5tZXRhS2V5PWEubWV0YUtleTt0aGlzLkE9ej9hLm1ldGFLZXk6YS5jdHJsS2V5O3RoaXMucG9pbnRlcklkPWEucG9pbnRlcklkfHwwO3RoaXMucG9pbnRlclR5cGU9InN0cmluZyI9PT10eXBlb2YgYS5wb2ludGVyVHlwZT9hLnBvaW50ZXJUeXBlOk1iW2EucG9pbnRlclR5cGVdfHwiIjt0aGlzLmI9YTthLmRlZmF1bHRQcmV2ZW50ZWQmJnRoaXMuZygpfX07bihHLEUpO3ZhciBOYj15YihbMSw0LDJdKSxNYj15Yih7MjoidG91Y2giLDM6InBlbiIsNDoibW91c2UifSksT2I9ZnVuY3Rpb24oYSl7cmV0dXJuIEliPzA9PWEuYi5idXR0b246ImNsaWNrIj09YS50eXBlPyEwOiEhKGEuYi5idXR0b24mTmJbMF0pfTtHLnByb3RvdHlwZS5qPWZ1bmN0aW9uKCl7Ry5pLmouY2FsbCh0aGlzKTt0aGlzLmIuc3RvcFByb3BhZ2F0aW9uP3RoaXMuYi5zdG9wUHJvcGFnYXRpb24oKTp0aGlzLmIuY2FuY2VsQnViYmxlPSEwfTsKRy5wcm90b3R5cGUuZz1mdW5jdGlvbigpe0cuaS5nLmNhbGwodGhpcyk7dmFyIGE9dGhpcy5iO2lmKGEucHJldmVudERlZmF1bHQpYS5wcmV2ZW50RGVmYXVsdCgpO2Vsc2UgaWYoYS5yZXR1cm5WYWx1ZT0hMSxLYil0cnl7aWYoYS5jdHJsS2V5fHwxMTI8PWEua2V5Q29kZSYmMTIzPj1hLmtleUNvZGUpYS5rZXlDb2RlPS0xfWNhdGNoKGIpe319O3ZhciBQYj0iY2xvc3VyZV9saXN0ZW5hYmxlXyIrKDFFNipNYXRoLnJhbmRvbSgpfDApLFFiPWZ1bmN0aW9uKGEpe3JldHVybiEoIWF8fCFhW1BiXSl9LFJiPTA7dmFyIFNiPWZ1bmN0aW9uKGEsYixjLGQsZSl7dGhpcy5saXN0ZW5lcj1hO3RoaXMuYT1udWxsO3RoaXMuc3JjPWI7dGhpcy50eXBlPWM7dGhpcy5jYXB0dXJlPSEhZDt0aGlzLm1hPWU7dGhpcy5rZXk9KytSYjt0aGlzLlg9dGhpcy5oYT0hMX0sVGI9ZnVuY3Rpb24oYSl7YS5YPSEwO2EubGlzdGVuZXI9bnVsbDthLmE9bnVsbDthLnNyYz1udWxsO2EubWE9bnVsbH07dmFyIFViPWZ1bmN0aW9uKGEpe3RoaXMuc3JjPWE7dGhpcy5hPXt9O3RoaXMuYj0wfTtVYi5wcm90b3R5cGUuYWRkPWZ1bmN0aW9uKGEsYixjLGQsZSl7dmFyIGY9YS50b1N0cmluZygpO2E9dGhpcy5hW2ZdO2F8fChhPXRoaXMuYVtmXT1bXSx0aGlzLmIrKyk7dmFyIGc9VmIoYSxiLGQsZSk7LTE8Zz8oYj1hW2ddLGN8fChiLmhhPSExKSk6KGI9bmV3IFNiKGIsdGhpcy5zcmMsZiwhIWQsZSksYi5oYT1jLGEucHVzaChiKSk7cmV0dXJuIGJ9Owp2YXIgV2I9ZnVuY3Rpb24oYSxiKXt2YXIgYz1iLnR5cGU7YyBpbiBhLmEmJkhhKGEuYVtjXSxiKSYmKFRiKGIpLDA9PWEuYVtjXS5sZW5ndGgmJihkZWxldGUgYS5hW2NdLGEuYi0tKSl9LFhiPWZ1bmN0aW9uKGEsYixjLGQsZSl7YT1hLmFbYi50b1N0cmluZygpXTtiPS0xO2EmJihiPVZiKGEsYyxkLGUpKTtyZXR1cm4tMTxiP2FbYl06bnVsbH0sVmI9ZnVuY3Rpb24oYSxiLGMsZCl7Zm9yKHZhciBlPTA7ZTxhLmxlbmd0aDsrK2Upe3ZhciBmPWFbZV07aWYoIWYuWCYmZi5saXN0ZW5lcj09YiYmZi5jYXB0dXJlPT0hIWMmJmYubWE9PWQpcmV0dXJuIGV9cmV0dXJuLTF9O3ZhciBZYj0iY2xvc3VyZV9sbV8iKygxRTYqTWF0aC5yYW5kb20oKXwwKSxaYj17fSwkYj0wLGJjPWZ1bmN0aW9uKGEsYixjLGQsZSl7aWYoZCYmZC5vbmNlKXJldHVybiBhYyhhLGIsYyxkLGUpO2lmKGZhKGIpKXtmb3IodmFyIGY9MDtmPGIubGVuZ3RoO2YrKyliYyhhLGJbZl0sYyxkLGUpO3JldHVybiBudWxsfWM9Y2MoYyk7cmV0dXJuIFFiKGEpP2RjKGEsYixjLG0oZCk/ISFkLmNhcHR1cmU6ISFkLGUpOmVjKGEsYixjLCExLGQsZSl9LGVjPWZ1bmN0aW9uKGEsYixjLGQsZSxmKXtpZighYil0aHJvdyBFcnJvcigiSW52YWxpZCBldmVudCB0eXBlIik7dmFyIGc9bShlKT8hIWUuY2FwdHVyZTohIWUsaz1mYyhhKTtrfHwoYVtZYl09az1uZXcgVWIoYSkpO2M9ay5hZGQoYixjLGQsZyxmKTtpZihjLmEpcmV0dXJuIGM7ZD1nYygpO2MuYT1kO2Quc3JjPWE7ZC5saXN0ZW5lcj1jO2lmKGEuYWRkRXZlbnRMaXN0ZW5lcilMYnx8KGU9Zyksdm9pZCAwPT09ZSYmKGU9ITEpLGEuYWRkRXZlbnRMaXN0ZW5lcihiLnRvU3RyaW5nKCksCmQsZSk7ZWxzZSBpZihhLmF0dGFjaEV2ZW50KWEuYXR0YWNoRXZlbnQoaGMoYi50b1N0cmluZygpKSxkKTtlbHNlIGlmKGEuYWRkTGlzdGVuZXImJmEucmVtb3ZlTGlzdGVuZXIpcSgiY2hhbmdlIj09PWIsIk1lZGlhUXVlcnlMaXN0IG9ubHkgaGFzIGEgY2hhbmdlIGV2ZW50IiksYS5hZGRMaXN0ZW5lcihkKTtlbHNlIHRocm93IEVycm9yKCJhZGRFdmVudExpc3RlbmVyIGFuZCBhdHRhY2hFdmVudCBhcmUgdW5hdmFpbGFibGUuIik7JGIrKztyZXR1cm4gY30sZ2M9ZnVuY3Rpb24oKXt2YXIgYT1pYyxiPUpiP2Z1bmN0aW9uKGMpe3JldHVybiBhLmNhbGwoYi5zcmMsYi5saXN0ZW5lcixjKX06ZnVuY3Rpb24oYyl7Yz1hLmNhbGwoYi5zcmMsYi5saXN0ZW5lcixjKTtpZighYylyZXR1cm4gY307cmV0dXJuIGJ9LGFjPWZ1bmN0aW9uKGEsYixjLGQsZSl7aWYoZmEoYikpe2Zvcih2YXIgZj0wO2Y8Yi5sZW5ndGg7ZisrKWFjKGEsYltmXSxjLGQsZSk7cmV0dXJuIG51bGx9Yz1jYyhjKTtyZXR1cm4gUWIoYSk/CmEuaC5hZGQoU3RyaW5nKGIpLGMsITAsbShkKT8hIWQuY2FwdHVyZTohIWQsZSk6ZWMoYSxiLGMsITAsZCxlKX0samM9ZnVuY3Rpb24oYSxiLGMsZCxlKXtpZihmYShiKSlmb3IodmFyIGY9MDtmPGIubGVuZ3RoO2YrKylqYyhhLGJbZl0sYyxkLGUpO2Vsc2UgZD1tKGQpPyEhZC5jYXB0dXJlOiEhZCxjPWNjKGMpLFFiKGEpPyhhPWEuaCxiPVN0cmluZyhiKS50b1N0cmluZygpLGIgaW4gYS5hJiYoZj1hLmFbYl0sYz1WYihmLGMsZCxlKSwtMTxjJiYoVGIoZltjXSkscShudWxsIT1mLmxlbmd0aCksQXJyYXkucHJvdG90eXBlLnNwbGljZS5jYWxsKGYsYywxKSwwPT1mLmxlbmd0aCYmKGRlbGV0ZSBhLmFbYl0sYS5iLS0pKSkpOmEmJihhPWZjKGEpKSYmKGM9WGIoYSxiLGMsZCxlKSkmJmtjKGMpfSxrYz1mdW5jdGlvbihhKXtpZigibnVtYmVyIiE9PXR5cGVvZiBhJiZhJiYhYS5YKXt2YXIgYj1hLnNyYztpZihRYihiKSlXYihiLmgsYSk7ZWxzZXt2YXIgYz1hLnR5cGUsZD1hLmE7Yi5yZW1vdmVFdmVudExpc3RlbmVyPwpiLnJlbW92ZUV2ZW50TGlzdGVuZXIoYyxkLGEuY2FwdHVyZSk6Yi5kZXRhY2hFdmVudD9iLmRldGFjaEV2ZW50KGhjKGMpLGQpOmIuYWRkTGlzdGVuZXImJmIucmVtb3ZlTGlzdGVuZXImJmIucmVtb3ZlTGlzdGVuZXIoZCk7JGItLTsoYz1mYyhiKSk/KFdiKGMsYSksMD09Yy5iJiYoYy5zcmM9bnVsbCxiW1liXT1udWxsKSk6VGIoYSl9fX0saGM9ZnVuY3Rpb24oYSl7cmV0dXJuIGEgaW4gWmI/WmJbYV06WmJbYV09Im9uIithfSxtYz1mdW5jdGlvbihhLGIsYyxkKXt2YXIgZT0hMDtpZihhPWZjKGEpKWlmKGI9YS5hW2IudG9TdHJpbmcoKV0pZm9yKGI9Yi5jb25jYXQoKSxhPTA7YTxiLmxlbmd0aDthKyspe3ZhciBmPWJbYV07ZiYmZi5jYXB0dXJlPT1jJiYhZi5YJiYoZj1sYyhmLGQpLGU9ZSYmITEhPT1mKX1yZXR1cm4gZX0sbGM9ZnVuY3Rpb24oYSxiKXt2YXIgYz1hLmxpc3RlbmVyLGQ9YS5tYXx8YS5zcmM7YS5oYSYma2MoYSk7cmV0dXJuIGMuY2FsbChkLGIpfSxpYz1mdW5jdGlvbihhLApiKXtpZihhLlgpcmV0dXJuITA7aWYoIUpiKXtpZighYilhOntiPVsid2luZG93IiwiZXZlbnQiXTtmb3IodmFyIGM9bCxkPTA7ZDxiLmxlbmd0aDtkKyspaWYoYz1jW2JbZF1dLG51bGw9PWMpe2I9bnVsbDticmVhayBhfWI9Y31kPWI7Yj1uZXcgRyhkLHRoaXMpO2M9ITA7aWYoISgwPmQua2V5Q29kZXx8dm9pZCAwIT1kLnJldHVyblZhbHVlKSl7YTp7dmFyIGU9ITE7aWYoMD09ZC5rZXlDb2RlKXRyeXtkLmtleUNvZGU9LTE7YnJlYWsgYX1jYXRjaChnKXtlPSEwfWlmKGV8fHZvaWQgMD09ZC5yZXR1cm5WYWx1ZSlkLnJldHVyblZhbHVlPSEwfWQ9W107Zm9yKGU9Yi5hO2U7ZT1lLnBhcmVudE5vZGUpZC5wdXNoKGUpO2E9YS50eXBlO2ZvcihlPWQubGVuZ3RoLTE7IWIuaCYmMDw9ZTtlLS0pe2IuYT1kW2VdO3ZhciBmPW1jKGRbZV0sYSwhMCxiKTtjPWMmJmZ9Zm9yKGU9MDshYi5oJiZlPGQubGVuZ3RoO2UrKyliLmE9ZFtlXSxmPW1jKGRbZV0sYSwhMSxiKSxjPWMmJmZ9cmV0dXJuIGN9cmV0dXJuIGxjKGEsCm5ldyBHKGIsdGhpcykpfSxmYz1mdW5jdGlvbihhKXthPWFbWWJdO3JldHVybiBhIGluc3RhbmNlb2YgVWI/YTpudWxsfSxuYz0iX19jbG9zdXJlX2V2ZW50c19mbl8iKygxRTkqTWF0aC5yYW5kb20oKT4+PjApLGNjPWZ1bmN0aW9uKGEpe3EoYSwiTGlzdGVuZXIgY2FuIG5vdCBiZSBudWxsLiIpO2lmKGlhKGEpKXJldHVybiBhO3EoYS5oYW5kbGVFdmVudCwiQW4gb2JqZWN0IGxpc3RlbmVyIG11c3QgaGF2ZSBoYW5kbGVFdmVudCBtZXRob2QuIik7YVtuY118fChhW25jXT1mdW5jdGlvbihiKXtyZXR1cm4gYS5oYW5kbGVFdmVudChiKX0pO3JldHVybiBhW25jXX07dmFyIG9jPWZ1bmN0aW9uKGEpe0MuY2FsbCh0aGlzKTt0aGlzLmI9YTt0aGlzLmE9e319O24ob2MsQyk7CnZhciBwYz1bXSxIPWZ1bmN0aW9uKGEsYixjLGQpe2ZhKGMpfHwoYyYmKHBjWzBdPWMudG9TdHJpbmcoKSksYz1wYyk7Zm9yKHZhciBlPTA7ZTxjLmxlbmd0aDtlKyspe3ZhciBmPWJjKGIsY1tlXSxkfHxhLmhhbmRsZUV2ZW50LCExLGEuYnx8YSk7aWYoIWYpYnJlYWs7YS5hW2Yua2V5XT1mfXJldHVybiBhfSxxYz1mdW5jdGlvbihhLGIsYyxkLGUsZil7aWYoZmEoYykpZm9yKHZhciBnPTA7ZzxjLmxlbmd0aDtnKyspcWMoYSxiLGNbZ10sZCxlLGYpO2Vsc2UgZD1kfHxhLmhhbmRsZUV2ZW50LGU9bShlKT8hIWUuY2FwdHVyZTohIWUsZj1mfHxhLmJ8fGEsZD1jYyhkKSxlPSEhZSxjPVFiKGIpP1hiKGIuaCxTdHJpbmcoYyksZCxlLGYpOmI/KGI9ZmMoYikpP1hiKGIsYyxkLGUsZik6bnVsbDpudWxsLGMmJihrYyhjKSxkZWxldGUgYS5hW2Mua2V5XSk7cmV0dXJuIGF9LHJjPWZ1bmN0aW9uKGEpe29hKGEuYSxmdW5jdGlvbihiLGMpe3RoaXMuYS5oYXNPd25Qcm9wZXJ0eShjKSYma2MoYil9LAphKTthLmE9e319O29jLnByb3RvdHlwZS51PWZ1bmN0aW9uKCl7b2MuaS51LmNhbGwodGhpcyk7cmModGhpcyl9O29jLnByb3RvdHlwZS5oYW5kbGVFdmVudD1mdW5jdGlvbigpe3Rocm93IEVycm9yKCJFdmVudEhhbmRsZXIuaGFuZGxlRXZlbnQgbm90IGltcGxlbWVudGVkIik7fTt2YXIgST1mdW5jdGlvbigpe0MuY2FsbCh0aGlzKTt0aGlzLmg9bmV3IFViKHRoaXMpO3RoaXMuTWE9dGhpczt0aGlzLnBhPW51bGx9O24oSSxDKTtJLnByb3RvdHlwZVtQYl09ITA7SS5wcm90b3R5cGUucmE9ZnVuY3Rpb24oYSl7dGhpcy5wYT1hfTtJLnByb3RvdHlwZS5yZW1vdmVFdmVudExpc3RlbmVyPWZ1bmN0aW9uKGEsYixjLGQpe2pjKHRoaXMsYSxiLGMsZCl9Owp2YXIgdWM9ZnVuY3Rpb24oYSxiKXtzYyhhKTt2YXIgYz1hLnBhO2lmKGMpe3ZhciBkPVtdO2Zvcih2YXIgZT0xO2M7Yz1jLnBhKWQucHVzaChjKSxxKDFFMz4rK2UsImluZmluaXRlIGxvb3AiKX1hPWEuTWE7Yz1iLnR5cGV8fGI7InN0cmluZyI9PT10eXBlb2YgYj9iPW5ldyBFKGIsYSk6YiBpbnN0YW5jZW9mIEU/Yi50YXJnZXQ9Yi50YXJnZXR8fGE6KGU9YixiPW5ldyBFKGMsYSksdWEoYixlKSk7ZT0hMDtpZihkKWZvcih2YXIgZj1kLmxlbmd0aC0xOyFiLmgmJjA8PWY7Zi0tKXt2YXIgZz1iLmE9ZFtmXTtlPXRjKGcsYywhMCxiKSYmZX1iLmh8fChnPWIuYT1hLGU9dGMoZyxjLCEwLGIpJiZlLGIuaHx8KGU9dGMoZyxjLCExLGIpJiZlKSk7aWYoZClmb3IoZj0wOyFiLmgmJmY8ZC5sZW5ndGg7ZisrKWc9Yi5hPWRbZl0sZT10YyhnLGMsITEsYikmJmU7cmV0dXJuIGV9OwpJLnByb3RvdHlwZS51PWZ1bmN0aW9uKCl7SS5pLnUuY2FsbCh0aGlzKTtpZih0aGlzLmgpe3ZhciBhPXRoaXMuaCxiPTAsYztmb3IoYyBpbiBhLmEpe2Zvcih2YXIgZD1hLmFbY10sZT0wO2U8ZC5sZW5ndGg7ZSsrKSsrYixUYihkW2VdKTtkZWxldGUgYS5hW2NdO2EuYi0tfX10aGlzLnBhPW51bGx9O3ZhciBkYz1mdW5jdGlvbihhLGIsYyxkLGUpe3NjKGEpO3JldHVybiBhLmguYWRkKFN0cmluZyhiKSxjLCExLGQsZSl9LHRjPWZ1bmN0aW9uKGEsYixjLGQpe2I9YS5oLmFbU3RyaW5nKGIpXTtpZighYilyZXR1cm4hMDtiPWIuY29uY2F0KCk7Zm9yKHZhciBlPSEwLGY9MDtmPGIubGVuZ3RoOysrZil7dmFyIGc9YltmXTtpZihnJiYhZy5YJiZnLmNhcHR1cmU9PWMpe3ZhciBrPWcubGlzdGVuZXIscD1nLm1hfHxnLnNyYztnLmhhJiZXYihhLmgsZyk7ZT0hMSE9PWsuY2FsbChwLGQpJiZlfX1yZXR1cm4gZSYmMCE9ZC5LYX0sc2M9ZnVuY3Rpb24oYSl7cShhLmgsIkV2ZW50IHRhcmdldCBpcyBub3QgaW5pdGlhbGl6ZWQuIERpZCB5b3UgY2FsbCB0aGUgc3VwZXJjbGFzcyAoZ29vZy5ldmVudHMuRXZlbnRUYXJnZXQpIGNvbnN0cnVjdG9yPyIpfTt2YXIgeGM9ZnVuY3Rpb24oYSxiLGMsZCxlLGYpe2lmKHkmJiFBKCI1MjUiKSlyZXR1cm4hMDtpZih6JiZlKXJldHVybiB2YyhhKTtpZihlJiYhZClyZXR1cm4hMTtpZigheCl7Im51bWJlciI9PT10eXBlb2YgYiYmKGI9d2MoYikpO3ZhciBnPTE3PT1ifHwxOD09Ynx8eiYmOTE9PWI7aWYoKCFjfHx6KSYmZ3x8eiYmMTY9PWImJihkfHxmKSlyZXR1cm4hMX1pZigoeXx8YmIpJiZkJiZjKXN3aXRjaChhKXtjYXNlIDIyMDpjYXNlIDIxOTpjYXNlIDIyMTpjYXNlIDE5MjpjYXNlIDE4NjpjYXNlIDE4OTpjYXNlIDE4NzpjYXNlIDE4ODpjYXNlIDE5MDpjYXNlIDE5MTpjYXNlIDE5MjpjYXNlIDIyMjpyZXR1cm4hMX1pZih2JiZkJiZiPT1hKXJldHVybiExO3N3aXRjaChhKXtjYXNlIDEzOnJldHVybiB4P2Z8fGU/ITE6IShjJiZkKTohMDtjYXNlIDI3OnJldHVybiEoeXx8YmJ8fHgpfXJldHVybiB4JiYoZHx8ZXx8Zik/ITE6dmMoYSl9LHZjPWZ1bmN0aW9uKGEpe2lmKDQ4PD1hJiY1Nz49YXx8Cjk2PD1hJiYxMDY+PWF8fDY1PD1hJiY5MD49YXx8KHl8fGJiKSYmMD09YSlyZXR1cm4hMDtzd2l0Y2goYSl7Y2FzZSAzMjpjYXNlIDQzOmNhc2UgNjM6Y2FzZSA2NDpjYXNlIDEwNzpjYXNlIDEwOTpjYXNlIDExMDpjYXNlIDExMTpjYXNlIDE4NjpjYXNlIDU5OmNhc2UgMTg5OmNhc2UgMTg3OmNhc2UgNjE6Y2FzZSAxODg6Y2FzZSAxOTA6Y2FzZSAxOTE6Y2FzZSAxOTI6Y2FzZSAyMjI6Y2FzZSAyMTk6Y2FzZSAyMjA6Y2FzZSAyMjE6Y2FzZSAxNjM6Y2FzZSA1ODpyZXR1cm4hMDtjYXNlIDE3MzpyZXR1cm4geDtkZWZhdWx0OnJldHVybiExfX0sd2M9ZnVuY3Rpb24oYSl7aWYoeClhPXljKGEpO2Vsc2UgaWYoeiYmeSlzd2l0Y2goYSl7Y2FzZSA5MzphPTkxfXJldHVybiBhfSx5Yz1mdW5jdGlvbihhKXtzd2l0Y2goYSl7Y2FzZSA2MTpyZXR1cm4gMTg3O2Nhc2UgNTk6cmV0dXJuIDE4NjtjYXNlIDE3MzpyZXR1cm4gMTg5O2Nhc2UgMjI0OnJldHVybiA5MTtjYXNlIDA6cmV0dXJuIDIyNDsKZGVmYXVsdDpyZXR1cm4gYX19O3ZhciBKPWZ1bmN0aW9uKGEsYil7SS5jYWxsKHRoaXMpO2EmJnpjKHRoaXMsYSxiKX07bihKLEkpO2g9Si5wcm90b3R5cGU7aC5TPW51bGw7aC5uYT1udWxsO2guQ2E9bnVsbDtoLm9hPW51bGw7aC53PS0xO2guRj0tMTtoLnVhPSExOwp2YXIgQWM9ezM6MTMsMTI6MTQ0LDYzMjMyOjM4LDYzMjMzOjQwLDYzMjM0OjM3LDYzMjM1OjM5LDYzMjM2OjExMiw2MzIzNzoxMTMsNjMyMzg6MTE0LDYzMjM5OjExNSw2MzI0MDoxMTYsNjMyNDE6MTE3LDYzMjQyOjExOCw2MzI0MzoxMTksNjMyNDQ6MTIwLDYzMjQ1OjEyMSw2MzI0NjoxMjIsNjMyNDc6MTIzLDYzMjQ4OjQ0LDYzMjcyOjQ2LDYzMjczOjM2LDYzMjc1OjM1LDYzMjc2OjMzLDYzMjc3OjM0LDYzMjg5OjE0NCw2MzMwMjo0NX0sQmM9e1VwOjM4LERvd246NDAsTGVmdDozNyxSaWdodDozOSxFbnRlcjoxMyxGMToxMTIsRjI6MTEzLEYzOjExNCxGNDoxMTUsRjU6MTE2LEY2OjExNyxGNzoxMTgsRjg6MTE5LEY5OjEyMCxGMTA6MTIxLEYxMToxMjIsRjEyOjEyMywiVSswMDdGIjo0NixIb21lOjM2LEVuZDozNSxQYWdlVXA6MzMsUGFnZURvd246MzQsSW5zZXJ0OjQ1fSxDYz0heXx8QSgiNTI1IiksRGM9eiYmeDsKSi5wcm90b3R5cGUuYT1mdW5jdGlvbihhKXtpZih5fHxiYilpZigxNz09dGhpcy53JiYhYS5jdHJsS2V5fHwxOD09dGhpcy53JiYhYS5hbHRLZXl8fHomJjkxPT10aGlzLncmJiFhLm1ldGFLZXkpdGhpcy5GPXRoaXMudz0tMTstMT09dGhpcy53JiYoYS5jdHJsS2V5JiYxNyE9YS5jP3RoaXMudz0xNzphLmFsdEtleSYmMTghPWEuYz90aGlzLnc9MTg6YS5tZXRhS2V5JiY5MSE9YS5jJiYodGhpcy53PTkxKSk7Q2MmJiF4YyhhLmMsdGhpcy53LGEuc2hpZnRLZXksYS5jdHJsS2V5LGEuYWx0S2V5LGEubWV0YUtleSk/dGhpcy5oYW5kbGVFdmVudChhKToodGhpcy5GPXdjKGEuYyksRGMmJih0aGlzLnVhPWEuYWx0S2V5KSl9O0oucHJvdG90eXBlLmI9ZnVuY3Rpb24oYSl7dGhpcy5GPXRoaXMudz0tMTt0aGlzLnVhPWEuYWx0S2V5fTsKSi5wcm90b3R5cGUuaGFuZGxlRXZlbnQ9ZnVuY3Rpb24oYSl7dmFyIGI9YS5iLGM9Yi5hbHRLZXk7aWYodiYmImtleXByZXNzIj09YS50eXBlKXt2YXIgZD10aGlzLkY7dmFyIGU9MTMhPWQmJjI3IT1kP2Iua2V5Q29kZTowfWVsc2UoeXx8YmIpJiYia2V5cHJlc3MiPT1hLnR5cGU/KGQ9dGhpcy5GLGU9MDw9Yi5jaGFyQ29kZSYmNjMyMzI+Yi5jaGFyQ29kZSYmdmMoZCk/Yi5jaGFyQ29kZTowKTphYiYmIXk/KGQ9dGhpcy5GLGU9dmMoZCk/Yi5rZXlDb2RlOjApOigia2V5cHJlc3MiPT1hLnR5cGU/KERjJiYoYz10aGlzLnVhKSxiLmtleUNvZGU9PWIuY2hhckNvZGU/MzI+Yi5rZXlDb2RlPyhkPWIua2V5Q29kZSxlPTApOihkPXRoaXMuRixlPWIuY2hhckNvZGUpOihkPWIua2V5Q29kZXx8dGhpcy5GLGU9Yi5jaGFyQ29kZXx8MCkpOihkPWIua2V5Q29kZXx8dGhpcy5GLGU9Yi5jaGFyQ29kZXx8MCkseiYmNjM9PWUmJjIyND09ZCYmKGQ9MTkxKSk7dmFyIGY9ZD13YyhkKTtkPzYzMjMyPD0KZCYmZCBpbiBBYz9mPUFjW2RdOjI1PT1kJiZhLnNoaWZ0S2V5JiYoZj05KTpiLmtleUlkZW50aWZpZXImJmIua2V5SWRlbnRpZmllciBpbiBCYyYmKGY9QmNbYi5rZXlJZGVudGlmaWVyXSk7eCYmQ2MmJiJrZXlwcmVzcyI9PWEudHlwZSYmIXhjKGYsdGhpcy53LGEuc2hpZnRLZXksYS5jdHJsS2V5LGMsYS5tZXRhS2V5KXx8KGE9Zj09dGhpcy53LHRoaXMudz1mLGI9bmV3IEVjKGYsZSxhLGIpLGIuYWx0S2V5PWMsdWModGhpcyxiKSl9O0oucHJvdG90eXBlLmY9ZnVuY3Rpb24oKXtyZXR1cm4gdGhpcy5TfTsKdmFyIHpjPWZ1bmN0aW9uKGEsYixjKXthLm9hJiZGYyhhKTthLlM9YjthLm5hPWJjKGEuUywia2V5cHJlc3MiLGEsYyk7YS5DYT1iYyhhLlMsImtleWRvd24iLGEuYSxjLGEpO2Eub2E9YmMoYS5TLCJrZXl1cCIsYS5iLGMsYSl9LEZjPWZ1bmN0aW9uKGEpe2EubmEmJihrYyhhLm5hKSxrYyhhLkNhKSxrYyhhLm9hKSxhLm5hPW51bGwsYS5DYT1udWxsLGEub2E9bnVsbCk7YS5TPW51bGw7YS53PS0xO2EuRj0tMX07Si5wcm90b3R5cGUudT1mdW5jdGlvbigpe0ouaS51LmNhbGwodGhpcyk7RmModGhpcyl9O3ZhciBFYz1mdW5jdGlvbihhLGIsYyxkKXtHLmNhbGwodGhpcyxkKTt0aGlzLnR5cGU9ImtleSI7dGhpcy5jPWE7dGhpcy5yZXBlYXQ9Y307bihFYyxHKTt2YXIgR2M9eD8iTW96VXNlclNlbGVjdCI6eXx8YmI/IldlYmtpdFVzZXJTZWxlY3QiOm51bGwsSGM9ZnVuY3Rpb24oYSxiKXtiPWI/bnVsbDphLmdldEVsZW1lbnRzQnlUYWdOYW1lKCIqIik7aWYoR2Mpe3ZhciBjPSJub25lIjthLnN0eWxlJiYoYS5zdHlsZVtHY109Yyk7aWYoYil7YT0wO2Zvcih2YXIgZDtkPWJbYV07YSsrKWQuc3R5bGUmJihkLnN0eWxlW0djXT1jKX19ZWxzZSBpZih2fHxhYilpZihjPSJvbiIsYS5zZXRBdHRyaWJ1dGUoInVuc2VsZWN0YWJsZSIsYyksYilmb3IoYT0wO2Q9YlthXTthKyspZC5zZXRBdHRyaWJ1dGUoInVuc2VsZWN0YWJsZSIsYyl9O3ZhciBJYz1mdW5jdGlvbigpe307ZGEoSWMpO0ljLnByb3RvdHlwZS5hPTA7dmFyIEs9ZnVuY3Rpb24oYSl7SS5jYWxsKHRoaXMpO3RoaXMuaj1hfHx4YXx8KHhhPW5ldyB1Yik7dGhpcy5xYT1KYzt0aGlzLlk9bnVsbDt0aGlzLm09ITE7dGhpcy5hPW51bGw7dGhpcy5BPXZvaWQgMDt0aGlzLmc9dGhpcy5jPXRoaXMuYj1udWxsO3RoaXMuR2E9ITF9O24oSyxJKTtLLnByb3RvdHlwZS5PYT1JYy5SKCk7CnZhciBKYz1udWxsLEtjPWZ1bmN0aW9uKGEsYil7c3dpdGNoKGEpe2Nhc2UgMTpyZXR1cm4gYj8iZGlzYWJsZSI6ImVuYWJsZSI7Y2FzZSAyOnJldHVybiBiPyJoaWdobGlnaHQiOiJ1bmhpZ2hsaWdodCI7Y2FzZSA0OnJldHVybiBiPyJhY3RpdmF0ZSI6ImRlYWN0aXZhdGUiO2Nhc2UgODpyZXR1cm4gYj8ic2VsZWN0IjoidW5zZWxlY3QiO2Nhc2UgMTY6cmV0dXJuIGI/ImNoZWNrIjoidW5jaGVjayI7Y2FzZSAzMjpyZXR1cm4gYj8iZm9jdXMiOiJibHVyIjtjYXNlIDY0OnJldHVybiBiPyJvcGVuIjoiY2xvc2UifXRocm93IEVycm9yKCJJbnZhbGlkIGNvbXBvbmVudCBzdGF0ZSIpO30sTGM9ZnVuY3Rpb24oYSl7cmV0dXJuIGEuWXx8KGEuWT0iOiIrKGEuT2EuYSsrKS50b1N0cmluZygzNikpfSxNYz1mdW5jdGlvbihhLGIpe2lmKGEuYiYmYS5iLmcpe3ZhciBjPWEuYi5nLGQ9YS5ZO2QgaW4gYyYmZGVsZXRlIGNbZF07cWEoYS5iLmcsYixhKX1hLlk9Yn07Sy5wcm90b3R5cGUuZj1mdW5jdGlvbigpe3JldHVybiB0aGlzLmF9Owp2YXIgTmM9ZnVuY3Rpb24oYSl7YT1hLmE7cShhLCJDYW4gbm90IGNhbGwgZ2V0RWxlbWVudFN0cmljdCBiZWZvcmUgcmVuZGVyaW5nL2RlY29yYXRpbmcuIik7cmV0dXJuIGF9LE9jPWZ1bmN0aW9uKGEpe2EuQXx8KGEuQT1uZXcgb2MoYSkpO3JldHVybiBxKGEuQSl9O0sucHJvdG90eXBlLnJhPWZ1bmN0aW9uKGEpe2lmKHRoaXMuYiYmdGhpcy5iIT1hKXRocm93IEVycm9yKCJNZXRob2Qgbm90IHN1cHBvcnRlZCIpO0suaS5yYS5jYWxsKHRoaXMsYSl9O0sucHJvdG90eXBlLmlhPWZ1bmN0aW9uKCl7dGhpcy5hPW5iKHRoaXMuai5hLCJESVYiKX07CnZhciBQYz1mdW5jdGlvbihhLGIpe2lmKGEubSl0aHJvdyBFcnJvcigiQ29tcG9uZW50IGFscmVhZHkgcmVuZGVyZWQiKTtpZihiJiZhLnhhKGIpKXthLkdhPSEwO3ZhciBjPXFiKGIpO2EuaiYmYS5qLmE9PWN8fChhLmo9Yj9uZXcgdWIocWIoYikpOnhhfHwoeGE9bmV3IHViKSk7YS52YShiKTthLkQoKX1lbHNlIHRocm93IEVycm9yKCJJbnZhbGlkIGVsZW1lbnQgdG8gZGVjb3JhdGUiKTt9O2g9Sy5wcm90b3R5cGU7aC54YT1mdW5jdGlvbigpe3JldHVybiEwfTtoLnZhPWZ1bmN0aW9uKGEpe3RoaXMuYT1hfTtoLkQ9ZnVuY3Rpb24oKXt0aGlzLm09ITA7UWModGhpcyxmdW5jdGlvbihhKXshYS5tJiZhLmYoKSYmYS5EKCl9KX07aC5QPWZ1bmN0aW9uKCl7UWModGhpcyxmdW5jdGlvbihhKXthLm0mJmEuUCgpfSk7dGhpcy5BJiZyYyh0aGlzLkEpO3RoaXMubT0hMX07CmgudT1mdW5jdGlvbigpe3RoaXMubSYmdGhpcy5QKCk7dGhpcy5BJiYodGhpcy5BLk4oKSxkZWxldGUgdGhpcy5BKTtRYyh0aGlzLGZ1bmN0aW9uKGEpe2EuTigpfSk7IXRoaXMuR2EmJnRoaXMuYSYmb2IodGhpcy5hKTt0aGlzLmI9dGhpcy5hPXRoaXMuZz10aGlzLmM9bnVsbDtLLmkudS5jYWxsKHRoaXMpfTtoLnNhPWZ1bmN0aW9uKGEsYil7dGhpcy50YShhLFJjKHRoaXMpLGIpfTsKaC50YT1mdW5jdGlvbihhLGIsYyl7cSghIWEsIlByb3ZpZGVkIGVsZW1lbnQgbXVzdCBub3QgYmUgbnVsbC4iKTtpZihhLm0mJihjfHwhdGhpcy5tKSl0aHJvdyBFcnJvcigiQ29tcG9uZW50IGFscmVhZHkgcmVuZGVyZWQiKTtpZigwPmJ8fGI+UmModGhpcykpdGhyb3cgRXJyb3IoIkNoaWxkIGNvbXBvbmVudCBpbmRleCBvdXQgb2YgYm91bmRzIik7dGhpcy5nJiZ0aGlzLmN8fCh0aGlzLmc9e30sdGhpcy5jPVtdKTtpZihhLmI9PXRoaXMpe3ZhciBkPUxjKGEpO3RoaXMuZ1tkXT1hO0hhKHRoaXMuYyxhKX1lbHNlIHFhKHRoaXMuZyxMYyhhKSxhKTtpZihhPT10aGlzKXRocm93IEVycm9yKCJVbmFibGUgdG8gc2V0IHBhcmVudCBjb21wb25lbnQiKTtpZihkPXRoaXMmJmEuYiYmYS5ZKXt2YXIgZT1hLmI7ZD1hLlk7ZS5nJiZkPyhlPWUuZyxkPShudWxsIT09ZSYmZCBpbiBlP2VbZF06dm9pZCAwKXx8bnVsbCk6ZD1udWxsfWlmKGQmJmEuYiE9dGhpcyl0aHJvdyBFcnJvcigiVW5hYmxlIHRvIHNldCBwYXJlbnQgY29tcG9uZW50Iik7CmEuYj10aGlzO0suaS5yYS5jYWxsKGEsdGhpcyk7TGEodGhpcy5jLGIsMCxhKTtpZihhLm0mJnRoaXMubSYmYS5iPT10aGlzKWM9dGhpcy5qYSgpLGI9Yy5jaGlsZE5vZGVzW2JdfHxudWxsLGIhPWEuZigpJiZjLmluc2VydEJlZm9yZShhLmYoKSxiKTtlbHNlIGlmKGMpe3RoaXMuYXx8dGhpcy5pYSgpO2M9TCh0aGlzLGIrMSk7Yj10aGlzLmphKCk7Yz1jP2MuYTpudWxsO2lmKGEubSl0aHJvdyBFcnJvcigiQ29tcG9uZW50IGFscmVhZHkgcmVuZGVyZWQiKTthLmF8fGEuaWEoKTtiP2IuaW5zZXJ0QmVmb3JlKGEuYSxjfHxudWxsKTphLmouYS5ib2R5LmFwcGVuZENoaWxkKGEuYSk7YS5iJiYhYS5iLm18fGEuRCgpfWVsc2UgdGhpcy5tJiYhYS5tJiZhLmEmJmEuYS5wYXJlbnROb2RlJiYxPT1hLmEucGFyZW50Tm9kZS5ub2RlVHlwZSYmYS5EKCl9O2guamE9ZnVuY3Rpb24oKXtyZXR1cm4gdGhpcy5hfTsKdmFyIFNjPWZ1bmN0aW9uKGEpe2lmKG51bGw9PWEucWEpe3ZhciBiPWEubT9hLmE6YS5qLmEuYm9keTthOnt2YXIgYz1xYihiKTtpZihjLmRlZmF1bHRWaWV3JiZjLmRlZmF1bHRWaWV3LmdldENvbXB1dGVkU3R5bGUmJihjPWMuZGVmYXVsdFZpZXcuZ2V0Q29tcHV0ZWRTdHlsZShiLG51bGwpKSl7Yz1jLmRpcmVjdGlvbnx8Yy5nZXRQcm9wZXJ0eVZhbHVlKCJkaXJlY3Rpb24iKXx8IiI7YnJlYWsgYX1jPSIifWEucWE9InJ0bCI9PShjfHwoYi5jdXJyZW50U3R5bGU/Yi5jdXJyZW50U3R5bGUuZGlyZWN0aW9uOm51bGwpfHxiLnN0eWxlJiZiLnN0eWxlLmRpcmVjdGlvbil9cmV0dXJuIGEucWF9LFJjPWZ1bmN0aW9uKGEpe3JldHVybiBhLmM/YS5jLmxlbmd0aDowfSxMPWZ1bmN0aW9uKGEsYil7cmV0dXJuIGEuYz9hLmNbYl18fG51bGw6bnVsbH0sUWM9ZnVuY3Rpb24oYSxiLGMpe2EuYyYmcihhLmMsYixjKX0sVGM9ZnVuY3Rpb24oYSxiKXtyZXR1cm4gYS5jJiZiP0RhKGEuYyxiKTotMX07dmFyIFZjPWZ1bmN0aW9uKGEsYil7aWYoIWEpdGhyb3cgRXJyb3IoIkludmFsaWQgY2xhc3MgbmFtZSAiK2EpO2lmKCFpYShiKSl0aHJvdyBFcnJvcigiSW52YWxpZCBkZWNvcmF0b3IgZnVuY3Rpb24gIitiKTtVY1thXT1ifSxXYz17fSxVYz17fTt2YXIgWGM9ZnVuY3Rpb24oYSl7dGhpcy5oPWF9O2RhKFhjKTt2YXIgWWM9ZnVuY3Rpb24oYSxiKXthJiYoYS50YWJJbmRleD1iPzA6LTEpfSwkYz1mdW5jdGlvbihhLGIsYyl7Yy5pZCYmTWMoYixjLmlkKTt2YXIgZD1hLmIoKSxlPSExLGY9RChjKTtmJiZyKGYsZnVuY3Rpb24oZyl7Zz09ZD9lPSEwOmcmJnRoaXMuZyhiLGcsZCl9LGEpO2V8fEViKGMsZCk7WmMoYixjKTtyZXR1cm4gY307WGMucHJvdG90eXBlLmc9ZnVuY3Rpb24oYSxiLGMpe2I9PWMrIi1kaXNhYmxlZCI/YS5aKCExKTpiPT1jKyItaG9yaXpvbnRhbCI/YWQoYSwiaG9yaXpvbnRhbCIpOmI9PWMrIi12ZXJ0aWNhbCImJmFkKGEsInZlcnRpY2FsIil9Owp2YXIgWmM9ZnVuY3Rpb24oYSxiKXtpZihiKWZvcih2YXIgYz1iLmZpcnN0Q2hpbGQsZDtjJiZjLnBhcmVudE5vZGU9PWI7KXtkPWMubmV4dFNpYmxpbmc7aWYoMT09Yy5ub2RlVHlwZSl7YTp7dmFyIGU9YztxKGUpO2U9RChlKTtmb3IodmFyIGY9MCxnPWUubGVuZ3RoO2Y8ZztmKyspe3ZhciBrPWVbZl07aWYoaz1rIGluIFVjP1VjW2tdKCk6bnVsbCl7ZT1rO2JyZWFrIGF9fWU9bnVsbH1lJiYoZS5hPWMsYS5pc0VuYWJsZWQoKXx8ZS5aKCExKSxhLnNhKGUpLFBjKGUsYykpfWVsc2UgYy5ub2RlVmFsdWUmJiIiIT1NYShjLm5vZGVWYWx1ZSl8fGIucmVtb3ZlQ2hpbGQoYyk7Yz1kfX0sYmQ9ZnVuY3Rpb24oYSxiKXtiPWIuZigpO3EoYiwiVGhlIGNvbnRhaW5lciBET00gZWxlbWVudCBjYW5ub3QgYmUgbnVsbC4iKTtIYyhiLHgpO3YmJihiLmhpZGVGb2N1cz0hMCk7KGE9YS5oKSYmdmIoYixhKX07WGMucHJvdG90eXBlLmI9ZnVuY3Rpb24oKXtyZXR1cm4iZ29vZy1jb250YWluZXIifTsKWGMucHJvdG90eXBlLmM9ZnVuY3Rpb24oYSl7dmFyIGI9dGhpcy5iKCksYz1bYiwiaG9yaXpvbnRhbCI9PWEuSD9iKyItaG9yaXpvbnRhbCI6YisiLXZlcnRpY2FsIl07YS5pc0VuYWJsZWQoKXx8Yy5wdXNoKGIrIi1kaXNhYmxlZCIpO3JldHVybiBjfTt2YXIgTT1mdW5jdGlvbigpe30sY2Q7ZGEoTSk7dmFyIGRkPXtidXR0b246InByZXNzZWQiLGNoZWNrYm94OiJjaGVja2VkIixtZW51aXRlbToic2VsZWN0ZWQiLG1lbnVpdGVtY2hlY2tib3g6ImNoZWNrZWQiLG1lbnVpdGVtcmFkaW86ImNoZWNrZWQiLHJhZGlvOiJjaGVja2VkIix0YWI6InNlbGVjdGVkIix0cmVlaXRlbToic2VsZWN0ZWQifTtNLnByb3RvdHlwZS5oPWZ1bmN0aW9uKCl7fTtNLnByb3RvdHlwZS5jPWZ1bmN0aW9uKGEpe3JldHVybiBhLmouYigiRElWIixlZCh0aGlzLGEpLmpvaW4oIiAiKSxhLmJhKX07dmFyIGdkPWZ1bmN0aW9uKGEsYixjKXtpZihhPWEuZj9hLmYoKTphKXt2YXIgZD1bYl07diYmIUEoIjciKSYmKGQ9ZmQoRChhKSxiKSxkLnB1c2goYikpOyhjP0ZiOkhiKShhLGQpfX07Ck0ucHJvdG90eXBlLmc9ZnVuY3Rpb24oYSxiKXtiLmlkJiZNYyhhLGIuaWQpO2ImJmIuZmlyc3RDaGlsZD9oZChhLGIuZmlyc3RDaGlsZC5uZXh0U2libGluZz9KYShiLmNoaWxkTm9kZXMpOmIuZmlyc3RDaGlsZCk6YS5iYT1udWxsO3ZhciBjPTAsZD10aGlzLmEoKSxlPXRoaXMuYSgpLGY9ITEsZz0hMSxrPSExLHA9SmEoRChiKSk7cihwLGZ1bmN0aW9uKEIpe2Z8fEIhPWQ/Z3x8QiE9ZT9jfD1pZCh0aGlzLEIpOmc9ITA6KGY9ITAsZT09ZCYmKGc9ITApKTsxPT1pZCh0aGlzLEIpJiYoQWEoYiksc2IoYikmJnRiKGIpJiZyYihiLCExKSl9LHRoaXMpO2Eubz1jO2Z8fChwLnB1c2goZCksZT09ZCYmKGc9ITApKTtnfHxwLnB1c2goZSk7KGE9YS53YSkmJnAucHVzaC5hcHBseShwLGEpO2lmKHYmJiFBKCI3Iikpe3ZhciB3PWZkKHApOzA8dy5sZW5ndGgmJihwLnB1c2guYXBwbHkocCx3KSxrPSEwKX1mJiZnJiYhYSYmIWt8fENiKGIscC5qb2luKCIgIikpO3JldHVybiBifTsKdmFyIGpkPWZ1bmN0aW9uKGEsYil7aWYoYT1hLmgoKSl7cShiLCJUaGUgZWxlbWVudCBwYXNzZWQgYXMgYSBmaXJzdCBwYXJhbWV0ZXIgY2Fubm90IGJlIG51bGwuIik7dmFyIGM9Yi5nZXRBdHRyaWJ1dGUoInJvbGUiKXx8bnVsbDthIT1jJiZ2YihiLGEpfX0sa2Q9ZnVuY3Rpb24oYSxiKXt2YXIgYztpZihhLnYmMzImJihjPWEuZigpKSl7aWYoIWImJmEubyYzMil7dHJ5e2MuYmx1cigpfWNhdGNoKGQpe31hLm8mMzImJmEuRmEobnVsbCl9KHNiKGMpJiZ0YihjKSkhPWImJnJiKGMsYil9fSxsZD1mdW5jdGlvbihhLGIsYyl7Y2R8fChjZD17MToiZGlzYWJsZWQiLDg6InNlbGVjdGVkIiwxNjoiY2hlY2tlZCIsNjQ6ImV4cGFuZGVkIn0pO3EoYSwiVGhlIGVsZW1lbnQgcGFzc2VkIGFzIGEgZmlyc3QgcGFyYW1ldGVyIGNhbm5vdCBiZSBudWxsLiIpO2I9Y2RbYl07dmFyIGQ9YS5nZXRBdHRyaWJ1dGUoInJvbGUiKXx8bnVsbDtkJiYoZD1kZFtkXXx8YixiPSJjaGVja2VkIj09Ynx8InNlbGVjdGVkIj09CmI/ZDpiKTtiJiZ4YihhLGIsYyl9O00ucHJvdG90eXBlLmE9ZnVuY3Rpb24oKXtyZXR1cm4iZ29vZy1jb250cm9sIn07CnZhciBlZD1mdW5jdGlvbihhLGIpe3ZhciBjPWEuYSgpLGQ9W2NdLGU9YS5hKCk7ZSE9YyYmZC5wdXNoKGUpO2M9Yi5vO2ZvcihlPVtdO2M7KXt2YXIgZj1jJi1jO2UucHVzaChtZChhLGYpKTtjJj1+Zn1kLnB1c2guYXBwbHkoZCxlKTsoYT1iLndhKSYmZC5wdXNoLmFwcGx5KGQsYSk7diYmIUEoIjciKSYmZC5wdXNoLmFwcGx5KGQsZmQoZCkpO3JldHVybiBkfSxmZD1mdW5jdGlvbihhLGIpe3ZhciBjPVtdO2ImJihhPUlhKGEsW2JdKSk7cihbXSxmdW5jdGlvbihkKXshRmEoZCxsYShHYSxhKSl8fGImJiFHYShkLGIpfHxjLnB1c2goZC5qb2luKCJfIikpfSk7cmV0dXJuIGN9LG1kPWZ1bmN0aW9uKGEsYil7YS5ifHxuZChhKTtyZXR1cm4gYS5iW2JdfSxpZD1mdW5jdGlvbihhLGIpe2Euanx8KGEuYnx8bmQoYSksYS5qPXNhKGEuYikpO2E9cGFyc2VJbnQoYS5qW2JdLDEwKTtyZXR1cm4gaXNOYU4oYSk/MDphfSxuZD1mdW5jdGlvbihhKXt2YXIgYj1hLmEoKSxjPSF0KGIucmVwbGFjZSgvXHhhMHxccy9nLAoiICIpLCIgIik7cShjLCJDb250cm9sUmVuZGVyZXIgaGFzIGFuIGludmFsaWQgY3NzIGNsYXNzOiAnIitiKyInIik7YS5iPXsxOmIrIi1kaXNhYmxlZCIsMjpiKyItaG92ZXIiLDQ6YisiLWFjdGl2ZSIsODpiKyItc2VsZWN0ZWQiLDE2OmIrIi1jaGVja2VkIiwzMjpiKyItZm9jdXNlZCIsNjQ6YisiLW9wZW4ifX07dmFyIE49ZnVuY3Rpb24oYSxiLGMpe0suY2FsbCh0aGlzLGMpO2lmKCFiKXtiPXRoaXMuY29uc3RydWN0b3I7Zm9yKHZhciBkO2I7KXtkPWJbamFdfHwoYltqYV09KytrYSk7aWYoZD1XY1tkXSlicmVhaztiPWIuaT9iLmkuY29uc3RydWN0b3I6bnVsbH1iPWQ/aWEoZC5SKT9kLlIoKTpuZXcgZDpudWxsfXRoaXMuQj1iO3RoaXMuYmE9dm9pZCAwIT09YT9hOm51bGx9O24oTixLKTtoPU4ucHJvdG90eXBlO2guYmE9bnVsbDtoLm89MDtoLnY9Mzk7aC5LPTA7aC5JYT0hMDtoLndhPW51bGw7aC55YT0hMDtoLmlhPWZ1bmN0aW9uKCl7dmFyIGE9dGhpcy5CLmModGhpcyk7dGhpcy5hPWE7amQodGhpcy5CLGEpO0hjKGEsIXYmJiFhYik7dGhpcy5pc1Zpc2libGUoKXx8KGEuc3R5bGUuZGlzcGxheT0ibm9uZSIsYSYmeGIoYSwiaGlkZGVuIiwhMCkpfTtoLmphPWZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMuZigpfTtoLnhhPWZ1bmN0aW9uKCl7cmV0dXJuITB9OwpoLnZhPWZ1bmN0aW9uKGEpe3RoaXMuYT1hPXRoaXMuQi5nKHRoaXMsYSk7amQodGhpcy5CLGEpO0hjKGEsIXYmJiFhYik7dGhpcy5JYT0ibm9uZSIhPWEuc3R5bGUuZGlzcGxheX07CmguRD1mdW5jdGlvbigpe04uaS5ELmNhbGwodGhpcyk7dmFyIGE9TmModGhpcyk7cSh0aGlzKTtxKGEpO3RoaXMuaXNWaXNpYmxlKCl8fHhiKGEsImhpZGRlbiIsIXRoaXMuaXNWaXNpYmxlKCkpO3RoaXMuaXNFbmFibGVkKCl8fGxkKGEsMSwhdGhpcy5pc0VuYWJsZWQoKSk7dGhpcy52JjgmJmxkKGEsOCwhISh0aGlzLm8mOCkpO3RoaXMudiYxNiYmbGQoYSwxNiwhISh0aGlzLm8mMTYpKTt0aGlzLnYmNjQmJmxkKGEsNjQsISEodGhpcy5vJjY0KSk7YT10aGlzLkI7U2ModGhpcykmJmdkKHRoaXMuZigpLGEuYSgpKyItcnRsIiwhMCk7dGhpcy5pc0VuYWJsZWQoKSYma2QodGhpcyx0aGlzLmlzVmlzaWJsZSgpKTtpZih0aGlzLnYmLTImJih0aGlzLnlhJiZvZCh0aGlzLCEwKSx0aGlzLnYmMzImJihhPXRoaXMuZigpKSkpe3ZhciBiPXRoaXMuV3x8KHRoaXMuVz1uZXcgSik7emMoYixhKTtIKEgoSChPYyh0aGlzKSxiLCJrZXkiLHRoaXMuZmEpLGEsImZvY3VzIix0aGlzLk5hKSxhLCJibHVyIiwKdGhpcy5GYSl9fTt2YXIgb2Q9ZnVuY3Rpb24oYSxiKXt2YXIgYz1PYyhhKSxkPWEuZigpO2I/KEgoSChIKEgoYyxkLEYuTCxhLmthKSxkLFtGLk0sRi5hYV0sYS5sYSksZCwibW91c2VvdmVyIixhLkFhKSxkLCJtb3VzZW91dCIsYS56YSksYS5lYSE9Y2EmJkgoYyxkLCJjb250ZXh0bWVudSIsYS5lYSksdiYmKEEoOSl8fEgoYyxkLCJkYmxjbGljayIsYS5KYSksYS5nYXx8KGEuZ2E9bmV3IHBkKGEpLHpiKGEsbGEoQWIsYS5nYSkpKSkpOihxYyhxYyhxYyhxYyhjLGQsRi5MLGEua2EpLGQsW0YuTSxGLmFhXSxhLmxhKSxkLCJtb3VzZW92ZXIiLGEuQWEpLGQsIm1vdXNlb3V0IixhLnphKSxhLmVhIT1jYSYmcWMoYyxkLCJjb250ZXh0bWVudSIsYS5lYSksdiYmKEEoOSl8fHFjKGMsZCwiZGJsY2xpY2siLGEuSmEpLEFiKGEuZ2EpLGEuZ2E9bnVsbCkpfTsKTi5wcm90b3R5cGUuUD1mdW5jdGlvbigpe04uaS5QLmNhbGwodGhpcyk7dGhpcy5XJiZGYyh0aGlzLlcpO3RoaXMuaXNWaXNpYmxlKCkmJnRoaXMuaXNFbmFibGVkKCkmJmtkKHRoaXMsITEpfTtOLnByb3RvdHlwZS51PWZ1bmN0aW9uKCl7Ti5pLnUuY2FsbCh0aGlzKTt0aGlzLlcmJih0aGlzLlcuTigpLGRlbGV0ZSB0aGlzLlcpO2RlbGV0ZSB0aGlzLkI7dGhpcy5nYT10aGlzLndhPXRoaXMuYmE9bnVsbH07dmFyIGhkPWZ1bmN0aW9uKGEsYil7YS5iYT1ifTtOLnByb3RvdHlwZS5pc1Zpc2libGU9ZnVuY3Rpb24oKXtyZXR1cm4gdGhpcy5JYX07Ti5wcm90b3R5cGUuaXNFbmFibGVkPWZ1bmN0aW9uKCl7cmV0dXJuISh0aGlzLm8mMSl9OwpOLnByb3RvdHlwZS5aPWZ1bmN0aW9uKGEpe3ZhciBiPXRoaXMuYjtiJiYiZnVuY3Rpb24iPT10eXBlb2YgYi5pc0VuYWJsZWQmJiFiLmlzRW5hYmxlZCgpfHwhTyh0aGlzLDEsIWEpfHwoYXx8KHFkKHRoaXMsITEpLFAodGhpcywhMSkpLHRoaXMuaXNWaXNpYmxlKCkmJmtkKHRoaXMsYSksUSh0aGlzLDEsIWEsITApKX07CnZhciBQPWZ1bmN0aW9uKGEsYil7TyhhLDIsYikmJlEoYSwyLGIpfSxxZD1mdW5jdGlvbihhLGIpe08oYSw0LGIpJiZRKGEsNCxiKX0scmQ9ZnVuY3Rpb24oYSxiKXtPKGEsOCxiKSYmUShhLDgsYil9LHNkPWZ1bmN0aW9uKGEsYil7TyhhLDY0LGIpJiZRKGEsNjQsYil9LFE9ZnVuY3Rpb24oYSxiLGMsZCl7aWYoIWQmJjE9PWIpYS5aKCFjKTtlbHNlIGlmKGEudiZiJiZjIT0hIShhLm8mYikpe3ZhciBlPWEuQjtpZihkPWEuZigpKShlPW1kKGUsYikpJiZnZChhLGUsYyksbGQoZCxiLGMpO2Eubz1jP2Eub3xiOmEubyZ+Yn19LHRkPWZ1bmN0aW9uKGEsYixjKXtpZihhLm0mJmEubyZiJiYhYyl0aHJvdyBFcnJvcigiQ29tcG9uZW50IGFscmVhZHkgcmVuZGVyZWQiKTshYyYmYS5vJmImJlEoYSxiLCExKTthLnY9Yz9hLnZ8YjphLnYmfmJ9LFI9ZnVuY3Rpb24oYSxiKXtyZXR1cm4hISgyNTUmYikmJiEhKGEudiZiKX0sTz1mdW5jdGlvbihhLGIsYyl7cmV0dXJuISEoYS52JmIpJiYhIShhLm8mCmIpIT1jJiYoIShhLksmYil8fHVjKGEsS2MoYixjKSkpJiYhYS5VfTtoPU4ucHJvdG90eXBlO2guQWE9ZnVuY3Rpb24oYSl7KCFhLnJlbGF0ZWRUYXJnZXR8fCFwYih0aGlzLmYoKSxhLnJlbGF0ZWRUYXJnZXQpKSYmdWModGhpcywiZW50ZXIiKSYmdGhpcy5pc0VuYWJsZWQoKSYmUih0aGlzLDIpJiZQKHRoaXMsITApfTtoLnphPWZ1bmN0aW9uKGEpe2EucmVsYXRlZFRhcmdldCYmcGIodGhpcy5mKCksYS5yZWxhdGVkVGFyZ2V0KXx8IXVjKHRoaXMsImxlYXZlIil8fChSKHRoaXMsNCkmJnFkKHRoaXMsITEpLFIodGhpcywyKSYmUCh0aGlzLCExKSl9O2guZWE9Y2E7Cmgua2E9ZnVuY3Rpb24oYSl7aWYodGhpcy5pc0VuYWJsZWQoKSYmKFIodGhpcywyKSYmUCh0aGlzLCEwKSxPYihhKSYmISh5JiZ6JiZhLmN0cmxLZXkpKSl7Uih0aGlzLDQpJiZxZCh0aGlzLCEwKTt2YXIgYjtpZihiPXRoaXMuQil7dmFyIGM7Yj10aGlzLnYmMzImJihjPXRoaXMuZigpKT9zYihjKSYmdGIoYyk6ITF9YiYmdGhpcy5mKCkuZm9jdXMoKX0hT2IoYSl8fHkmJnomJmEuY3RybEtleXx8YS5nKCl9O2gubGE9ZnVuY3Rpb24oYSl7dGhpcy5pc0VuYWJsZWQoKSYmKFIodGhpcywyKSYmUCh0aGlzLCEwKSx0aGlzLm8mNCYmdWQodGhpcyxhKSYmUih0aGlzLDQpJiZxZCh0aGlzLCExKSl9O2guSmE9ZnVuY3Rpb24oYSl7dGhpcy5pc0VuYWJsZWQoKSYmdWQodGhpcyxhKX07CnZhciB1ZD1mdW5jdGlvbihhLGIpe2lmKFIoYSwxNikpe3ZhciBjPSEoYS5vJjE2KTtPKGEsMTYsYykmJlEoYSwxNixjKX1SKGEsOCkmJnJkKGEsITApO1IoYSw2NCkmJnNkKGEsIShhLm8mNjQpKTtjPW5ldyBFKCJhY3Rpb24iLGEpO2ImJihjLmFsdEtleT1iLmFsdEtleSxjLmN0cmxLZXk9Yi5jdHJsS2V5LGMubWV0YUtleT1iLm1ldGFLZXksYy5zaGlmdEtleT1iLnNoaWZ0S2V5LGMuQT1iLkEpO3JldHVybiB1YyhhLGMpfTtOLnByb3RvdHlwZS5OYT1mdW5jdGlvbigpe1IodGhpcywzMikmJk8odGhpcywzMiwhMCkmJlEodGhpcywzMiwhMCl9O04ucHJvdG90eXBlLkZhPWZ1bmN0aW9uKCl7Uih0aGlzLDQpJiZxZCh0aGlzLCExKTtSKHRoaXMsMzIpJiZPKHRoaXMsMzIsITEpJiZRKHRoaXMsMzIsITEpfTsKTi5wcm90b3R5cGUuZmE9ZnVuY3Rpb24oYSl7cmV0dXJuIHRoaXMuaXNWaXNpYmxlKCkmJnRoaXMuaXNFbmFibGVkKCkmJjEzPT1hLmMmJnVkKHRoaXMsYSk/KGEuZygpLGEuaigpLCEwKTohMX07aWYoIWlhKE4pKXRocm93IEVycm9yKCJJbnZhbGlkIGNvbXBvbmVudCBjbGFzcyAiK04pO2lmKCFpYShNKSl0aHJvdyBFcnJvcigiSW52YWxpZCByZW5kZXJlciBjbGFzcyAiK00pO3ZhciB2ZD1OW2phXXx8KE5bamFdPSsra2EpO1djW3ZkXT1NO1ZjKCJnb29nLWNvbnRyb2wiLGZ1bmN0aW9uKCl7cmV0dXJuIG5ldyBOKG51bGwpfSk7dmFyIHBkPWZ1bmN0aW9uKGEpe0MuY2FsbCh0aGlzKTt0aGlzLmI9YTt0aGlzLmE9ITE7dGhpcy5jPW5ldyBvYyh0aGlzKTt6Yih0aGlzLGxhKEFiLHRoaXMuYykpO2E9TmModGhpcy5iKTtIKEgoSCh0aGlzLmMsYSxGLkwsdGhpcy5oKSxhLEYuTSx0aGlzLmopLGEsImNsaWNrIix0aGlzLmcpfTtuKHBkLEMpO3ZhciB3ZD0hdnx8OTw9TnVtYmVyKGliKTsKcGQucHJvdG90eXBlLmg9ZnVuY3Rpb24oKXt0aGlzLmE9ITF9O3BkLnByb3RvdHlwZS5qPWZ1bmN0aW9uKCl7dGhpcy5hPSEwfTt2YXIgeGQ9ZnVuY3Rpb24oYSxiKXtpZighd2QpcmV0dXJuIGEuYnV0dG9uPTAsYS50eXBlPWIsYTt2YXIgYz1kb2N1bWVudC5jcmVhdGVFdmVudCgiTW91c2VFdmVudHMiKTtjLmluaXRNb3VzZUV2ZW50KGIsYS5idWJibGVzLGEuY2FuY2VsYWJsZSxhLnZpZXd8fG51bGwsYS5kZXRhaWwsYS5zY3JlZW5YLGEuc2NyZWVuWSxhLmNsaWVudFgsYS5jbGllbnRZLGEuY3RybEtleSxhLmFsdEtleSxhLnNoaWZ0S2V5LGEubWV0YUtleSwwLGEucmVsYXRlZFRhcmdldHx8bnVsbCk7cmV0dXJuIGN9OwpwZC5wcm90b3R5cGUuZz1mdW5jdGlvbihhKXtpZih0aGlzLmEpdGhpcy5hPSExO2Vsc2V7dmFyIGI9YS5iLGM9Yi5idXR0b24sZD1iLnR5cGUsZT14ZChiLCJtb3VzZWRvd24iKTt0aGlzLmIua2EobmV3IEcoZSxhLmEpKTtlPXhkKGIsIm1vdXNldXAiKTt0aGlzLmIubGEobmV3IEcoZSxhLmEpKTt3ZHx8KGIuYnV0dG9uPWMsYi50eXBlPWQpfX07cGQucHJvdG90eXBlLnU9ZnVuY3Rpb24oKXt0aGlzLmI9bnVsbDtwZC5pLnUuY2FsbCh0aGlzKX07dmFyIFM9ZnVuY3Rpb24oYSxiLGMpe0suY2FsbCh0aGlzLGMpO3RoaXMuZGE9Ynx8WGMuUigpO3RoaXMuSD1hfHwidmVydGljYWwifTtuKFMsSyk7aD1TLnByb3RvdHlwZTtoLkRhPW51bGw7aC5jYT1udWxsO2guZGE9bnVsbDtoLkg9bnVsbDtoLlQ9ITA7aC5PPSEwO2gubD0tMTtoLnM9bnVsbDtoLlY9ITE7aC5JPW51bGw7dmFyIHlkPWZ1bmN0aW9uKGEpe3JldHVybiBhLkRhfHxhLmYoKX07aD1TLnByb3RvdHlwZTtoLmlhPWZ1bmN0aW9uKCl7dGhpcy5hPXRoaXMuai5iKCJESVYiLHRoaXMuZGEuYyh0aGlzKS5qb2luKCIgIikpfTtoLmphPWZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMuZigpfTtoLnhhPWZ1bmN0aW9uKGEpe3JldHVybiJESVYiPT1hLnRhZ05hbWV9O2gudmE9ZnVuY3Rpb24oYSl7dGhpcy5hPSRjKHRoaXMuZGEsdGhpcyxhKTsibm9uZSI9PWEuc3R5bGUuZGlzcGxheSYmKHRoaXMuVD0hMSl9OwpoLkQ9ZnVuY3Rpb24oKXtTLmkuRC5jYWxsKHRoaXMpO1FjKHRoaXMsZnVuY3Rpb24oYil7Yi5tJiZ6ZCh0aGlzLGIpfSx0aGlzKTt2YXIgYT10aGlzLmYoKTtiZCh0aGlzLmRhLHRoaXMpO0FkKHRoaXMsdGhpcy5UKTtIKEgoSChIKEgoSChIKEgoT2ModGhpcyksdGhpcywiZW50ZXIiLHRoaXMuV2EpLHRoaXMsImhpZ2hsaWdodCIsdGhpcy5YYSksdGhpcywidW5oaWdobGlnaHQiLHRoaXMuY2IpLHRoaXMsIm9wZW4iLHRoaXMuWWEpLHRoaXMsImNsb3NlIix0aGlzLlVhKSxhLEYuTCx0aGlzLlNhKSxxYihhKSxbRi5NLEYuYWFdLHRoaXMuVmEpLGEsW0YuTCxGLk0sRi5hYSwibW91c2VvdmVyIiwibW91c2VvdXQiLCJjb250ZXh0bWVudSJdLHRoaXMuVGEpO0JkKHRoaXMpfTt2YXIgQmQ9ZnVuY3Rpb24oYSl7dmFyIGI9T2MoYSksYz15ZChhKTtIKEgoSChiLGMsImZvY3VzIixhLkhhKSxjLCJibHVyIixhLlFhKSxhLmNhfHwoYS5jYT1uZXcgSih5ZChhKSkpLCJrZXkiLGEuUmEpfTtoPVMucHJvdG90eXBlOwpoLlA9ZnVuY3Rpb24oKXtDZCh0aGlzLC0xKTt0aGlzLnMmJnNkKHRoaXMucywhMSk7dGhpcy5WPSExO1MuaS5QLmNhbGwodGhpcyl9O2gudT1mdW5jdGlvbigpe1MuaS51LmNhbGwodGhpcyk7dGhpcy5jYSYmKHRoaXMuY2EuTigpLHRoaXMuY2E9bnVsbCk7dGhpcy5kYT10aGlzLnM9dGhpcy5JPXRoaXMuRGE9bnVsbH07aC5XYT1mdW5jdGlvbigpe3JldHVybiEwfTsKaC5YYT1mdW5jdGlvbihhKXt2YXIgYj1UYyh0aGlzLGEudGFyZ2V0KTtpZigtMTxiJiZiIT10aGlzLmwpe3ZhciBjPUwodGhpcyx0aGlzLmwpO2MmJlAoYywhMSk7dGhpcy5sPWI7Yz1MKHRoaXMsdGhpcy5sKTt0aGlzLlYmJnFkKGMsITApO3RoaXMucyYmYyE9dGhpcy5zJiYoYy52JjY0P3NkKGMsITApOnNkKHRoaXMucywhMSkpfWI9dGhpcy5mKCk7cShiLCJUaGUgRE9NIGVsZW1lbnQgZm9yIHRoZSBjb250YWluZXIgY2Fubm90IGJlIG51bGwuIik7bnVsbCE9YS50YXJnZXQuZigpJiZ4YihiLCJhY3RpdmVkZXNjZW5kYW50IixhLnRhcmdldC5mKCkuaWQpfTtoLmNiPWZ1bmN0aW9uKGEpe2EudGFyZ2V0PT1MKHRoaXMsdGhpcy5sKSYmKHRoaXMubD0tMSk7YT10aGlzLmYoKTtxKGEsIlRoZSBET00gZWxlbWVudCBmb3IgdGhlIGNvbnRhaW5lciBjYW5ub3QgYmUgbnVsbC4iKTthLnJlbW92ZUF0dHJpYnV0ZSh3YigiYWN0aXZlZGVzY2VuZGFudCIpKX07CmguWWE9ZnVuY3Rpb24oYSl7KGE9YS50YXJnZXQpJiZhIT10aGlzLnMmJmEuYj09dGhpcyYmKHRoaXMucyYmc2QodGhpcy5zLCExKSx0aGlzLnM9YSl9O2guVWE9ZnVuY3Rpb24oYSl7YS50YXJnZXQ9PXRoaXMucyYmKHRoaXMucz1udWxsKTt2YXIgYj10aGlzLmYoKSxjPWEudGFyZ2V0LmYoKTtiJiZhLnRhcmdldC5vJjImJmMmJihhPSIiLGMmJihhPWMuaWQscShhLCJUaGUgYWN0aXZlIGVsZW1lbnQgc2hvdWxkIGhhdmUgYW4gaWQuIikpLHhiKGIsImFjdGl2ZWRlc2NlbmRhbnQiLGEpKX07aC5TYT1mdW5jdGlvbihhKXt0aGlzLk8mJih0aGlzLlY9ITApO3ZhciBiPXlkKHRoaXMpO2ImJnNiKGIpJiZ0YihiKT9iLmZvY3VzKCk6YS5nKCl9O2guVmE9ZnVuY3Rpb24oKXt0aGlzLlY9ITF9OwpoLlRhPWZ1bmN0aW9uKGEpe2E6e3ZhciBiPWEudGFyZ2V0O2lmKHRoaXMuSSlmb3IodmFyIGM9dGhpcy5mKCk7YiYmYiE9PWM7KXt2YXIgZD1iLmlkO2lmKGQgaW4gdGhpcy5JKXtiPXRoaXMuSVtkXTticmVhayBhfWI9Yi5wYXJlbnROb2RlfWI9bnVsbH1pZihiKXN3aXRjaChhLnR5cGUpe2Nhc2UgRi5MOmIua2EoYSk7YnJlYWs7Y2FzZSBGLk06Y2FzZSBGLmFhOmIubGEoYSk7YnJlYWs7Y2FzZSAibW91c2VvdmVyIjpiLkFhKGEpO2JyZWFrO2Nhc2UgIm1vdXNlb3V0IjpiLnphKGEpO2JyZWFrO2Nhc2UgImNvbnRleHRtZW51IjpiLmVhKGEpfX07aC5IYT1mdW5jdGlvbigpe307aC5RYT1mdW5jdGlvbigpe0NkKHRoaXMsLTEpO3RoaXMuVj0hMTt0aGlzLnMmJnNkKHRoaXMucywhMSl9OwpoLlJhPWZ1bmN0aW9uKGEpe3JldHVybiB0aGlzLmlzRW5hYmxlZCgpJiZ0aGlzLmlzVmlzaWJsZSgpJiYoMCE9UmModGhpcyl8fHRoaXMuRGEpJiZEZCh0aGlzLGEpPyhhLmcoKSxhLmooKSwhMCk6ITF9Owp2YXIgRGQ9ZnVuY3Rpb24oYSxiKXt2YXIgYz1MKGEsYS5sKTtpZihjJiYiZnVuY3Rpb24iPT10eXBlb2YgYy5mYSYmYy5mYShiKXx8YS5zJiZhLnMhPWMmJiJmdW5jdGlvbiI9PXR5cGVvZiBhLnMuZmEmJmEucy5mYShiKSlyZXR1cm4hMDtpZihiLnNoaWZ0S2V5fHxiLmN0cmxLZXl8fGIubWV0YUtleXx8Yi5hbHRLZXkpcmV0dXJuITE7c3dpdGNoKGIuYyl7Y2FzZSAyNzp5ZChhKS5ibHVyKCk7YnJlYWs7Y2FzZSAzNjpFZChhKTticmVhaztjYXNlIDM1OkZkKGEpO2JyZWFrO2Nhc2UgMzg6aWYoInZlcnRpY2FsIj09YS5IKUdkKGEpO2Vsc2UgcmV0dXJuITE7YnJlYWs7Y2FzZSAzNzppZigiaG9yaXpvbnRhbCI9PWEuSClTYyhhKT9IZChhKTpHZChhKTtlbHNlIHJldHVybiExO2JyZWFrO2Nhc2UgNDA6aWYoInZlcnRpY2FsIj09YS5IKUhkKGEpO2Vsc2UgcmV0dXJuITE7YnJlYWs7Y2FzZSAzOTppZigiaG9yaXpvbnRhbCI9PWEuSClTYyhhKT9HZChhKTpIZChhKTtlbHNlIHJldHVybiExOwpicmVhaztkZWZhdWx0OnJldHVybiExfXJldHVybiEwfSx6ZD1mdW5jdGlvbihhLGIpe3ZhciBjPWIuZigpO2M9Yy5pZHx8KGMuaWQ9TGMoYikpO2EuSXx8KGEuST17fSk7YS5JW2NdPWJ9O1MucHJvdG90eXBlLnNhPWZ1bmN0aW9uKGEsYil7Q2EoYSxOLCJUaGUgY2hpbGQgb2YgYSBjb250YWluZXIgbXVzdCBiZSBhIGNvbnRyb2wiKTtTLmkuc2EuY2FsbCh0aGlzLGEsYil9O1MucHJvdG90eXBlLnRhPWZ1bmN0aW9uKGEsYixjKXtDYShhLE4pO2EuS3w9MjthLkt8PTY0O3RkKGEsMzIsITEpO2EubSYmMCE9YS55YSYmb2QoYSwhMSk7YS55YT0hMTt2YXIgZD1hLmI9PXRoaXM/VGModGhpcyxhKTotMTtTLmkudGEuY2FsbCh0aGlzLGEsYixjKTthLm0mJnRoaXMubSYmemQodGhpcyxhKTthPWQ7LTE9PWEmJihhPVJjKHRoaXMpKTthPT10aGlzLmw/dGhpcy5sPU1hdGgubWluKFJjKHRoaXMpLTEsYik6YT50aGlzLmwmJmI8PXRoaXMubD90aGlzLmwrKzphPHRoaXMubCYmYj50aGlzLmwmJnRoaXMubC0tfTsKdmFyIGFkPWZ1bmN0aW9uKGEsYil7aWYoYS5mKCkpdGhyb3cgRXJyb3IoIkNvbXBvbmVudCBhbHJlYWR5IHJlbmRlcmVkIik7YS5IPWJ9O1MucHJvdG90eXBlLmlzVmlzaWJsZT1mdW5jdGlvbigpe3JldHVybiB0aGlzLlR9O3ZhciBBZD1mdW5jdGlvbihhLGIpe2EuVD1iO3ZhciBjPWEuZigpO2MmJihjLnN0eWxlLmRpc3BsYXk9Yj8iIjoibm9uZSIsWWMoeWQoYSksYS5PJiZhLlQpKX07Uy5wcm90b3R5cGUuaXNFbmFibGVkPWZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMuT307Uy5wcm90b3R5cGUuWj1mdW5jdGlvbihhKXt0aGlzLk8hPWEmJnVjKHRoaXMsYT8iZW5hYmxlIjoiZGlzYWJsZSIpJiYoYT8odGhpcy5PPSEwLFFjKHRoaXMsZnVuY3Rpb24oYil7Yi5MYT9kZWxldGUgYi5MYTpiLlooITApfSkpOihRYyh0aGlzLGZ1bmN0aW9uKGIpe2IuaXNFbmFibGVkKCk/Yi5aKCExKTpiLkxhPSEwfSksdGhpcy5WPXRoaXMuTz0hMSksWWMoeWQodGhpcyksYSYmdGhpcy5UKSl9Owp2YXIgQ2Q9ZnVuY3Rpb24oYSxiKXsoYj1MKGEsYikpP1AoYiwhMCk6LTE8YS5sJiZQKEwoYSxhLmwpLCExKX0sRWQ9ZnVuY3Rpb24oYSl7SWQoYSxmdW5jdGlvbihiLGMpe3JldHVybihiKzEpJWN9LFJjKGEpLTEpfSxGZD1mdW5jdGlvbihhKXtJZChhLGZ1bmN0aW9uKGIsYyl7Yi0tO3JldHVybiAwPmI/Yy0xOmJ9LDApfSxIZD1mdW5jdGlvbihhKXtJZChhLGZ1bmN0aW9uKGIsYyl7cmV0dXJuKGIrMSklY30sYS5sKX0sR2Q9ZnVuY3Rpb24oYSl7SWQoYSxmdW5jdGlvbihiLGMpe2ItLTtyZXR1cm4gMD5iP2MtMTpifSxhLmwpfSxJZD1mdW5jdGlvbihhLGIsYyl7Yz0wPmM/VGMoYSxhLnMpOmM7dmFyIGQ9UmMoYSk7Yz1iLmNhbGwoYSxjLGQpO2Zvcih2YXIgZT0wO2U8PWQ7KXt2YXIgZj1MKGEsYyksZztpZihnPWYpZz1mLmlzVmlzaWJsZSgpJiZmLmlzRW5hYmxlZCgpJiYhIShmLnYmMik7aWYoZyl7YS5FYShjKTticmVha31lKys7Yz1iLmNhbGwoYSxjLGQpfX07ClMucHJvdG90eXBlLkVhPWZ1bmN0aW9uKGEpe0NkKHRoaXMsYSl9O3ZhciBUPWZ1bmN0aW9uKCl7fTtuKFQsTSk7ZGEoVCk7VC5wcm90b3R5cGUuYT1mdW5jdGlvbigpe3JldHVybiJnb29nLXRhYiJ9O1QucHJvdG90eXBlLmg9ZnVuY3Rpb24oKXtyZXR1cm4idGFiIn07VC5wcm90b3R5cGUuYz1mdW5jdGlvbihhKXt2YXIgYj1ULmkuYy5jYWxsKHRoaXMsYSk7KGE9YS5QYSkmJmImJihiLnRpdGxlPWF8fCIiKTtyZXR1cm4gYn07VC5wcm90b3R5cGUuZz1mdW5jdGlvbihhLGIpe2I9VC5pLmcuY2FsbCh0aGlzLGEsYik7dmFyIGM9Yi50aXRsZXx8IiI7YyYmKGEuUGE9Yyk7YS5vJjgmJihjPWEuYikmJmlhKGMuRykmJihRKGEsOCwhMSksYy5HKGEpKTtyZXR1cm4gYn07dmFyIEpkPWZ1bmN0aW9uKGEsYixjKXtOLmNhbGwodGhpcyxhLGJ8fFQuUigpLGMpO3RkKHRoaXMsOCwhMCk7dGhpcy5LfD05fTtuKEpkLE4pO1ZjKCJnb29nLXRhYiIsZnVuY3Rpb24oKXtyZXR1cm4gbmV3IEpkKG51bGwpfSk7dmFyIFU9ZnVuY3Rpb24oKXt0aGlzLmg9InRhYmxpc3QifTtuKFUsWGMpO2RhKFUpO1UucHJvdG90eXBlLmI9ZnVuY3Rpb24oKXtyZXR1cm4iZ29vZy10YWItYmFyIn07VS5wcm90b3R5cGUuZz1mdW5jdGlvbihhLGIsYyl7dGhpcy5qfHwodGhpcy5hfHxLZCh0aGlzKSx0aGlzLmo9c2EodGhpcy5hKSk7dmFyIGQ9dGhpcy5qW2JdO2Q/KGFkKGEsTGQoZCkpLGEuQj1kKTpVLmkuZy5jYWxsKHRoaXMsYSxiLGMpfTtVLnByb3RvdHlwZS5jPWZ1bmN0aW9uKGEpe3ZhciBiPVUuaS5jLmNhbGwodGhpcyxhKTt0aGlzLmF8fEtkKHRoaXMpO2IucHVzaCh0aGlzLmFbYS5CXSk7cmV0dXJuIGJ9O3ZhciBLZD1mdW5jdGlvbihhKXt2YXIgYj1hLmIoKTthLmE9e3RvcDpiKyItdG9wIixib3R0b206YisiLWJvdHRvbSIsc3RhcnQ6YisiLXN0YXJ0IixlbmQ6YisiLWVuZCJ9fTt2YXIgVj1mdW5jdGlvbihhLGIsYyl7YT1hfHwidG9wIjthZCh0aGlzLExkKGEpKTt0aGlzLkI9YTtTLmNhbGwodGhpcyx0aGlzLkgsYnx8VS5SKCksYyk7TWQodGhpcyl9O24oVixTKTtoPVYucHJvdG90eXBlO2guQz1udWxsO2guRD1mdW5jdGlvbigpe1YuaS5ELmNhbGwodGhpcyk7TWQodGhpcyl9O2gudT1mdW5jdGlvbigpe1YuaS51LmNhbGwodGhpcyk7dGhpcy5DPW51bGx9O2guRWE9ZnVuY3Rpb24oYSl7Vi5pLkVhLmNhbGwodGhpcyxhKTt0aGlzLkcoTCh0aGlzLGEpKX07aC5HPWZ1bmN0aW9uKGEpe2E/cmQoYSwhMCk6dGhpcy5DJiZyZCh0aGlzLkMsITEpfTsKdmFyIE5kPWZ1bmN0aW9uKGEsYil7aWYoYiYmYj09YS5DKXtmb3IodmFyIGM9VGMoYSxiKSxkPWMtMTtiPUwoYSxkKTtkLS0pe3ZhciBlPWI7aWYoZS5pc1Zpc2libGUoKSYmZS5pc0VuYWJsZWQoKSl7YS5HKGIpO3JldHVybn19Zm9yKGMrPTE7Yj1MKGEsYyk7YysrKWlmKGQ9YixkLmlzVmlzaWJsZSgpJiZkLmlzRW5hYmxlZCgpKXthLkcoYik7cmV0dXJufWEuRyhudWxsKX19O2g9Vi5wcm90b3R5cGU7aC5hYj1mdW5jdGlvbihhKXt0aGlzLkMmJnRoaXMuQyE9YS50YXJnZXQmJnJkKHRoaXMuQywhMSk7dGhpcy5DPWEudGFyZ2V0fTtoLmJiPWZ1bmN0aW9uKGEpe2EudGFyZ2V0PT10aGlzLkMmJih0aGlzLkM9bnVsbCl9O2guWmE9ZnVuY3Rpb24oYSl7TmQodGhpcyxhLnRhcmdldCl9O2guJGE9ZnVuY3Rpb24oYSl7TmQodGhpcyxhLnRhcmdldCl9O2guSGE9ZnVuY3Rpb24oKXtMKHRoaXMsdGhpcy5sKXx8Q2QodGhpcyxUYyh0aGlzLHRoaXMuQ3x8TCh0aGlzLDApKSl9Owp2YXIgTWQ9ZnVuY3Rpb24oYSl7SChIKEgoSChPYyhhKSxhLCJzZWxlY3QiLGEuYWIpLGEsInVuc2VsZWN0IixhLmJiKSxhLCJkaXNhYmxlIixhLlphKSxhLCJoaWRlIixhLiRhKX0sTGQ9ZnVuY3Rpb24oYSl7cmV0dXJuInN0YXJ0Ij09YXx8ImVuZCI9PWE/InZlcnRpY2FsIjoiaG9yaXpvbnRhbCJ9O1ZjKCJnb29nLXRhYi1iYXIiLGZ1bmN0aW9uKCl7cmV0dXJuIG5ldyBWfSk7ZnVuY3Rpb24gT2QoYSl7dmFyIGI9e3RvcDoiYm90dG9tIixib3R0b206InRvcCIsc3RhcnQ6InJpZ2h0IixlbmQ6ImxlZnQifVthLmxvY2F0aW9uXSxjPWEuZWxlbWVudElkLGQ9ZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgic3R5bGUiKTtkLnRleHRDb250ZW50PSJcbiAgICBmaWVsZHNldCB7XG4gICAgICBwYWRkaW5nOiAxMHB4O1xuICAgICAgYm9yZGVyOiAxcHggc29saWQgIzM2OTtcbiAgICB9XG5cbiAgICAjIitjKyIgLmdvb2ctdGFiLWNvbnRlbnQge1xuICAgICAgbWluLWhlaWdodDogM2VtO1xuICAgICAgbWFyZ2luOiAwO1xuICAgICAgYm9yZGVyOiAiK2EuYm9yZGVyKyIgc29saWQgIithLmJvcmRlckNvbG9yKyI7XG4gICAgICBib3JkZXItdG9wOiAwO1xuICAgICAgaGVpZ2h0OiAiK2EuY29udGVudEhlaWdodCsiO1xuICAgICAgcGFkZGluZzogNHB4IDhweDtcbiAgICAgIG1hcmdpbi1yaWdodDogNHB4O1xuICAgICAgYmFja2dyb3VuZDogdmFyKC0tY29sYWItcHJpbWFyeS1zdXJmYWNlLWNvbG9yKTtcbiAgICAgIG92ZXJmbG93OiBhdXRvO1xuICAgIH1cblxuICAgICMiKwpjKyIgLmdvb2ctdGFiLWJhci0iK2EubG9jYXRpb24rIiAuZ29vZy10YWItc2VsZWN0ZWQge1xuICAgICAgYmFja2dyb3VuZC1jb2xvcjogdmFyKC0tY29sYWItcHJpbWFyeS1zdXJmYWNlLWNvbG9yKTtcbiAgICAgIGJvcmRlcjogMXB4IHNvbGlkICIrYS5ib3JkZXJDb2xvcisiO1xuICAgICAgYm9yZGVyLSIrYisiOiAwcHg7XG4gICAgfVxuXG4gICAgIyIrYysiIC5nb29nLXRhYi1iYXItIithLmxvY2F0aW9uKyIge1xuICAgICAgcGFkZGluZy0iK2EubG9jYXRpb24rIjogNXB4ICFpbXBvcnRhbnQ7XG4gICAgICBib3JkZXItIitiKyI6IDFweCBzb2xpZCAiK2EuYm9yZGVyQ29sb3IrIiAhaW1wb3J0YW50O1xuICAgICAgYmFja2dyb3VuZDogdmFyKC0tY29sYWItcHJpbWFyeS1zdXJmYWNlLWNvbG9yKTtcbiAgICB9XG5cbiAgICAjIitjKyIgLmdvb2ctdGFiLWJhciB7XG4gICAgICAgbWFyZ2luOiAwO1xuICAgICAgIGJvcmRlcjogMDtcbiAgICAgICBwYWRkaW5nOiAwO1xuICAgICAgIGxpc3Qtc3R5bGU6IG5vbmU7XG4gICAgICAgY3Vyc29yOiBkZWZhdWx0O1xuICAgICAgIG91dGxpbmU6IG5vbmU7XG4gICAgICAgYmFja2dyb3VuZDogdmFyKC0tY29sYWItcHJpbWFyeS1zdXJmYWNlLWNvbG9yKTtcbiAgICAgICBtYXJnaW4tcmlnaHQ6IDRweDtcbiAgICAgIH1cblxuICAgICAjIisKYysiIC5nb29nLXRhYiB7XG4gICAgICAgcG9zaXRpb246IHJlbGF0aXZlO1xuICAgICAgIHBhZGRpbmc6IDRweCA4cHg7XG4gICAgICAgY29sb3I6IHZhcigtLWNvbGFiLXByaW1hcnktdGV4dC1jb2xvcik7XG4gICAgICAgdGV4dC1kZWNvcmF0aW9uOiBpbml0aWFsO1xuICAgICAgIGN1cnNvcjogZGVmYXVsdDtcbiAgICAgIH1cblxuICAgICAgIyIrYysiIC5nb29nLXRhYi1ob3ZlciB7XG4gICAgICAgIGJhY2tncm91bmQtY29sb3I6IHZhcigtLWNvbGFiLWhpZ2hsaWdodGVkLXN1cmZhY2UtY29sb3IpO1xuICAgICAgfVxuICAgICAgIjtyZXR1cm4gZH0KdmFyIFBkPWZ1bmN0aW9uKGEpe3ZhciBiPWEuZWxlbWVudElkLGM9YS50YWJOYW1lcyxkPWEuc2VsZWN0ZWRJbmRleDsiY29udGVudEJvcmRlciJpbiBhfHwoYS5jb250ZW50Qm9yZGVyPSIwcHgiKTsiY29udGVudEhlaWdodCJpbiBhfHwoYS5jb250ZW50SGVpZ2h0PSJpbml0aWFsIik7ImJvcmRlckNvbG9yImluIGF8fChhLmJvcmRlckNvbG9yPSJ2YXIoLS1jb2xhYi1ib3JkZXItY29sb3IpIik7YS5sb2NhdGlvbnx8KGEubG9jYXRpb249InRvcCIpO3ZhciBlPWRvY3VtZW50LnF1ZXJ5U2VsZWN0b3IoIiMiK2IpLGY9ZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgiZGl2Iik7Zi5jbGFzc0xpc3QuYWRkKCJnb29nLXRhYi1iYXIiKTt2YXIgZz1hLmxvY2F0aW9uO2YuY2xhc3NMaXN0LmFkZCgiZ29vZy10YWItYmFyLSIrZyk7Zm9yKHZhciBrPVtdLHA9YmEoYyksdz1wLm5leHQoKTshdy5kb25lO3c9cC5uZXh0KCkpe3c9dy52YWx1ZTt2YXIgQj1kb2N1bWVudC5jcmVhdGVFbGVtZW50KCJkaXYiKTsKQi5jbGFzc0xpc3QuYWRkKCJnb29nLXRhYiIpO0IudGV4dENvbnRlbnQ9dztmLmFwcGVuZENoaWxkKEIpO2sucHVzaChCKX0iYm90dG9tIiE9ZyYmZS5hcHBlbmRDaGlsZChmKTtwPW51bGw7aWYoInRvcCI9PWd8fCJib3R0b20iPT1nKXA9ZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgiZGl2IikscC5jbGFzc0xpc3QuYWRkKCJnb29nLXRhYi1iYXItY2xlYXIiKTsidG9wIj09ZyYmcCYmZS5hcHBlbmRDaGlsZChwKTtCPWRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoImRpdiIpO0IuY2xhc3NMaXN0LmFkZCgiZ29vZy10YWItY29udGVudCIpO3ZhciByYT1bXTtjPWJhKGMpO2Zvcih3PWMubmV4dCgpOyF3LmRvbmU7dz1jLm5leHQoKSl3PWRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoImRpdiIpLHcuaWQ9ZS5pZCsiX2NvbnRlbnRfIityYS5sZW5ndGgsdy5zdHlsZS5kaXNwbGF5PSJub25lIixCLmFwcGVuZENoaWxkKHcpLHJhLnB1c2godyk7ZS5hcHBlbmRDaGlsZChCKTsiYm90dG9tIj09ZyYmKHAmJgplLmFwcGVuZENoaWxkKHApLGUuYXBwZW5kQ2hpbGQoZikpO3ZhciBYPW5ldyBWKGcpO1BjKFgsZik7dmFyIFk9LTE7ZGMoWCwic2VsZWN0IixmdW5jdGlvbihaKXtaPWsuaW5kZXhPZihaLnRhcmdldC5mKCkpO1ohPVkmJigwPD1ZJiZZPHJhLmxlbmd0aCYmKHJhW1ldLnN0eWxlLmRpc3BsYXk9Im5vbmUiLFk9LTEpLDA8PVomJlo8cmEubGVuZ3RoJiYoWT1aLHJhW1ldLnN0eWxlLmRpc3BsYXk9ImlubGluZSIsd2luZG93LmRpc3BhdGNoRXZlbnQobmV3IEV2ZW50KCJyZXNpemUiKSksZ29vZ2xlLmNvbGFiLm91dHB1dC5yZXNpemVJZnJhbWVUb0NvbnRlbnQoKSksWC5HKEwoWCxZKSkpfSk7WC5HKEwoWCxkKSk7d2luZG93W2JdPXtzZXRTZWxlY3RlZFRhYkluZGV4OmZ1bmN0aW9uKFope1guRyhMKFgsWikpfX07ZG9jdW1lbnQuaGVhZC5hcHBlbmRDaGlsZChPZChhKSl9LFFkPVsiY29sYWJfbGliIiwiY3JlYXRlVGFiQmFyIl0sVz1sOwpRZFswXWluIFd8fCJ1bmRlZmluZWQiPT10eXBlb2YgVy5leGVjU2NyaXB0fHxXLmV4ZWNTY3JpcHQoInZhciAiK1FkWzBdKTtmb3IodmFyIFJkO1FkLmxlbmd0aCYmKFJkPVFkLnNoaWZ0KCkpOylRZC5sZW5ndGh8fHZvaWQgMD09PVBkP1dbUmRdJiZXW1JkXSE9PU9iamVjdC5wcm90b3R5cGVbUmRdP1c9V1tSZF06Vz1XW1JkXT17fTpXW1JkXT1QZDt9KS5jYWxsKHRoaXMpOwovLyBjbGFuZy1mb3JtYXQgb24K","headers":[["content-type","application/javascript"]],"ok":true,"status":200,"status_text":""}}},"executionInfo":{"elapsed":1049,"status":"ok","timestamp":1712935210811,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"S0aT2rYMslyi","outputId":"593aa1e8-78af-4506-effe-af5a2b183fbf"},"outputs":[{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["<link rel=stylesheet type=text/css href='/nbextensions/google.colab/tabbar.css'></link>"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["<script src='/nbextensions/google.colab/tabbar_main.min.js'></script>"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["<div id=\"id2\"></div>"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"2689a342-f8e0-11ee-89e5-0242ac1c000c\"] = colab_lib.createTabBar({\"location\": \"top\", \"elementId\": \"id2\", \"tabNames\": [\"ner_posology\", \"ner_posology_greedy\", \"viz_posology\", \"viz_posology_greedy\"], \"initialSelection\": 0, \"contentBorder\": [\"0px\"], \"contentHeight\": [\"initial\"], \"borderColor\": [\"#a7a7a7\"]});\n","//# sourceURL=js_0d182b6b71"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"2689ec30-f8e0-11ee-89e5-0242ac1c000c\"] = window[\"id2\"].setSelectedTabIndex(0);\n","//# sourceURL=js_1bbecea4a1"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"268bd234-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.getActiveOutputArea();\n","//# sourceURL=js_07cb08bd2f"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"268c3382-f8e0-11ee-89e5-0242ac1c000c\"] = document.querySelector(\"#id2_content_0\");\n","//# sourceURL=js_ab2e944178"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"268c9106-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"268c3382-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_267748ec94"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"268cdc42-f8e0-11ee-89e5-0242ac1c000c\"] = window[\"id2\"].setSelectedTabIndex(0);\n","//# sourceURL=js_78ededc1fd"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["  entities_ner_chunk entities_ner_chunk_class entities_ner_chunk_begin  \\\n","0                  1                   DOSAGE                       27   \n","0            capsule                     FORM                       29   \n","0              Advil                     DRUG                       40   \n","0         for 5 days                 DURATION                       46   \n","0           40 units                   DOSAGE                      126   \n","0   insulin glargine                     DRUG                      138   \n","0           at night                FREQUENCY                      155   \n","0           12 units                   DOSAGE                      166   \n","0     insulin lispro                     DRUG                      178   \n","0         with meals                FREQUENCY                      193   \n","\n","  entities_ner_chunk_end  \n","0                     27  \n","0                     35  \n","0                     44  \n","0                     55  \n","0                    133  \n","0                    153  \n","0                    162  \n","0                    173  \n","0                    191  \n","0                    202  "],"text/html":["\n","  <div id=\"df-53c8b2b9-442a-46c4-bdc5-25353bd1eb6c\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>entities_ner_chunk</th>\n","      <th>entities_ner_chunk_class</th>\n","      <th>entities_ner_chunk_begin</th>\n","      <th>entities_ner_chunk_end</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>DOSAGE</td>\n","      <td>27</td>\n","      <td>27</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>capsule</td>\n","      <td>FORM</td>\n","      <td>29</td>\n","      <td>35</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>Advil</td>\n","      <td>DRUG</td>\n","      <td>40</td>\n","      <td>44</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>for 5 days</td>\n","      <td>DURATION</td>\n","      <td>46</td>\n","      <td>55</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>40 units</td>\n","      <td>DOSAGE</td>\n","      <td>126</td>\n","      <td>133</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>insulin glargine</td>\n","      <td>DRUG</td>\n","      <td>138</td>\n","      <td>153</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>at night</td>\n","      <td>FREQUENCY</td>\n","      <td>155</td>\n","      <td>162</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>12 units</td>\n","      <td>DOSAGE</td>\n","      <td>166</td>\n","      <td>173</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>insulin lispro</td>\n","      <td>DRUG</td>\n","      <td>178</td>\n","      <td>191</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>with meals</td>\n","      <td>FREQUENCY</td>\n","      <td>193</td>\n","      <td>202</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-53c8b2b9-442a-46c4-bdc5-25353bd1eb6c')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-53c8b2b9-442a-46c4-bdc5-25353bd1eb6c button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-53c8b2b9-442a-46c4-bdc5-25353bd1eb6c');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-b38e483f-f506-4d67-919a-a69391a6c38e\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b38e483f-f506-4d67-919a-a69391a6c38e')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-b38e483f-f506-4d67-919a-a69391a6c38e button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","summary":"{\n  \"name\": \"    visualiser\",\n  \"rows\": 10,\n  \"fields\": [\n    {\n      \"column\": \"entities_ner_chunk\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 10,\n        \"samples\": [\n          \"insulin lispro\",\n          \"capsule\",\n          \"insulin glargine\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_class\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"FORM\",\n          \"FREQUENCY\",\n          \"DRUG\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_begin\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 27,\n        \"max\": 193,\n        \"num_unique_values\": 10,\n        \"samples\": [\n          178,\n          29,\n          138\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_end\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 27,\n        \"max\": 202,\n        \"num_unique_values\": 10,\n        \"samples\": [\n          191,\n          35,\n          153\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"26907d20-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"268bd234-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_b380798799"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"2692be6e-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.getActiveOutputArea();\n","//# sourceURL=js_5f43418a24"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"26930f36-f8e0-11ee-89e5-0242ac1c000c\"] = document.querySelector(\"#id2_content_1\");\n","//# sourceURL=js_cf4ad85a85"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"26935f0e-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"26930f36-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_aacbe2569d"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"2693aa18-f8e0-11ee-89e5-0242ac1c000c\"] = window[\"id2\"].setSelectedTabIndex(1);\n","//# sourceURL=js_9bc62bb468"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["      entities_ner_chunk_greedy entities_ner_chunk_greedy_class  \\\n","0            1 capsule of Advil                            DRUG   \n","0                    for 5 days                        DURATION   \n","0  40 units of insulin glargine                            DRUG   \n","0                      at night                       FREQUENCY   \n","0    12 units of insulin lispro                            DRUG   \n","0                    with meals                       FREQUENCY   \n","0             metformin 1000 mg                            DRUG   \n","0               two times a day                       FREQUENCY   \n","0              SGLT2 inhibitors                            DRUG   \n","0                  for 3 months                        DURATION   \n","\n","  entities_ner_chunk_greedy_begin entities_ner_chunk_greedy_end  \n","0                              27                            44  \n","0                              46                            55  \n","0                             126                           153  \n","0                             155                           162  \n","0                             166                           191  \n","0                             193                           202  \n","0                             210                           226  \n","0                             228                           242  \n","0                             273                           288  \n","0                             326                           337  "],"text/html":["\n","  <div id=\"df-04a93bc4-c489-46d1-bc7a-4e5216b4d6c0\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>entities_ner_chunk_greedy</th>\n","      <th>entities_ner_chunk_greedy_class</th>\n","      <th>entities_ner_chunk_greedy_begin</th>\n","      <th>entities_ner_chunk_greedy_end</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>1 capsule of Advil</td>\n","      <td>DRUG</td>\n","      <td>27</td>\n","      <td>44</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>for 5 days</td>\n","      <td>DURATION</td>\n","      <td>46</td>\n","      <td>55</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>40 units of insulin glargine</td>\n","      <td>DRUG</td>\n","      <td>126</td>\n","      <td>153</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>at night</td>\n","      <td>FREQUENCY</td>\n","      <td>155</td>\n","      <td>162</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>12 units of insulin lispro</td>\n","      <td>DRUG</td>\n","      <td>166</td>\n","      <td>191</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>with meals</td>\n","      <td>FREQUENCY</td>\n","      <td>193</td>\n","      <td>202</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>metformin 1000 mg</td>\n","      <td>DRUG</td>\n","      <td>210</td>\n","      <td>226</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>two times a day</td>\n","      <td>FREQUENCY</td>\n","      <td>228</td>\n","      <td>242</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>SGLT2 inhibitors</td>\n","      <td>DRUG</td>\n","      <td>273</td>\n","      <td>288</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>for 3 months</td>\n","      <td>DURATION</td>\n","      <td>326</td>\n","      <td>337</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-04a93bc4-c489-46d1-bc7a-4e5216b4d6c0')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-04a93bc4-c489-46d1-bc7a-4e5216b4d6c0 button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-04a93bc4-c489-46d1-bc7a-4e5216b4d6c0');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-e3788426-c164-404f-9d00-a6d69b577aad\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e3788426-c164-404f-9d00-a6d69b577aad')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-e3788426-c164-404f-9d00-a6d69b577aad button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","summary":"{\n  \"name\": \"    visualiser\",\n  \"rows\": 10,\n  \"fields\": [\n    {\n      \"column\": \"entities_ner_chunk_greedy\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 10,\n        \"samples\": [\n          \"SGLT2 inhibitors\",\n          \"for 5 days\",\n          \"with meals\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_greedy_class\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"DRUG\",\n          \"DURATION\",\n          \"FREQUENCY\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_greedy_begin\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 27,\n        \"max\": 326,\n        \"num_unique_values\": 10,\n        \"samples\": [\n          273,\n          46,\n          193\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_greedy_end\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 44,\n        \"max\": 337,\n        \"num_unique_values\": 10,\n        \"samples\": [\n          288,\n          55,\n          202\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"26971432-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"2692be6e-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_5c74cc0308"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"2699a21a-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.getActiveOutputArea();\n","//# sourceURL=js_4e0b1e5b10"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"269a0318-f8e0-11ee-89e5-0242ac1c000c\"] = document.querySelector(\"#id2_content_2\");\n","//# sourceURL=js_28557ae651"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"269a6524-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"269a0318-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_491e2d52d1"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"269acb54-f8e0-11ee-89e5-0242ac1c000c\"] = window[\"id2\"].setSelectedTabIndex(2);\n","//# sourceURL=js_32dc27fdbf"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["\n","<style>\n","    @import url('https://fonts.googleapis.com/css2?family=Montserrat:wght@300;400;500;600;700&display=swap');\n","    @import url('https://fonts.googleapis.com/css2?family=Vistol Regular:wght@300;400;500;600;700&display=swap');\n","    \n","    .spark-nlp-display-scroll-entities {\n","        border: 1px solid #E7EDF0;\n","        border-radius: 3px;\n","        text-align: justify;\n","        \n","    }\n","    .spark-nlp-display-scroll-entities span {  \n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #536B76;\n","        font-family: 'Montserrat', sans-serif !important;\n","    }\n","    \n","    .spark-nlp-display-entity-wrapper{\n","    \n","        display: inline-grid;\n","        text-align: center;\n","        border-radius: 4px;\n","        margin: 0 2px 5px 2px;\n","        padding: 1px\n","    }\n","    .spark-nlp-display-entity-name{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        background: #f1f2f3;\n","        border-width: medium;\n","        text-align: center;\n","        \n","        font-weight: 400;\n","        \n","        border-radius: 5px;\n","        padding: 2px 5px;\n","        display: block;\n","        margin: 3px 2px;\n","    \n","    }\n","    .spark-nlp-display-entity-type{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-entity-resolution{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Vistol Regular', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-others{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        font-weight: 400;\n","    }\n","\n","</style>\n"," <span class=\"spark-nlp-display-others\" style=\"background-color: white\">The patient was prescribed </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #3DB198\"><span class=\"spark-nlp-display-entity-name\">1 </span><span class=\"spark-nlp-display-entity-type\">DOSAGE</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #3424BB\"><span class=\"spark-nlp-display-entity-name\">capsule </span><span class=\"spark-nlp-display-entity-type\">FORM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8B668B\"><span class=\"spark-nlp-display-entity-name\">Advil </span><span class=\"spark-nlp-display-entity-type\">DRUG</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #C60C17\"><span class=\"spark-nlp-display-entity-name\">for 5 days </span><span class=\"spark-nlp-display-entity-type\">DURATION</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> . He was seen by the endocrinology service and she was discharged on </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #3DB198\"><span class=\"spark-nlp-display-entity-name\">40 units </span><span class=\"spark-nlp-display-entity-type\">DOSAGE</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8B668B\"><span class=\"spark-nlp-display-entity-name\">insulin glargine </span><span class=\"spark-nlp-display-entity-type\">DRUG</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #658F85\"><span class=\"spark-nlp-display-entity-name\">at night </span><span class=\"spark-nlp-display-entity-type\">FREQUENCY</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #3DB198\"><span class=\"spark-nlp-display-entity-name\">12 units </span><span class=\"spark-nlp-display-entity-type\">DOSAGE</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8B668B\"><span class=\"spark-nlp-display-entity-name\">insulin lispro </span><span class=\"spark-nlp-display-entity-type\">DRUG</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #658F85\"><span class=\"spark-nlp-display-entity-name\">with meals </span><span class=\"spark-nlp-display-entity-type\">FREQUENCY</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> , and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8B668B\"><span class=\"spark-nlp-display-entity-name\">metformin </span><span class=\"spark-nlp-display-entity-type\">DRUG</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #A2AB7A\"><span class=\"spark-nlp-display-entity-name\">1000 mg </span><span class=\"spark-nlp-display-entity-type\">STRENGTH</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #658F85\"><span class=\"spark-nlp-display-entity-name\">two times a day </span><span class=\"spark-nlp-display-entity-type\">FREQUENCY</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> . It was determined that all </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8B668B\"><span class=\"spark-nlp-display-entity-name\">SGLT2 inhibitors </span><span class=\"spark-nlp-display-entity-type\">DRUG</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> should be discontinued indefinitely </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #C60C17\"><span class=\"spark-nlp-display-entity-name\">for 3 months </span><span class=\"spark-nlp-display-entity-type\">DURATION</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> .</span></div>"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"269d13b4-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"2699a21a-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_264e1594b7"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"269fbefc-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.getActiveOutputArea();\n","//# sourceURL=js_300e868790"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"26a034d6-f8e0-11ee-89e5-0242ac1c000c\"] = document.querySelector(\"#id2_content_3\");\n","//# sourceURL=js_ed6498b6af"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"26a09c96-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"26a034d6-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_ac23a9de15"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"26a1049c-f8e0-11ee-89e5-0242ac1c000c\"] = window[\"id2\"].setSelectedTabIndex(3);\n","//# sourceURL=js_5d95bfc20d"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["\n","<style>\n","    @import url('https://fonts.googleapis.com/css2?family=Montserrat:wght@300;400;500;600;700&display=swap');\n","    @import url('https://fonts.googleapis.com/css2?family=Vistol Regular:wght@300;400;500;600;700&display=swap');\n","    \n","    .spark-nlp-display-scroll-entities {\n","        border: 1px solid #E7EDF0;\n","        border-radius: 3px;\n","        text-align: justify;\n","        \n","    }\n","    .spark-nlp-display-scroll-entities span {  \n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #536B76;\n","        font-family: 'Montserrat', sans-serif !important;\n","    }\n","    \n","    .spark-nlp-display-entity-wrapper{\n","    \n","        display: inline-grid;\n","        text-align: center;\n","        border-radius: 4px;\n","        margin: 0 2px 5px 2px;\n","        padding: 1px\n","    }\n","    .spark-nlp-display-entity-name{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        background: #f1f2f3;\n","        border-width: medium;\n","        text-align: center;\n","        \n","        font-weight: 400;\n","        \n","        border-radius: 5px;\n","        padding: 2px 5px;\n","        display: block;\n","        margin: 3px 2px;\n","    \n","    }\n","    .spark-nlp-display-entity-type{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-entity-resolution{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Vistol Regular', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-others{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        font-weight: 400;\n","    }\n","\n","</style>\n"," <span class=\"spark-nlp-display-others\" style=\"background-color: white\">The patient was prescribed </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8B668B\"><span class=\"spark-nlp-display-entity-name\">1 capsule of Advil </span><span class=\"spark-nlp-display-entity-type\">DRUG</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #6F491B\"><span class=\"spark-nlp-display-entity-name\">for 5 days </span><span class=\"spark-nlp-display-entity-type\">DURATION</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> . He was seen by the endocrinology service and she was discharged on </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8B668B\"><span class=\"spark-nlp-display-entity-name\">40 units of insulin glargine </span><span class=\"spark-nlp-display-entity-type\">DRUG</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #3291C6\"><span class=\"spark-nlp-display-entity-name\">at night </span><span class=\"spark-nlp-display-entity-type\">FREQUENCY</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8B668B\"><span class=\"spark-nlp-display-entity-name\">12 units of insulin lispro </span><span class=\"spark-nlp-display-entity-type\">DRUG</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #3291C6\"><span class=\"spark-nlp-display-entity-name\">with meals </span><span class=\"spark-nlp-display-entity-type\">FREQUENCY</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> , and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8B668B\"><span class=\"spark-nlp-display-entity-name\">metformin 1000 mg </span><span class=\"spark-nlp-display-entity-type\">DRUG</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #3291C6\"><span class=\"spark-nlp-display-entity-name\">two times a day </span><span class=\"spark-nlp-display-entity-type\">FREQUENCY</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> . It was determined that all </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8B668B\"><span class=\"spark-nlp-display-entity-name\">SGLT2 inhibitors </span><span class=\"spark-nlp-display-entity-type\">DRUG</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> should be discontinued indefinitely </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #6F491B\"><span class=\"spark-nlp-display-entity-name\">for 3 months </span><span class=\"spark-nlp-display-entity-type\">DURATION</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> .</span></div>"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["window[\"26a2fe3c-f8e0-11ee-89e5-0242ac1c000c\"] = google.colab.output.setActiveOutputArea(window[\"269fbefc-f8e0-11ee-89e5-0242ac1c000c\"]);\n","//# sourceURL=js_b868fead1c"]},"metadata":{}}],"source":["posology_light_result= posology_light_model.fullAnnotate(text)\n","\n","from google.colab import widgets\n","\n","t = widgets.TabBar([\"ner_posology\", \"ner_posology_greedy\", \"viz_posology\", \"viz_posology_greedy\"])\n","\n","with t.output_to(0):\n","    display(posology_result_df.head(10))\n","\n","with t.output_to(1):\n","    display(posology_result_greedy_df.head(10))\n","\n","with t.output_to(2):\n","    visualiser.display(posology_light_result[0], label_col='ner_chunk', document_col='document')\n","\n","with t.output_to(3):\n","    visualiser.display(posology_light_result[0], label_col='ner_chunk_greedy', document_col='document')"]},{"cell_type":"markdown","metadata":{"id":"0vWIDpk50QfG"},"source":["## Writing a generic NER function"]},{"cell_type":"markdown","metadata":{"id":"mDXfsBokuq4h"},"source":["**Generic NER Function with LightPipeline**"]},{"cell_type":"code","execution_count":56,"metadata":{"id":"kd3wwfajfyDb","executionInfo":{"status":"ok","timestamp":1712935210811,"user_tz":-180,"elapsed":5,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["def get_light_model(embeddings, model_name = 'ner_clinical'):\n","\n","  documentAssembler = nlp.DocumentAssembler()\\\n","      .setInputCol(\"text\")\\\n","      .setOutputCol(\"document\")\n","\n","  sentenceDetector = nlp.SentenceDetector()\\\n","      .setInputCols([\"document\"])\\\n","      .setOutputCol(\"sentence\")\n","\n","  tokenizer = nlp.Tokenizer()\\\n","      .setInputCols([\"sentence\"])\\\n","      .setOutputCol(\"token\")\n","\n","  word_embeddings = nlp.WordEmbeddingsModel.pretrained(embeddings, \"en\", \"clinical/models\")\\\n","      .setInputCols([\"sentence\", \"token\"])\\\n","      .setOutputCol(\"embeddings\")\n","\n","  loaded_ner_model = medical.NerModel.pretrained(model_name, \"en\", \"clinical/models\") \\\n","      .setInputCols([\"sentence\", \"token\", \"embeddings\"]) \\\n","      .setOutputCol(\"ner\")\n","\n","  ner_converter = medical.NerConverterInternal() \\\n","      .setInputCols([\"sentence\", \"token\", \"ner\"]) \\\n","      .setOutputCol(\"ner_chunk\")\n","\n","  nlpPipeline = nlp.Pipeline(stages=[\n","      documentAssembler,\n","      sentenceDetector,\n","      tokenizer,\n","      word_embeddings,\n","      loaded_ner_model,\n","      ner_converter])\n","\n","  model = nlpPipeline.fit(spark.createDataFrame([[\"\"]]).toDF(\"text\"))\n","\n","  return nlp.LightPipeline(model)"]},{"cell_type":"code","execution_count":57,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":9793,"status":"ok","timestamp":1712935220600,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"V81zaAe13qLU","outputId":"0feed353-7d13-44db-c190-d6b7c0b7e17a"},"outputs":[{"output_type":"stream","name":"stdout","text":["embeddings_clinical download started this may take some time.\n","Approximate size to download 1.6 GB\n","[OK!]\n","ner_clinical download started this may take some time.\n","[OK!]\n"]}],"source":["embeddings = 'embeddings_clinical'\n","\n","model_name = 'ner_clinical'\n","\n","light_model = get_light_model(embeddings, model_name)\n","\n","text = \"I had a headache yesterday and took an Advil.\""]},{"cell_type":"code","execution_count":58,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":167},"executionInfo":{"elapsed":1634,"status":"ok","timestamp":1712935222230,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"M53Wcu6pre0j","outputId":"232d981a-562b-49ea-ea7e-ccd72e7c9481"},"outputs":[{"output_type":"stream","name":"stdout","text":["Warning::Spark Session already created, some configs may not take.\n","Warning::Spark Session already created, some configs may not take.\n"]},{"output_type":"execute_result","data":{"text/plain":["  entities_ner_chunk_origin_sentence entities_ner_chunk_begin  \\\n","0                                  0                        6   \n","1                                  0                       36   \n","\n","  entities_ner_chunk_end entities_ner_chunk entities_ner_chunk_class  \n","0                     15         a headache                  PROBLEM  \n","1                     43           an Advil                TREATMENT  "],"text/html":["\n","  <div id=\"df-c6e54e45-2bba-4cfc-a0f9-7ac0ac60ccfa\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>entities_ner_chunk_origin_sentence</th>\n","      <th>entities_ner_chunk_begin</th>\n","      <th>entities_ner_chunk_end</th>\n","      <th>entities_ner_chunk</th>\n","      <th>entities_ner_chunk_class</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0</td>\n","      <td>6</td>\n","      <td>15</td>\n","      <td>a headache</td>\n","      <td>PROBLEM</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>0</td>\n","      <td>36</td>\n","      <td>43</td>\n","      <td>an Advil</td>\n","      <td>TREATMENT</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c6e54e45-2bba-4cfc-a0f9-7ac0ac60ccfa')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-c6e54e45-2bba-4cfc-a0f9-7ac0ac60ccfa button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-c6e54e45-2bba-4cfc-a0f9-7ac0ac60ccfa');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-33f647c7-dff4-485a-b351-8c4cbbfff0d9\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-33f647c7-dff4-485a-b351-8c4cbbfff0d9')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-33f647c7-dff4-485a-b351-8c4cbbfff0d9 button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","summary":"{\n  \"name\": \"nlp\",\n  \"rows\": 2,\n  \"fields\": [\n    {\n      \"column\": \"entities_ner_chunk_origin_sentence\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 1,\n        \"samples\": [\n          \"0\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_begin\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 6,\n        \"max\": 36,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          36\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_end\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 15,\n        \"max\": 43,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          43\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"an Advil\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_class\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"TREATMENT\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":58}],"source":["cols = [\n","        'entities_ner_chunk_origin_sentence',\n","        'entities_ner_chunk_begin',\n","        'entities_ner_chunk_end',\n","        'entities_ner_chunk',\n","        'entities_ner_chunk_class',\n","       ]\n","\n","nlp.nlu.to_pretty_df(light_model, text, positions=True, output_level='chunk')[cols].reset_index(drop=True)"]},{"cell_type":"code","execution_count":59,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":5345,"status":"ok","timestamp":1712935227570,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"LmLPlfCw5hh-","outputId":"681a18e5-f326-409e-84f8-5fb0f99ccd23"},"outputs":[{"output_type":"stream","name":"stdout","text":["embeddings_clinical download started this may take some time.\n","Approximate size to download 1.6 GB\n","[OK!]\n","ner_posology download started this may take some time.\n","[OK!]\n"]}],"source":["text ='''The patient was prescribed 1 capsule of Parol with meals .\n","He was seen by the endocrinology service and she was discharged on 40 units of insulin glargine at night , 12 units of insulin lispro with meals , and metformin 1000 mg two times a day .\n","It was determined that all SGLT2 inhibitors should be discontinued indefinitely fro 3 months .'''\n","\n","embeddings = 'embeddings_clinical'\n","\n","model_name = 'ner_posology'\n","\n","light_model = get_light_model(embeddings, model_name)"]},{"cell_type":"code","execution_count":60,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":543},"executionInfo":{"elapsed":2270,"status":"ok","timestamp":1712935229838,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"DLnWyjNrrm4v","outputId":"d9776df5-102f-4fd5-cf36-262a95fe9db2"},"outputs":[{"output_type":"stream","name":"stdout","text":["Warning::Spark Session already created, some configs may not take.\n","Warning::Spark Session already created, some configs may not take.\n"]},{"output_type":"execute_result","data":{"text/plain":["   entities_ner_chunk_origin_sentence entities_ner_chunk_begin  \\\n","0                                   0                       27   \n","1                                   0                       29   \n","2                                   0                       40   \n","3                                   0                       46   \n","4                                   1                      126   \n","5                                   1                      138   \n","6                                   1                      155   \n","7                                   1                      166   \n","8                                   1                      178   \n","9                                   1                      193   \n","10                                  1                      210   \n","11                                  1                      220   \n","12                                  1                      228   \n","13                                  2                      273   \n","\n","   entities_ner_chunk_end entities_ner_chunk entities_ner_chunk_class  \n","0                      27                  1                   DOSAGE  \n","1                      35            capsule                     FORM  \n","2                      44              Parol                     DRUG  \n","3                      55         with meals                FREQUENCY  \n","4                     133           40 units                   DOSAGE  \n","5                     153   insulin glargine                     DRUG  \n","6                     162           at night                FREQUENCY  \n","7                     173           12 units                   DOSAGE  \n","8                     191     insulin lispro                     DRUG  \n","9                     202         with meals                FREQUENCY  \n","10                    218          metformin                     DRUG  \n","11                    226            1000 mg                 STRENGTH  \n","12                    242    two times a day                FREQUENCY  \n","13                    288   SGLT2 inhibitors                     DRUG  "],"text/html":["\n","  <div id=\"df-9441cb1b-967d-4596-b9cc-8a9df8f6cecb\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>entities_ner_chunk_origin_sentence</th>\n","      <th>entities_ner_chunk_begin</th>\n","      <th>entities_ner_chunk_end</th>\n","      <th>entities_ner_chunk</th>\n","      <th>entities_ner_chunk_class</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0</td>\n","      <td>27</td>\n","      <td>27</td>\n","      <td>1</td>\n","      <td>DOSAGE</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>0</td>\n","      <td>29</td>\n","      <td>35</td>\n","      <td>capsule</td>\n","      <td>FORM</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>0</td>\n","      <td>40</td>\n","      <td>44</td>\n","      <td>Parol</td>\n","      <td>DRUG</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>0</td>\n","      <td>46</td>\n","      <td>55</td>\n","      <td>with meals</td>\n","      <td>FREQUENCY</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>1</td>\n","      <td>126</td>\n","      <td>133</td>\n","      <td>40 units</td>\n","      <td>DOSAGE</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>1</td>\n","      <td>138</td>\n","      <td>153</td>\n","      <td>insulin glargine</td>\n","      <td>DRUG</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>1</td>\n","      <td>155</td>\n","      <td>162</td>\n","      <td>at night</td>\n","      <td>FREQUENCY</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>1</td>\n","      <td>166</td>\n","      <td>173</td>\n","      <td>12 units</td>\n","      <td>DOSAGE</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>1</td>\n","      <td>178</td>\n","      <td>191</td>\n","      <td>insulin lispro</td>\n","      <td>DRUG</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>1</td>\n","      <td>193</td>\n","      <td>202</td>\n","      <td>with meals</td>\n","      <td>FREQUENCY</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>1</td>\n","      <td>210</td>\n","      <td>218</td>\n","      <td>metformin</td>\n","      <td>DRUG</td>\n","    </tr>\n","    <tr>\n","      <th>11</th>\n","      <td>1</td>\n","      <td>220</td>\n","      <td>226</td>\n","      <td>1000 mg</td>\n","      <td>STRENGTH</td>\n","    </tr>\n","    <tr>\n","      <th>12</th>\n","      <td>1</td>\n","      <td>228</td>\n","      <td>242</td>\n","      <td>two times a day</td>\n","      <td>FREQUENCY</td>\n","    </tr>\n","    <tr>\n","      <th>13</th>\n","      <td>2</td>\n","      <td>273</td>\n","      <td>288</td>\n","      <td>SGLT2 inhibitors</td>\n","      <td>DRUG</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9441cb1b-967d-4596-b9cc-8a9df8f6cecb')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-9441cb1b-967d-4596-b9cc-8a9df8f6cecb button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-9441cb1b-967d-4596-b9cc-8a9df8f6cecb');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-a728bcda-dfea-4c01-b0e5-5f9fa82e3989\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-a728bcda-dfea-4c01-b0e5-5f9fa82e3989')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-a728bcda-dfea-4c01-b0e5-5f9fa82e3989 button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","summary":"{\n  \"name\": \"nlp\",\n  \"rows\": 14,\n  \"fields\": [\n    {\n      \"column\": \"entities_ner_chunk_origin_sentence\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"0\",\n          \"1\",\n          \"2\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_begin\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 27,\n        \"max\": 273,\n        \"num_unique_values\": 14,\n        \"samples\": [\n          193,\n          220,\n          27\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_end\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 27,\n        \"max\": 288,\n        \"num_unique_values\": 14,\n        \"samples\": [\n          202,\n          226,\n          27\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 13,\n        \"samples\": [\n          \"two times a day\",\n          \"metformin\",\n          \"1\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_class\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"FORM\",\n          \"STRENGTH\",\n          \"DRUG\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":60}],"source":["nlp.nlu.to_pretty_df(light_model, text,positions=True, output_level='chunk')[cols].reset_index(drop=True)"]},{"cell_type":"markdown","metadata":{"id":"gQZP4CvL_qha"},"source":["## PHI NER\n","\n","**Entities**\n","- AGE\n","- CONTACT\n","- DATE\n","- ID\n","- LOCATION\n","- NAME\n","- PROFESSION"]},{"cell_type":"code","execution_count":61,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":9020,"status":"ok","timestamp":1712935238852,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"ZGGTicqg6Ii-","outputId":"ea620118-52bd-4843-ae00-1f796cfd6c6d"},"outputs":[{"output_type":"stream","name":"stdout","text":["embeddings_clinical download started this may take some time.\n","Approximate size to download 1.6 GB\n","[OK!]\n","ner_deid_subentity_augmented download started this may take some time.\n","[OK!]\n"]}],"source":["embeddings = 'embeddings_clinical'\n","\n","model_name = 'ner_deid_subentity_augmented'\n","\n","# deidentify_dl\n","# ner_deid_sd\n","# ner_deid_large\n","# ner_deid_generic_augmented\n","# ner_deid_subentity_augmented\n","# ner_deid_subentity_augmented_i2b2\n","\n","text = \"\"\"Miriam BRAY is a 41-year-old female from Vietnam and she was admitted for a right-sided pleural effusion for thoracentesis on Monday by Dr. X. Her Coumadin was placed on hold.\n","She was instructed to followup with Dr. XYZ in the office to check her INR On August 24, 2007 .\"\"\"\n","\n","light_model = get_light_model(embeddings, model_name)"]},{"cell_type":"code","execution_count":62,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":323},"executionInfo":{"elapsed":1634,"status":"ok","timestamp":1712935240473,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"Z3claKyp6S_D","outputId":"bb0310ef-501e-4583-8bda-067fe567f950"},"outputs":[{"output_type":"stream","name":"stdout","text":["Warning::Spark Session already created, some configs may not take.\n","Warning::Spark Session already created, some configs may not take.\n"]},{"output_type":"execute_result","data":{"text/plain":["  entities_ner_chunk_origin_sentence entities_ner_chunk_begin  \\\n","0                                  0                        0   \n","1                                  0                       17   \n","2                                  0                       41   \n","3                                  0                      126   \n","4                                  0                      140   \n","5                                  1                      216   \n","6                                  1                      254   \n","\n","  entities_ner_chunk_end entities_ner_chunk entities_ner_chunk_class  \n","0                     10        Miriam BRAY                  PATIENT  \n","1                     27        41-year-old                      AGE  \n","2                     47            Vietnam                  COUNTRY  \n","3                    131             Monday                     DATE  \n","4                    140                  X                   DOCTOR  \n","5                    218                XYZ                   DOCTOR  \n","6                    268    August 24, 2007                     DATE  "],"text/html":["\n","  <div id=\"df-2f50287a-fcae-4c09-b22a-cfcac9918995\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>entities_ner_chunk_origin_sentence</th>\n","      <th>entities_ner_chunk_begin</th>\n","      <th>entities_ner_chunk_end</th>\n","      <th>entities_ner_chunk</th>\n","      <th>entities_ner_chunk_class</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>10</td>\n","      <td>Miriam BRAY</td>\n","      <td>PATIENT</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>0</td>\n","      <td>17</td>\n","      <td>27</td>\n","      <td>41-year-old</td>\n","      <td>AGE</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>0</td>\n","      <td>41</td>\n","      <td>47</td>\n","      <td>Vietnam</td>\n","      <td>COUNTRY</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>0</td>\n","      <td>126</td>\n","      <td>131</td>\n","      <td>Monday</td>\n","      <td>DATE</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>0</td>\n","      <td>140</td>\n","      <td>140</td>\n","      <td>X</td>\n","      <td>DOCTOR</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>1</td>\n","      <td>216</td>\n","      <td>218</td>\n","      <td>XYZ</td>\n","      <td>DOCTOR</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>1</td>\n","      <td>254</td>\n","      <td>268</td>\n","      <td>August 24, 2007</td>\n","      <td>DATE</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-2f50287a-fcae-4c09-b22a-cfcac9918995')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-2f50287a-fcae-4c09-b22a-cfcac9918995 button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-2f50287a-fcae-4c09-b22a-cfcac9918995');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-08f808c7-8318-4b5b-bd1e-a2320da86329\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-08f808c7-8318-4b5b-bd1e-a2320da86329')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-08f808c7-8318-4b5b-bd1e-a2320da86329 button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","summary":"{\n  \"name\": \"nlp\",\n  \"rows\": 7,\n  \"fields\": [\n    {\n      \"column\": \"entities_ner_chunk_origin_sentence\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"1\",\n          \"0\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_begin\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 0,\n        \"max\": 254,\n        \"num_unique_values\": 7,\n        \"samples\": [\n          0,\n          17\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_end\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 10,\n        \"max\": 268,\n        \"num_unique_values\": 7,\n        \"samples\": [\n          10,\n          27\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 7,\n        \"samples\": [\n          \"Miriam BRAY\",\n          \"41-year-old\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_class\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"AGE\",\n          \"DOCTOR\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":62}],"source":["nlp.nlu.to_pretty_df(light_model, text,positions=True, output_level='chunk')[cols].reset_index(drop=True)"]},{"cell_type":"markdown","metadata":{"id":"qVeR4zew_urR"},"source":["## Oncology NER\n","\n","There are several Oncology NER models in Spark NLP. We will show an example of `new_oncology_wip` model here.\n","\n","**Entities**\n","\n","| | | | |\n","|-|-|-|-|\n","| Invasion | Dosage | Hormonal_Therapy | Site_Lung |\n","| Biomarker_Result | Response_To_Treatment | Oncogene | Relative_Date |\n","| Site_Bone | Histological_Type | Radiation_Dose | Age |\n","| Duration | Grade | Tumor_Size | Line_Of_Therapy |\n","| Site_Breast | Cancer_Surgery | Smoking_Status | Imaging_Test |\n","| Adenopathy | Targeted_Therapy | Radiotherapy | Metastasis |\n","| Date | Cycle_Number | Pathology_Result | Cancer_Score |\n","| Cancer_Dx | Death_Entity | Site_Liver | Site_Brain |\n","| Pathology_Test | Chemotherapy | Biomarker | Frequency |\n","| Site_Lymph_Node | Performance_Status | Unspecific_Therapy | Cycle_Count |\n","| Race_Ethnicity | Gender | Staging | Immunotherapy |\n","| Site_Other_Body_Part | Tumor_Finding | Route | Cycle_Day |\n","| Direction| | | |\n","\n","You can check [Oncology Model Notebook](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/27.Oncology_Model.ipynb) for more examples.\n"]},{"cell_type":"code","execution_count":63,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":175},"executionInfo":{"elapsed":5,"status":"ok","timestamp":1712935240473,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"xhOvypXqSjfd","outputId":"c31dd468-f15e-4cd0-866c-bc6f0ba45f59"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["'Sample Type / Medical Specialty:\\nHematology - Oncology\\nSample Name:\\nDischarge Summary - Mesothelioma - 1\\nDescription:\\nMesothelioma, pleural effusion, atrial fibrillation, anemia, ascites, esophageal reflux, and history of deep venous thrombosis.\\n(Medical Transcription Sample Report)\\nPRINCIPAL DIAGNOSIS:\\nMesothelioma.\\nSECONDARY DIAGNOSES:\\nPleural effusion, atrial fibrillation, anemia, ascites, esophageal reflux, and history of deep venous thrombosis.\\nPROCEDURES\\n1. On August 24, 2007, decortication of the lung with pleural biopsy and transpleural fluoroscopy.\\n2. On August 20, 2007, thoracentesis.\\n3. On August 31, 2007, Port-A-Cath placement.\\nHISTORY AND PHYSICAL:\\nThe patient is a 41-year-old Vietnamese female with a nonproductive cough that started last week. She has had right-sided chest pain radiating to her back with fever starting yesterday. She has a history of pericarditis and pericardectomy in May 2006 and developed cough with right-sided chest pain, and went to an urgent care center. Chest x-ray revealed right-sided pleural effusion.\\nPAST MEDICAL HISTORY\\n1. Pericardectomy.\\n2. Pericarditis.\\n2. Atrial fibrillation.\\n4. RNCA with intracranial thrombolytic treatment.\\n5 PTA of MCA.\\n6. Mesenteric venous thrombosis.\\n7. Pericardial window.\\n8. Cholecystectomy.\\n9. Left thoracentesis.\\nFAMILY HISTORY:\\nNo family history of coronary artery disease, CVA, diabetes, CHF or MI. The patient has one family member, a sister, with history of cancer.\\nSOCIAL HISTORY:\\nShe is married. Employed with the US Post Office. She is a mother of three. Denies tobacco, alcohol or illicit drug use.\\nMEDICATIONS\\n1. Coumadin 1 mg daily. Last INR was on Tuesday, August 14, 2007, and her INR was 2.3.\\n2. Amiodarone 100 mg p.o. daily.\\nREVIEW OF SYSTEMS:\\nComplete review of systems negative except as in pulmonary as noted above. The patient also reports occasional numbness and tingling of her left arm.\\nPHYSICAL EXAMINATION\\nVITAL SIGNS: Blood pressure 123/95, heart rate 83, respirations 20, temperature 97, and oxygen saturation 97%.\\nGENERAL: Positive nonproductive cough and pain with coughing.\\nHEENT: Pupils are equal and reactive to light and accommodation. Tympanic membranes are clear.\\nNECK: Supple. No lymphadenopathy. No masses.\\nRESPIRATORY: Pleural friction rub is noted.\\nGI: Soft, nondistended, and nontender. Positive bowel sounds. No organomegaly.\\nEXTREMITIES: No edema, no clubbing, no cyanosis, no tenderness. Full range of motion. Normal pulses in all extremities.\\nSKIN: No breakdown or lesions. No ulcers.\\nNEUROLOGIC: Grossly intact. No focal deficits. Awake, alert, and oriented to person, place, and time.\\nLABORATORY DATA:\\nLabs are pending.\\nHOSPITAL COURSE:\\nThe patient was admitted for a right-sided pleural effusion for thoracentesis on Monday by Dr. X. Her Coumadin was placed on hold. A repeat echocardiogram was checked. She was started on prophylaxis for DVT with Lovenox 40 mg subcutaneously. Her history dated back to March 2005 when she first sought medical attention for evidence of pericarditis, which was treated with pericardial window in an outside hospital, at that time she was also found to have mesenteric pain and thrombosis, is now anticoagulated. Her pericardial fluid was accumulated and she was seen by Dr. Y. At that time, she was recommended for pericardectomy, which was performed by Dr. Z. Review of her CT scan from March 2006 prior to her pericardectomy, already shows bilateral plural effusions. The patient improved clinically after the pericardectomy with resolution of her symptoms. Recently, she was readmitted to the hospital with chest pain and found to have bilateral pleural effusion, the right greater than the left. CT of the chest also revealed a large mediastinal lymph node. We reviewed the pathology obtained from the pericardectomy in March 2006, which was diagnostic of mesothelioma. At this time, chest tube placement for drainage of the fluid occurred and thoracoscopy with fluid biopsies, which were performed, which revealed epithelioid malignant mesothelioma. The patient was then stained with a PET CT, which showed extensive uptake in the chest, bilateral pleural pericardial effusions, and lymphadenopathy. She also had acidic fluid, pectoral and intramammary lymph nodes and uptake in L4 with SUV of 4. This was consistent with stage III disease. Her repeat echocardiogram showed an ejection fraction of 45% to 49%. She was transferred to Oncology service and started on chemotherapy on September 1, 2007 with cisplatin 75 mg/centimeter squared equaling 109 mg IV piggyback over 2 hours on September 1, 2007, Alimta 500 mg/ centimeter squared equaling 730 mg IV piggyback over 10 minutes. This was all initiated after a Port-A-Cath was placed. The chemotherapy was well tolerated and the patient was discharged the following day after discontinuing IV fluid and IV. Her Port-A-Cath was packed with heparin according to protocol.\\nDISCHARGE MEDICATIONS:\\nZofran, Phenergan, Coumadin, and Lovenox, and Vicodin\\nDISCHARGE INSTRUCTIONS:\\nShe was instructed to followup with Dr. XYZ in the office to check her INR on Tuesday. She was instructed to call if she had any other questions or concerns in the interim.\\nKeywords:\\nhematology - oncology, mesothelioma, pleural effusion, atrial fibrillation, anemia, ascites, esophageal reflux, deep venous thrombosis, port-a-cath placement, port a cath, iv piggyback, venous thrombosis, atrial, thrombosis, pericardial, lymphadenopathy, fluid, pericardectomy, chest, pleural,'"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"string"}},"metadata":{},"execution_count":63}],"source":["mt_samples_df.limit(1).collect()[0][\"text\"]"]},{"cell_type":"code","execution_count":64,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":11970,"status":"ok","timestamp":1712935252439,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"y1Ffv9qT_AYV","outputId":"9d08a093-4d4f-4c31-b22b-c227ceac8dae"},"outputs":[{"output_type":"stream","name":"stdout","text":["embeddings_clinical download started this may take some time.\n","Approximate size to download 1.6 GB\n","[OK!]\n","ner_oncology download started this may take some time.\n","[OK!]\n"]}],"source":["embeddings = 'embeddings_clinical'\n","\n","model_name = 'ner_oncology'\n","\n","text =  mt_samples_df.limit(1).collect()[0][\"text\"]\n","\n","light_model = get_light_model(embeddings, model_name)\n"]},{"cell_type":"code","execution_count":65,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":748},"executionInfo":{"elapsed":4259,"status":"ok","timestamp":1712935256686,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"0jIPze7e48Of","outputId":"06458ad6-0336-40ec-b466-fbd21e6984ea"},"outputs":[{"output_type":"stream","name":"stdout","text":["Warning::Spark Session already created, some configs may not take.\n","Warning::Spark Session already created, some configs may not take.\n"]},{"output_type":"execute_result","data":{"text/plain":["   entities_ner_chunk_origin_sentence entities_ner_chunk_begin  \\\n","0                                   0                      118   \n","1                                   4                      471   \n","2                                   4                      488   \n","3                                   4                      509   \n","4                                   4                      519   \n","5                                   4                      527   \n","6                                   4                      538   \n","7                                   5                      570   \n","8                                   6                      608   \n","9                                   7                      687   \n","10                                  7                      699   \n","11                                  7                      710   \n","12                                  7                      757   \n","13                                  8                      768   \n","14                                  8                      816   \n","15                                  9                      856   \n","16                                  9                      894   \n","17                                  9                      912   \n","18                                 10                     1005   \n","19                                 19                     1260   \n","\n","   entities_ner_chunk_end        entities_ner_chunk entities_ner_chunk_class  \n","0                     129              Mesothelioma                Cancer_Dx  \n","1                     485           August 24, 2007                     Date  \n","2                     500             decortication           Cancer_Surgery  \n","3                     512                      lung                Site_Lung  \n","4                     525                   pleural                Site_Lung  \n","5                     532                    biopsy           Pathology_Test  \n","6                     561  transpleural fluoroscopy             Imaging_Test  \n","7                     584           August 20, 2007                     Date  \n","8                     622           August 31, 2007                     Date  \n","9                     697               41-year-old                   Gender  \n","10                    708                Vietnamese           Race_Ethnicity  \n","11                    715                    female                   Gender  \n","12                    765                 last week            Relative_Date  \n","13                    770                       She                   Gender  \n","14                    818                       her                   Gender  \n","15                    858                       She                   Gender  \n","16                    907            pericardectomy           Cancer_Surgery  \n","17                    919                  May 2006                     Date  \n","18                   1015               Chest x-ray             Imaging_Test  \n","19                   1274           Cholecystectomy           Cancer_Surgery  "],"text/html":["\n","  <div id=\"df-35c645c7-20c9-4765-bbdc-927813194a7c\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>entities_ner_chunk_origin_sentence</th>\n","      <th>entities_ner_chunk_begin</th>\n","      <th>entities_ner_chunk_end</th>\n","      <th>entities_ner_chunk</th>\n","      <th>entities_ner_chunk_class</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0</td>\n","      <td>118</td>\n","      <td>129</td>\n","      <td>Mesothelioma</td>\n","      <td>Cancer_Dx</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>4</td>\n","      <td>471</td>\n","      <td>485</td>\n","      <td>August 24, 2007</td>\n","      <td>Date</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>4</td>\n","      <td>488</td>\n","      <td>500</td>\n","      <td>decortication</td>\n","      <td>Cancer_Surgery</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>4</td>\n","      <td>509</td>\n","      <td>512</td>\n","      <td>lung</td>\n","      <td>Site_Lung</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>4</td>\n","      <td>519</td>\n","      <td>525</td>\n","      <td>pleural</td>\n","      <td>Site_Lung</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>4</td>\n","      <td>527</td>\n","      <td>532</td>\n","      <td>biopsy</td>\n","      <td>Pathology_Test</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>4</td>\n","      <td>538</td>\n","      <td>561</td>\n","      <td>transpleural fluoroscopy</td>\n","      <td>Imaging_Test</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>5</td>\n","      <td>570</td>\n","      <td>584</td>\n","      <td>August 20, 2007</td>\n","      <td>Date</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>6</td>\n","      <td>608</td>\n","      <td>622</td>\n","      <td>August 31, 2007</td>\n","      <td>Date</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>7</td>\n","      <td>687</td>\n","      <td>697</td>\n","      <td>41-year-old</td>\n","      <td>Gender</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>7</td>\n","      <td>699</td>\n","      <td>708</td>\n","      <td>Vietnamese</td>\n","      <td>Race_Ethnicity</td>\n","    </tr>\n","    <tr>\n","      <th>11</th>\n","      <td>7</td>\n","      <td>710</td>\n","      <td>715</td>\n","      <td>female</td>\n","      <td>Gender</td>\n","    </tr>\n","    <tr>\n","      <th>12</th>\n","      <td>7</td>\n","      <td>757</td>\n","      <td>765</td>\n","      <td>last week</td>\n","      <td>Relative_Date</td>\n","    </tr>\n","    <tr>\n","      <th>13</th>\n","      <td>8</td>\n","      <td>768</td>\n","      <td>770</td>\n","      <td>She</td>\n","      <td>Gender</td>\n","    </tr>\n","    <tr>\n","      <th>14</th>\n","      <td>8</td>\n","      <td>816</td>\n","      <td>818</td>\n","      <td>her</td>\n","      <td>Gender</td>\n","    </tr>\n","    <tr>\n","      <th>15</th>\n","      <td>9</td>\n","      <td>856</td>\n","      <td>858</td>\n","      <td>She</td>\n","      <td>Gender</td>\n","    </tr>\n","    <tr>\n","      <th>16</th>\n","      <td>9</td>\n","      <td>894</td>\n","      <td>907</td>\n","      <td>pericardectomy</td>\n","      <td>Cancer_Surgery</td>\n","    </tr>\n","    <tr>\n","      <th>17</th>\n","      <td>9</td>\n","      <td>912</td>\n","      <td>919</td>\n","      <td>May 2006</td>\n","      <td>Date</td>\n","    </tr>\n","    <tr>\n","      <th>18</th>\n","      <td>10</td>\n","      <td>1005</td>\n","      <td>1015</td>\n","      <td>Chest x-ray</td>\n","      <td>Imaging_Test</td>\n","    </tr>\n","    <tr>\n","      <th>19</th>\n","      <td>19</td>\n","      <td>1260</td>\n","      <td>1274</td>\n","      <td>Cholecystectomy</td>\n","      <td>Cancer_Surgery</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-35c645c7-20c9-4765-bbdc-927813194a7c')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-35c645c7-20c9-4765-bbdc-927813194a7c button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-35c645c7-20c9-4765-bbdc-927813194a7c');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-9a956a3b-8739-449c-9d97-b9196da7e650\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-9a956a3b-8739-449c-9d97-b9196da7e650')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-9a956a3b-8739-449c-9d97-b9196da7e650 button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","summary":"{\n  \"name\": \"nlp\",\n  \"rows\": 20,\n  \"fields\": [\n    {\n      \"column\": \"entities_ner_chunk_origin_sentence\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 9,\n        \"samples\": [\n          \"10\",\n          \"4\",\n          \"8\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_begin\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 118,\n        \"max\": 1260,\n        \"num_unique_values\": 20,\n        \"samples\": [\n          118,\n          912,\n          856\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_end\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 129,\n        \"max\": 1274,\n        \"num_unique_values\": 20,\n        \"samples\": [\n          129,\n          919,\n          858\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 19,\n        \"samples\": [\n          \"Mesothelioma\",\n          \"biopsy\",\n          \"female\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_ner_chunk_class\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 9,\n        \"samples\": [\n          \"Race_Ethnicity\",\n          \"Date\",\n          \"Imaging_Test\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":65}],"source":["nlp.nlu.to_pretty_df(light_model, text, positions=True, output_level='chunk')[cols].head(20).reset_index(drop=True)"]},{"cell_type":"markdown","metadata":{"id":"a8W07CdCZJmO"},"source":["## NER Chunker\n","We can extract phrases that fits into a known pattern using the NER tags. NerChunker would be quite handy to extract entity groups with neighboring tokens when there is no pretrained NER model to address certain issues. Lets say we want to extract drug and frequency together as a single chunk even if there are some unwanted tokens between them."]},{"cell_type":"code","execution_count":66,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":3629,"status":"ok","timestamp":1712935260303,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"WdK6LdDxvVSt","outputId":"bb68963c-3328-40e3-887c-a5423aafb699"},"outputs":[{"output_type":"stream","name":"stdout","text":["ner_posology download started this may take some time.\n","[OK!]\n"]}],"source":["posology_ner = medical.NerModel.pretrained(\"ner_posology\", \"en\", \"clinical/models\") \\\n","    .setInputCols([\"sentence\", \"token\", \"embeddings\"]) \\\n","    .setOutputCol(\"ner\")\n","\n","ner_chunker = medical.NerChunker()\\\n","    .setInputCols([\"sentence\",\"ner\"])\\\n","    .setOutputCol(\"ner_chunk\")\\\n","    .setRegexParsers([\"<DRUG>.*<FREQUENCY>\"])\n","\n","nlpPipeline = nlp.Pipeline(stages=[\n","    documentAssembler,\n","    sentenceDetector,\n","    tokenizer,\n","    word_embeddings,\n","    posology_ner,\n","    ner_chunker])\n","\n","empty_data = spark.createDataFrame([[\"\"]]).toDF(\"text\")\n","\n","ner_chunker_model = nlpPipeline.fit(empty_data)"]},{"cell_type":"code","execution_count":67,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":21,"status":"ok","timestamp":1712935260304,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"wseg1Ko2vgKQ","outputId":"867ff310-2100-4f97-e44d-1df564b99928"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["['O',\n"," 'B-DOSAGE',\n"," 'B-STRENGTH',\n"," 'I-STRENGTH',\n"," 'B-ROUTE',\n"," 'B-FREQUENCY',\n"," 'I-FREQUENCY',\n"," 'B-DRUG',\n"," 'I-DRUG',\n"," 'B-FORM',\n"," 'I-DOSAGE',\n"," 'B-DURATION',\n"," 'I-DURATION',\n"," 'I-FORM',\n"," 'I-ROUTE']"]},"metadata":{},"execution_count":67}],"source":["posology_ner.getClasses()"]},{"cell_type":"code","execution_count":68,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":18,"status":"ok","timestamp":1712935260304,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"d1IvoqP7wC9b","outputId":"09be36b6-418e-480f-f651-21d03988e8d8"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["[('The', 'O'),\n"," ('patient', 'O'),\n"," ('was', 'O'),\n"," ('prescribed', 'O'),\n"," ('1', 'B-DOSAGE'),\n"," ('capsule', 'B-FORM'),\n"," ('of', 'O'),\n"," ('Advil', 'B-DRUG'),\n"," ('for', 'B-DURATION'),\n"," ('5', 'I-DURATION'),\n"," ('days', 'I-DURATION'),\n"," ('.', 'O'),\n"," ('He', 'O'),\n"," ('was', 'O'),\n"," ('seen', 'O'),\n"," ('by', 'O'),\n"," ('the', 'O'),\n"," ('endocrinology', 'O'),\n"," ('service', 'O'),\n"," ('and', 'O'),\n"," ('she', 'O'),\n"," ('was', 'O'),\n"," ('discharged', 'O'),\n"," ('on', 'O'),\n"," ('40', 'B-DOSAGE'),\n"," ('units', 'I-DOSAGE'),\n"," ('of', 'O'),\n"," ('insulin', 'B-DRUG'),\n"," ('glargine', 'I-DRUG'),\n"," ('at', 'B-FREQUENCY'),\n"," ('night', 'I-FREQUENCY'),\n"," (',', 'O'),\n"," ('12', 'B-DOSAGE'),\n"," ('units', 'I-DOSAGE'),\n"," ('of', 'O'),\n"," ('insulin', 'B-DRUG'),\n"," ('lispro', 'I-DRUG'),\n"," ('with', 'B-FREQUENCY'),\n"," ('meals', 'I-FREQUENCY'),\n"," (',', 'O'),\n"," ('metformin', 'B-DRUG'),\n"," ('1000', 'B-STRENGTH'),\n"," ('mg', 'I-STRENGTH'),\n"," ('two', 'B-FREQUENCY'),\n"," ('times', 'I-FREQUENCY'),\n"," ('a', 'I-FREQUENCY'),\n"," ('day', 'I-FREQUENCY'),\n"," ('.', 'O'),\n"," ('It', 'O'),\n"," ('was', 'O'),\n"," ('determined', 'O'),\n"," ('that', 'O'),\n"," ('all', 'O'),\n"," ('SGLT2', 'B-DRUG'),\n"," ('inhibitors', 'I-DRUG'),\n"," ('should', 'O'),\n"," ('be', 'O'),\n"," ('discontinued', 'O'),\n"," ('indefinitely', 'O'),\n"," ('fro', 'O'),\n"," ('3', 'O'),\n"," ('months', 'O'),\n"," ('.', 'O')]"]},"metadata":{},"execution_count":68}],"source":["light_model = nlp.LightPipeline(ner_chunker_model)\n","\n","text ='The patient was prescribed 1 capsule of Advil for 5 days . He was seen by the endocrinology service and she was discharged on 40 units of insulin glargine at night , 12 units of insulin lispro with meals , metformin 1000 mg two times a day . It was determined that all SGLT2 inhibitors should be discontinued indefinitely fro 3 months .'\n","\n","light_result = light_model.annotate(text)\n","\n","list(zip(light_result['token'], light_result['ner']))"]},{"cell_type":"code","execution_count":69,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":16,"status":"ok","timestamp":1712935260304,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"w3iyF9PVTcUX","outputId":"b6c218e3-aa63-40c7-dfae-b3cf99de9cfd"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["['insulin glargine at night , 12 units of insulin lispro with meals , metformin 1000 mg two times a day']"]},"metadata":{},"execution_count":69}],"source":["light_result[\"ner_chunk\"]"]},{"cell_type":"markdown","metadata":{"id":"YgSZ4ghNlDbV"},"source":["## Chunk Filterer\n","ChunkFilterer will allow you to filter out named entities by some conditions or predefined look-up lists, so that you can feed these entities to other annotators like Assertion Status or Entity Resolvers. It can be used with two criteria: isin and regex."]},{"cell_type":"code","execution_count":70,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2247,"status":"ok","timestamp":1712935262537,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"mK5A8IozCk-5","outputId":"210670fd-502e-4664-d9c0-b71b0e572aa3"},"outputs":[{"output_type":"stream","name":"stdout","text":["ner_posology download started this may take some time.\n","[OK!]\n"]}],"source":["posology_ner = medical.NerModel.pretrained(\"ner_posology\", \"en\", \"clinical/models\") \\\n","    .setInputCols([\"sentence\", \"token\", \"embeddings\"]) \\\n","    .setOutputCol(\"ner\")\n","\n","ner_converter = medical.NerConverterInternal()\\\n","    .setInputCols([\"sentence\",\"token\",\"ner\"])\\\n","    .setOutputCol(\"ner_chunk\")\n","\n","chunk_filterer = medical.ChunkFilterer()\\\n","    .setInputCols(\"sentence\",\"ner_chunk\")\\\n","    .setOutputCol(\"chunk_filtered\")\\\n","    .setCriteria(\"isin\")\\\n","    .setWhiteList(['ADVIL','Metformin', 'Insulin Lispro'])\\\n","    .setCaseSensitive(False)\n","\n","nlpPipeline = nlp.Pipeline(stages=[\n","    documentAssembler,\n","    sentenceDetector,\n","    tokenizer,\n","    word_embeddings,\n","    posology_ner,\n","    ner_converter,\n","    chunk_filterer])\n","\n","empty_data = spark.createDataFrame([[\"\"]]).toDF(\"text\")\n","\n","chunk_filter_model = nlpPipeline.fit(empty_data)"]},{"cell_type":"code","execution_count":71,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":8,"status":"ok","timestamp":1712935262537,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"NnerfAA4C9Oe","outputId":"53c6a7e1-9295-4d45-8eb1-75a55797428b"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["dict_keys(['document', 'ner_chunk', 'chunk_filtered', 'token', 'ner', 'embeddings', 'sentence'])"]},"metadata":{},"execution_count":71}],"source":["light_model = nlp.LightPipeline(chunk_filter_model)\n","\n","text ='The patient was prescribed 1 capsule of Advil for 5 days . He was seen by the endocrinology service and she was discharged on 40 units of insulin glargine at night , 12 units of insulin lispro with meals , metformin 1000 mg two times a day . It was determined that all SGLT2 inhibitors should be discontinued indefinitely fro 3 months .'\n","\n","light_result = light_model.annotate(text)\n","\n","light_result.keys()"]},{"cell_type":"code","execution_count":72,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":6,"status":"ok","timestamp":1712935262537,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"uXGq4DBJDCu7","outputId":"cf702563-4bf3-4ec0-f10b-c14971d9ba4d"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["['1',\n"," 'capsule',\n"," 'Advil',\n"," 'for 5 days',\n"," '40 units',\n"," 'insulin glargine',\n"," 'at night',\n"," '12 units',\n"," 'insulin lispro',\n"," 'with meals',\n"," 'metformin',\n"," '1000 mg',\n"," 'two times a day',\n"," 'SGLT2 inhibitors']"]},"metadata":{},"execution_count":72}],"source":["light_result['ner_chunk']"]},{"cell_type":"code","execution_count":73,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":5,"status":"ok","timestamp":1712935262537,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"Tt-rCxecDeqY","outputId":"61284670-d909-4148-ff9d-4e4169b123ee"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["['Advil', 'insulin lispro', 'metformin']"]},"metadata":{},"execution_count":73}],"source":["light_result['chunk_filtered']"]},{"cell_type":"code","execution_count":74,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2133,"status":"ok","timestamp":1712935264666,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"3hmPFLKrlZoh","outputId":"1549a4df-926f-451c-c530-99d231d1dc92"},"outputs":[{"output_type":"stream","name":"stdout","text":["ner_clinical download started this may take some time.\n","[OK!]\n"]}],"source":["ner_model = medical.NerModel.pretrained(\"ner_clinical\", \"en\", \"clinical/models\")\\\n","    .setInputCols(\"sentence\",\"token\",\"embeddings\")\\\n","    .setOutputCol(\"ner\")\n","\n","ner_converter = medical.NerConverterInternal()\\\n","    .setInputCols([\"sentence\",\"token\",\"ner\"])\\\n","    .setOutputCol(\"ner_chunk\")\n","\n","chunk_filterer = medical.ChunkFilterer()\\\n","    .setInputCols(\"sentence\",\"ner_chunk\")\\\n","    .setOutputCol(\"chunk_filtered\")\\\n","    .setCriteria(\"isin\")\\\n","    .setWhiteList(['severe fever','sore throat'])\\\n","    .setCaseSensitive(False)\n","\n","\n","#   .setCriteria(\"regex\")\\\n","\n","nlpPipeline = nlp.Pipeline(stages=[\n","    documentAssembler,\n","    sentenceDetector,\n","    tokenizer,\n","    word_embeddings,\n","    ner_model,\n","    ner_converter,\n","    chunk_filterer])\n","\n","empty_data = spark.createDataFrame([[\"\"]]).toDF(\"text\")\n","\n","chunk_filter_model = nlpPipeline.fit(empty_data)"]},{"cell_type":"code","execution_count":75,"metadata":{"id":"xYkwr0Iwp5KU","executionInfo":{"status":"ok","timestamp":1712935265487,"user_tz":-180,"elapsed":823,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["text = 'Patient with severe fever, severe cough, sore throat, stomach pain, and a headache.'\n","\n","filter_df = spark.createDataFrame([[text]]).toDF(\"text\")\n","\n","chunk_filter_result = chunk_filter_model.transform(filter_df)"]},{"cell_type":"code","execution_count":76,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":670,"status":"ok","timestamp":1712935266155,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"3CTm6zFtApR_","outputId":"daad45b3-bf98-42a1-a2c3-288c581fadee"},"outputs":[{"output_type":"stream","name":"stdout","text":["+-------------------------------------------------------------------+---------------------------+\n","|result                                                             |result                     |\n","+-------------------------------------------------------------------+---------------------------+\n","|[severe fever, severe cough, sore throat, stomach pain, a headache]|[severe fever, sore throat]|\n","+-------------------------------------------------------------------+---------------------------+\n","\n"]}],"source":["chunk_filter_result.select('ner_chunk.result','chunk_filtered.result').show(truncate=False)"]},{"cell_type":"markdown","source":["## ChunkFiltererApproach\n","ChunkFiltererApproach will allow you to enhance filtering by using JSON-Based Entity Confidence Configuration.\n","\n","  + `.setEntitiesConfidenceResourceAsJsonString`: You can   fine tune entity confidence levels using a JSON configuration."],"metadata":{"id":"d8_64ISkufkR"}},{"cell_type":"code","source":["chunk_filterer = medical.ChunkFiltererApproach()\\\n","    .setInputCols(\"sentence\",\"ner_chunk\")\\\n","    .setOutputCol(\"chunk_filtered\")\\\n","    .setFilterEntity(\"entity\")\\\n","    .setEntitiesConfidenceResourceAsJsonString(\"\"\"{'DURATION':'0.9',\n","                                                  'DOSAGE':'0.9',\n","                                                  'FREQUENCY':'0.9',\n","                                                  'STRENGTH':'0.9',\n","                                                  'DRUG':'0.9'}\"\"\")\n","\n","text ='The patient was prescribed 1 capsule of Advil for 5 days . He was seen by the endocrinology service and she was discharged on 40 units of insulin glargine at night , 12 units of insulin lispro with meals , metformin 1000 mg two times a day . It was determined that all SGLT2 inhibitors should be discontinued indefinitely fro 3 months .'"],"metadata":{"id":"IW0BYMkRuo9v","executionInfo":{"status":"ok","timestamp":1712935266155,"user_tz":-180,"elapsed":3,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"execution_count":77,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"4WtnMBdJyE1K"},"source":["## Changing entity labels with `NerConverterInternal()`\n","\n","There are two parameters in `NerConverterInternal()` annotator for replacing the labels.\n","\n","+ `.setReplaceLabels`: You can just create a dictionary with the labels and their new values in this parameter.\n","\n","```python\n","jsl_ner_converter_internal = medical.NerConverterInternal()\\\n","    .setInputCols([\"sentence\",\"token\",\"jsl_ner\"])\\\n","    .setOutputCol(\"replaced_ner_chunk\")\\\n","    .setReplaceLabels({\"Drug_BrandName\": \"Drug\",\n","                       \"Frequency\": \"Drug_Frequency\",\n","                       \"Dosage\": \"Drug_Dosage\",\n","                       \"Strength\": \"Drug_Strength\"})\n","```\n","\n","+ `.setReplaceDictResource`: You can feed a file that contains the labels and their new values in a dictionary format.\n","\n","```python\n","replace_dict = \"\"\"Drug_BrandName,Drug\n","Frequency,Drug_Frequency\n","Dosage,Drug_Dosage\n","Strength,Drug_Strength\n","\"\"\"\n","with open('replace_dict.csv', 'w') as f:\n","    f.write(replace_dict)\n","\n","...\n","jsl_ner_converter_internal = medical.NerConverterInternal()\\\n","    .setInputCols([\"sentence\",\"token\",\"jsl_ner\"])\\\n","    .setOutputCol(\"replaced_ner_chunk\")\\\n","    .setReplaceDictResource(\"replace_dict.csv\",\"text\", {\"delimiter\":\",\"})\n","...\n","```\n","\n","\n","\n","You can check [NerConverterInternal Annotator Page](https://nlp.johnsnowlabs.com/docs/en/licensed_annotators#nerconverterinternal) for more details."]},{"cell_type":"code","execution_count":78,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":11113,"status":"ok","timestamp":1712935277265,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"YYMqNMnJyEBk","outputId":"3bc49dbe-91fe-46a6-ef14-0c8b0174b0b4"},"outputs":[{"output_type":"stream","name":"stdout","text":["sentence_detector_dl_healthcare download started this may take some time.\n","Approximate size to download 367.3 KB\n","[OK!]\n","embeddings_clinical download started this may take some time.\n","Approximate size to download 1.6 GB\n","[OK!]\n","ner_jsl download started this may take some time.\n","[OK!]\n"]}],"source":["documentAssembler = nlp.DocumentAssembler()\\\n","    .setInputCol(\"text\")\\\n","    .setOutputCol(\"document\")\n","\n","sentenceDetector = nlp.SentenceDetectorDLModel.pretrained(\"sentence_detector_dl_healthcare\",\"en\",\"clinical/models\") \\\n","    .setInputCols([\"document\"]) \\\n","    .setOutputCol(\"sentence\")\n","\n","tokenizer = nlp.Tokenizer()\\\n","    .setInputCols([\"sentence\"])\\\n","    .setOutputCol(\"token\")\n","\n","word_embeddings = nlp.WordEmbeddingsModel.pretrained(\"embeddings_clinical\", \"en\", \"clinical/models\")\\\n","    .setInputCols([\"sentence\", \"token\"])\\\n","    .setOutputCol(\"embeddings\")\n","\n","jsl_ner = medical.NerModel.pretrained(\"ner_jsl\", \"en\", \"clinical/models\") \\\n","    .setInputCols([\"sentence\", \"token\", \"embeddings\"]) \\\n","    .setOutputCol(\"jsl_ner\")\n","\n","jsl_ner_converter = medical.NerConverterInternal() \\\n","    .setInputCols([\"sentence\", \"token\", \"jsl_ner\"]) \\\n","    .setOutputCol(\"jsl_ner_chunk\")\n","\n","jsl_ner_converter_internal = medical.NerConverterInternal()\\\n","    .setInputCols([\"sentence\",\"token\",\"jsl_ner\"])\\\n","    .setOutputCol(\"replaced_ner_chunk\")\\\n","    .setReplaceLabels({\"Drug_BrandName\": \"Drug\",\n","                       \"Frequency\": \"Drug_Frequency\",\n","                       \"Dosage\": \"Drug_Dosage\",\n","                       \"Strength\": \"Drug_Strength\"})\n","\n","nlpPipeline = nlp.Pipeline(stages=[\n","    documentAssembler,\n","    sentenceDetector,\n","    tokenizer,\n","    word_embeddings,\n","    jsl_ner,\n","    jsl_ner_converter,\n","    jsl_ner_converter_internal\n","    ])\n","\n","empty_data = spark.createDataFrame([[\"\"]]).toDF(\"text\")\n","ner_converter_model = nlpPipeline.fit(empty_data)"]},{"cell_type":"code","execution_count":79,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":637},"executionInfo":{"elapsed":15,"status":"ok","timestamp":1712935277265,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"JUwrYrviz3J0","outputId":"d022b902-fb55-45c5-b5fb-7c085935e80e"},"outputs":[{"output_type":"stream","name":"stdout","text":["Warning::Spark Session already created, some configs may not take.\n","Warning::Spark Session already created, some configs may not take.\n"]},{"output_type":"execute_result","data":{"text/plain":["  entities_jsl_ner_chunk_origin_sentence entities_jsl_ner_chunk_begin  \\\n","0                                      0                           27   \n","0                                      0                           40   \n","0                                      1                           58   \n","0                                      1                           77   \n","0                                      1                          103   \n","0                                      1                          111   \n","0                                      1                          125   \n","0                                      1                          137   \n","0                                      1                          154   \n","0                                      1                          165   \n","0                                      1                          177   \n","0                                      1                          192   \n","0                                      1                          209   \n","0                                      1                          219   \n","0                                      1                          227   \n","0                                      2                          272   \n","0                                      2                          325   \n","\n","  entities_jsl_ner_chunk_end entities_jsl_ner_chunk  \\\n","0                         35              1 capsule   \n","0                         44                  Parol   \n","0                         59                     He   \n","0                         97  endocrinology service   \n","0                        105                    she   \n","0                        120             discharged   \n","0                        132               40 units   \n","0                        152       insulin glargine   \n","0                        161               at night   \n","0                        172               12 units   \n","0                        190         insulin lispro   \n","0                        201             with meals   \n","0                        217              metformin   \n","0                        225                1000 mg   \n","0                        241        two times a day   \n","0                        287       SGLT2 inhibitors   \n","0                        336           fro 3 months   \n","\n","  entities_jsl_ner_chunk_class entities_replaced_ner_chunk_class  \n","0                       Dosage                       Drug_Dosage  \n","0               Drug_BrandName                              Drug  \n","0                       Gender                            Gender  \n","0                Clinical_Dept                     Clinical_Dept  \n","0                       Gender                            Gender  \n","0          Admission_Discharge               Admission_Discharge  \n","0                       Dosage                       Drug_Dosage  \n","0              Drug_Ingredient                   Drug_Ingredient  \n","0                    Frequency                    Drug_Frequency  \n","0                       Dosage                       Drug_Dosage  \n","0              Drug_Ingredient                   Drug_Ingredient  \n","0                    Frequency                    Drug_Frequency  \n","0              Drug_Ingredient                   Drug_Ingredient  \n","0                     Strength                     Drug_Strength  \n","0                    Frequency                    Drug_Frequency  \n","0              Drug_Ingredient                   Drug_Ingredient  \n","0                     Duration                          Duration  "],"text/html":["\n","  <div id=\"df-721b9151-7dc1-4abe-8bfe-e41284d46e48\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>entities_jsl_ner_chunk_origin_sentence</th>\n","      <th>entities_jsl_ner_chunk_begin</th>\n","      <th>entities_jsl_ner_chunk_end</th>\n","      <th>entities_jsl_ner_chunk</th>\n","      <th>entities_jsl_ner_chunk_class</th>\n","      <th>entities_replaced_ner_chunk_class</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0</td>\n","      <td>27</td>\n","      <td>35</td>\n","      <td>1 capsule</td>\n","      <td>Dosage</td>\n","      <td>Drug_Dosage</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>0</td>\n","      <td>40</td>\n","      <td>44</td>\n","      <td>Parol</td>\n","      <td>Drug_BrandName</td>\n","      <td>Drug</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>58</td>\n","      <td>59</td>\n","      <td>He</td>\n","      <td>Gender</td>\n","      <td>Gender</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>77</td>\n","      <td>97</td>\n","      <td>endocrinology service</td>\n","      <td>Clinical_Dept</td>\n","      <td>Clinical_Dept</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>103</td>\n","      <td>105</td>\n","      <td>she</td>\n","      <td>Gender</td>\n","      <td>Gender</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>111</td>\n","      <td>120</td>\n","      <td>discharged</td>\n","      <td>Admission_Discharge</td>\n","      <td>Admission_Discharge</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>125</td>\n","      <td>132</td>\n","      <td>40 units</td>\n","      <td>Dosage</td>\n","      <td>Drug_Dosage</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>137</td>\n","      <td>152</td>\n","      <td>insulin glargine</td>\n","      <td>Drug_Ingredient</td>\n","      <td>Drug_Ingredient</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>154</td>\n","      <td>161</td>\n","      <td>at night</td>\n","      <td>Frequency</td>\n","      <td>Drug_Frequency</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>165</td>\n","      <td>172</td>\n","      <td>12 units</td>\n","      <td>Dosage</td>\n","      <td>Drug_Dosage</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>177</td>\n","      <td>190</td>\n","      <td>insulin lispro</td>\n","      <td>Drug_Ingredient</td>\n","      <td>Drug_Ingredient</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>192</td>\n","      <td>201</td>\n","      <td>with meals</td>\n","      <td>Frequency</td>\n","      <td>Drug_Frequency</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>209</td>\n","      <td>217</td>\n","      <td>metformin</td>\n","      <td>Drug_Ingredient</td>\n","      <td>Drug_Ingredient</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>219</td>\n","      <td>225</td>\n","      <td>1000 mg</td>\n","      <td>Strength</td>\n","      <td>Drug_Strength</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>227</td>\n","      <td>241</td>\n","      <td>two times a day</td>\n","      <td>Frequency</td>\n","      <td>Drug_Frequency</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>2</td>\n","      <td>272</td>\n","      <td>287</td>\n","      <td>SGLT2 inhibitors</td>\n","      <td>Drug_Ingredient</td>\n","      <td>Drug_Ingredient</td>\n","    </tr>\n","    <tr>\n","      <th>0</th>\n","      <td>2</td>\n","      <td>325</td>\n","      <td>336</td>\n","      <td>fro 3 months</td>\n","      <td>Duration</td>\n","      <td>Duration</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-721b9151-7dc1-4abe-8bfe-e41284d46e48')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-721b9151-7dc1-4abe-8bfe-e41284d46e48 button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-721b9151-7dc1-4abe-8bfe-e41284d46e48');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-6fb275f9-51b2-4ee6-9e0f-ad9969426000\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-6fb275f9-51b2-4ee6-9e0f-ad9969426000')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-6fb275f9-51b2-4ee6-9e0f-ad9969426000 button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","summary":"{\n  \"name\": \"nlp\",\n  \"rows\": 17,\n  \"fields\": [\n    {\n      \"column\": \"entities_jsl_ner_chunk_origin_sentence\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"0\",\n          \"1\",\n          \"2\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_jsl_ner_chunk_begin\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 27,\n        \"max\": 325,\n        \"num_unique_values\": 17,\n        \"samples\": [\n          27,\n          40,\n          111\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_jsl_ner_chunk_end\",\n      \"properties\": {\n        \"dtype\": \"date\",\n        \"min\": 35,\n        \"max\": 336,\n        \"num_unique_values\": 17,\n        \"samples\": [\n          35,\n          44,\n          120\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_jsl_ner_chunk\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 17,\n        \"samples\": [\n          \"1 capsule\",\n          \"Parol\",\n          \"discharged\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_jsl_ner_chunk_class\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 9,\n        \"samples\": [\n          \"Strength\",\n          \"Drug_BrandName\",\n          \"Drug_Ingredient\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"entities_replaced_ner_chunk_class\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 9,\n        \"samples\": [\n          \"Drug_Strength\",\n          \"Drug\",\n          \"Drug_Ingredient\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":79}],"source":["text ='The patient was prescribed 1 capsule of Parol with meals. He was seen by the endocrinology service and she was discharged on 40 units of insulin glargine at night , 12 units of insulin lispro with meals , and metformin 1000 mg two times a day . It was determined that all SGLT2 inhibitors should be discontinued indefinitely fro 3 months .'\n","\n","#light_model = LightPipeline(ner_converter_model)\n","\n","cols = [\n","     'entities_jsl_ner_chunk_origin_sentence',\n","     'entities_jsl_ner_chunk_begin',\n","     'entities_jsl_ner_chunk_end',\n","     'entities_jsl_ner_chunk',\n","     'entities_jsl_ner_chunk_class',\n","     'entities_replaced_ner_chunk_class'\n","]\n","nlp.nlu.to_pretty_df(ner_converter_model,text,positions=True,output_level='chunk')[cols].head(20)"]},{"cell_type":"markdown","metadata":{"id":"7zkx4ZL2udiS"},"source":["### Adding `setIgnoreStopWords` parameter to control chunk generation.\n","By providing `setIgnoreStopWords`, we can ignore certain words or characters while creating a chunk. <br/>\n","\n","It should be a list of tokens/words or characters, and when two entities of the same type are separated by those words, these entities can be combined to produce a single, larger chunk.\n","\n","\n","\n","\n"]},{"cell_type":"markdown","metadata":{"id":"LjXlxH9AwUd6"},"source":["Firstly let's create `NerConverterInternal()` without this parameter"]},{"cell_type":"code","execution_count":80,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":9083,"status":"ok","timestamp":1712935286335,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"m9gUWSQBwBzG","outputId":"6a695da4-98c5-473c-e476-587f8e45e359"},"outputs":[{"output_type":"stream","name":"stdout","text":["ner_deid_generic_augmented download started this may take some time.\n","[OK!]\n"]}],"source":["jsl_ner= medical.NerModel.pretrained(\"ner_deid_generic_augmented\", \"en\", \"clinical/models\")\\\n","    .setInputCols([\"sentence\", \"token\", \"embeddings\"])\\\n","    .setOutputCol(\"ner_deid\")\\\n","    .setLabelCasing('upper')\n","\n","jsl_ner_converter_internal= medical.NerConverterInternal()\\\n","    .setInputCols([\"sentence\", \"token\", \"ner_deid\"])\\\n","    .setOutputCol(\"chunk_deid\")\\\n","    .setGreedyMode(True)\\\n","    .setWhiteList(['LOCATION'])\n","\n","nlpPipeline = nlp.Pipeline(stages=[\n","    documentAssembler,\n","    sentenceDetector,\n","    tokenizer,\n","    word_embeddings,\n","    jsl_ner,\n","    jsl_ner_converter_internal\n","    ])\n","\n","empty_data = spark.createDataFrame([[\"\"]]).toDF(\"text\")\n","ner_converter_model = nlpPipeline.fit(empty_data)"]},{"cell_type":"code","execution_count":81,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":207},"executionInfo":{"elapsed":12,"status":"ok","timestamp":1712935286335,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"D09_j4bPt_MJ","outputId":"36b93794-d736-4a9a-9f7b-b9f1c81b306b"},"outputs":[{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["\n","<style>\n","    @import url('https://fonts.googleapis.com/css2?family=Montserrat:wght@300;400;500;600;700&display=swap');\n","    @import url('https://fonts.googleapis.com/css2?family=Vistol Regular:wght@300;400;500;600;700&display=swap');\n","    \n","    .spark-nlp-display-scroll-entities {\n","        border: 1px solid #E7EDF0;\n","        border-radius: 3px;\n","        text-align: justify;\n","        \n","    }\n","    .spark-nlp-display-scroll-entities span {  \n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #536B76;\n","        font-family: 'Montserrat', sans-serif !important;\n","    }\n","    \n","    .spark-nlp-display-entity-wrapper{\n","    \n","        display: inline-grid;\n","        text-align: center;\n","        border-radius: 4px;\n","        margin: 0 2px 5px 2px;\n","        padding: 1px\n","    }\n","    .spark-nlp-display-entity-name{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        background: #f1f2f3;\n","        border-width: medium;\n","        text-align: center;\n","        \n","        font-weight: 400;\n","        \n","        border-radius: 5px;\n","        padding: 2px 5px;\n","        display: block;\n","        margin: 3px 2px;\n","    \n","    }\n","    .spark-nlp-display-entity-type{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-entity-resolution{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Vistol Regular', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-others{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        font-weight: 400;\n","    }\n","\n","</style>\n"," <span class=\"spark-nlp-display-others\" style=\"background-color: white\"><br>The address of the manufacturer:<br></span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #02C84B\"><span class=\"spark-nlp-display-entity-name\">R K Industry House </span><span class=\"spark-nlp-display-entity-type\">LOCATION</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #02C84B\"><span class=\"spark-nlp-display-entity-name\">Walbhat Rd </span><span class=\"spark-nlp-display-entity-type\">LOCATION</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"><br></span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #02C84B\"><span class=\"spark-nlp-display-entity-name\">Mumbai </span><span class=\"spark-nlp-display-entity-type\">LOCATION</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #02C84B\"><span class=\"spark-nlp-display-entity-name\">Maharashtra </span><span class=\"spark-nlp-display-entity-type\">LOCATION</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\">, </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #02C84B\"><span class=\"spark-nlp-display-entity-name\">India </span><span class=\"spark-nlp-display-entity-type\">LOCATION</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"><br></span></div>"]},"metadata":{}}],"source":["#from sparknlp_display import NerVisualizer\n","\n","text = \"\"\"\n","The address of the manufacturer:\n","R K Industry House, Walbhat Rd\n","Mumbai, Maharashtra, India\n","\"\"\"\n","\n","lmodel= nlp.LightPipeline(ner_converter_model)\n","res = lmodel.fullAnnotate(text)[0]\n","\n","nlp.viz.NerVisualizer().display(res, 'chunk_deid')"]},{"cell_type":"markdown","metadata":{"id":"YX1EApSwx2jn"},"source":["Now, let's define some characters and words with the `setIgnoreStopWords()` parameter and see the difference between the chunks."]},{"cell_type":"code","execution_count":82,"metadata":{"id":"w5hDhPlTxmbS","executionInfo":{"status":"ok","timestamp":1712935288928,"user_tz":-180,"elapsed":2603,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["jsl_ner_converter_internal= medical.NerConverterInternal()\\\n","    .setInputCols([\"sentence\", \"token\", \"ner_deid\"])\\\n","    .setOutputCol(\"chunk_deid\")\\\n","    .setGreedyMode(True)\\\n","    .setWhiteList(['LOCATION'])\\\n","    .setIgnoreStopWords(['\\n', ',', \"and\", 'or', '.'])\n","\n","nlpPipeline = nlp.Pipeline(\n","    stages=[\n","        documentAssembler,\n","        sentenceDetector,\n","        tokenizer,\n","        word_embeddings,\n","        jsl_ner,\n","        jsl_ner_converter_internal\n","])\n","\n","empty_data = spark.createDataFrame([[\"\"]]).toDF(\"text\")\n","ner_converter_model = nlpPipeline.fit(empty_data)"]},{"cell_type":"code","execution_count":83,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":207},"executionInfo":{"elapsed":6,"status":"ok","timestamp":1712935288928,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"EDMNk__vxmOS","outputId":"dd8dc740-5cec-4b8f-deb2-87d9c47f0508"},"outputs":[{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["\n","<style>\n","    @import url('https://fonts.googleapis.com/css2?family=Montserrat:wght@300;400;500;600;700&display=swap');\n","    @import url('https://fonts.googleapis.com/css2?family=Vistol Regular:wght@300;400;500;600;700&display=swap');\n","    \n","    .spark-nlp-display-scroll-entities {\n","        border: 1px solid #E7EDF0;\n","        border-radius: 3px;\n","        text-align: justify;\n","        \n","    }\n","    .spark-nlp-display-scroll-entities span {  \n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #536B76;\n","        font-family: 'Montserrat', sans-serif !important;\n","    }\n","    \n","    .spark-nlp-display-entity-wrapper{\n","    \n","        display: inline-grid;\n","        text-align: center;\n","        border-radius: 4px;\n","        margin: 0 2px 5px 2px;\n","        padding: 1px\n","    }\n","    .spark-nlp-display-entity-name{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        background: #f1f2f3;\n","        border-width: medium;\n","        text-align: center;\n","        \n","        font-weight: 400;\n","        \n","        border-radius: 5px;\n","        padding: 2px 5px;\n","        display: block;\n","        margin: 3px 2px;\n","    \n","    }\n","    .spark-nlp-display-entity-type{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-entity-resolution{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Vistol Regular', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-others{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        font-weight: 400;\n","    }\n","\n","</style>\n"," <span class=\"spark-nlp-display-others\" style=\"background-color: white\"><br>The address of the manufacturer:<br></span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #017F33\"><span class=\"spark-nlp-display-entity-name\">R K Industry House, Walbhat Rd </span><span class=\"spark-nlp-display-entity-type\">LOCATION</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"><br></span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #017F33\"><span class=\"spark-nlp-display-entity-name\">Mumbai, Maharashtra, India </span><span class=\"spark-nlp-display-entity-type\">LOCATION</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"><br></span></div>"]},"metadata":{}}],"source":["lmodel= nlp.LightPipeline(ner_converter_model)\n","res = lmodel.fullAnnotate(text)[0]\n","\n","nlp.viz.NerVisualizer().display(res, 'chunk_deid')"]},{"cell_type":"markdown","metadata":{"id":"c9-Oy0t25tu6"},"source":["## Downloading Pretrained Models\n","\n","- When we use `.pretrained` method, model is downloaded to  a folder named `cache_pretrained` automatically and it is loaded from this folder if you run it again.\n","\n","- In order to download the models manually to any folder, you can use `ResourceDownloader.downloadModelDirectly` from `sparknlp.pretrained` or AWS CLI (steps below). In this case you should use `.load()` method.\n","\n","  - Install AWS CLI to your local computer following the steps [here](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-linux.html) for Linux and [here](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-mac.html) for MacOS.\n","\n","  - Then configure your AWS credentials.\n","\n","  - Go to models hub and look for the model you need.\n","\n","  - Select the model you found and you will see the model card that shows all the details about that model.\n","\n","  - Hover the Download button on that page and you will see the download link from the S3 bucket.\n"]},{"cell_type":"code","execution_count":84,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":5,"status":"ok","timestamp":1712935288928,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"4T3_nTI4QZRV","outputId":"30bca97b-e530-4276-c94e-4226d0454e92"},"outputs":[{"output_type":"stream","name":"stdout","text":["cache_pretrained\n"]}],"source":["!cd /root && ls"]},{"cell_type":"code","execution_count":85,"metadata":{"id":"ni6oA3MqP6cM","executionInfo":{"status":"ok","timestamp":1712935290305,"user_tz":-180,"elapsed":1380,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["#The first argument is the path to the zip file and the second one is the folder.\n","nlp.ResourceDownloader.downloadModelDirectly(\"clinical/models/ner_jsl_en_3.1.0_2.4_1624566960534.zip\", \"cache_pretrained\")\n","\n","#or you can use the classic AWS CLI\n","# !aws s3 cp --region us-east-2 s3://auxdata.johnsnowlabs.com/clinical/models/ner_jsl_en_3.1.0_2.4_1624566960534.zip ."]},{"cell_type":"markdown","metadata":{"id":"FcOwTCgB-VnC"},"source":["## Training a Clinical NER (NCBI Disease Dataset)"]},{"cell_type":"markdown","metadata":{"id":"LxOVZe5dgFaW"},"source":["`CoNLL(includeDocId=True).readDataset(spark, \"conll_file.txt\")`  this methos can allow if you have doc_id information in the conll file, you can add this information to the dataframe as a column.\n","\n","```\n","conll=\"\"\"-DOCSTART- -X- -1- O\n","\n","EU NNP B-NP B-ORG\n","rejects VBZ B-VP O\n","\n","-DOCSTART- -X- 2 O\n","\n","Rare NNP B-NP O\n","Hendrix NNP I-NP B-PER\n","\n","-DOCSTART- -X- -3-1- O\n","\n","China NNP B-NP B-LOC\n","says VBZ B-VP O\n","\n","-DOCSTART-\n","\n","China NNP B-NP B-LOC\n","says VBZ B-VP O\n","\"\"\"\n","```\n","```\n","with open('conll_file.txt', 'w') as f:\n","    f.write(conll)\n","\n","data = CoNLL(includeDocId=True).readDataset(spark, \"conll_file.txt\")\n","\n","data.show()\n","```\n","\n","\n","\n","```\n","+------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+\n","|doc_id|                text|            document|            sentence|               token|                 pos|               label|\n","+------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+\n","|     1|EU rejects German...|[{document, 0, 28...|[{document, 0, 47...|[{token, 0, 1, EU...|[{pos, 0, 1, NNP,...|[{named_entity, 0...|\n","|     2|Rare Hendrix song...|[{document, 0, 97...|[{document, 0, 50...|[{token, 0, 3, Ra...|[{pos, 0, 3, NNP,...|[{named_entity, 0...|\n","|   3-1|China says Taiwan...|[{document, 0, 13...|[{document, 0, 46...|[{token, 0, 4, Ch...|[{pos, 0, 4, NNP,...|[{named_entity, 0...|\n","|     X|China says Taiwan...|[{document, 0, 13...|[{document, 0, 46...|[{token, 0, 4, Ch...|[{pos, 0, 4, NNP,...|[{named_entity, 0...|\n","+------+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+\n","```"]},{"cell_type":"code","execution_count":86,"metadata":{"id":"vxPi21kH-Zbj","executionInfo":{"status":"ok","timestamp":1712935291675,"user_tz":-180,"elapsed":1372,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["!wget -q https://raw.githubusercontent.com/JohnSnowLabs/spark-nlp-workshop/master/tutorials/Certification_Trainings/Healthcare/data/NER_NCBIconlltrain.txt\n","!wget -q https://raw.githubusercontent.com/JohnSnowLabs/spark-nlp-workshop/master/tutorials/Certification_Trainings/Healthcare/data/NER_NCBIconlltest.txt"]},{"cell_type":"code","execution_count":87,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2621,"status":"ok","timestamp":1712935294295,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"thHY_1pi-cLu","outputId":"e503e557-3d04-4745-813e-549e822ae88d"},"outputs":[{"output_type":"stream","name":"stdout","text":["+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+\n","|                text|            document|            sentence|               token|                 pos|               label|\n","+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+\n","|Identification of...|[{document, 0, 89...|[{document, 0, 89...|[{token, 0, 13, I...|[{pos, 0, 13, NN,...|[{named_entity, 0...|\n","|The adenomatous p...|[{document, 0, 21...|[{document, 0, 21...|[{token, 0, 2, Th...|[{pos, 0, 2, NN, ...|[{named_entity, 0...|\n","|Complex formation...|[{document, 0, 63...|[{document, 0, 63...|[{token, 0, 6, Co...|[{pos, 0, 6, NN, ...|[{named_entity, 0...|\n","+--------------------+--------------------+--------------------+--------------------+--------------------+--------------------+\n","only showing top 3 rows\n","\n"]}],"source":["conll_data = nlp.CoNLL().readDataset(spark, 'NER_NCBIconlltrain.txt')\n","\n","conll_data.show(3)"]},{"cell_type":"code","execution_count":88,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1184,"status":"ok","timestamp":1712935295476,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"83cbUZVM-9L1","outputId":"05090553-8912-4419-b020-2261e5094114"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["3266"]},"metadata":{},"execution_count":88}],"source":["conll_data.count()"]},{"cell_type":"code","execution_count":89,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1290,"status":"ok","timestamp":1712935296764,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"nh5enx4p-spM","outputId":"7d4d6410-a70e-483c-d913-8368864391a1"},"outputs":[{"output_type":"stream","name":"stdout","text":["+------------+-----+\n","|ground_truth|count|\n","+------------+-----+\n","|O           |75093|\n","|I-Disease   |3547 |\n","|B-Disease   |3093 |\n","+------------+-----+\n","\n"]}],"source":["\n","conll_data.select(F.explode(F.arrays_zip(conll_data.token.result,\n","                                         conll_data.label.result)).alias(\"cols\")) \\\n","          .select(F.expr(\"cols['0']\").alias(\"token\"),\n","                  F.expr(\"cols['1']\").alias(\"ground_truth\"))\\\n","          .groupBy('ground_truth')\\\n","          .count()\\\n","          .orderBy('count', ascending=False)\\\n","          .show(100,truncate=False)"]},{"cell_type":"code","execution_count":90,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1151,"status":"ok","timestamp":1712935297913,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"48XUc9WX_fjs","outputId":"af783abf-9af8-40f5-e7a5-ed51cfae3b0b"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["1537"]},"metadata":{},"execution_count":90}],"source":["conll_data.select(\"label.result\").distinct().count()"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Rkg9IcR--_PF"},"outputs":[],"source":["'''\n","As you can see, there are too many `O` labels in the dataset.\n","To make it more balanced, we can drop the sentences have only O labels.\n","(`c>1` means we drop all the sentences that have no valuable labels other than `O`)\n","'''\n","\n","'''\n","conll_data = conll_data.withColumn('unique', F.array_distinct(\"label.result\"))\\\n","                       .withColumn('c', F.size('unique'))\\\n","                       .filter(F.col('c')>1)\n","\n","conll_data.select(F.explode(F.arrays_zip(conll_data.token.result,conll_data.label.result)).alias(\"cols\")) \\\n","          .select(F.expr(\"cols['0']\").alias(\"token\"),\n","                  F.expr(\"cols['1']\").alias(\"ground_truth\"))\\\n","          .groupBy('ground_truth')\\\n","          .count()\\\n","          .orderBy('count', ascending=False)\\\n","          .show(100,truncate=False)\n","'''"]},{"cell_type":"code","execution_count":92,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":4085,"status":"ok","timestamp":1712935301994,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"mTopTnOJAV51","outputId":"73203ea6-aa14-46c1-a082-1c30503e4708"},"outputs":[{"output_type":"stream","name":"stdout","text":["embeddings_clinical download started this may take some time.\n","Approximate size to download 1.6 GB\n","[OK!]\n"]}],"source":["# Clinical word embeddings trained on PubMED dataset\n","clinical_embeddings = nlp.WordEmbeddingsModel.pretrained('embeddings_clinical', \"en\", \"clinical/models\")\\\n","    .setInputCols([\"sentence\", \"token\"])\\\n","    .setOutputCol(\"embeddings\")"]},{"cell_type":"code","execution_count":93,"metadata":{"id":"lRNBxjqkQO1U","executionInfo":{"status":"ok","timestamp":1712935305322,"user_tz":-180,"elapsed":3337,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["test_data = nlp.CoNLL().readDataset(spark, 'NER_NCBIconlltest.txt')\n","\n","test_data = clinical_embeddings.transform(test_data)\n","\n","test_data.write.mode(\"overwrite\").parquet('NER_NCBIconlltest.parquet')"]},{"cell_type":"markdown","metadata":{"id":"dPNI5AQ4W4oE"},"source":["### NERDL Graph\n"]},{"cell_type":"markdown","metadata":{"id":"XA2JSVK-wDBy"},"source":["We will use `TFGraphBuilder` annotator which can be used to create graphs in the model training pipeline. `TFGraphBuilder` inspects the data and creates the proper graph if a suitable version of TensorFlow is available. The graph is stored in the defined folder and loaded by the `MedicalNerApproach` annotator."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"9JaZJg06hLM0"},"outputs":[],"source":["!pip install -q tensorflow==2.12.0\n","!pip install -q tensorflow-addons"]},{"cell_type":"code","execution_count":95,"metadata":{"id":"wSQ4X33bwZL1","executionInfo":{"status":"ok","timestamp":1712935379447,"user_tz":-180,"elapsed":9,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["graph_folder_path = \"medical_ner_graphs\"\n","\n","ner_graph_builder = medical.TFGraphBuilder()\\\n","    .setModelName(\"ner_dl\")\\\n","    .setInputCols([\"sentence\", \"token\", \"embeddings\"]) \\\n","    .setLabelColumn(\"label\")\\\n","    .setGraphFolder(graph_folder_path)\\\n","    .setGraphFile(\"auto\")\\\n","    .setHiddenUnitsNumber(20)\\\n","    .setIsLicensed(True) # False -> if you want to use TFGraphBuilder with NerDLApproach"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"znd3vD3_0y8L"},"outputs":[],"source":["# TensorFlow graph file (`.pb` extension) can be produced for NER training externally.\n","# If this method is used, graph folder should be added to MedicalNerApproach training\n","# model as `.setGraphFolder(graph_folder_path)` .\n","\n","'''\n","medical.tf_graph.print_model_params(\"ner_dl\")\n","\n","graph_folder_path = \"medical_ner_graphs\"\n","\n","medical.tf_graph.build(\"ner_dl\",\n","               build_params={\"embeddings_dim\": 200,\n","                             \"nchars\": 85,\n","                             \"ntags\": 3,\n","                             \"is_medical\": 1},\n","               model_location=graph_folder_path,\n","               model_filename=\"auto\")\n","'''\n","\n","# for open source users\n","\n","'''\n","!wget -q https://raw.githubusercontent.com/JohnSnowLabs/spark-nlp-workshop/master/jupyter/training/english/dl-ner/nerdl-graph/create_graph.py\n","!wget -q https://raw.githubusercontent.com/JohnSnowLabs/spark-nlp-workshop/master/jupyter/training/english/dl-ner/nerdl-graph/dataset_encoder.py\n","!wget -q https://raw.githubusercontent.com/JohnSnowLabs/spark-nlp-workshop/master/jupyter/training/english/dl-ner/nerdl-graph/ner_model.py\n","!wget -q https://raw.githubusercontent.com/JohnSnowLabs/spark-nlp-workshop/master/jupyter/training/english/dl-ner/nerdl-graph/ner_model_saver.py\n","!wget -q https://raw.githubusercontent.com/JohnSnowLabs/spark-nlp-workshop/master/jupyter/training/english/dl-ner/nerdl-graph/sentence_grouper.py\n","\n","!pip -q install tensorflow==1.15.0\n","\n","import create_graph\n","\n","ntags = 3 # number of labels\n","embeddings_dim = 200\n","nchars =83\n","\n","create_graph.create_graph(ntags, embeddings_dim, nchars)\n","'''"]},{"cell_type":"markdown","metadata":{"id":"Vd8mR5eAYwc7"},"source":["We will use `setEarlyStoppingCriterion` and `setEarlyStoppingPatience` parameters to stop training if it doesn't improve any more. Also, these parameters will provide us to save time and resources.\n","\n","- `setEarlyStoppingCriterion(float)`: This parameter is used to set the minimal improvement of the test metric to terminate training. The metric monitored is the same which is used to save the best best model (macro F1 if using test or validation set, loss otherwise). If it is F1, then training is stopped if `F1_current - F1_previous < earlyStoppingCriterion`. If monitoring loss, then training is stopped if `loss_previous - loss_current < earlyStoppingCriterion`. If earlyStoppingCriterion is `0`, then no early stopping is applied. Default is `0`.\n","\n","- `setEarlyStoppingPatience(int)`: This is the number of epoch without improvement which will be tolerated. Default is `0`, which means that early stopping will occur at the first time when performance in the current epoch is no better than in the previous epoch (taking earlyStoppingCriterion into account)."]},{"cell_type":"code","execution_count":98,"metadata":{"id":"WzVgfInoJS3I","executionInfo":{"status":"ok","timestamp":1712935379448,"user_tz":-180,"elapsed":8,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["nerTagger = medical.NerApproach()\\\n","    .setInputCols([\"sentence\", \"token\", \"embeddings\"])\\\n","    .setLabelColumn(\"label\")\\\n","    .setOutputCol(\"ner\")\\\n","    .setMaxEpochs(30)\\\n","    .setBatchSize(64)\\\n","    .setRandomSeed(0)\\\n","    .setVerbose(1)\\\n","    .setValidationSplit(0.2)\\\n","    .setEvaluationLogExtended(True) \\\n","    .setEnableOutputLogs(True)\\\n","    .setIncludeConfidence(True)\\\n","    .setOutputLogsPath('ner_logs')\\\n","    .setTestDataset(\"NER_NCBIconlltest.parquet\")\\\n","    .setUseBestModel(True)\\\n","    .setEarlyStoppingCriterion(0.04)\\\n","    .setEarlyStoppingPatience(3)\\\n","    .setGraphFolder(graph_folder_path)\n","   # .setEnableMemoryOptimizer(True) #>> if you have a limited memory and a large conll file, you can set this True to train batch by batch\n","\n","ner_pipeline = nlp.Pipeline(stages=[\n","          clinical_embeddings,\n","          ner_graph_builder,\n","          nerTagger\n"," ])"]},{"cell_type":"markdown","metadata":{"id":"goG7cYUi2_09"},"source":["You can visit [1.4.Resume_MedicalNer_Model_Training.ipynb](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/1.4.Resume_MedicalNer_Model_Training.ipynb) notebook for fine-tuning pretrained NER models and more details of `MedicalNerApproach()` parameters."]},{"cell_type":"code","execution_count":99,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":234693,"status":"ok","timestamp":1712935614133,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"aiXOjTzpBdrQ","outputId":"23965c7f-378b-4ca8-c7dc-825d52e9aca8"},"outputs":[{"output_type":"stream","name":"stdout","text":["TF Graph Builder configuration:\n","Model name: ner_dl\n","Graph folder: medical_ner_graphs\n","Graph file name: auto\n","Build params: {'ntags': 3, 'embeddings_dim': 200, 'nchars': 85, 'is_medical': True, 'lstm_size': 20}\n"]},{"output_type":"stream","name":"stderr","text":["WARNING:tensorflow:From /usr/local/lib/python3.10/dist-packages/tensorflow/python/compat/v2_compat.py:107: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.\n","Instructions for updating:\n","non-resource variables are not supported in the long term\n","WARNING:tensorflow:From /usr/local/lib/python3.10/dist-packages/tensorflow/python/ops/init_ops.py:93: calling GlorotUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Call initializer instance with the dtype argument instead of passing it to the constructor\n","WARNING:tensorflow:From /usr/local/lib/python3.10/dist-packages/tensorflow/python/ops/init_ops.py:93: calling Orthogonal.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Call initializer instance with the dtype argument instead of passing it to the constructor\n","WARNING:tensorflow:From /usr/local/lib/python3.10/dist-packages/tensorflow/python/ops/init_ops.py:93: calling Zeros.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n","Instructions for updating:\n","Call initializer instance with the dtype argument instead of passing it to the constructor\n"]},{"output_type":"stream","name":"stdout","text":["ner_dl graph exported to medical_ner_graphs/blstm_3_200_20_85.pb\n","CPU times: user 15.1 s, sys: 614 ms, total: 15.7 s\n","Wall time: 3min 54s\n"]}],"source":["%%time\n","ner_model = ner_pipeline.fit(conll_data)"]},{"cell_type":"markdown","metadata":{"id":"y-SOgKz4yow0"},"source":["If you get an error for incompatible TF graph, use [4.1 NerDL-Graph](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings_JSL/Public/4.1_NerDL_Graph.ipynb) notebook in public folder to create a graph\n","\n","Licensed users can also use [17.Graph_builder_for_DL_models.ipynb](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/17.Graph_builder_for_DL_models.ipynb) to create TF graphs easily."]},{"cell_type":"markdown","metadata":{"id":"Huet0xVDzRee"},"source":["\n","`getTrainingClassDistribution()` parameter returns the distribution of labels used when training the NER model."]},{"cell_type":"code","execution_count":100,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":17,"status":"ok","timestamp":1712935614133,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"CbWpSBNBLxx2","outputId":"d0aa48a0-7bde-4fe7-f9bc-436020c0d967"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'O': 75093, 'B-Disease': 3093, 'I-Disease': 3547}"]},"metadata":{},"execution_count":100}],"source":["ner_model.stages[2].getTrainingClassDistribution()"]},{"cell_type":"markdown","metadata":{"id":"wG1Md0T-u7He"},"source":["Let's check the results saved in the log file."]},{"cell_type":"code","execution_count":101,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":16,"status":"ok","timestamp":1712935614133,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"TtrxZvAwtZIo","outputId":"58a5486a-122e-421e-aa79-3fb4441a3e7e"},"outputs":[{"output_type":"stream","name":"stdout","text":["Name of the selected graph: /content/medical_ner_graphs/blstm_3_200_20_85.pb\n","Training started - total epochs: 30 - lr: 0.001 - batch size: 64 - labels: 3 - chars: 84 - training examples: 3266\n","\n","\n","Epoch 1/30 started, lr: 0.001, dataset size: 3266\n","\n","\n","Epoch 1/30 - 14.82s - loss: 637.2786 - avg training loss: 14.820434 - batches: 43\n","Quality on validation dataset (20.0%), validation examples = 653\n","time to finish evaluation: 1.61s\n","Total validation loss: 100.9458\tAvg validation loss: 7.2104\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 1\t 0\t 697\t 1.0\t 0.0014326648\t 0.0028612304\n","B-Disease\t 0\t 0\t 663\t 0.0\t 0.0\t 0.0\n","tp: 1 fp: 0 fn: 1360 labels: 2\n","Macro-average\t prec: 0.5, rec: 7.163324E-4, f1: 0.0014306152\n","Micro-average\t prec: 1.0, rec: 7.3475385E-4, f1: 0.0014684286\n","Quality on test dataset: \n","time to finish evaluation: 1.01s\n","Total test loss: 105.8674\tAvg test loss: 7.5620\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 0\t 0\t 789\t 0.0\t 0.0\t 0.0\n","B-Disease\t 0\t 0\t 708\t 0.0\t 0.0\t 0.0\n","tp: 0 fp: 0 fn: 1497 labels: 2\n","Macro-average\t prec: 0.0, rec: 0.0, f1: NaN\n","Micro-average\t prec: 0.0, rec: 0.0, f1: 0.0\n","\n","\n","Epoch 2/30 started, lr: 9.950249E-4, dataset size: 3266\n","\n","\n","Epoch 2/30 - 11.07s - loss: 344.18954 - avg training loss: 8.004408 - batches: 43\n","Quality on validation dataset (20.0%), validation examples = 653\n","time to finish evaluation: 1.06s\n","Total validation loss: 77.0325\tAvg validation loss: 5.5023\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 33\t 6\t 665\t 0.84615386\t 0.04727794\t 0.089552246\n","B-Disease\t 37\t 10\t 626\t 0.78723407\t 0.05580694\t 0.10422535\n","tp: 70 fp: 16 fn: 1291 labels: 2\n","Macro-average\t prec: 0.81669396, rec: 0.05154244, f1: 0.09696529\n","Micro-average\t prec: 0.81395346, rec: 0.05143277, f1: 0.0967519\n","Quality on test dataset: \n","time to finish evaluation: 1.01s\n","Total test loss: 79.9457\tAvg test loss: 5.7104\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 33\t 3\t 756\t 0.9166667\t 0.041825093\t 0.08\n","B-Disease\t 32\t 16\t 676\t 0.6666667\t 0.04519774\t 0.08465609\n","tp: 65 fp: 19 fn: 1432 labels: 2\n","Macro-average\t prec: 0.7916667, rec: 0.043511417, f1: 0.08248909\n","Micro-average\t prec: 0.77380955, rec: 0.043420173, f1: 0.08222643\n","\n","\n","Epoch 3/30 started, lr: 9.90099E-4, dataset size: 3266\n","\n","\n","Epoch 3/30 - 16.58s - loss: 256.67395 - avg training loss: 5.9691615 - batches: 43\n","Quality on validation dataset (20.0%), validation examples = 653\n","time to finish evaluation: 1.11s\n","Total validation loss: 55.7582\tAvg validation loss: 3.9827\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 232\t 84\t 466\t 0.73417723\t 0.3323782\t 0.45759365\n","B-Disease\t 214\t 58\t 449\t 0.7867647\t 0.32277527\t 0.457754\n","tp: 446 fp: 142 fn: 915 labels: 2\n","Macro-average\t prec: 0.760471, rec: 0.32757676, f1: 0.45790753\n","Micro-average\t prec: 0.7585034, rec: 0.32770023, f1: 0.4576706\n","Quality on test dataset: \n","time to finish evaluation: 1.03s\n","Total test loss: 59.0398\tAvg test loss: 4.2171\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 248\t 95\t 541\t 0.72303206\t 0.31432194\t 0.43816257\n","B-Disease\t 231\t 68\t 477\t 0.77257526\t 0.32627118\t 0.45878845\n","tp: 479 fp: 163 fn: 1018 labels: 2\n","Macro-average\t prec: 0.7478037, rec: 0.32029656, f1: 0.44849527\n","Micro-average\t prec: 0.7461059, rec: 0.3199733, f1: 0.44787288\n","\n","\n","Epoch 4/30 started, lr: 9.852217E-4, dataset size: 3266\n","\n","\n","Epoch 4/30 - 11.71s - loss: 200.46725 - avg training loss: 4.6620293 - batches: 43\n","Quality on validation dataset (20.0%), validation examples = 653\n","time to finish evaluation: 1.03s\n","Total validation loss: 47.3427\tAvg validation loss: 3.3816\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 319\t 118\t 379\t 0.72997713\t 0.45702004\t 0.56211454\n","B-Disease\t 301\t 85\t 362\t 0.7797927\t 0.453997\t 0.5738799\n","tp: 620 fp: 203 fn: 741 labels: 2\n","Macro-average\t prec: 0.75488496, rec: 0.45550853, f1: 0.56817317\n","Micro-average\t prec: 0.75334144, rec: 0.4555474, f1: 0.5677656\n","Quality on test dataset: \n","time to finish evaluation: 1.02s\n","Total test loss: 51.2355\tAvg test loss: 3.6597\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 359\t 138\t 430\t 0.722334\t 0.45500633\t 0.5583204\n","B-Disease\t 316\t 90\t 392\t 0.77832514\t 0.4463277\t 0.567325\n","tp: 675 fp: 228 fn: 822 labels: 2\n","Macro-average\t prec: 0.7503296, rec: 0.45066702, f1: 0.56311363\n","Micro-average\t prec: 0.7475083, rec: 0.4509018, f1: 0.56250006\n","\n","\n","Epoch 5/30 started, lr: 9.803922E-4, dataset size: 3266\n","\n","\n","Epoch 5/30 - 10.82s - loss: 180.083 - avg training loss: 4.1879764 - batches: 43\n","Quality on validation dataset (20.0%), validation examples = 653\n","time to finish evaluation: 1.02s\n","Total validation loss: 41.7652\tAvg validation loss: 2.9832\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 410\t 149\t 288\t 0.7334526\t 0.58739257\t 0.6523469\n","B-Disease\t 398\t 109\t 265\t 0.78500986\t 0.6003017\t 0.68034184\n","tp: 808 fp: 258 fn: 553 labels: 2\n","Macro-average\t prec: 0.7592312, rec: 0.59384716, f1: 0.6664319\n","Micro-average\t prec: 0.75797373, rec: 0.5936811, f1: 0.66584265\n","Quality on test dataset: \n","time to finish evaluation: 1.00s\n","Total test loss: 43.6450\tAvg test loss: 3.1175\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 460\t 174\t 329\t 0.725552\t 0.58301646\t 0.6465214\n","B-Disease\t 414\t 117\t 294\t 0.779661\t 0.58474576\t 0.66828084\n","tp: 874 fp: 291 fn: 623 labels: 2\n","Macro-average\t prec: 0.7526065, rec: 0.58388114, f1: 0.6575934\n","Micro-average\t prec: 0.7502146, rec: 0.58383435, f1: 0.6566491\n","\n","\n","Epoch 6/30 started, lr: 9.756098E-4, dataset size: 3266\n","\n","\n","Epoch 6/30 - 10.61s - loss: 164.66263 - avg training loss: 3.8293633 - batches: 43\n","Quality on validation dataset (20.0%), validation examples = 653\n","time to finish evaluation: 1.04s\n","Total validation loss: 38.6400\tAvg validation loss: 2.7600\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 350\t 53\t 348\t 0.86848634\t 0.50143266\t 0.63578564\n","B-Disease\t 408\t 107\t 255\t 0.792233\t 0.61538464\t 0.69269955\n","tp: 758 fp: 160 fn: 603 labels: 2\n","Macro-average\t prec: 0.8303597, rec: 0.5584086, f1: 0.66775715\n","Micro-average\t prec: 0.82570803, rec: 0.5569434, f1: 0.66520405\n","Quality on test dataset: \n","time to finish evaluation: 1.00s\n","Total test loss: 45.2877\tAvg test loss: 3.2348\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 387\t 71\t 402\t 0.84497815\t 0.4904943\t 0.62068963\n","B-Disease\t 431\t 127\t 277\t 0.77240145\t 0.6087571\t 0.68088466\n","tp: 818 fp: 198 fn: 679 labels: 2\n","Macro-average\t prec: 0.80868983, rec: 0.5496257, f1: 0.65445286\n","Micro-average\t prec: 0.8051181, rec: 0.5464262, f1: 0.6510147\n","\n","\n","Epoch 7/30 started, lr: 9.7087387E-4, dataset size: 3266\n","\n","\n","Epoch 7/30 - 11.49s - loss: 155.29387 - avg training loss: 3.6114852 - batches: 43\n","Quality on validation dataset (20.0%), validation examples = 653\n","time to finish evaluation: 1.07s\n","Total validation loss: 35.6490\tAvg validation loss: 2.5464\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 476\t 191\t 222\t 0.7136432\t 0.6819484\t 0.6974359\n","B-Disease\t 451\t 124\t 212\t 0.78434783\t 0.68024135\t 0.7285946\n","tp: 927 fp: 315 fn: 434 labels: 2\n","Macro-average\t prec: 0.74899554, rec: 0.6810949, f1: 0.71343327\n","Micro-average\t prec: 0.7463768, rec: 0.6811168, f1: 0.7122551\n","Quality on test dataset: \n","time to finish evaluation: 1.00s\n","Total test loss: 39.6128\tAvg test loss: 2.8295\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 525\t 217\t 264\t 0.7075472\t 0.66539925\t 0.68582624\n","B-Disease\t 471\t 124\t 237\t 0.79159665\t 0.66525424\t 0.722947\n","tp: 996 fp: 341 fn: 501 labels: 2\n","Macro-average\t prec: 0.7495719, rec: 0.6653267, f1: 0.7049413\n","Micro-average\t prec: 0.74495137, rec: 0.66533065, f1: 0.70289344\n","\n","\n","Epoch 8/30 started, lr: 9.6618367E-4, dataset size: 3266\n","\n","\n","Epoch 8/30 - 11.49s - loss: 142.10468 - avg training loss: 3.30476 - batches: 43\n","Quality on validation dataset (20.0%), validation examples = 653\n","time to finish evaluation: 1.06s\n","Total validation loss: 36.0479\tAvg validation loss: 2.5749\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 392\t 55\t 306\t 0.8769575\t 0.56160456\t 0.6847161\n","B-Disease\t 437\t 109\t 226\t 0.8003663\t 0.6591252\t 0.7229115\n","tp: 829 fp: 164 fn: 532 labels: 2\n","Macro-average\t prec: 0.8386619, rec: 0.6103649, f1: 0.7065291\n","Micro-average\t prec: 0.83484393, rec: 0.60911095, f1: 0.704333\n","Quality on test dataset: \n","time to finish evaluation: 1.00s\n","Total test loss: 38.7947\tAvg test loss: 2.7711\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 433\t 71\t 356\t 0.859127\t 0.54879594\t 0.6697602\n","B-Disease\t 451\t 117\t 257\t 0.7940141\t 0.6370056\t 0.7068965\n","tp: 884 fp: 188 fn: 613 labels: 2\n","Macro-average\t prec: 0.8265705, rec: 0.59290075, f1: 0.6905026\n","Micro-average\t prec: 0.82462686, rec: 0.59051436, f1: 0.68820554\n","\n","\n","Epoch 9/30 started, lr: 9.615385E-4, dataset size: 3266\n","\n","\n","Epoch 9/30 - 12.18s - loss: 128.1541 - avg training loss: 2.9803278 - batches: 43\n","Quality on validation dataset (20.0%), validation examples = 653\n","time to finish evaluation: 1.05s\n","Total validation loss: 35.1161\tAvg validation loss: 2.5083\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 384\t 49\t 314\t 0.88683605\t 0.55014324\t 0.6790451\n","B-Disease\t 429\t 93\t 234\t 0.8218391\t 0.64705884\t 0.7240507\n","tp: 813 fp: 142 fn: 548 labels: 2\n","Macro-average\t prec: 0.8543376, rec: 0.59860104, f1: 0.70396286\n","Micro-average\t prec: 0.8513089, rec: 0.5973549, f1: 0.70207256\n","Quality on test dataset: \n","time to finish evaluation: 1.10s\n","Total test loss: 41.4514\tAvg test loss: 2.9608\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 424\t 68\t 365\t 0.86178863\t 0.5373891\t 0.6619828\n","B-Disease\t 441\t 107\t 267\t 0.80474454\t 0.62288135\t 0.70222926\n","tp: 865 fp: 175 fn: 632 labels: 2\n","Macro-average\t prec: 0.8332666, rec: 0.5801352, f1: 0.6840338\n","Micro-average\t prec: 0.8317308, rec: 0.5778223, f1: 0.6819078\n","\n","\n","Epoch 10/30 started, lr: 9.569379E-4, dataset size: 3266\n","\n","\n","Epoch 10/30 - 12.31s - loss: 126.72783 - avg training loss: 2.9471588 - batches: 43\n","Quality on validation dataset (20.0%), validation examples = 653\n","time to finish evaluation: 1.11s\n","Total validation loss: 29.9304\tAvg validation loss: 2.1379\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 452\t 78\t 246\t 0.8528302\t 0.6475645\t 0.7361564\n","B-Disease\t 496\t 135\t 167\t 0.7860539\t 0.74811465\t 0.76661515\n","tp: 948 fp: 213 fn: 413 labels: 2\n","Macro-average\t prec: 0.81944203, rec: 0.69783956, f1: 0.75376797\n","Micro-average\t prec: 0.81653744, rec: 0.6965467, f1: 0.7517843\n","Quality on test dataset: \n","time to finish evaluation: 1.00s\n","Total test loss: 32.4125\tAvg test loss: 2.3152\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 510\t 93\t 279\t 0.84577113\t 0.6463878\t 0.7327586\n","B-Disease\t 536\t 137\t 172\t 0.79643387\t 0.75706214\t 0.77624905\n","tp: 1046 fp: 230 fn: 451 labels: 2\n","Macro-average\t prec: 0.8211025, rec: 0.701725, f1: 0.7567346\n","Micro-average\t prec: 0.81974924, rec: 0.69873077, f1: 0.75441754\n","\n","\n","Epoch 11/30 started, lr: 9.5238106E-4, dataset size: 3266\n","\n","\n","Epoch 11/30 - 12.08s - loss: 123.620995 - avg training loss: 2.8749068 - batches: 43\n","Quality on validation dataset (20.0%), validation examples = 653\n","time to finish evaluation: 1.05s\n","Total validation loss: 29.4633\tAvg validation loss: 2.1045\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 550\t 190\t 148\t 0.7432432\t 0.7879656\t 0.76495135\n","B-Disease\t 504\t 134\t 159\t 0.78996867\t 0.760181\t 0.7747886\n","tp: 1054 fp: 324 fn: 307 labels: 2\n","Macro-average\t prec: 0.766606, rec: 0.7740733, f1: 0.7703216\n","Micro-average\t prec: 0.7648766, rec: 0.7744306, f1: 0.76962394\n","Quality on test dataset: \n","time to finish evaluation: 1.02s\n","Total test loss: 31.9479\tAvg test loss: 2.2820\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 590\t 228\t 199\t 0.7212714\t 0.747782\t 0.7342875\n","B-Disease\t 537\t 135\t 171\t 0.79910713\t 0.7584746\t 0.77826095\n","tp: 1127 fp: 363 fn: 370 labels: 2\n","Macro-average\t prec: 0.7601893, rec: 0.7531283, f1: 0.75664234\n","Micro-average\t prec: 0.75637585, rec: 0.752839, f1: 0.7546033\n","\n","\n","Epoch 12/30 started, lr: 9.478674E-4, dataset size: 3266\n","\n","\n","Epoch 12/30 - 11.13s - loss: 114.46572 - avg training loss: 2.6619935 - batches: 43\n","Quality on validation dataset (20.0%), validation examples = 653\n","time to finish evaluation: 1.18s\n","Total validation loss: 30.4022\tAvg validation loss: 2.1716\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 452\t 64\t 246\t 0.875969\t 0.6475645\t 0.7446458\n","B-Disease\t 469\t 98\t 194\t 0.8271605\t 0.70739067\t 0.7626017\n","tp: 921 fp: 162 fn: 440 labels: 2\n","Macro-average\t prec: 0.85156476, rec: 0.6774776, f1: 0.75461096\n","Micro-average\t prec: 0.8504155, rec: 0.6767083, f1: 0.75368243\n","Quality on test dataset: \n","time to finish evaluation: 1.07s\n","Total test loss: 31.4122\tAvg test loss: 2.2437\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 505\t 81\t 284\t 0.86177474\t 0.6400507\t 0.73454547\n","B-Disease\t 493\t 104\t 215\t 0.82579565\t 0.6963277\t 0.75555557\n","tp: 998 fp: 185 fn: 499 labels: 2\n","Macro-average\t prec: 0.84378517, rec: 0.66818917, f1: 0.74579054\n","Micro-average\t prec: 0.8436179, rec: 0.6666667, f1: 0.7447761\n","\n","\n","Epoch 13/30 started, lr: 9.433963E-4, dataset size: 3266\n","\n","\n","Epoch 13/30 - 11.46s - loss: 105.74969 - avg training loss: 2.459295 - batches: 43\n","Quality on validation dataset (20.0%), validation examples = 653\n","time to finish evaluation: 1.03s\n","Total validation loss: 26.0859\tAvg validation loss: 1.8633\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 446\t 54\t 252\t 0.892\t 0.63896847\t 0.7445743\n","B-Disease\t 493\t 125\t 170\t 0.7977346\t 0.74358976\t 0.76971114\n","tp: 939 fp: 179 fn: 422 labels: 2\n","Macro-average\t prec: 0.84486735, rec: 0.6912791, f1: 0.76039517\n","Micro-average\t prec: 0.8398927, rec: 0.6899339, f1: 0.75756353\n","Quality on test dataset: \n","time to finish evaluation: 1.00s\n","Total test loss: 30.6030\tAvg test loss: 2.1859\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 508\t 70\t 281\t 0.8788927\t 0.64385295\t 0.7432334\n","B-Disease\t 539\t 124\t 169\t 0.81297135\t 0.76129943\t 0.78628737\n","tp: 1047 fp: 194 fn: 450 labels: 2\n","Macro-average\t prec: 0.845932, rec: 0.70257616, f1: 0.7676184\n","Micro-average\t prec: 0.8436745, rec: 0.6993988, f1: 0.76479185\n","\n","\n","Epoch 14/30 started, lr: 9.389671E-4, dataset size: 3266\n","\n","\n","Epoch 14/30 - 11.48s - loss: 101.847885 - avg training loss: 2.3685555 - batches: 43\n","Quality on validation dataset (20.0%), validation examples = 653\n","time to finish evaluation: 1.05s\n","Total validation loss: 25.2069\tAvg validation loss: 1.8005\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 470\t 57\t 228\t 0.89184064\t 0.6733524\t 0.7673469\n","B-Disease\t 497\t 108\t 166\t 0.8214876\t 0.74962294\t 0.78391176\n","tp: 967 fp: 165 fn: 394 labels: 2\n","Macro-average\t prec: 0.8566641, rec: 0.71148765, f1: 0.77735585\n","Micro-average\t prec: 0.8542403, rec: 0.710507, f1: 0.7757721\n","Quality on test dataset: \n","time to finish evaluation: 0.98s\n","Total test loss: 29.0728\tAvg test loss: 2.0766\n","label\t tp\t fp\t fn\t prec\t rec\t f1\n","I-Disease\t 524\t 70\t 265\t 0.8821549\t 0.6641318\t 0.7577729\n","B-Disease\t 538\t 114\t 170\t 0.82515335\t 0.759887\t 0.79117644\n","tp: 1062 fp: 184 fn: 435 labels: 2\n","Macro-average\t prec: 0.85365415, rec: 0.71200943, f1: 0.7764245\n","Micro-average\t prec: 0.85232747, rec: 0.70941883, f1: 0.77433467\n","Early stopping: test set macro F1 has not improved for more than 3 epochs\n","\n"]}],"source":["import os\n","log_file= os.listdir(\"ner_logs\")[0]\n","\n","with open (f\"./ner_logs/{log_file}\") as f:\n","  print(f.read())"]},{"cell_type":"markdown","metadata":{"id":"0_rSimOoUa8v"},"source":["As you see above, our **earlyStopping** feature worked, trainining was terminated before 30th epoch."]},{"cell_type":"markdown","metadata":{"id":"rUehS3qTdHUh"},"source":["### Evaluate your model"]},{"cell_type":"code","execution_count":102,"metadata":{"id":"5GV4YaeHcsHB","executionInfo":{"status":"ok","timestamp":1712935614134,"user_tz":-180,"elapsed":15,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["pred_df = ner_model.stages[2].transform(test_data)"]},{"cell_type":"code","execution_count":103,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":14,"status":"ok","timestamp":1712935614134,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"tICjE0Yqc2Ri","outputId":"c84aaf24-fe80-4657-a04f-f7c80619d7bb"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["['text', 'document', 'sentence', 'token', 'pos', 'label', 'embeddings', 'ner']"]},"metadata":{},"execution_count":103}],"source":["pred_df.columns"]},{"cell_type":"code","execution_count":104,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":8751,"status":"ok","timestamp":1712935622879,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"B9Qc_oukcj94","outputId":"a3a57e0d-9517-4560-9e4c-03573e1192eb"},"outputs":[{"output_type":"stream","name":"stdout","text":["+-------+-----+-----+-----+-----+---------+------+------+\n","| entity|   tp|   fp|   fn|total|precision|recall|    f1|\n","+-------+-----+-----+-----+-----+---------+------+------+\n","|Disease|515.0|187.0|189.0|704.0|   0.7336|0.7315|0.7326|\n","+-------+-----+-----+-----+-----+---------+------+------+\n","\n","+-----------------+\n","|            macro|\n","+-----------------+\n","|0.732574679943101|\n","+-----------------+\n","\n","None\n","+-----------------+\n","|            micro|\n","+-----------------+\n","|0.732574679943101|\n","+-----------------+\n","\n","None\n"]}],"source":["evaler = medical.NerDLMetrics(mode=\"full_chunk\")\n","\n","eval_result = evaler.computeMetricsFromDF(pred_df.select(\"label\",\"ner\"), prediction_col=\"ner\", label_col=\"label\", drop_o = True, case_sensitive = True).cache()\n","\n","eval_result.withColumn(\"precision\", F.round(eval_result[\"precision\"],4))\\\n","           .withColumn(\"recall\", F.round(eval_result[\"recall\"],4))\\\n","           .withColumn(\"f1\", F.round(eval_result[\"f1\"],4)).show(100)\n","\n","print(eval_result.selectExpr(\"avg(f1) as macro\").show())\n","print (eval_result.selectExpr(\"sum(f1*total) as sumprod\",\"sum(total) as sumtotal\").selectExpr(\"sumprod/sumtotal as micro\").show())"]},{"cell_type":"code","execution_count":105,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":7878,"status":"ok","timestamp":1712935630749,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"nh1LY2cwdApE","outputId":"f209c88b-d332-42a9-905e-9ba28e9c96e0"},"outputs":[{"output_type":"stream","name":"stdout","text":["+-------+------+----+-----+------+---------+------+------+\n","| entity|    tp|  fp|   fn| total|precision|recall|    f1|\n","+-------+------+----+-----+------+---------+------+------+\n","|Disease|1149.0|99.0|348.0|1497.0|   0.9207|0.7675|0.8372|\n","+-------+------+----+-----+------+---------+------+------+\n","\n","+------------------+\n","|             macro|\n","+------------------+\n","|0.8371584699453551|\n","+------------------+\n","\n","None\n","+------------------+\n","|             micro|\n","+------------------+\n","|0.8371584699453551|\n","+------------------+\n","\n","None\n"]}],"source":["evaler = medical.NerDLMetrics(mode=\"partial_chunk_per_token\")\n","\n","eval_result = evaler.computeMetricsFromDF(pred_df.select(\"label\",\"ner\"), prediction_col=\"ner\", label_col=\"label\", drop_o = True, case_sensitive = True).cache()\n","\n","eval_result.withColumn(\"precision\", F.round(eval_result[\"precision\"],4))\\\n","           .withColumn(\"recall\", F.round(eval_result[\"recall\"],4))\\\n","           .withColumn(\"f1\", F.round(eval_result[\"f1\"],4)).show(100)\n","\n","print(eval_result.selectExpr(\"avg(f1) as macro\").show())\n","print (eval_result.selectExpr(\"sum(f1*total) as sumprod\",\"sum(total) as sumtotal\").selectExpr(\"sumprod/sumtotal as micro\").show())"]},{"cell_type":"markdown","metadata":{"id":"_iNzZYVz0P5B"},"source":["`ner_utils`: This new module is used after NER training to calculate mertic chunkbase and plot training logs.\n","\n","`evaluate`: if verbose, returns overall performance, as well as performance per chunk type; otherwise, simply returns overall precision, recall, f1 scores\n","\n","`loss_plot`: Plots the figure of loss vs epochs\n","\n","`get_charts` : Plots the figures of metrics ( precision, recall, f1) vs epochs\n","\n","```\n","import sparknlp_jsl\n","from sparknlp_jsl.training_log_parser import ner_log_parser\n","parser = ner_log_parser()\n","\n","pred_df = ner_model.stages[2].transform(test_data)\n","\n","pred_df = pred_df.select(F.explode(F.arrays_zip(pred_df.ner.result,\n","                                                pred_df.label.result)).alias(\"cols\"))\\\n","                 .select(F.expr(\"cols['0']\").alias(\"prediction\"),\n","                         F.expr(\"cols['1']\").alias(\"ground_truth\"))\n","                 \n","df = pred_df.toPandas()\n","\n","\n","metrics = parser.evaluate( df['prediction'].values, df['ground_truth'].values)\n","\n","parser.loss_plot(f\"./ner_logs/{log_file}\")\n","\n","parser.get_charts('./ner_logs/'+log_file)\n","```"]},{"cell_type":"code","execution_count":106,"metadata":{"id":"SdlTQl4eJmvl","executionInfo":{"status":"ok","timestamp":1712935631581,"user_tz":-180,"elapsed":834,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["ner_model.stages[2].write().overwrite().save('models/custom_NER_model')"]},{"cell_type":"code","execution_count":107,"metadata":{"id":"ZEzG0T1vEZvV","executionInfo":{"status":"ok","timestamp":1712935633952,"user_tz":-180,"elapsed":2374,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["document = nlp.DocumentAssembler()\\\n","    .setInputCol(\"text\")\\\n","    .setOutputCol(\"document\")\n","\n","sentence = nlp.SentenceDetector()\\\n","    .setInputCols(['document'])\\\n","    .setOutputCol('sentence')\n","\n","token = nlp.Tokenizer()\\\n","    .setInputCols(['sentence'])\\\n","    .setOutputCol('token')\n","\n","loaded_ner_model = medical.NerModel.load(\"models/custom_NER_model\")\\\n","    .setInputCols([\"sentence\", \"token\", \"embeddings\"])\\\n","    .setOutputCol(\"ner\")\n","\n","converter = medical.NerConverterInternal()\\\n","    .setInputCols([\"document\", \"token\", \"ner\"])\\\n","    .setOutputCol(\"ner_span\")\n","\n","ner_prediction_pipeline = nlp.Pipeline(stages=[\n","    document,\n","    sentence,\n","    token,\n","    clinical_embeddings,\n","    loaded_ner_model,\n","    converter])\n","\n","empty_data = spark.createDataFrame([['']]).toDF(\"text\")\n","\n","prediction_model = ner_prediction_pipeline.fit(empty_data)\n","\n","\n","light_model = nlp.LightPipeline(prediction_model)"]},{"cell_type":"code","execution_count":108,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":6,"status":"ok","timestamp":1712935633953,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"kR4172tKJxX3","outputId":"11a65659-8942-4da1-edc7-57314153bcf9"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["[('metastatic breast cancer', 'Disease')]"]},"metadata":{},"execution_count":108}],"source":["text = \"She has a metastatic breast cancer\"\n","\n","result = light_model.fullAnnotate(text)[0]\n","\n","[(i.result, i.metadata['entity']) for i in result['ner_span']]"]},{"cell_type":"markdown","metadata":{"id":"c_T8KEpnij8c"},"source":["## BertForTokenClassification NER models"]},{"cell_type":"code","execution_count":109,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":47242,"status":"ok","timestamp":1712935681190,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"rkhS3P6Hij8d","outputId":"c2998741-b62d-43d6-f9a3-43a86d420ffd"},"outputs":[{"output_type":"stream","name":"stdout","text":["sentence_detector_dl_healthcare download started this may take some time.\n","Approximate size to download 367.3 KB\n","[OK!]\n","bert_token_classifier_ner_clinical download started this may take some time.\n","[OK!]\n"]}],"source":["documentAssembler = nlp.DocumentAssembler()\\\n","    .setInputCol(\"text\")\\\n","    .setOutputCol(\"document\")\n","\n","sentenceDetector = nlp.SentenceDetectorDLModel.pretrained(\"sentence_detector_dl_healthcare\",\"en\",\"clinical/models\")\\\n","    .setInputCols([\"document\"])\\\n","    .setOutputCol(\"sentence\")\n","\n","tokenizer = nlp.Tokenizer()\\\n","    .setInputCols(\"sentence\")\\\n","    .setOutputCol(\"token\")\n","\n","tokenClassifier = medical.BertForTokenClassification.pretrained(\"bert_token_classifier_ner_clinical\", \"en\", \"clinical/models\")\\\n","    .setInputCols(\"token\", \"sentence\")\\\n","    .setOutputCol(\"ner\")\\\n","    .setCaseSensitive(True)\n","\n","ner_converter = medical.NerConverterInternal()\\\n","    .setInputCols([\"sentence\",\"token\",\"ner\"])\\\n","    .setOutputCol(\"ner_chunk\")\n","\n","pipeline =  nlp.Pipeline(stages=[\n","    documentAssembler,\n","    sentenceDetector,\n","    tokenizer,\n","    tokenClassifier,\n","    ner_converter\n","    ])\n","\n","model = pipeline.fit(spark.createDataFrame([[\"\"]]).toDF(\"text\"))\n","\n","text = \"\"\"A 28-year-old female with a history of gestational diabetes mellitus diagnosed eight years prior to presentation and subsequent type two diabetes mellitus ( T2DM ), one prior episode of HTG-induced pancreatitis three years prior to presentation , associated with an acute hepatitis , and obesity with a body mass index ( BMI ) of 33.5 kg/m2 , presented with a one-week history of polyuria , polydipsia , poor appetite , and vomiting .\n","Two weeks prior to presentation , she was treated with a five-day course of amoxicillin for a respiratory tract infection .\n","She was on metformin , glipizide , and dapagliflozin for T2DM and atorvastatin and gemfibrozil for HTG . She had been on dapagliflozin for six months at the time of presentation .\n","Physical examination on presentation was significant for dry oral mucosa ; significantly , her abdominal examination was benign with no tenderness , guarding , or rigidity .\n","Pertinent laboratory findings on admission were : serum glucose 111 mg/dl , bicarbonate 18 mmol/l , anion gap 20 , creatinine 0.4 mg/dL , triglycerides 508 mg/dL , total cholesterol 122 mg/dL , glycated hemoglobin ( HbA1c ) 10% , and venous pH 7.27 .\n","Serum lipase was normal at 43 U/L . Serum acetone levels could not be assessed as blood samples kept hemolyzing due to significant lipemia .\n","The patient was initially admitted for starvation ketosis , as she reported poor oral intake for three days prior to admission .\n","However , serum chemistry obtained six hours after presentation revealed her glucose was 186 mg/dL , the anion gap was still elevated at 21 , serum bicarbonate was 16 mmol/L , triglyceride level peaked at 2050 mg/dL , and lipase was 52 U/L .\n","The β-hydroxybutyrate level was obtained and found to be elevated at 5.29 mmol/L - the original sample was centrifuged and the chylomicron layer removed prior to analysis due to interference from turbidity caused by lipemia again .\n","The patient was treated with an insulin drip for euDKA and HTG with a reduction in the anion gap to 13 and triglycerides to 1400 mg/dL , within 24 hours .\n","Her euDKA was thought to be precipitated by her respiratory tract infection in the setting of SGLT2 inhibitor use .\n","The patient was seen by the endocrinology service and she was discharged on 40 units of insulin glargine at night , 12 units of insulin lispro with meals , and metformin 1000 mg two times a day .\n","It was determined that all SGLT2 inhibitors should be discontinued indefinitely . She had close follow-up with endocrinology post discharge .\"\"\"\n","\n","res = model.transform(spark.createDataFrame([[text]]).toDF(\"text\"))"]},{"cell_type":"code","execution_count":110,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":4616,"status":"ok","timestamp":1712935685797,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"ciii_Ptkij8d","outputId":"c6981c9a-996c-4653-f161-c744ee68e9a8"},"outputs":[{"output_type":"stream","name":"stdout","text":["+-----------+-----------------------------+---+---------+\n","|sentence_id|chunk                        |end|ner_label|\n","+-----------+-----------------------------+---+---------+\n","|0          |gestational diabetes mellitus|67 |PROBLEM  |\n","|0          |type two diabetes mellitus   |153|PROBLEM  |\n","|0          |T2DM                         |160|PROBLEM  |\n","|0          |HTG-induced pancreatitis     |209|PROBLEM  |\n","|0          |an acute hepatitis           |280|PROBLEM  |\n","|0          |obesity                      |294|PROBLEM  |\n","|0          |a body mass index            |317|TEST     |\n","|0          |BMI                          |323|TEST     |\n","|0          |polyuria                     |387|PROBLEM  |\n","|0          |polydipsia                   |400|PROBLEM  |\n","|0          |poor appetite                |416|PROBLEM  |\n","|0          |vomiting                     |431|PROBLEM  |\n","|1          |amoxicillin                  |521|TREATMENT|\n","|1          |a respiratory tract infection|555|PROBLEM  |\n","|2          |metformin                    |578|TREATMENT|\n","|2          |glipizide                    |590|TREATMENT|\n","|2          |dapagliflozin                |610|TREATMENT|\n","|2          |T2DM                         |619|PROBLEM  |\n","|2          |atorvastatin                 |636|TREATMENT|\n","|2          |gemfibrozil                  |652|TREATMENT|\n","+-----------+-----------------------------+---+---------+\n","only showing top 20 rows\n","\n"]}],"source":["res.select(F.explode(F.arrays_zip(res.ner_chunk.result, res.ner_chunk.begin, res.ner_chunk.end, res.ner_chunk.metadata)).alias(\"cols\")) \\\n","   .select(F.expr(\"cols['3']['sentence']\").alias(\"sentence_id\"),\n","           F.expr(\"cols['0']\").alias(\"chunk\"),\n","           F.expr(\"cols['2']\").alias(\"end\"),\n","           F.expr(\"cols['3']['entity']\").alias(\"ner_label\"))\\\n","   .filter(\"ner_label!='O'\")\\\n","   .show(truncate=False)"]},{"cell_type":"code","execution_count":111,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000},"executionInfo":{"elapsed":2632,"status":"ok","timestamp":1712935688419,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"WFw1PgGP_K1S","outputId":"17f9a1f7-8fca-4877-8c2e-4f0fc220e984"},"outputs":[{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["\n","<style>\n","    @import url('https://fonts.googleapis.com/css2?family=Montserrat:wght@300;400;500;600;700&display=swap');\n","    @import url('https://fonts.googleapis.com/css2?family=Vistol Regular:wght@300;400;500;600;700&display=swap');\n","    \n","    .spark-nlp-display-scroll-entities {\n","        border: 1px solid #E7EDF0;\n","        border-radius: 3px;\n","        text-align: justify;\n","        \n","    }\n","    .spark-nlp-display-scroll-entities span {  \n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #536B76;\n","        font-family: 'Montserrat', sans-serif !important;\n","    }\n","    \n","    .spark-nlp-display-entity-wrapper{\n","    \n","        display: inline-grid;\n","        text-align: center;\n","        border-radius: 4px;\n","        margin: 0 2px 5px 2px;\n","        padding: 1px\n","    }\n","    .spark-nlp-display-entity-name{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        background: #f1f2f3;\n","        border-width: medium;\n","        text-align: center;\n","        \n","        font-weight: 400;\n","        \n","        border-radius: 5px;\n","        padding: 2px 5px;\n","        display: block;\n","        margin: 3px 2px;\n","    \n","    }\n","    .spark-nlp-display-entity-type{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-entity-resolution{\n","        font-size: 14px;\n","        line-height: 24px;\n","        color: #ffffff;\n","        font-family: 'Vistol Regular', sans-serif !important;\n","        \n","        text-transform: uppercase;\n","        \n","        font-weight: 500;\n","\n","        display: block;\n","        padding: 3px 5px;\n","    }\n","    \n","    .spark-nlp-display-others{\n","        font-size: 14px;\n","        line-height: 24px;\n","        font-family: 'Montserrat', sans-serif !important;\n","        \n","        font-weight: 400;\n","    }\n","\n","</style>\n"," <span class=\"spark-nlp-display-others\" style=\"background-color: white\">A 28-year-old female with a history of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">gestational diabetes mellitus </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> diagnosed eight years prior to presentation and subsequent </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">type two diabetes mellitus </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> ( </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">T2DM </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> ), one prior episode of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">HTG-induced pancreatitis </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> three years prior to presentation , associated with </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">an acute hepatitis </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> , and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">obesity </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> with </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">a body mass index </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> ( </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">BMI </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> ) of 33.5 kg/m2 , presented with a one-week history of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">polyuria </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">polydipsia </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">poor appetite </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> , and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">vomiting </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> .<br>Two weeks prior to presentation , she was treated with a five-day course of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">amoxicillin </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">a respiratory tract infection </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> .<br>She was on </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">metformin </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">glipizide </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> , and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">dapagliflozin </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">T2DM </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">atorvastatin </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">gemfibrozil </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">HTG </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> . She had been on </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">dapagliflozin </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> for six months at the time of presentation .<br></span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">Physical examination </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> on presentation was significant for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">dry oral mucosa </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> ; significantly , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">her abdominal examination </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> was benign with no </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">tenderness </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">guarding </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> , or </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">rigidity </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> .<br>Pertinent laboratory findings on admission were : </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">serum glucose </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 111 mg/dl , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">bicarbonate </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 18 mmol/l , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">anion gap </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 20 , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">creatinine </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 0.4 mg/dL , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">triglycerides </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 508 mg/dL , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">total cholesterol </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 122 mg/dL , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">glycated hemoglobin ( </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">HbA1c </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">) </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 10% , and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">venous pH </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 7.27 .<br></span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">Serum lipase </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> was normal at 43 U/L . </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">Serum acetone levels </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> could not be assessed as </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">blood samples </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> kept </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">hemolyzing </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> due to </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">significant lipemia </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> .<br>The patient was initially admitted for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">starvation ketosis </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> , as she reported </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">poor oral intake </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> for three days prior to admission .<br>However , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">serum chemistry </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> obtained six hours after presentation revealed </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">her glucose </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> was 186 mg/dL , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">the anion gap </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> was still </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">elevated </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> at 21 , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">serum bicarbonate </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> was 16 mmol/L , </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">triglyceride level </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> peaked at 2050 mg/dL , and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">lipase </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> was 52 U/L .<br></span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">The β-hydroxybutyrate level </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> was obtained and found to be </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">elevated </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> at 5.29 mmol/L - the original sample was centrifuged and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">the chylomicron layer </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> removed prior to </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">analysis </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> due to interference from </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">turbidity </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> caused by </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">lipemia </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> again .<br>The patient was treated with </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">an insulin drip </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> for </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">euDKA </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">HTG </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> with a reduction in </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">the anion gap </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> to 13 and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #77b5fe\"><span class=\"spark-nlp-display-entity-name\">triglycerides </span><span class=\"spark-nlp-display-entity-type\">TEST</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> to 1400 mg/dL , within 24 hours .<br></span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">Her euDKA </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> was thought to be precipitated by </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #800080\"><span class=\"spark-nlp-display-entity-name\">her respiratory tract infection </span><span class=\"spark-nlp-display-entity-type\">PROBLEM</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> in the setting of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">SGLT2 inhibitor use </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> .<br>The patient was seen by the endocrinology service and she was discharged on 40 units of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">insulin glargine </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> at night , 12 units of </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">insulin lispro </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> with meals , and </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">metformin </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> 1000 mg two times a day .<br>It was determined that </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">all </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> </span><span class=\"spark-nlp-display-entity-wrapper\" style=\"background-color: #8b6673\"><span class=\"spark-nlp-display-entity-name\">SGLT2 inhibitors </span><span class=\"spark-nlp-display-entity-type\">TREATMENT</span></span><span class=\"spark-nlp-display-others\" style=\"background-color: white\"> should be discontinued indefinitely . She had close follow-up with endocrinology post discharge .</span></div>"]},"metadata":{}}],"source":["light_model = nlp.LightPipeline(model)\n","\n","light_result = light_model.fullAnnotate(text)\n","\n","\n","visualiser = nlp.viz.NerVisualizer()\n","\n","visualiser.display(light_result[0], label_col='ner_chunk', document_col='document', save_path=\"display_bert_result.html\")"]},{"cell_type":"markdown","metadata":{"id":"M3Whd9kamRqw"},"source":["**Training Clinical BertForTokenClassification Model**\n","\n","For training own BertForTokenClassification NER model, you can check [this notebook](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/1.5.BertForTokenClassification_NER_SparkNLP_with_Transformers.ipynb)"]},{"cell_type":"markdown","metadata":{"id":"QBDG2m3z_A0M"},"source":["## Pretrained NER Profiling Pipelines\n","\n","We can use pretrained NER profiling pipelines for exploring all the available pretrained NER models at once. In Spark NLP we have two different NER profiling pipelines;\n","\n","- `ner_profiling_clinical` : Returns results for clinical NER models trained with `embeddings_clinical`.\n","- `ner_profiling_biobert` : Returns results for clinical NER models trained with `biobert_pubmed_base_cased`.\n","\n","For more examples, please check [this notebook](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/11.2.Pretrained_NER_Profiling_Pipelines.ipynb).\n","\n","\n","\n"]},{"cell_type":"markdown","metadata":{"id":"wUoe5mEEO5Fv"},"source":["<center><b>NER Profiling Clinical Model List</b>\n","\n","|| | | |\n","|--------------|-----------------|-----------------|-----------------|\n","| jsl_ner_wip_clinical               | jsl_ner_wip_greedy_clinical                    | jsl_rd_ner_wip_greedy_clinical   | jsl_ner_wip_modifier_clinical |\n","| ner_abbreviation_clinical          | ner_ade_binary                                 | ner_ade_clinical                 | ner_anatomy          |\n","| ner_anatomy_coarse                  | ner_bacterial_species                          | ner_biomedical_bc2gm             | ner_biomarker        |\n","| ner_bionlp                          | ner_chemicals                                  | ner_chemd_clinical               | ner_chemprot_clinical|\n","| ner_chexpert                        | ner_clinical                                   | ner_clinical_large               | ner_clinical_trials_abstracts  |\n","| ner_covid_trials                    | ner_deid_augmented                             | ner_deid_enriched                | ner_deid_generic_augmented |\n","| ner_deid_large                      | ner_deid_sd                                    | ner_deid_sd_large                | ner_deid_subentity_augmented |\n","| ner_deid_subentity_augmented        | ner_deid_subentity_augmented_i2b2              | ner_diseases                     | ner_diseases_large   |\n","| ner_drugprot_clinical               | ner_drugs                                      | ner_drugs_greedy                 | ner_drugs_large      |\n","| ner_eu_clinical_case                | ner_eu_clinical_condition                       | ner_events_admission_clinical    | ner_events_clinical  |\n","| ner_genetic_variants                | ner_human_phenotype_gene_clinical               | ner_human_phenotype_go_clinical  | ner_jsl              |\n","| ner_jsl_enriched                    | ner_jsl_greedy                                 | ner_jsl_slim                     | ner_living_species   |\n","| ner_measurements_clinical           | ner_medmentions_coarse                          | ner_nature_nero_clinical         | ner_nihss            |\n","| ner_oncology                        | ner_oncology_anatomy_general                    | ner_oncology_anatomy_granular    | ner_oncology_biomarker |\n","| ner_oncology_demographics           | ner_oncology_diagnosis                          | ner_oncology_posology            | ner_oncology_response_to_treatment |\n","| ner_oncology_tnm                    | ner_oncology_therapy                            | ner_oncology_unspecific_posology | ner_oncology_wip     |\n","| ner_pathogen                        | ner_posology                                   | ner_posology_experimental        | ner_posology_greedy  |\n","| ner_posology_large                  | ner_posology_small                              | ner_posology_wip                 | ner_radiology        |\n","| ner_radiology_wip_clinical          | ner_risk_factors                               | ner_sdoh_access_to_healthcare_wip| ner_sdoh_community_condition_wip |\n","| ner_sdoh_demographics_wip           | ner_sdoh_health_behaviours_problems_wip         | ner_sdoh_income_social_status_wip| ner_sdoh_mentions    |\n","| ner_sdoh_social_environment_wip     | ner_sdoh_substance_usage_wip                    | ner_sdoh_wip                     | ner_supplement_clinical |\n","| ner_token                           | ner_vop_anatomy_wip                            | ner_vop_clinical_dept_wip        | ner_vop_demographic_wip |\n","| ner_vop_problem_reduced_wip         | ner_vop_problem_wip                            | ner_vop_slim_wip                 | ner_vop_temporal_wip |\n","| ner_vop_test_wip                    | ner_vop_treatment_wip                           | ner_vop_wip                      |                       |\n","\n","\n","<b>NER Profiling BioBert Model List</b>\n","\n","| | |\n","|-|-|\n","| ner_cellular_biobert           | ner_clinical_biobert             |\n","| ner_diseases_biobert           | ner_anatomy_coarse_biobert       |\n","| ner_events_biobert             | ner_human_phenotype_gene_biobert |\n","| ner_bionlp_biobert             | ner_posology_large_biobert       |\n","| ner_jsl_greedy_biobert         | jsl_rd_ner_wip_greedy_biobert    |\n","| ner_jsl_biobert                | ner_posology_biobert             |\n","| ner_anatomy_biobert            | jsl_ner_wip_greedy_biobert       |\n","| ner_jsl_enriched_biobert       | ner_chemprot_biobert             |\n","| ner_human_phenotype_go_biobert | ner_ade_biobert                  |\n","| ner_deid_biobert               | ner_risk_factors_biobert         |\n","| ner_deid_enriched_biobert      | ner_living_species_biobert                                |\n","\n","\n","</center>"]},{"cell_type":"code","execution_count":112,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":381799,"status":"ok","timestamp":1712936070214,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"y5VEZIdBINmn","outputId":"ee75e666-f7d0-4cb1-e109-29be660abfa0"},"outputs":[{"output_type":"stream","name":"stdout","text":["ner_profiling_clinical download started this may take some time.\n","Approx size to download 2.9 GB\n","[OK!]\n"]}],"source":["clinical_profiling_pipeline = nlp.PretrainedPipeline(\"ner_profiling_clinical\", \"en\", \"clinical/models\")"]},{"cell_type":"code","execution_count":113,"metadata":{"id":"8GYCnYmMnU-H","executionInfo":{"status":"ok","timestamp":1712936070214,"user_tz":-180,"elapsed":11,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["text = '''A 28-year-old female with a history of gestational diabetes mellitus diagnosed eight years prior to presentation and subsequent type two diabetes mellitus ( T2DM ), one prior episode of HTG-induced pancreatitis three years prior to presentation , associated with an acute hepatitis , and obesity with a body mass index ( BMI ) of 33.5 kg/m2 , presented with a one-week history of polyuria , polydipsia , poor appetite , and vomiting .'''"]},{"cell_type":"code","execution_count":114,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":62801,"status":"ok","timestamp":1712936133004,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"5GJFg2AKpxam","outputId":"09dfd04b-0549-4b99-dc36-a868e42139fd"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["dict_keys(['ner_vop_problem_wip_chunks', 'ner_sdoh_demographics_wip_chunks', 'ner_ade_clinical_chunks', 'ner_deid_augmented', 'ner_deid_subentity_augmented_i2b2', 'ner_posology_greedy_chunks', 'ner_vop_demographic_wip_chunks', 'ner_sdoh_community_condition_wip_chunks', 'ner_radiology_wip_clinical', 'ner_vop_slim_wip', 'ner_oncology_diagnosis_chunks', 'ner_vop_test_wip_chunks', 'ner_jsl_slim', 'ner_vop_wip', 'ner_risk_factors_chunks', 'jsl_ner_wip_clinical_chunks', 'ner_oncology_unspecific_posology_chunks', 'ner_vop_problem_reduced_wip', 'ner_vop_anatomy_wip', 'ner_vop_problem_reduced_wip_chunks', 'ner_eu_clinical_case_chunks', 'ner_oncology_test_chunks', 'ner_deid_synthetic', 'ner_oncology_posology_chunks', 'ner_oncology_tnm_chunks', 'ner_oncology_anatomy_general_chunks', 'ner_drugs_greedy', 'ner_abbreviation_clinical_chunks', 'ner_covid_trials_chunks', 'ner_human_phenotype_gene_clinical_chunks', 'ner_events_admission_clinical', 'jsl_ner_wip_greedy_clinical_chunks', 'ner_posology_greedy', 'ner_cellular_chunks', 'ner_vop_clinical_dept_wip', 'ner_cancer_genetics_chunks', 'ner_biomedical_bc2gm_chunks', 'ner_oncology', 'ner_jsl_greedy', 'jsl_ner_wip_modifier_clinical_chunks', 'ner_drugs_greedy_chunks', 'ner_sdoh_access_to_healthcare_wip', 'ner_deid_sd_large_chunks', 'ner_diseases_chunks', 'ner_diseases_large', 'ner_chemprot_clinical', 'ner_posology_large', 'nerdl_tumour_demo_chunks', 'ner_oncology_chunks', 'ner_deid_subentity_augmented_chunks', 'ner_jsl_enriched_chunks', 'ner_genetic_variants_chunks', 'ner_sdoh_income_social_status_wip_chunks', 'ner_chexpert', 'ner_bionlp_chunks', 'ner_eu_clinical_condition_chunks', 'ner_measurements_clinical_chunks', 'ner_diseases_large_chunks', 'ner_drugs_large', 'ner_living_species_chunks', 'ner_oncology_therapy_chunks', 'ner_clinical_large', 'ner_chemicals', 'ner_sdoh_slim_wip', 'ner_radiology_chunks', 'ner_bacterial_species', 'ner_deid_augmented_chunks', 'ner_bionlp', 'ner_drugprot_clinical', 'ner_anatomy_chunks', 'ner_deid_subentity_augmented_i2b2_chunks', 'ner_deid_large', 'jsl_ner_wip_modifier_clinical', 'ner_vop_slim_wip_chunks', 'ner_chemprot_clinical_chunks', 'ner_eu_clinical_condition', 'nerdl_tumour_demo', 'ner_posology_experimental_chunks', 'ner_cancer_genetics', 'ner_biomarker', 'ner_drugprot_clinical_chunks', 'ner_vop_problem_wip', 'ner_sdoh_mentions', 'ner_posology_small', 'ner_vop_wip_chunks', 'ner_oncology_biomarker_chunks', 'ner_drugs_chunks', 'ner_deid_sd_chunks', 'ner_chemd_clinical', 'ner_human_phenotype_gene_clinical', 'ner_ade_clinical', 'ner_human_phenotype_go_clinical', 'ner_sdoh_wip_chunks', 'ner_sdoh_social_environment_wip', 'ner_sdoh_mentions_chunks', 'ner_risk_factors', 'ner_clinical', 'ner_posology_large_chunks', 'ner_cellular', 'ner_nature_nero_clinical_chunks', 'ner_deid_large_chunks', 'ner_clinical_trials_abstracts_chunks', 'ner_deid_generic_augmented_chunks', 'ner_posology_chunks', 'ner_oncology_anatomy_general', 'ner_sdoh_social_environment_wip_chunks', 'ner_supplement_clinical_chunks', 'ner_oncology_response_to_treatment_chunks', 'ner_deid_enriched_chunks', 'ner_chemd_clinical_chunks', 'ner_deid_sd_large', 'ner_oncology_biomarker', 'jsl_ner_wip_greedy_clinical', 'ner_sdoh_slim_wip_chunks', 'ner_bacterial_species_chunks', 'ner_sdoh_health_behaviours_problems_wip_chunks', 'ner_sdoh_demographics_wip', 'ner_sdoh_community_condition_wip', 'ner_vop_anatomy_wip_chunks', 'ner_living_species', 'ner_abbreviation_clinical', 'ner_oncology_test', 'ner_vop_temporal_wip_chunks', 'ner_diseases', 'jsl_rd_ner_wip_greedy_clinical', 'ner_oncology_response_to_treatment', 'ner_drugs', 'ner_drugs_large_chunks', 'ner_vop_treatment_wip', 'ner_biomarker_chunks', 'ner_clinical_trials_abstracts', 'ner_clinical_large_chunks', 'ner_anatomy_coarse', 'ner_nature_nero_clinical', 'token', 'ner_oncology_anatomy_granular', 'ner_medmentions_coarse', 'ner_oncology_therapy', 'ner_deid_sd', 'jsl_rd_ner_wip_greedy_clinical_chunks', 'ner_oncology_demographics', 'ner_deid_generic_augmented', 'ner_pathogen_chunks', 'ner_medmentions_coarse_chunks', 'ner_anatomy', 'ner_deid_enriched', 'ner_deid_subentity_augmented', 'ner_radiology_wip_clinical_chunks', 'ner_clinical_chunks', 'ner_oncology_diagnosis', 'ner_sdoh_health_behaviours_problems_wip', 'ner_measurements_clinical', 'ner_ade_binary', 'ner_nihss', 'ner_chexpert_chunks', 'ner_chemicals_chunks', 'ner_pathogen', 'ner_vop_test_wip', 'ner_oncology_demographics_chunks', 'ner_jsl_enriched', 'ner_sdoh_access_to_healthcare_wip_chunks', 'ner_oncology_tnm', 'ner_vop_treatment_wip_chunks', 'ner_deid_synthetic_chunks', 'ner_events_clinical_chunks', 'ner_posology_experimental', 'jsl_ner_wip_clinical', 'ner_oncology_anatomy_granular_chunks', 'ner_biomedical_bc2gm', 'ner_vop_temporal_wip', 'ner_posology_small_chunks', 'ner_oncology_wip', 'ner_nihss_chunks', 'ner_anatomy_coarse_chunks', 'ner_human_phenotype_go_clinical_chunks', 'ner_vop_demographic_wip', 'ner_sdoh_substance_usage_wip_chunks', 'ner_sdoh_income_social_status_wip', 'ner_jsl', 'ner_sdoh_substance_usage_wip', 'ner_events_clinical', 'ner_supplement_clinical', 'ner_jsl_slim_chunks', 'ner_jsl_chunks', 'ner_jsl_greedy_chunks', 'ner_oncology_wip_chunks', 'ner_oncology_unspecific_posology', 'ner_genetic_variants', 'ner_sdoh_wip', 'ner_radiology', 'ner_eu_clinical_case', 'ner_posology', 'ner_vop_clinical_dept_wip_chunks', 'sentence', 'ner_oncology_posology', 'ner_covid_trials', 'ner_ade_binary_chunks', 'ner_events_admission_clinical_chunks'])"]},"metadata":{},"execution_count":114}],"source":["clinical_result = clinical_profiling_pipeline.fullAnnotate(text)[0]\n","clinical_result.keys()"]},{"cell_type":"code","execution_count":115,"metadata":{"id":"C99PTQiNKZtw","executionInfo":{"status":"ok","timestamp":1712936133005,"user_tz":-180,"elapsed":8,"user":{"displayName":"Monster C","userId":"08787989274818793476"}}},"outputs":[],"source":["def get_token_results(light_result):\n","\n","  tokens = [j.result for j in light_result[\"token\"]]\n","  sentences = [j.metadata[\"sentence\"] for j in light_result[\"token\"]]\n","  begins = [j.begin for j in light_result[\"token\"]]\n","  ends = [j.end for j in light_result[\"token\"]]\n","  model_list = [ a for a in light_result.keys() if (a not in [\"sentence\", \"token\"] and \"_chunks\" not in a)]\n","\n","  df = pd.DataFrame({'sentence':sentences, 'begin': begins, 'end': ends, 'token':tokens})\n","\n","  for model_name in model_list:\n","\n","    temp_df = pd.DataFrame(light_result[model_name])\n","    temp_df[\"jsl_label\"] = temp_df.iloc[:,0].apply(lambda x : x.result)\n","    temp_df = temp_df[[\"jsl_label\"]]\n","\n","    # temp_df = get_ner_result(model_name)\n","    temp_df.columns = [model_name]\n","    df = pd.concat([df, temp_df], axis=1)\n","\n","  return df"]},{"cell_type":"code","execution_count":116,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":478},"executionInfo":{"elapsed":7,"status":"ok","timestamp":1712936133005,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"trBEKiZRLiff","outputId":"c14afbc7-43e8-46c9-ccc4-b7f4016fed77"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["   sentence  begin  end        token ner_deid_augmented  \\\n","0         0      0    0            A                  O   \n","1         0      2   12  28-year-old                  O   \n","2         0     14   19       female                  O   \n","3         0     21   24         with                  O   \n","4         0     26   26            a                  O   \n","..      ...    ...  ...          ...                ...   \n","68        0    409  416     appetite                  O   \n","69        0    418  418            ,                  O   \n","70        0    420  422          and                  O   \n","71        0    424  431     vomiting                  O   \n","72        0    433  433            .                  O   \n","\n","   ner_deid_subentity_augmented_i2b2 ner_radiology_wip_clinical  \\\n","0                                  O                          O   \n","1                                  O                          O   \n","2                                  O                          O   \n","3                                  O                          O   \n","4                                  O                          O   \n","..                               ...                        ...   \n","68                                 O                  I-Symptom   \n","69                                 O                          O   \n","70                                 O                          O   \n","71                                 O                  B-Symptom   \n","72                                 O                          O   \n","\n","   ner_vop_slim_wip    ner_jsl_slim ner_vop_wip  ... ner_events_clinical  \\\n","0                 O               O           O  ...                   O   \n","1                 O           B-Age       B-Age  ...                   O   \n","2          B-Gender  B-Demographics    B-Gender  ...                   O   \n","3                 O               O           O  ...                   O   \n","4                 O               O           O  ...                   O   \n","..              ...             ...         ...  ...                 ...   \n","68        I-Symptom       I-Symptom   I-Symptom  ...           I-PROBLEM   \n","69                O               O           O  ...                   O   \n","70                O               O           O  ...                   O   \n","71        B-Symptom       B-Symptom   B-Symptom  ...           B-PROBLEM   \n","72                O               O           O  ...                   O   \n","\n","   ner_supplement_clinical ner_oncology_unspecific_posology  \\\n","0                        O                                O   \n","1                        O                                O   \n","2                        O                                O   \n","3                        O                                O   \n","4                        O                                O   \n","..                     ...                              ...   \n","68                       O                                O   \n","69                       O                                O   \n","70                       O                                O   \n","71             B-CONDITION                                O   \n","72                       O                                O   \n","\n","   ner_genetic_variants ner_sdoh_wip ner_radiology  ner_eu_clinical_case  \\\n","0                     O            O             O             B-patient   \n","1                     O     B-Gender             O             I-patient   \n","2                     O     B-Gender             O             I-patient   \n","3                     O            O             O                     O   \n","4                     O            O             O                     O   \n","..                  ...          ...           ...                   ...   \n","68                    O            O     I-Symptom  I-clinical_condition   \n","69                    O            O             O                     O   \n","70                    O            O             O                     O   \n","71                    O            O     B-Symptom  B-clinical_condition   \n","72                    O            O             O                     O   \n","\n","   ner_posology ner_oncology_posology ner_covid_trials  \n","0             O                     O                O  \n","1             O                     O         B-Gender  \n","2             O                     O         B-Gender  \n","3             O                     O                O  \n","4             O                     O                O  \n","..          ...                   ...              ...  \n","68            O                     O                O  \n","69            O                     O                O  \n","70            O                     O                O  \n","71            O                     O                O  \n","72            O                     O                O  \n","\n","[73 rows x 102 columns]"],"text/html":["\n","  <div id=\"df-6cfaad85-c5be-41fa-95ee-14858a89b88a\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>sentence</th>\n","      <th>begin</th>\n","      <th>end</th>\n","      <th>token</th>\n","      <th>ner_deid_augmented</th>\n","      <th>ner_deid_subentity_augmented_i2b2</th>\n","      <th>ner_radiology_wip_clinical</th>\n","      <th>ner_vop_slim_wip</th>\n","      <th>ner_jsl_slim</th>\n","      <th>ner_vop_wip</th>\n","      <th>...</th>\n","      <th>ner_events_clinical</th>\n","      <th>ner_supplement_clinical</th>\n","      <th>ner_oncology_unspecific_posology</th>\n","      <th>ner_genetic_variants</th>\n","      <th>ner_sdoh_wip</th>\n","      <th>ner_radiology</th>\n","      <th>ner_eu_clinical_case</th>\n","      <th>ner_posology</th>\n","      <th>ner_oncology_posology</th>\n","      <th>ner_covid_trials</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>A</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>...</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>B-patient</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>12</td>\n","      <td>28-year-old</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>B-Age</td>\n","      <td>B-Age</td>\n","      <td>...</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>B-Gender</td>\n","      <td>O</td>\n","      <td>I-patient</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>B-Gender</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>0</td>\n","      <td>14</td>\n","      <td>19</td>\n","      <td>female</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>B-Gender</td>\n","      <td>B-Demographics</td>\n","      <td>B-Gender</td>\n","      <td>...</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>B-Gender</td>\n","      <td>O</td>\n","      <td>I-patient</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>B-Gender</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>0</td>\n","      <td>21</td>\n","      <td>24</td>\n","      <td>with</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>...</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>0</td>\n","      <td>26</td>\n","      <td>26</td>\n","      <td>a</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>...</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","    </tr>\n","    <tr>\n","      <th>...</th>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","    </tr>\n","    <tr>\n","      <th>68</th>\n","      <td>0</td>\n","      <td>409</td>\n","      <td>416</td>\n","      <td>appetite</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>I-Symptom</td>\n","      <td>I-Symptom</td>\n","      <td>I-Symptom</td>\n","      <td>I-Symptom</td>\n","      <td>...</td>\n","      <td>I-PROBLEM</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>I-Symptom</td>\n","      <td>I-clinical_condition</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","    </tr>\n","    <tr>\n","      <th>69</th>\n","      <td>0</td>\n","      <td>418</td>\n","      <td>418</td>\n","      <td>,</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>...</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","    </tr>\n","    <tr>\n","      <th>70</th>\n","      <td>0</td>\n","      <td>420</td>\n","      <td>422</td>\n","      <td>and</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>...</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","    </tr>\n","    <tr>\n","      <th>71</th>\n","      <td>0</td>\n","      <td>424</td>\n","      <td>431</td>\n","      <td>vomiting</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>B-Symptom</td>\n","      <td>B-Symptom</td>\n","      <td>B-Symptom</td>\n","      <td>B-Symptom</td>\n","      <td>...</td>\n","      <td>B-PROBLEM</td>\n","      <td>B-CONDITION</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>B-Symptom</td>\n","      <td>B-clinical_condition</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","    </tr>\n","    <tr>\n","      <th>72</th>\n","      <td>0</td>\n","      <td>433</td>\n","      <td>433</td>\n","      <td>.</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>...</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","      <td>O</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>73 rows × 102 columns</p>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6cfaad85-c5be-41fa-95ee-14858a89b88a')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-6cfaad85-c5be-41fa-95ee-14858a89b88a button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-6cfaad85-c5be-41fa-95ee-14858a89b88a');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-8319fac8-63c7-4692-899d-16d0d0755d11\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-8319fac8-63c7-4692-899d-16d0d0755d11')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-8319fac8-63c7-4692-899d-16d0d0755d11 button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","  <div id=\"id_13daa13b-475a-454c-baa7-3c0a87e3c06e\">\n","    <style>\n","      .colab-df-generate {\n","        background-color: #E8F0FE;\n","        border: none;\n","        border-radius: 50%;\n","        cursor: pointer;\n","        display: none;\n","        fill: #1967D2;\n","        height: 32px;\n","        padding: 0 0 0 0;\n","        width: 32px;\n","      }\n","\n","      .colab-df-generate:hover {\n","        background-color: #E2EBFA;\n","        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","        fill: #174EA6;\n","      }\n","\n","      [theme=dark] .colab-df-generate {\n","        background-color: #3B4455;\n","        fill: #D2E3FC;\n","      }\n","\n","      [theme=dark] .colab-df-generate:hover {\n","        background-color: #434B5C;\n","        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","        fill: #FFFFFF;\n","      }\n","    </style>\n","    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('result_df')\"\n","            title=\"Generate code using this dataframe.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n","  </svg>\n","    </button>\n","    <script>\n","      (() => {\n","      const buttonEl =\n","        document.querySelector('#id_13daa13b-475a-454c-baa7-3c0a87e3c06e button.colab-df-generate');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      buttonEl.onclick = () => {\n","        google.colab.notebook.generateWithVariable('result_df');\n","      }\n","      })();\n","    </script>\n","  </div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"result_df"}},"metadata":{},"execution_count":116}],"source":["result_df = get_token_results(clinical_result)\n","result_df"]},{"cell_type":"markdown","metadata":{"id":"cuv1WRBjLv_J"},"source":["**Lets plot the labelled entity counts by NER models.**"]},{"cell_type":"code","execution_count":117,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000},"executionInfo":{"elapsed":619,"status":"ok","timestamp":1712936133618,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"OhBqL-nPE5zr","outputId":"a3eebc81-ab6e-487e-dc22-4470f04e5609"},"outputs":[{"output_type":"display_data","data":{"text/html":["<html>\n","<head><meta charset=\"utf-8\" /></head>\n","<body>\n","    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n","        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.24.1.min.js\"></script>                <div id=\"3250d36e-5c35-4f13-9728-db39c377da79\" class=\"plotly-graph-div\" style=\"height:1500px; width:1500px;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"3250d36e-5c35-4f13-9728-db39c377da79\")) {                    Plotly.newPlot(                        \"3250d36e-5c35-4f13-9728-db39c377da79\",                        [{\"orientation\":\"h\",\"x\":[0,0,23,25,34,32,16,0,0,0,35,0,0,8,34,2,12,0,0,0,0,25,0,12,0,2,0,0,34,13,0,0,2,16,0,0,0,5,4,3,0,7,25,0,0,0,0,34,2,2,1,2,0,16,35,0,0,0,10,1,41,0,22,0,0,2,1,1,0,1,0,14,2,6,0,15,8,35,0,0,33,0,6,8,2,0,33,1,35,15,0,0,16,24,32,0,0,18],\"y\":[\"ner_deid_augmented\",\"ner_deid_subentity_augmented_i2b2\",\"ner_radiology_wip_clinical\",\"ner_vop_slim_wip\",\"ner_jsl_slim\",\"ner_vop_wip\",\"ner_vop_problem_reduced_wip\",\"ner_vop_anatomy_wip\",\"ner_deid_synthetic\",\"ner_drugs_greedy\",\"ner_events_admission_clinical\",\"ner_posology_greedy\",\"ner_vop_clinical_dept_wip\",\"ner_oncology\",\"ner_jsl_greedy\",\"ner_sdoh_access_to_healthcare_wip\",\"ner_diseases_large\",\"ner_chemprot_clinical\",\"ner_posology_large\",\"ner_chexpert\",\"ner_drugs_large\",\"ner_clinical_large\",\"ner_chemicals\",\"ner_sdoh_slim_wip\",\"ner_bacterial_species\",\"ner_bionlp\",\"ner_drugprot_clinical\",\"ner_deid_large\",\"jsl_ner_wip_modifier_clinical\",\"ner_eu_clinical_condition\",\"nerdl_tumour_demo\",\"ner_cancer_genetics\",\"ner_biomarker\",\"ner_vop_problem_wip\",\"ner_sdoh_mentions\",\"ner_posology_small\",\"ner_chemd_clinical\",\"ner_human_phenotype_gene_clinical\",\"ner_ade_clinical\",\"ner_human_phenotype_go_clinical\",\"ner_sdoh_social_environment_wip\",\"ner_risk_factors\",\"ner_clinical\",\"ner_cellular\",\"ner_oncology_anatomy_general\",\"ner_deid_sd_large\",\"ner_oncology_biomarker\",\"jsl_ner_wip_greedy_clinical\",\"ner_sdoh_demographics_wip\",\"ner_sdoh_community_condition_wip\",\"ner_living_species\",\"ner_abbreviation_clinical\",\"ner_oncology_test\",\"ner_diseases\",\"jsl_rd_ner_wip_greedy_clinical\",\"ner_oncology_response_to_treatment\",\"ner_drugs\",\"ner_vop_treatment_wip\",\"ner_clinical_trials_abstracts\",\"ner_anatomy_coarse\",\"ner_nature_nero_clinical\",\"ner_oncology_anatomy_granular\",\"ner_medmentions_coarse\",\"ner_oncology_therapy\",\"ner_deid_sd\",\"ner_oncology_demographics\",\"ner_deid_generic_augmented\",\"ner_anatomy\",\"ner_deid_enriched\",\"ner_deid_subentity_augmented\",\"ner_oncology_diagnosis\",\"ner_sdoh_health_behaviours_problems_wip\",\"ner_measurements_clinical\",\"ner_ade_binary\",\"ner_nihss\",\"ner_pathogen\",\"ner_vop_test_wip\",\"ner_jsl_enriched\",\"ner_oncology_tnm\",\"ner_posology_experimental\",\"jsl_ner_wip_clinical\",\"ner_biomedical_bc2gm\",\"ner_vop_temporal_wip\",\"ner_oncology_wip\",\"ner_vop_demographic_wip\",\"ner_sdoh_income_social_status_wip\",\"ner_jsl\",\"ner_sdoh_substance_usage_wip\",\"ner_events_clinical\",\"ner_supplement_clinical\",\"ner_oncology_unspecific_posology\",\"ner_genetic_variants\",\"ner_sdoh_wip\",\"ner_radiology\",\"ner_eu_clinical_case\",\"ner_posology\",\"ner_oncology_posology\",\"ner_covid_trials\"],\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"margin\":{\"l\":50,\"r\":50,\"b\":100,\"t\":100,\"pad\":4},\"title\":{\"font\":{\"size\":30},\"text\":\"Counts of Labelled Entities\",\"y\":0.98,\"x\":0.5,\"xanchor\":\"center\",\"yanchor\":\"top\"},\"yaxis\":{\"title\":{\"font\":{\"size\":30},\"text\":\"Clinical NER Models\"}},\"autosize\":false,\"width\":1500,\"height\":1500,\"paper_bgcolor\":\"LightSteelBlue\"},                        {\"responsive\": true}                    ).then(function(){\n","                            \n","var gd = document.getElementById('3250d36e-5c35-4f13-9728-db39c377da79');\n","var x = new MutationObserver(function (mutations, observer) {{\n","        var display = window.getComputedStyle(gd).display;\n","        if (!display || display === 'none') {{\n","            console.log([gd, 'removed!']);\n","            Plotly.purge(gd);\n","            observer.disconnect();\n","        }}\n","}});\n","\n","// Listen for the removal of the full notebook cells\n","var notebookContainer = gd.closest('#notebook-container');\n","if (notebookContainer) {{\n","    x.observe(notebookContainer, {childList: true});\n","}}\n","\n","// Listen for the clearing of the current output cell\n","var outputEl = gd.closest('.output');\n","if (outputEl) {{\n","    x.observe(outputEl, {childList: true});\n","}}\n","\n","                        })                };                            </script>        </div>\n","</body>\n","</html>"]},"metadata":{}}],"source":["import plotly.graph_objects as go\n","\n","label_result_df = result_df[result_df.columns[4:]].copy()\n","label_count_list = [label_result_df[label_result_df[i] != \"O\"].shape[0] for i in label_result_df.columns]\n","\n","fig = go.Figure(go.Bar(x=label_count_list, y=label_result_df.columns, orientation='h'))\n","\n","fig.update_layout(\n","    autosize = False,\n","    width = 1500,\n","    height = 1500,\n","    margin = dict(l=50, r=50, b=100, t=100, pad=4),\n","    paper_bgcolor = \"LightSteelBlue\",\n","    title={'text': \"Counts of Labelled Entities\", 'y':0.98, 'x':0.5, 'xanchor': 'center', 'yanchor': 'top'},\n","    titlefont=dict(size=30),\n","    yaxis=dict(title_text=\"Clinical NER Models\", titlefont=dict(size=30)),\n",")\n","\n","fig.show()"]},{"cell_type":"markdown","metadata":{"id":"P24PYkhmwTLm"},"source":["## NER Model Finder Pretrained Pipeline\n","`ner_model_finder`  pretrained pipeline trained with bert embeddings that can be used to find the most appropriate NER model given the entity name."]},{"cell_type":"code","execution_count":118,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":20922,"status":"ok","timestamp":1712936154533,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"c3L49OPfwOj0","outputId":"d13fbb62-498f-4157-93a6-0e6102090a7c"},"outputs":[{"output_type":"stream","name":"stdout","text":["ner_model_finder download started this may take some time.\n","Approx size to download 148.7 MB\n","[OK!]\n"]}],"source":["finder_pipeline = nlp.PretrainedPipeline(\"ner_model_finder\", \"en\", \"clinical/models\")"]},{"cell_type":"code","execution_count":119,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":27,"status":"ok","timestamp":1712936154533,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"yBEVDDmSAUhC","outputId":"4038ae4a-5a31-426b-ea61-b00dabbe85bc"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["dict_keys(['model_names'])"]},"metadata":{},"execution_count":119}],"source":["result = finder_pipeline.fullAnnotate(\"oncology\")[0]\n","result.keys()"]},{"cell_type":"markdown","metadata":{"id":"TVfVAsplCKG4"},"source":["From the metadata in the `model_names` column, we'll get to the top models to the given 'oncology' entity and oncology related categories."]},{"cell_type":"code","execution_count":120,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":206},"executionInfo":{"elapsed":25,"status":"ok","timestamp":1712936154533,"user":{"displayName":"Monster C","userId":"08787989274818793476"},"user_tz":-180},"id":"TisrPdBJohGy","outputId":"4455f9ec-793e-4f4a-a5b0-b2c8c557daa3"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["              category                                         top_models\n","0     oncology therapy  ['ner_jsl', 'jsl_rd_ner_wip_greedy_clinical', ...\n","1  clinical department  ['ner_jsl', 'jsl_rd_ner_wip_greedy_clinical', ...\n","2      biomedical unit                  ['ner_clinical_trials_abstracts']\n","3      cancer genetics                            ['ner_cancer_genetics']\n","4              anatomy  ['ner_bionlp', 'ner_medmentions_coarse', 'ner_..."],"text/html":["\n","  <div id=\"df-4645930a-71ee-48ed-9c80-2ca937e75212\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>category</th>\n","      <th>top_models</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>oncology therapy</td>\n","      <td>['ner_jsl', 'jsl_rd_ner_wip_greedy_clinical', ...</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>clinical department</td>\n","      <td>['ner_jsl', 'jsl_rd_ner_wip_greedy_clinical', ...</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>biomedical unit</td>\n","      <td>['ner_clinical_trials_abstracts']</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>cancer genetics</td>\n","      <td>['ner_cancer_genetics']</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>anatomy</td>\n","      <td>['ner_bionlp', 'ner_medmentions_coarse', 'ner_...</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4645930a-71ee-48ed-9c80-2ca937e75212')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-4645930a-71ee-48ed-9c80-2ca937e75212 button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-4645930a-71ee-48ed-9c80-2ca937e75212');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-84b6b073-8a1c-44c1-977d-f2b8d8dd1b20\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-84b6b073-8a1c-44c1-977d-f2b8d8dd1b20')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-84b6b073-8a1c-44c1-977d-f2b8d8dd1b20 button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"df","summary":"{\n  \"name\": \"df\",\n  \"rows\": 17,\n  \"fields\": [\n    {\n      \"column\": \"category\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 17,\n        \"samples\": [\n          \"oncology therapy\",\n          \"clinical department\",\n          \"cancer surgery\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"top_models\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 17,\n        \"samples\": [\n          \"['ner_jsl', 'jsl_rd_ner_wip_greedy_clinical', 'jsl_ner_wip_modifier_clinical', 'ner_jsl_enriched', 'ner_jsl_greedy', 'ner_jsl_slim']\",\n          \"['ner_jsl', 'jsl_rd_ner_wip_greedy_clinical', 'jsl_ner_wip_modifier_clinical', 'ner_events_clinical', 'ner_jsl_enriched', 'ner_jsl_greedy', 'ner_jsl_slim', 'ner_events_healthcare', 'ner_events_admission_clinical']\",\n          \"['ner_biomarker']\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":120}],"source":["df= pd.DataFrame(zip(result[\"model_names\"][0].metadata[\"all_k_resolutions\"].split(\":::\"),\n","                     result[\"model_names\"][0].metadata[\"all_k_results\"].split(\":::\")),\n","                 columns=[\"category\", \"top_models\"])\n","\n","df.head()"]},{"cell_type":"markdown","source":["## Extra Informations"],"metadata":{"id":"V-HFYgcl840I"}},{"cell_type":"markdown","source":["**ExceptionHandling**\n","\n","If true, try to compute batch-wise as usual.\n","If within a batch an exception occurs, the batch will be processed row-wise and for any exception occurring during row-processing an Error Annotation is emitted.\n","This means only the bad rows are lost instead of the entire batch.\n","This comes with a performance penalty.\n","\n","*Example*:\n","```python\n","clinical_ner = MedicalNerModel.pretrained(\"ner_oncology\", \"en\", \"clinical/models\") \\\n","    .setInputCols([\"sentence\", \"token\", \"embeddings\"]) \\\n","    .setOutputCol(\"ner\")\\\n","    .setDoExceptionHandling(True)\n","```"],"metadata":{"id":"l77rOIZP95Id"}}],"metadata":{"colab":{"machine_shape":"hm","provenance":[{"file_id":"https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/healthcare-nlp/01.0.Clinical_Named_Entity_Recognition_Model.ipynb","timestamp":1709049185107},{"file_id":"https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/healthcare-nlp/01.0.Clinical_Named_Entity_Recognition_Model.ipynb","timestamp":1688557759347}],"toc_visible":true},"kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.9.1"}},"nbformat":4,"nbformat_minor":0}