From ec1d25d131a6c228a96cfa425c494b11218d3e4b Mon Sep 17 00:00:00 2001 From: jsl-models <74001263+jsl-models@users.noreply.github.com> Date: Tue, 4 Jul 2023 19:40:21 +0700 Subject: [PATCH 1/3] Add model 2023-07-04-leggen_contract_nli_en (#442) Co-authored-by: gadde5300 --- .../2023-07-04-leggen_contract_nli_en.md | 91 +++++++++++++++++++ 1 file changed, 91 insertions(+) create mode 100644 docs/_posts/gadde5300/2023-07-04-leggen_contract_nli_en.md diff --git a/docs/_posts/gadde5300/2023-07-04-leggen_contract_nli_en.md b/docs/_posts/gadde5300/2023-07-04-leggen_contract_nli_en.md new file mode 100644 index 0000000000..c98d973633 --- /dev/null +++ b/docs/_posts/gadde5300/2023-07-04-leggen_contract_nli_en.md @@ -0,0 +1,91 @@ +--- +layout: model +title: Legal Contract NLI Text Generation +author: John Snow Labs +name: leggen_contract_nli +date: 2023-07-04 +tags: [legal, en, text_generation, contract_nli, licensed, tensorflow] +task: Text Generation +language: en +edition: Legal NLP 1.0.0 +spark_version: 3.0 +supported: true +engine: tensorflow +annotator: LegalTextGenerator +article_header: + type: cover +use_language_switcher: "Python-Scala-Java" +--- + +## Description + +This is a text generation model that has undergone fine-tuning on flan-t5 using the Contract-NLI dataset. The ContractNLI dataset is designed for document-level natural language inference (NLI) on contracts, aiming to streamline and expedite the contract review process. The objective of this task is to provide a system with a set of hypotheses, like "Some obligations of Agreement may survive termination," along with a contract, and task it with classifying whether each hypothesis is entailed, contradicted, or not mentioned (neutral) by the contract. + +## Predicted Entities + +`Contradiction`, `Entailment`, `Neutral` + +{:.btn-box} + + +[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/legal/models/leggen_contract_nli_en_1.0.0_3.0_1688441488610.zip){:.button.button-orange.button-orange-trans.arr.button-icon.hidden} +[Copy S3 URI](s3://auxdata.johnsnowlabs.com/legal/models/leggen_contract_nli_en_1.0.0_3.0_1688441488610.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3} + +## How to use + + + +
+{% include programmingLanguageSelectScalaPythonNLU.html %} +```python +document_assembler = nlp.DocumentAssembler()\ + .setInputCol("text")\ + .setOutputCol("document") + +flant5 = legal.TextGenerator.pretrained('leggen_contract_nli','en','legal/models')\ + .setInputCols(["document"])\ + .setMaxNewTokens(256)\ + .setStopAtEos(True)\ + .setOutputCol("generated") + +pipeline = nlp.Pipeline(stages=[document_assembler, flant5]) + +data = spark.createDataFrame([ + [1, '''mnli hypothesis: +Receiving Party shall not use any Confidential Information for any purpose other than the purposes stated in Agreement. +premise: +(a) furnish only that portion of the Proprietary Information which in its reasonable opinion, based upon advice of counsel, it is legally compelled to disclose, and (b) at the other Party's cost, cooperate with the efforts of the Party to obtain order or other reliable assurance that confidential treat will he accorded to such portion of the Proprietary Information as may be disclosed. +''']]).toDF('id', 'text') + +results = pipeline.fit(data).transform(data) + +results.select("generated.result").show(truncate=False) +``` + +
+ +## Results + +```bash ++---------+ +|result | ++---------+ +|[Neutral]| ++---------+ +``` + +{:.model-param} +## Model Information + +{:.table-model} +|---|---| +|Model Name:|leggen_contract_nli| +|Compatibility:|Legal NLP 1.0.0+| +|License:|Licensed| +|Edition:|Official| +|Language:|en| +|Size:|1.6 GB| + +## References + +This model has been fine tuned using: https://stanfordnlp.github.io/contract-nli/ \ No newline at end of file From 1f7927f561c43cbc2b31cdce2289087ef8b4860d Mon Sep 17 00:00:00 2001 From: jsl-models <74001263+jsl-models@users.noreply.github.com> Date: Sat, 5 Aug 2023 18:53:55 +0700 Subject: [PATCH 2/3] Add model 2023-08-04-legclf_law_stack_exchange_en (#499) Co-authored-by: Mary-Sci --- ...2023-08-04-legclf_law_stack_exchange_en.md | 117 ++++++++++++++++++ 1 file changed, 117 insertions(+) create mode 100644 docs/_posts/Mary-Sci/2023-08-04-legclf_law_stack_exchange_en.md diff --git a/docs/_posts/Mary-Sci/2023-08-04-legclf_law_stack_exchange_en.md b/docs/_posts/Mary-Sci/2023-08-04-legclf_law_stack_exchange_en.md new file mode 100644 index 0000000000..e48a7fd350 --- /dev/null +++ b/docs/_posts/Mary-Sci/2023-08-04-legclf_law_stack_exchange_en.md @@ -0,0 +1,117 @@ +--- +layout: model +title: Legal Law Stack Exchange Classifier in Domain-Specific Documents +author: John Snow Labs +name: legclf_law_stack_exchange +date: 2023-08-04 +tags: [en, licensed, classification, legal, tensorflow] +task: Text Classification +language: en +edition: Legal NLP 1.0.0 +spark_version: 3.0 +supported: true +engine: tensorflow +annotator: LegalBertForSequenceClassification +article_header: + type: cover +use_language_switcher: "Python-Scala-Java" +--- + +## Description + +This model is a multi-class classification model that can classify a wide variety of legal issues. The model demonstrates remarkable proficiency in predicting `business`, `constitutional-law`, `contract-law`, `copyright`, `criminal-law`, `employment`, `liability`, `privacy`, `tax-law`, and `trademark`. + +## Predicted Entities + +`business`, `constitutional-law`, `contract-law`, `copyright`, `criminal-law`, `employment`, `liability`, `privacy`, `tax-law`, `trademark` + +{:.btn-box} + + +[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/legal/models/legclf_law_stack_exchange_en_1.0.0_3.0_1691173181059.zip){:.button.button-orange.button-orange-trans.arr.button-icon.hidden} +[Copy S3 URI](s3://auxdata.johnsnowlabs.com/legal/models/legclf_law_stack_exchange_en_1.0.0_3.0_1691173181059.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3} + +## How to use + + + +
+{% include programmingLanguageSelectScalaPythonNLU.html %} +```python +document_assembler = nlp.DocumentAssembler() \ + .setInputCol('text') \ + .setOutputCol('document') + +tokenizer = nlp.Tokenizer() \ + .setInputCols(['document']) \ + .setOutputCol('token') + +sequenceClassifier = legal.BertForSequenceClassification.pretrained("legclf_law_stack_exchange", "en", "legal/models") \ + .setInputCols(["document", "token"]) \ + .setOutputCol("class") + +pipeline = nlp.Pipeline(stages=[ + document_assembler, + tokenizer, + sequenceClassifier +]) + +# couple of simple examples +example = spark.createDataFrame([["I have been helping a nonprofit by developing a piece of software that they needed. The software is more-or-less built to their specs in a 'functional' way, but I wrote 100% of the code: they are not programmers. Anyhow, we didn't make any kind of contract at the beginning verbally or otherwise. Who owns the copyright to all of this? Do they have any rights to it at all for providing 'ideas'?"]]).toDF("text") + +result = pipeline.fit(example).transform(example) + +# result is a DataFrame +result.select("text", "class.result").show(truncate=100) +``` + +
+ +## Results + +```bash ++----------------------------------------------------------------------------------------------------+-----------+ +| text| result| ++----------------------------------------------------------------------------------------------------+-----------+ +|I have been helping a nonprofit by developing a piece of software that they needed. The software ...|[copyright]| ++----------------------------------------------------------------------------------------------------+-----------+ +``` + +{:.model-param} +## Model Information + +{:.table-model} +|---|---| +|Model Name:|legclf_law_stack_exchange| +|Compatibility:|Legal NLP 1.0.0+| +|License:|Licensed| +|Edition:|Official| +|Input Labels:|[document, token]| +|Output Labels:|[class]| +|Language:|en| +|Size:|410.1 MB| +|Case sensitive:|true| +|Max sentence length:|512| + +## References + +Train dataset available [here](https://huggingface.co/datasets/jonathanli/law-stack-exchange) + +## Benchmarking + +```bash +label precision recall f1-score support +business 0.50 0.24 0.32 17 +constitutional-law 0.94 0.68 0.79 25 +contract-law 0.88 0.85 0.86 91 +copyright 0.91 0.97 0.94 151 +criminal-law 0.80 0.91 0.85 75 +employment 0.74 0.93 0.82 30 +liability 0.67 0.31 0.42 13 +privacy 0.77 0.82 0.79 28 +tax-law 0.93 0.78 0.85 32 +trademark 0.89 0.91 0.90 44 +accuracy - - 0.86 506 +macro-avg 0.80 0.74 0.75 506 +weighted-avg 0.85 0.86 0.85 506 +``` \ No newline at end of file From 62447c32990b8db24ba58b5423fc6b86ef9b63a3 Mon Sep 17 00:00:00 2001 From: jsl-models <74001263+jsl-models@users.noreply.github.com> Date: Tue, 8 Aug 2023 01:54:20 +0700 Subject: [PATCH 3/3] 2023-08-07-legner_bert_subpoenas_sm_en (#502) * Add model 2023-08-07-legner_bert_subpoenas_sm_en * Update 2023-08-07-legner_bert_subpoenas_sm_en.md --------- Co-authored-by: gadde5300 Co-authored-by: GADDE SAI SHAILESH <69344247+gadde5300@users.noreply.github.com> --- .../2023-08-07-legner_bert_subpoenas_sm_en.md | 165 ++++++++++++++++++ 1 file changed, 165 insertions(+) create mode 100644 docs/_posts/gadde5300/2023-08-07-legner_bert_subpoenas_sm_en.md diff --git a/docs/_posts/gadde5300/2023-08-07-legner_bert_subpoenas_sm_en.md b/docs/_posts/gadde5300/2023-08-07-legner_bert_subpoenas_sm_en.md new file mode 100644 index 0000000000..d60b1a71f5 --- /dev/null +++ b/docs/_posts/gadde5300/2023-08-07-legner_bert_subpoenas_sm_en.md @@ -0,0 +1,165 @@ +--- +layout: model +title: Legal NER on Subpoenas (Small) +author: John Snow Labs +name: legner_bert_subpoenas_sm +date: 2023-08-07 +tags: [en, licensed, tensorflow] +task: Named Entity Recognition +language: en +edition: Legal NLP 1.0.0 +spark_version: 3.0 +supported: true +engine: tensorflow +annotator: LegalBertForTokenClassification +article_header: + type: cover +use_language_switcher: "Python-Scala-Java" +--- + +## Description + +This is a Legal NER model aimed to extract 19 entities from subpoenas. This is called a small version because it has been trained on more generic labels. The larger versions of this model will be available on models hub. + +## Predicted Entities + +`COURT`, `APPOINTMENT_DATE`, `DEADLINE_DATE`, `DOCUMENT_DATE_FROM`, `ADDRESS`, `APPOINTMENT_HOUR`, `DOCUMENT_DATE_TO`, `DOCUMENT_PERSON`, `DOCUMENT_DATE_YEAR`, `STATE`, `MATTER_VS`, `CASE`, `COUNTY`, `DOCUMENT_TOPIC`, `MATTER`, `SUBPOENA_DATE`, `SIGNER`, `RECEIVER`, `DOCUMENT_TYPE` + +{:.btn-box} + + +[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/legal/models/legner_bert_subpoenas_sm_en_1.0.0_3.0_1691423741988.zip){:.button.button-orange.button-orange-trans.arr.button-icon.hidden} +[Copy S3 URI](s3://auxdata.johnsnowlabs.com/legal/models/legner_bert_subpoenas_sm_en_1.0.0_3.0_1691423741988.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3} + +## How to use + + + +
+{% include programmingLanguageSelectScalaPythonNLU.html %} +```python +from pyspark.sql import functions as F + +document_assembler = nlp.DocumentAssembler()\ + .setInputCol("text")\ + .setOutputCol("document")\ + +sentence_detector = nlp.SentenceDetector()\ + .setInputCols(["document"])\ + .setOutputCol("sentence")\ + +tokenizer = nlp.Tokenizer() \ + .setInputCols(["sentence"]) \ + .setOutputCol("token") + +ner_model = legal.BertForTokenClassification.pretrained("legner_bert_subpoenas_sm", "en", "legal/models")\ + .setInputCols(["sentence", "token"])\ + .setOutputCol("ner")\ + .setCaseSensitive(True)\ + .setMaxSentenceLength(512) + +ner_converter = nlp.NerConverter()\ + .setInputCols(["sentence", "token", "ner"])\ + .setOutputCol("ner_chunk") + +pipeline = nlp.Pipeline(stages=[ + document_assembler, + sentence_detector, + tokenizer, + ner_model, + ner_converter +]) + + +empty_data = spark.createDataFrame([[""]]).toDF("text") + +model = pipeline.fit(empty_data) + +text = """In addition , in an earlier motion for summary disposition in which all Respondents joined , and which this Court denied in its Order of April30 , 2013 , Respondent Deloitte Touche Tohmatsu Certified Public Accountants Ltd .""" +data = spark.createDataFrame([[text]]).toDF("text") + +result = model.transform(data) + +result.select(F.explode(F.arrays_zip('ner_chunk.result', 'ner_chunk.metadata')).alias("cols")) \ + .select(F.expr("cols['0']").alias("chunk"), + F.expr("cols['1']['entity']").alias("label")).show(50, truncate = False) +``` + +
+ +## Results + +```bash ++------------------------+---------------+ +|chunk |label | ++------------------------+---------------+ +|summary disposition |DOCUMENT_TYPE | +|Deloitte Touche Tohmatsu|DOCUMENT_PERSON| ++------------------------+---------------+ +``` + +{:.model-param} +## Model Information + +{:.table-model} +|---|---| +|Model Name:|legner_bert_subpoenas_sm| +|Compatibility:|Legal NLP 1.0.0+| +|License:|Licensed| +|Edition:|Official| +|Input Labels:|[document, token]| +|Output Labels:|[ner]| +|Language:|en| +|Size:|401.1 MB| +|Case sensitive:|true| +|Max sentence length:|128| + +## References + +In House annotated dataset + +## Benchmarking + +```bash +label precision recall f1-score support + B-COURT 1.00 0.60 0.75 30 + I-APPOINTMENT_DATE 0.57 0.65 0.60 20 + I-COURT 0.93 0.89 0.91 166 + B-APPOINTMENT_DATE 0.67 0.44 0.53 9 + I-DEADLINE_DATE 0.83 0.26 0.40 19 +B-DOCUMENT_DATE_FROM 0.80 1.00 0.89 16 + I-ADDRESS 0.87 0.94 0.90 1046 + B-APPOINTMENT_HOUR 0.43 0.92 0.59 13 + B-DOCUMENT_DATE_TO 0.88 1.00 0.93 7 + I-APPOINTMENT_HOUR 1.00 0.15 0.26 20 + B-DOCUMENT_PERSON 0.79 0.84 0.82 2919 +B-DOCUMENT_DATE_YEAR 0.00 0.00 0.00 5 + B-STATE 0.59 0.79 0.68 24 + I-MATTER_VS 0.65 0.79 0.71 150 + I-CASE 0.00 0.00 0.00 11 + I-COUNTY 0.00 0.00 0.00 0 + B-DOCUMENT_TOPIC 0.64 0.77 0.70 208 + B-COUNTY 0.00 0.00 0.00 0 + B-MATTER 0.85 0.86 0.86 328 +I-DOCUMENT_DATE_FROM 0.87 1.00 0.93 48 + I-SUBPOENA_DATE 0.56 0.28 0.38 53 + I-SIGNER 0.56 0.46 0.50 59 + I-DOCUMENT_DATE_TO 0.83 1.00 0.91 25 + I-RECEIVER 0.71 0.52 0.60 98 + B-SIGNER 0.76 0.49 0.59 39 + I-DOCUMENT_TOPIC 0.83 0.80 0.81 725 + I-STATE 0.67 0.29 0.40 14 + B-MATTER_VS 0.78 0.82 0.80 136 + I-DOCUMENT_TYPE 0.83 0.87 0.85 621 + B-DEADLINE_DATE 0.00 0.00 0.00 6 + I-MATTER 0.88 0.82 0.85 479 + B-DOCUMENT_TYPE 0.87 0.90 0.88 1714 + B-ADDRESS 0.81 0.83 0.82 101 + B-SUBPOENA_DATE 0.42 0.28 0.33 18 + B-CASE 0.91 0.97 0.94 312 + I-DOCUMENT_PERSON 0.80 0.83 0.81 3672 + B-RECEIVER 0.76 0.63 0.69 46 + micro-avg 0.82 0.84 0.83 13157 + macro-avg 0.66 0.61 0.61 13157 + weighted-avg 0.82 0.84 0.83 13157 +```