Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Models hub legal #523

Merged
merged 4 commits into from
Aug 16, 2023
Merged

Models hub legal #523

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ec1d25d
Add model 2023-07-04-leggen_contract_nli_en (#442)
jsl-models Jul 4, 2023
1f7927f
Add model 2023-08-04-legclf_law_stack_exchange_en (#499)
jsl-models Aug 5, 2023
62447c3
2023-08-07-legner_bert_subpoenas_sm_en (#502)
jsl-models Aug 7, 2023
35a8f10
Merge branch 'main' into models_hub_legal
dcecchini Aug 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 117 additions & 0 deletions docs/_posts/Mary-Sci/2023-08-04-legclf_law_stack_exchange_en.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
layout: model
title: Legal Law Stack Exchange Classifier in Domain-Specific Documents
author: John Snow Labs
name: legclf_law_stack_exchange
date: 2023-08-04
tags: [en, licensed, classification, legal, tensorflow]
task: Text Classification
language: en
edition: Legal NLP 1.0.0
spark_version: 3.0
supported: true
engine: tensorflow
annotator: LegalBertForSequenceClassification
article_header:
type: cover
use_language_switcher: "Python-Scala-Java"
---

## Description

This model is a multi-class classification model that can classify a wide variety of legal issues. The model demonstrates remarkable proficiency in predicting `business`, `constitutional-law`, `contract-law`, `copyright`, `criminal-law`, `employment`, `liability`, `privacy`, `tax-law`, and `trademark`.

## Predicted Entities

`business`, `constitutional-law`, `contract-law`, `copyright`, `criminal-law`, `employment`, `liability`, `privacy`, `tax-law`, `trademark`

{:.btn-box}
<button class="button button-orange" disabled>Live Demo</button>
<button class="button button-orange" disabled>Open in Colab</button>
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/legal/models/legclf_law_stack_exchange_en_1.0.0_3.0_1691173181059.zip){:.button.button-orange.button-orange-trans.arr.button-icon.hidden}
[Copy S3 URI](s3://auxdata.johnsnowlabs.com/legal/models/legclf_law_stack_exchange_en_1.0.0_3.0_1691173181059.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}

## How to use



<div class="tabs-box" markdown="1">
{% include programmingLanguageSelectScalaPythonNLU.html %}
```python
document_assembler = nlp.DocumentAssembler() \
.setInputCol('text') \
.setOutputCol('document')

tokenizer = nlp.Tokenizer() \
.setInputCols(['document']) \
.setOutputCol('token')

sequenceClassifier = legal.BertForSequenceClassification.pretrained("legclf_law_stack_exchange", "en", "legal/models") \
.setInputCols(["document", "token"]) \
.setOutputCol("class")

pipeline = nlp.Pipeline(stages=[
document_assembler,
tokenizer,
sequenceClassifier
])

# couple of simple examples
example = spark.createDataFrame([["I have been helping a nonprofit by developing a piece of software that they needed. The software is more-or-less built to their specs in a 'functional' way, but I wrote 100% of the code: they are not programmers. Anyhow, we didn't make any kind of contract at the beginning verbally or otherwise. Who owns the copyright to all of this? Do they have any rights to it at all for providing 'ideas'?"]]).toDF("text")

result = pipeline.fit(example).transform(example)

# result is a DataFrame
result.select("text", "class.result").show(truncate=100)
```

</div>

## Results

```bash
+----------------------------------------------------------------------------------------------------+-----------+
| text| result|
+----------------------------------------------------------------------------------------------------+-----------+
|I have been helping a nonprofit by developing a piece of software that they needed. The software ...|[copyright]|
+----------------------------------------------------------------------------------------------------+-----------+
```

{:.model-param}
## Model Information

{:.table-model}
|---|---|
|Model Name:|legclf_law_stack_exchange|
|Compatibility:|Legal NLP 1.0.0+|
|License:|Licensed|
|Edition:|Official|
|Input Labels:|[document, token]|
|Output Labels:|[class]|
|Language:|en|
|Size:|410.1 MB|
|Case sensitive:|true|
|Max sentence length:|512|

## References

Train dataset available [here](https://huggingface.co/datasets/jonathanli/law-stack-exchange)

## Benchmarking

```bash
label precision recall f1-score support
business 0.50 0.24 0.32 17
constitutional-law 0.94 0.68 0.79 25
contract-law 0.88 0.85 0.86 91
copyright 0.91 0.97 0.94 151
criminal-law 0.80 0.91 0.85 75
employment 0.74 0.93 0.82 30
liability 0.67 0.31 0.42 13
privacy 0.77 0.82 0.79 28
tax-law 0.93 0.78 0.85 32
trademark 0.89 0.91 0.90 44
accuracy - - 0.86 506
macro-avg 0.80 0.74 0.75 506
weighted-avg 0.85 0.86 0.85 506
```
165 changes: 165 additions & 0 deletions docs/_posts/gadde5300/2023-08-07-legner_bert_subpoenas_sm_en.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
---
layout: model
title: Legal NER on Subpoenas (Small)
author: John Snow Labs
name: legner_bert_subpoenas_sm
date: 2023-08-07
tags: [en, licensed, tensorflow]
task: Named Entity Recognition
language: en
edition: Legal NLP 1.0.0
spark_version: 3.0
supported: true
engine: tensorflow
annotator: LegalBertForTokenClassification
article_header:
type: cover
use_language_switcher: "Python-Scala-Java"
---

## Description

This is a Legal NER model aimed to extract 19 entities from subpoenas. This is called a small version because it has been trained on more generic labels. The larger versions of this model will be available on models hub.

## Predicted Entities

`COURT`, `APPOINTMENT_DATE`, `DEADLINE_DATE`, `DOCUMENT_DATE_FROM`, `ADDRESS`, `APPOINTMENT_HOUR`, `DOCUMENT_DATE_TO`, `DOCUMENT_PERSON`, `DOCUMENT_DATE_YEAR`, `STATE`, `MATTER_VS`, `CASE`, `COUNTY`, `DOCUMENT_TOPIC`, `MATTER`, `SUBPOENA_DATE`, `SIGNER`, `RECEIVER`, `DOCUMENT_TYPE`

{:.btn-box}
<button class="button button-orange" disabled>Live Demo</button>
<button class="button button-orange" disabled>Open in Colab</button>
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/legal/models/legner_bert_subpoenas_sm_en_1.0.0_3.0_1691423741988.zip){:.button.button-orange.button-orange-trans.arr.button-icon.hidden}
[Copy S3 URI](s3://auxdata.johnsnowlabs.com/legal/models/legner_bert_subpoenas_sm_en_1.0.0_3.0_1691423741988.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}

## How to use



<div class="tabs-box" markdown="1">
{% include programmingLanguageSelectScalaPythonNLU.html %}
```python
from pyspark.sql import functions as F

document_assembler = nlp.DocumentAssembler()\
.setInputCol("text")\
.setOutputCol("document")\

sentence_detector = nlp.SentenceDetector()\
.setInputCols(["document"])\
.setOutputCol("sentence")\

tokenizer = nlp.Tokenizer() \
.setInputCols(["sentence"]) \
.setOutputCol("token")

ner_model = legal.BertForTokenClassification.pretrained("legner_bert_subpoenas_sm", "en", "legal/models")\
.setInputCols(["sentence", "token"])\
.setOutputCol("ner")\
.setCaseSensitive(True)\
.setMaxSentenceLength(512)

ner_converter = nlp.NerConverter()\
.setInputCols(["sentence", "token", "ner"])\
.setOutputCol("ner_chunk")

pipeline = nlp.Pipeline(stages=[
document_assembler,
sentence_detector,
tokenizer,
ner_model,
ner_converter
])


empty_data = spark.createDataFrame([[""]]).toDF("text")

model = pipeline.fit(empty_data)

text = """In addition , in an earlier motion for summary disposition in which all Respondents joined , and which this Court denied in its Order of April30 , 2013 , Respondent Deloitte Touche Tohmatsu Certified Public Accountants Ltd ."""
data = spark.createDataFrame([[text]]).toDF("text")

result = model.transform(data)

result.select(F.explode(F.arrays_zip('ner_chunk.result', 'ner_chunk.metadata')).alias("cols")) \
.select(F.expr("cols['0']").alias("chunk"),
F.expr("cols['1']['entity']").alias("label")).show(50, truncate = False)
```

</div>

## Results

```bash
+------------------------+---------------+
|chunk |label |
+------------------------+---------------+
|summary disposition |DOCUMENT_TYPE |
|Deloitte Touche Tohmatsu|DOCUMENT_PERSON|
+------------------------+---------------+
```

{:.model-param}
## Model Information

{:.table-model}
|---|---|
|Model Name:|legner_bert_subpoenas_sm|
|Compatibility:|Legal NLP 1.0.0+|
|License:|Licensed|
|Edition:|Official|
|Input Labels:|[document, token]|
|Output Labels:|[ner]|
|Language:|en|
|Size:|401.1 MB|
|Case sensitive:|true|
|Max sentence length:|128|

## References

In House annotated dataset

## Benchmarking

```bash
label precision recall f1-score support
B-COURT 1.00 0.60 0.75 30
I-APPOINTMENT_DATE 0.57 0.65 0.60 20
I-COURT 0.93 0.89 0.91 166
B-APPOINTMENT_DATE 0.67 0.44 0.53 9
I-DEADLINE_DATE 0.83 0.26 0.40 19
B-DOCUMENT_DATE_FROM 0.80 1.00 0.89 16
I-ADDRESS 0.87 0.94 0.90 1046
B-APPOINTMENT_HOUR 0.43 0.92 0.59 13
B-DOCUMENT_DATE_TO 0.88 1.00 0.93 7
I-APPOINTMENT_HOUR 1.00 0.15 0.26 20
B-DOCUMENT_PERSON 0.79 0.84 0.82 2919
B-DOCUMENT_DATE_YEAR 0.00 0.00 0.00 5
B-STATE 0.59 0.79 0.68 24
I-MATTER_VS 0.65 0.79 0.71 150
I-CASE 0.00 0.00 0.00 11
I-COUNTY 0.00 0.00 0.00 0
B-DOCUMENT_TOPIC 0.64 0.77 0.70 208
B-COUNTY 0.00 0.00 0.00 0
B-MATTER 0.85 0.86 0.86 328
I-DOCUMENT_DATE_FROM 0.87 1.00 0.93 48
I-SUBPOENA_DATE 0.56 0.28 0.38 53
I-SIGNER 0.56 0.46 0.50 59
I-DOCUMENT_DATE_TO 0.83 1.00 0.91 25
I-RECEIVER 0.71 0.52 0.60 98
B-SIGNER 0.76 0.49 0.59 39
I-DOCUMENT_TOPIC 0.83 0.80 0.81 725
I-STATE 0.67 0.29 0.40 14
B-MATTER_VS 0.78 0.82 0.80 136
I-DOCUMENT_TYPE 0.83 0.87 0.85 621
B-DEADLINE_DATE 0.00 0.00 0.00 6
I-MATTER 0.88 0.82 0.85 479
B-DOCUMENT_TYPE 0.87 0.90 0.88 1714
B-ADDRESS 0.81 0.83 0.82 101
B-SUBPOENA_DATE 0.42 0.28 0.33 18
B-CASE 0.91 0.97 0.94 312
I-DOCUMENT_PERSON 0.80 0.83 0.81 3672
B-RECEIVER 0.76 0.63 0.69 46
micro-avg 0.82 0.84 0.83 13157
macro-avg 0.66 0.61 0.61 13157
weighted-avg 0.82 0.84 0.83 13157
```