Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unzip param to downloadModelDirectly in ResourceDownloader #13796

Conversation

mehmetbutgul
Copy link
Contributor

This PR adds a unzip argument to downloadModelDirectly in ResourceDownloader

Description

issue: https://github.com/JohnSnowLabs/spark-nlp-internal/issues/1837

  1. Added unzip argument to downloadModelDirectly() in ResourceDownloader.
    BEFORE: downloadModelDirectly(name, remote_loc="public/models")
    AFTER: downloadModelDirectly(name, remote_loc="public/models", unzip=True)

  2. Updated also name argument
    BEFORE: clinical/models/deidentify_enriched_clinical_en_2.7.2_2.4_1611917177874.zip
    AFTER: 1--> clinical/models/deidentify_enriched_clinical_en_2.7.2_2.4_1611917177874.zip
    2--> s3://auxdata.johnsnowlabs.com/clinical/models/deidentify_enriched_clinical_en_2.7.2_2.4_1611917177874.zip

  3. deleted duplicate codes
    Note: supported Backward compatibility

How Has This Been Tested?

Tested via Scala and Python tests locally

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • [x ] Code improvements with no or little impact
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • [x ] I have updated the documentation accordingly.
  • I have read the CONTRIBUTING page.
  • I have added tests to cover my changes.
  • [] All new and existing tests passed.

@mehmetbutgul mehmetbutgul added enhancement new-feature Introducing a new feature labels May 11, 2023
@maziyarpanahi maziyarpanahi added the DON'T MERGE Do not merge this PR label May 11, 2023
@mehmetbutgul
Copy link
Contributor Author

I cannot find a new release-candidate branch.

@coveralls
Copy link

Pull Request Test Coverage Report for Build 4947838414

  • 11 of 39 (28.21%) changed or added relevant lines in 2 files are covered.
  • 12 unchanged lines in 9 files lost coverage.
  • Overall coverage decreased (-0.006%) to 65.838%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/main/scala/com/johnsnowlabs/nlp/pretrained/ResourceDownloader.scala 11 14 78.57%
src/main/scala/com/johnsnowlabs/nlp/pretrained/S3ResourceDownloader.scala 0 25 0.0%
Files with Coverage Reduction New Missed Lines %
src/main/scala/com/johnsnowlabs/nlp/pretrained/S3ResourceDownloader.scala 1 34.29%
src/main/scala/com/johnsnowlabs/nlp/util/GraphBuilder.scala 1 97.73%
src/main/scala/com/johnsnowlabs/storage/HasStorageModel.scala 1 87.5%
src/main/scala/com/johnsnowlabs/storage/StorageHelper.scala 1 59.18%
src/main/scala/com/johnsnowlabs/util/Benchmark.scala 1 76.47%
src/main/scala/com/johnsnowlabs/util/FileHelper.scala 1 32.0%
src/main/scala/com/johnsnowlabs/nlp/annotators/sentence_detector_dl/SentenceDetectorDLModel.scala 2 82.72%
src/main/scala/com/johnsnowlabs/nlp/util/io/ResourceHelper.scala 2 44.99%
src/main/scala/com/johnsnowlabs/storage/HasStorage.scala 2 50.0%
Totals Coverage Status
Change from base Build 4945586143: -0.006%
Covered Lines: 8632
Relevant Lines: 13111

💛 - Coveralls

@maziyarpanahi maziyarpanahi changed the base branch from master to release/443-release-candidate May 24, 2023 11:53
Copy link
Contributor

@danilojsl danilojsl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGFM

@maziyarpanahi maziyarpanahi merged commit f24c85f into release/443-release-candidate May 25, 2023
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DON'T MERGE Do not merge this PR enhancement new-feature Introducing a new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants