Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added import task to import a single annotated doc. Also added a test. #110

Merged
merged 4 commits into from
May 13, 2023

Conversation

JSv4
Copy link
Owner

@JSv4 JSv4 commented May 11, 2023

Summary

This PR borrows code from the import_corpus task and creates a new task called import_document_to_corpus. The new task is designed to import a single document with its annotations and labels into an existing corpus. To achieve this, we introduced a new data type called OpenContractsAnnotatedDocumentImportType, which represents the JSON data structure for importing a single document along with its annotations and labels. The PDF for the document is converted to a base64 string and provided as a property in this data type.

Major changes in this PR include:

  • Introduced OpenContractsAnnotatedDocumentImportType data type for importing a single document and its annotations and labels.
  • Refactored import_corpus task into import_document_to_corpus task, which accepts the id of the target corpus and the new data type OpenContractsAnnotatedDocumentImportType.
  • The task checks if new labels need to be created, creates them if necessary, imports the document, and finally imports the annotations for the document.
  • Added a test for the import_document_to_corpus task in test_tasks.py.
  • Added mutation UploadAnnotatedDocument: This mutation is responsible for uploading a single annotated document to an existing corpus. It takes two required arguments: target_corpus_id and document_import_data. The target_corpus_id is the ID of the target corpus to which the document will be added. The document_import_data is a string representation of the OpenContractsAnnotatedDocumentImportType object that contains the necessary information about the document, annotations, and labels. The mutation calls the import_document_to_corpus task with the provided arguments and applies it asynchronously.
  • Added test test_upload_annotated_document: This test is designed to check whether the UploadAnnotatedDocument mutation works as expected. It sets up the necessary test environment with prerequisites such as a user, a corpus, and sample import data. The test then prepares the upload_annotated_document_mutation to be executed with the given variables (corpus ID and document import data). After executing the mutation, the test checks whether the response contains "ok" as True and the message as "SUCCESS`, indicating that the task started successfully.

@codecov
Copy link

codecov bot commented May 11, 2023

Codecov Report

Merging #110 (c8d0d4d) into main (26cdf7a) will increase coverage by 0.82%.
The diff coverage is 81.91%.

@@            Coverage Diff             @@
##             main     #110      +/-   ##
==========================================
+ Coverage   64.68%   65.50%   +0.82%     
==========================================
  Files          48       48              
  Lines        2047     2128      +81     
==========================================
+ Hits         1324     1394      +70     
- Misses        723      734      +11     
Impacted Files Coverage Δ
opencontractserver/tasks/export_tasks.py 26.53% <ø> (+0.26%) ⬆️
opencontractserver/utils/etl.py 20.16% <33.33%> (+0.50%) ⬆️
opencontractserver/tasks/import_tasks.py 39.24% <78.87%> (+22.58%) ⬆️
opencontractserver/tasks/__init__.py 100.00% <100.00%> (ø)
opencontractserver/tasks/doc_tasks.py 48.16% <100.00%> (+0.38%) ⬆️
opencontractserver/types/dicts.py 100.00% <100.00%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@JSv4 JSv4 merged commit c2b2902 into main May 13, 2023
5 checks passed
@JSv4 JSv4 deleted the JSv4/add-ability-to-import-single-annotated-doc branch May 13, 2023 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant