Plain-text seq2seq models #26
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix enumeration of input files when no data channel manifest provided
Resolve issue where mlm get_task() and class TextractLayoutLMDataCollatorForLanguageModelling would throw errors if model_param_names argument was missing, even though it was marked as optional. These will now raise warnings instead.
Text-only seq2seq field normalization model using T5 to normalize date fields to YYYY-MM-DD format. Includes integration to pipeline (via postproc Lambda + field configuration SSM param) and setup NB (in Optional Extras.ipnyb), but no updates to readme/customization guide yet.
Allow non-fast tokenizers for seq2seq modelling tasks, which is necessary if wanting to use byT5 model in seq2seq mode. The previous restriction requiring Fast Tokenizers matters for ner and mlm tasks where custom data collation expects a particular API, but not for seq2seq.
Re-weight sample date re-formatting task from UK date formats towards US (month-first) formats, for better consistency with the CFPB credit cards sample documents.
Enable/streamline creating a SMGT labelling job using the custom (transcription reviews) template from data preparation notebook. With addition of the seq2seq model, more users are likely to be interested in collecting text normalizations.
Date normalization generators were not correctly adding ordinal suffixes to day numbers in some formats as intended: Just decorating the format name without changing the actual format string.
Explicitly set up Python logging level in pre/post SMGT Lambda functions to ensure (only) expected messages are generated. Fixes issue of .info() calls not coming through from postproc Lambda by default.
Simplify and improve the output format of labelling jobs using the custom (bboxes + OCR transcript reviews) task UI. The new template avoids duplicating the bboxes and recording Textract word IDs (which are very verbose), and makes it easier to pull out source/OCR text vs target/corrected text for each field (useful for seq2seq).
Update the training script data loaders to support training seq2seq model (still plain text) from SMGT custom task UI entity OCR validation results. Update seq2seq in Extras notebook to match setup used in demo. Add mention of SMGT data usage to Extras notebook.
Update README and CUSTOMIZATION_GUIDE to mention new seq2seq entity text normalization training option.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available: N/A
Description of changes:
This PR introduces trainable sequence-to-sequence models for generative tasks like OCR error correction or field re-formatting (e.g. date normalization).
Convert dates to YYYY-MM-DD: Dec 31 1999to yield1999-12-31)Normalize Agreement Effective Date: Dec 31 1999to yield1999-12-31, or however labelers have standardised the field)Some important caveats:
Testing done:
Re-verified notebooks on an existing environment (didn't re-build from scratch)
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.