Add table evaluation for table datasets, and hyperscaler predictors #47
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 7 commits
March 14, 2025 01:50
Need to work on image dimensions as the visualizations seem to be off
added 3 commits
March 19, 2025 18:37
- Save the hyperscaler API jsons
- Save the docling document formats
- Temporary files are stored as below:
- `intermediate_files` -- for parquet files
- `microsoft` - Root folder that contains API output files
- `docling_document` - docling document output of the MS outputs
- `visualizations` - Output of table visualizations
- Prediction table output converted to doclingDocument (has a bug in the spans) - Save the GT doclingDocument as well for debugging - Some cleanup
26 tasks
- New implementation for S3 source - Save the intermediate files - prediction and doclingDocuments for pred+groundTruth for debugging - Implement the run for TEDS metric for (limited) Fintabnet dataset from cos; If the API predictions are available, will use them instead of calling the API - Needed to pin library versions of pydantic, url + s3 library
Contributor
|
This PR will be superseded by #47, which ports the key features from this branch to the final docling-eval API. |
Contributor
Author
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Working on Fintabnet datasetbuilder + Azure Provider.
//Creating a draft as requested, only for initial review.
Changes include