Skip to content

feat: Add DocVQA dataset builder #54

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 100 commits into from
Apr 1, 2025
Merged

feat: Add DocVQA dataset builder #54

merged 100 commits into from
Apr 1, 2025

Conversation

@cau-git
Copy link
Contributor

@cau-git cau-git commented Apr 1, 2025 •
edited
Loading

Currently the DocVQA dataset builder only extracts the page images but none of the questions are stored. There is no layout annotation.

PeterStaar-IBM and others added 30 commits February 12, 2025 12:48
@PeterStaar-IBM
correct mpy
3053b56 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@PeterStaar-IBM
reformatting
a7869e5 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@PeterStaar-IBM
Merge branch 'main' into fix/docling-dpbench
9b03f64
@PeterStaar-IBM
adding the script to make an initial dataset from pdf's
31f7a1a 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@PeterStaar-IBM
before switching to specific docling-core branch
86efb2d 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@PeterStaar-IBM
rebased on kv-items and updated the create script in CVAT
b504372 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@PeterStaar-IBM
fixed the cvat
68f8b1d 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@PeterStaar-IBM
Merge branch 'main' into fix/docling-dpbench
f045f0b
@PeterStaar-IBM
added the annotation description on CVAT
7f59ff2 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@PeterStaar-IBM
added the annotation description on CVAT (2)
3baf9f1 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@PeterStaar-IBM
added the annotation description on CVAT (3)
c45d2e3 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@cau-git
[WIP] Crafting new dataset builder and prediction provider API
d8a8a59 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Merge from main
b8cb738 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Restructure to docling_eval_next
23834fb 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Fix mypy
2206a97 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Merge branch 'main' of github.com:DS4SD/docling-eval into cau/new-cla…
ea02901 
…ss-design
@cau-git
Fix f-strings
a6295bc 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Merge from main
684fd27 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Merge branch 'main' of github.com:DS4SD/docling-eval into cau/new-cla…
415d767 
…ss-design
@cau-git
Changes for prediction_provider interface, to support all cases.
3b62bc6 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Add omnidocbench DatasetBuilder
0555485 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Add doclaynet v1, funsd
3027ba8 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Fixes
12f025f 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Add XFUND, more fixes
4693b2c 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@Saidgurbuz
update the kv cell creation to prevent false positives
f8bd070 
Signed-off-by: Saidgurbuz <said.gurbuz@epfl.ch>
@nikos-livathinos
chore: Fixing imports
57df7bb 
Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
@nikos-livathinos
chore: Update docling-core version
1250f5d 
Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
@nikos-livathinos
feat: Introduce new design for Evaluators based on BaseEvaluator that…
51260fd 
… accept external predictions.

And utility adapters.

Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
@cau-git
Factor PredictionProvider out of dataset builder, many fixes on Datas…
6812656 
…etRecord

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Merge branch 'cau/new-class-design' of github.com:DS4SD/docling-eval …
8df9157 
…into cau/new-class-design
cau-git and others added 23 commits March 31, 2025 16:58
@cau-git
Fixes, pass tests
e898ab5 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@nikos-livathinos
feat: Refactor the TableEvaluator to use the new class design.
7be89ef 
Move common evaluator code to BaseEvaluator.
Add more unit tests. Introduce pytest dependencies.

Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
@cau-git
Update lockfile
a7f830b 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Update lockfile
8c81563 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Make pytest CI output more verbose
3e607ee 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@nikos-livathinos
feat: Refactor the ReadingOrderEvaluator to use the new class design.
af87bad 
Remove the BaseReadingOrderEvaluator. Add unit test.

Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
@cau-git
Optimize GT downloading behaviour
6af49cd 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Add file sources
c5b4a24 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Allow pytest output on CI
f165e79 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Disable tests in CI
79cb068 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Reenable tests in CI
153e5b9 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Add correct @pytest.mark.dependency()
7e83d61 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@nikos-livathinos
feat: Introduce TypeVars for the UnitEvaluation and DatasetEvaluation…
40c6d97 
… used by the BaseEvaluator.

Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
@cau-git
Merge branch 'cau/new-class-design' of github.com:DS4SD/docling-eval …
fe96106 
…into cau/new-class-design
@cau-git
Minimize tests in CI
880ecf2 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@nikos-livathinos
feat: Refactor BboxTestEvaluator to use the new design. Introduce uni…
a23f7ea 
…t test.

Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
@cau-git
Remove streaming in DocLaynet v1
77b3bed 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Merge branch 'cau/new-class-design' of github.com:DS4SD/docling-eval …
9739dd6 
…into cau/new-class-design
@cau-git
Add back test dependency
71511d7 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Add DocVQA dataset builder
fddc215 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Bugfixes
97ef6ab 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Remove prints
9c437e8 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Merge from main
c836959 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git cau-git requested a review from PeterStaar-IBM April 1, 2025 11:17
cau-git added 2 commits April 1, 2025 13:18
@cau-git
Cleanup
e6655b1 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@cau-git
Add DocVQA to CLI
604e4a7 
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@nikos-livathinos nikos-livathinos self-requested a review April 1, 2025 11:31
@cau-git cau-git merged commit 09c40f1 into main Apr 1, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PeterStaar-IBM PeterStaar-IBM PeterStaar-IBM approved these changes

@nikos-livathinos nikos-livathinos nikos-livathinos approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@cau-git @PeterStaar-IBM @nikos-livathinos @Saidgurbuz