Skip to content

Conversation

@percevalw
Copy link
Member

Description

Added

  • Add multi-modal transformers (huggingface-embedding) with windowing options
  • Add render_page option to pdfminer extractor, for multi-modal PDF features
  • Add inference utilities (accelerators), with simple mono process support and multi gpu / cpu support
  • Packaging utils (pipeline.package(...)) to make a pip installable package from a pipeline

Changed

  • Updated API to follow EDS-NLP's refactoring
  • Updated confit to 0.4.2 (better errors) and foldedtensor to 0.3.0 (better multiprocess support)
  • Removed pipeline.score. You should use pipeline.pipe, a custom scorer and pipeline.select_pipes instead.
  • Better test coverage

Fixed

  • Fixed attrs dependency only being installed in dev mode

Checklist

  • If this PR is a bug fix, the bug is documented in the test suite.
  • Changes were documented in the changelog (pending section).
  • If necessary, changes were made to the documentation.

@codecov
Copy link

codecov bot commented Sep 7, 2023

Codecov Report

Patch coverage is 100.00% of modified lines.

Files Changed Coverage
edspdf/__init__.py 100.00%

📢 Thoughts on this report? Let us know!.

@percevalw percevalw merged commit dc15622 into main Sep 7, 2023
@percevalw percevalw deleted the v0.8.0 branch September 7, 2023 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants