Skip to content

Refactoring of package management#442

Merged
geritwagner merged 23 commits intomainfrom
internal_packages
Jul 25, 2024
Merged

Refactoring of package management#442
geritwagner merged 23 commits intomainfrom
internal_packages

Conversation

@geritwagner
Copy link
Collaborator

@geritwagner geritwagner commented May 18, 2024

The package management is refactored to make sure that colrev packages are installed as full Python packages with their own dependencies, and that package discovery uses importlib functionality (plugin endpoints associated with the "colrev" group). This means that fully independent colrev packages could be developed and used in the colrev workflow. It would also allow us to reduce dependencies in the core by moving them to the colrev packages.

While the package management is set up for external packages, most colrev packages will remain in the colrev repository. Such a monorepo approach should allow us to refactor the codebase more efficiently, and to ensure consistency across colrev packages. Once the development of the core matures, colrev packages may be extracted to separate repositories (standalone packages).

The plugin management systems of pluggy and llm serve as an inspiration.

  • Handling colrev-specific package metadata and extracting it for the documentation (colrev.package_manager.doc_registry_manager)
    • Package metadata should ideally be added to the pyproject.toml
    • When packages are installed, custom metadata in the pyproject.toml is not made available (e.g., for the doc_registry_manager)
  • The package development documentation must be updated accordingly

@geritwagner geritwagner added this to the v0.14.0 milestone May 18, 2024
@geritwagner geritwagner requested a review from julianprester May 18, 2024 17:33
@geritwagner geritwagner force-pushed the main branch 4 times, most recently from 0a8e615 to 5546058 Compare May 26, 2024 15:10
@geritwagner geritwagner force-pushed the main branch 2 times, most recently from 01965ad to b0bc6fa Compare June 25, 2024 08:30
@geritwagner geritwagner force-pushed the internal_packages branch 3 times, most recently from 4766ee3 to 21ac8c9 Compare July 22, 2024 10:07
@geritwagner geritwagner force-pushed the internal_packages branch 4 times, most recently from 535f35a to d031818 Compare July 24, 2024 19:59
@geritwagner geritwagner merged commit 47e947c into main Jul 25, 2024
geritwagner added a commit that referenced this pull request Jul 25, 2024
* update internal packages (extras with dependencies)

* update Package structure

* update docs/package development

* update poetry.lock

* load packages through python/plugin entrypoints

* test importlib import

* test

* add notes

* format

* deactivate discover_packages() in cli

* implement colrev install ...

* update

* separate package documentation based on pyproject.toml

* minor fixes

* rewrite doc-registry-manager

* fix springer/synergy

* move dependencies to packages

* refactor doc-registry-manager

* Use more informative commit messages

* update header for package docs

* update docs
@geritwagner geritwagner deleted the internal_packages branch August 28, 2024 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant