Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add background task to reindex all artifacts #1407

Closed
adpi2 opened this issue May 27, 2024 · 0 comments
Closed

Add background task to reindex all artifacts #1407

adpi2 opened this issue May 27, 2024 · 0 comments
Assignees

Comments

@adpi2
Copy link
Member

adpi2 commented May 27, 2024

Motivation

Each time we update the Artifact model we need to re-download all POM files to get the new values and populate the new fields in the database.

Identified Obstacles

  • We need a new endpoint to trigger the task. (The difference between a scheduled job and a task is: a scheduled job runs automatically, a task is triggered by an admin once in a while).
  • We should make sure that downloading all POM files will not fill the disk completely. (Probably it won't because POM files are quite small, smaller than JAR files).
  • What if we cannot download some artifact: should we remove it from Scaladex?

Implementation Guideline

  • In Task, add a new task update-maven-artifacts
  • In AdminPage, add a new endpoint to trigger this task.
  • In AdminService create a method updateMavenArtifacts that will download each POM file and reindex it. Maybe create a new class File MavenArtifactUpdater to implement this method. It should
    • Query the database to get all Maven artifact references.
    • Use CoursierResolver to download the POM file
    • Use PublishProcess to publish the POM file.

Expectations

We should be able to trigger the task in the admin page and it should update all artifacts one by one without failing. If there is a failure it should appear in the admin page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants