Skip to content

Parallelize candidate lookup for metadata plugins.#6546

Merged
semohr merged 2 commits into
masterfrom
parallel_lookup
Apr 19, 2026
Merged

Parallelize candidate lookup for metadata plugins.#6546
semohr merged 2 commits into
masterfrom
parallel_lookup

Conversation

@semohr
Copy link
Copy Markdown
Contributor

@semohr semohr commented Apr 18, 2026

This pull request introduces concurrent execution of metadata source plugin searches and lookups, which significantly improves performance when multiple plugins are enabled. Instead of running each plugin sequentially, the code now uses threads to perform plugin lookups in parallel, reducing overall wait time for I/O-bound operations.

For me this improved lookup times by up to 5s each! The improvement should roughly scale linear with the number of enabled metadata plugins.

Copilot AI review requested due to automatic review settings April 18, 2026 15:27
@semohr semohr requested a review from a team as a code owner April 18, 2026 15:27
@semohr semohr added metadata_plugins An issue that is generally related to metadata plugin logic. performance labels Apr 18, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

PR make metadata source plugin search/lookup run in parallel threads, so total wait time go down when many plugins enabled.

Changes:

  • Run candidates / item_candidates / albums_for_ids / tracks_for_ids across metadata source plugins using ThreadPoolExecutor.
  • Add changelog entry for concurrent metadata plugin execution.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
docs/changelog.rst Document new concurrent metadata plugin searches/lookups feature.
beets/metadata_plugins.py Replace sequential plugin iteration with threaded execution using futures.

Comment thread beets/metadata_plugins.py
Comment thread beets/metadata_plugins.py
Comment thread beets/metadata_plugins.py
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.19%. Comparing base (be33782) to head (f7ed69b).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #6546   +/-   ##
=======================================
  Coverage   71.18%   71.19%           
=======================================
  Files         150      150           
  Lines       19172    19175    +3     
  Branches     3084     3084           
=======================================
+ Hits        13648    13651    +3     
  Misses       4864     4864           
  Partials      660      660           
Files with missing lines Coverage Δ
beets/metadata_plugins.py 84.47% <100.00%> (+0.29%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@snejus
Copy link
Copy Markdown
Member

snejus commented Apr 18, 2026

image
squinting-skeptical.mp4

Or maybe it's just our nice design 😆

@semohr semohr force-pushed the parallel_lookup branch 3 times, most recently from 3caa9f8 to c9604c1 Compare April 18, 2026 17:55
@semohr
Copy link
Copy Markdown
Contributor Author

semohr commented Apr 18, 2026

@snejus We should consider doing something similar for send as the events handling could also be parallelized. For me the event handling takes more time than the lookup itself :/

Comment thread docs/changelog.rst Outdated
@semohr semohr enabled auto-merge April 19, 2026 13:16
@semohr semohr merged commit 82384da into master Apr 19, 2026
19 checks passed
@semohr semohr deleted the parallel_lookup branch April 19, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

metadata_plugins An issue that is generally related to metadata plugin logic. performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants