Skip to content

Do not write unchanged items to the library in FtInTitle #5718

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 14, 2025

Conversation

peterjdolan
Copy link
Contributor

@peterjdolan peterjdolan commented Apr 10, 2025

Description

FtInTitle performs a library store operation for every item it processes, whether or not the item has changed. By limiting the item.store() call to only those cases when the item has changed, the plugin’s performance when processing an entire library improves by two to three orders of magnitude.

To Do

  • Documentation. (If you've added a new command-line flag, for example, find the appropriate page under docs/ to describe it.)
  • Changelog. (Add an entry to docs/changelog.rst to the bottom of one of the lists near the top of the document.)
  • Tests. (Very much encouraged but not strictly required.)

@peterjdolan peterjdolan changed the title Parallelize several built-in plugin operations Parallelize FtInTitle operations Apr 10, 2025
@peterjdolan peterjdolan changed the title Parallelize FtInTitle operations Parallelize FtInTitle Apr 10, 2025
Copy link

Thank you for the PR! The changelog has not been updated, so here is a friendly reminder to check if you need to add an entry.

@wisp3rwind
Copy link
Member

FtInTitle, on the other hand, gains a substantial improvement, going from 2 titles / second to 200-2000 titles / second on a computer with 32 cores.

That doesn't make a whole lot of sense, the speedup should be closer to a factor 32 then. It seems like more is going here than just parallelization, possibly the behavior actually changed in some unintended way?

In any case, this PR seems to contain a bunch of unrelated changes and it's quite hard to get a reasonable overview of what really changed. It would be helpful if it could be focused on just introducing the parallelism without any extra changes to the plugin logic or logging (if that is actually relevant for the speed-up, it would be useful to point that out).

@peterjdolan
Copy link
Contributor Author

Yes, you're absolutely right. I looked at it again and I think the main performance improvement comes from only calling item.store() when the item was changed. I'll rewrite this to include only that modification.

@peterjdolan peterjdolan changed the title Parallelize FtInTitle Do not write unchanged items to the library in FtInTitle Apr 14, 2025
@snejus snejus requested a review from Copilot April 14, 2025 18:15
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 1 out of 2 changed files in this pull request and generated no comments.

Files not reviewed (1)
  • docs/changelog.rst: Language not supported

Copy link
Member

@wisp3rwind wisp3rwind left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks!

I went ahead, rebased this on master & squashed.

@wisp3rwind wisp3rwind enabled auto-merge (squash) April 14, 2025 18:21
@wisp3rwind wisp3rwind merged commit 447cc82 into beetbox:master Apr 14, 2025
14 checks passed
@wisp3rwind wisp3rwind mentioned this pull request Apr 14, 2025
@wisp3rwind
Copy link
Member

For reference, the fact that this allows for such a large speed-up makes me wonder:

  • What is the impact of Model.store() in general? Given that there has been a bunch of dbcore optimizations, it would be surprising if this remains inefficient. (But maybe optimizations mainly concerned queries?)
  • Should there be an early exit in Model.store() if there are no dirty fields?
  • Is this not really an issue with model.store(), but rather with LibItem.store() which notifies plugins of database_change combined with a plugin that performs some slow operation upon receiving that event? @peterjdolan is there anything particular about your beets configuration that would make this likely?

wisp3rwind added a commit that referenced this pull request Apr 15, 2025
some work following up my review of #5718
- add typings
- refactor the logic in the core `ft_in_title` function, motivated by the
fact that I found this more difficult to read than necessary during the
review
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants