New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Early stage pipeline plugin functions #2814

Merged
merged 5 commits into from Feb 22, 2018

Conversation

Projects
None yet
2 participants
@autrimpo
Contributor

autrimpo commented Feb 18, 2018

This is a bug report together with a PR because I like to scratch my own itch :)
Something has been bugging me for quite some time after switching to Opus - some of the albums I've imported didn't have replaygain calculated, even though the replaygain plugin did definitely run. After lots of debugging the issue seems to be as follows:

  1. User tries to import a FLAC album
  2. Since the plugins don't run in a deterministic order (as far as I've noticed), ReplayGain is sometimes calculated on the FLAC files
  3. Files get converted to Opus
  4. There are ReplayGain values calculated from FLAC but Opus only takes R128
  5. The imported files need to have replaygain calculated again, manually

I've added a (quite ugly) hack to always convert the file first, which from my testing seems to reliably solve the issue. If you have better idea how to solve this (I've considered dividing the plugins into groups by 'priority' but that felt like an overkill), I'll happily implement it.

@sampsyo

This comment has been minimized.

Show comment
Hide comment
@sampsyo

sampsyo Feb 18, 2018

Member

Got it! Thanks for explaining why it's nice for this to run first. I can see it being a good idea in general for convert to run before other operations.

It would be good to come up with a general way to prioritize import stages. We don't have to go crazy with anything too elaborate (such as a priority score), but how about just a mechanism for asking to be inserted at the beginning of the list? For example, we could add an early_import_stages alternative to import_stages that does this.

Member

sampsyo commented Feb 18, 2018

Got it! Thanks for explaining why it's nice for this to run first. I can see it being a good idea in general for convert to run before other operations.

It would be good to come up with a general way to prioritize import stages. We don't have to go crazy with anything too elaborate (such as a priority score), but how about just a mechanism for asking to be inserted at the beginning of the list? For example, we could add an early_import_stages alternative to import_stages that does this.

@autrimpo

This comment has been minimized.

Show comment
Hide comment
@autrimpo

autrimpo Feb 18, 2018

Contributor

So if I understand correctly, plugins would be either 'regular' or 'early' and 'early' ones would always run before 'regular' ones but other than that the order would be still non-deterministic? That's very similar to what I was trying with the priority groups, except I had three and this only has two. I'll have a go at it tomorrow.

Contributor

autrimpo commented Feb 18, 2018

So if I understand correctly, plugins would be either 'regular' or 'early' and 'early' ones would always run before 'regular' ones but other than that the order would be still non-deterministic? That's very similar to what I was trying with the priority groups, except I had three and this only has two. I'll have a go at it tomorrow.

@sampsyo

This comment has been minimized.

Show comment
Hide comment
@sampsyo

sampsyo Feb 18, 2018

Member

Yeah, you got it exactly. I could also see three groups, but two might suffice—I don't quite see a use case at the moment for running as late as possible.

Member

sampsyo commented Feb 18, 2018

Yeah, you got it exactly. I could also see three groups, but two might suffice—I don't quite see a use case at the moment for running as late as possible.

@autrimpo

This comment has been minimized.

Show comment
Hide comment
@autrimpo

autrimpo Feb 18, 2018

Contributor

It can always be added later if the need arises.

Contributor

autrimpo commented Feb 18, 2018

It can always be added later if the need arises.

@autrimpo

This comment has been minimized.

Show comment
Hide comment
@autrimpo

autrimpo Feb 19, 2018

Contributor

I've added functionality for early stage functions, decided it might be even better because plugins can now request to have part of their functions ran early and the rest at the normal time.

I think it might be worthwhile to add a test for this as well, have a plugin with 1 early and 1 normal function in the pipeline and check that they run in the correct order. I'll add one after we agree on the final implementation.

Contributor

autrimpo commented Feb 19, 2018

I've added functionality for early stage functions, decided it might be even better because plugins can now request to have part of their functions ran early and the rest at the normal time.

I think it might be worthwhile to add a test for this as well, have a plugin with 1 early and 1 normal function in the pipeline and check that they run in the correct order. I'll add one after we agree on the final implementation.

@autrimpo autrimpo changed the title from Make 'convert' plugin run first in the pipeline to Early stage pipeline plugin functions Feb 19, 2018

@sampsyo

Looks perfect! To finish this off, could you please document the new API in the plugin docs (and add a quick changelog entry)?

Show outdated Hide outdated beets/plugins.py Outdated
@autrimpo

This comment has been minimized.

Show comment
Hide comment
@autrimpo

autrimpo Feb 21, 2018

Contributor

Done, though I haven't written a test as I initially planned because unfortunately I don't know how.

Contributor

autrimpo commented Feb 21, 2018

Done, though I haven't written a test as I initially planned because unfortunately I don't know how.

@sampsyo sampsyo merged commit bbadb5f into beetbox:master Feb 22, 2018

1 check failed

continuous-integration/appveyor/pr AppVeyor was unable to build non-mergeable pull request
Details

sampsyo added a commit that referenced this pull request Feb 22, 2018

Merge pull request #2814 from autrimpo/replaygain-after-convert
Early stage pipeline plugin functions

sampsyo added a commit that referenced this pull request Feb 22, 2018

sampsyo added a commit that referenced this pull request Feb 22, 2018

@sampsyo

This comment has been minimized.

Show comment
Hide comment
@sampsyo

sampsyo Feb 22, 2018

Member

Awesome; thank you! 🐟 🌹 Merged.

Member

sampsyo commented Feb 22, 2018

Awesome; thank you! 🐟 🌹 Merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment