Refactor of metadata plugin and opt in all metadata plugins to new baseclass

[semohr]

Description

At the moment the MetaDataSourcePlugin has multiple responsibilities:

• fetch data via _search api
• defines contract for interaction within the beets autotag lookup

I propose splitting these responsibilities, as it would enable us to use the MetaDataSourcePlugin baseclass with plugins that use external packages to fetch data.

This follows from discussion in #5761 and #5748 (comment).

Feedback is highly appreciated, as this is mainly architectural decision and I would prefer if the new behavior is a shared consensus.

To Do

• Opt in plugins into the new MetaDataSourcePlugin
  • Spotify
  • Musicbrainz
  • Deezer
  • Beatport
  • Chroma
  • Disccogs
• Remove old MetaDataSourcePlugin and related functions
• Documentation on the ontology of plugins
• Changelog

This PR was initially #5764 and was accidentally closed as the target branch was deleted. Wasn't able to recover the original PR.

[github-actions]

Thank you for the PR! The changelog has not been updated, so here is a friendly reminder to check if you need to add an entry.

[Copilot]

Pull Request Overview

This PR refactors the metadata lookup architecture by introducing a new beets.metadata_plugins module with a single base interface and migrating all existing source plugins to use it.

• Added beets/metadata_plugins.py defining MetadataSourcePluginNext and SearchApiMetadataSourcePluginNext
• Updated all core and third-party plugins (Spotify, MusicBrainz, Discogs, Deezer, Beatport, Chroma/Acoustid, mbsync, missing) to inherit from the new base classes
• Adjusted tests and documentation to patch and reference beets.metadata_plugins instead of the old beets.plugins metadata methods

Reviewed Changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
test/test_ui.py	Adjusted plugin-loading assertion to use new API
test/test_importer.py	Patches updated to beets.metadata_plugins lookups
test/plugins/test_mbsync.py	References to plugins.track_for_id/album_for_id replaced
docs/dev/plugins.rst	Documentation revised for new metadata plugin interface
docs/dev/index.rst	Added Confuse library link
beets/util/id_extractors.py	Fixed key lookup to use lowercase source names
beets/plugins.py	Removed old metadata plugin base and helper methods
beets/metadata_plugins.py	New metadata plugin interface and loader functions
beetsplug/spotify.py	Migrated Spotify plugin to SearchApiMetadataSourcePluginNext
beetsplug/musicbrainz.py	Updated MusicBrainz plugin to MetadataSourcePluginNext
beetsplug/missing.py	Switched plugins.* calls to metadata_plugins.*
beetsplug/mbsync.py	Updated track/album lookup to new API
beetsplug/discogs.py	Refactored Discogs plugin to new metadata interface
beetsplug/deezer.py	Migrated Deezer plugin to SearchApiMetadataSourcePluginNext
beetsplug/chroma.py	Adapted Acoustid plugin to new base and import patterns
beetsplug/beatport.py	Refactored Beatport plugin to new base class and models

Comments suppressed due to low confidence (2)

test/test_ui.py:1062

• [nitpick] The variable name plugs is ambiguous; consider renaming to found_plugins or loaded_plugins for clarity.

plugs = plugins.find_plugins()

beetsplug/chroma.py:29

• TrackInfo is not exported by beets.metadata_plugins at runtime; import it from beets.autotag.hooks instead.

from beets.metadata_plugins import MetadataSourcePluginNext, TrackInfo

[snejus]

Good stuff! I added a couple of comments in the interfaces, however I'm unable to see how this affects the plugins because they have also been refactored.

Could you move optional changes to a separate PR to minimize any side effects this change may cause? I tried testing it and I found spotify stuck in some authentication exception loop, for example:

...done.
Success. Distance: 0.58
Sending event: albuminfo_received
Candidate: DRS - Mid Mic Crisis (6817507)
Computing track assignment...
...done.
Success. Distance: 0.58
spotify: Searching Spotify for 'album:Skeptical Answers artist:Vulkanski'
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.

[semohr]

Good stuff! I added a couple of comments in the interfaces, however I'm unable to see how this affects the plugins because they have also been refactored.

Could you move optional changes to a separate PR to minimize any side effects this change may cause? I tried testing it and I found spotify stuck in some authentication exception loop, for example:

...done.
Success. Distance: 0.58
Sending event: albuminfo_received
Candidate: DRS - Mid Mic Crisis (6817507)
Computing track assignment...
...done.
Success. Distance: 0.58
spotify: Searching Spotify for 'album:Skeptical Answers artist:Vulkanski'
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.

That is a good point. How about we keep it for now (at least) until we are happy with the metadata_plugin.py file and after that I slowly start to create PRs to prepare the specific metadataplugins for adaption. Once all plugins are migrated we should than be able to merge this PR.

[snejus]

slowly start to create PRs to prepare the specific metadata plugins for adaption

If you undo MetadataSourcePlugin.get_artist -> artists_to_artist_str replacement and _get_id removal, you should be left with very little that needs adapting.

[snejus]

This may be my last comment 😅 Check the build for typing error, I think there's one popping up:

Argument 1 to "append" of "list" has incompatible type "BeetsPlugin"; expected "MetadataSourcePlugin"

[semohr]

This may be my last comment 😅 Check the build for typing error, I think there's one popping up:

Argument 1 to "append" of "list" has incompatible type "BeetsPlugin"; expected "MetadataSourcePlugin"

Seemly like this was an issue introduced with the legacy support. Sadly the fix needs a type ignore but we should be able to remove it with v3.0.0.

[snejus]

Have just tested it locally, seems to work. Let's go!!!

[snejus]

Just rebase it and it will get merged.