Refactor of metadata plugin and opt in all metadata plugins to new baseclass

[semohr]

Description

At the moment the MetaDataSourcePlugin has multiple responsibilities:

• fetch data via _search api
• defines contract for interaction within the beets autotag lookup

I propose splitting these responsibilities, as it would enable us to use the MetaDataSourcePlugin baseclass with plugins that use external packages to fetch data.

This follows from discussion in #5761 and #5748 (comment).

Feedback is highly appreciated, as this is mainly architectural decision and I would prefer if the new behavior is a shared consensus.

To Do

• Opt in plugins into the new MetaDataSourcePlugin
  • Spotify
  • Musicbrainz
  • Deezer
  • Beatport
  • Chroma
  • Disccogs
• Remove old MetaDataSourcePlugin and related functions
• Documentation on the ontology of plugins
• Changelog

This PR was initially #5764 and was accidentally closed as the target branch was deleted. Wasn't able to recover the original PR.

[github-actions]

Thank you for the PR! The changelog has not been updated, so here is a friendly reminder to check if you need to add an entry.

[Copilot]

Pull Request Overview

This PR refactors the metadata lookup architecture by introducing a new beets.metadata_plugins module with a single base interface and migrating all existing source plugins to use it.

• Added beets/metadata_plugins.py defining MetadataSourcePluginNext and SearchApiMetadataSourcePluginNext
• Updated all core and third-party plugins (Spotify, MusicBrainz, Discogs, Deezer, Beatport, Chroma/Acoustid, mbsync, missing) to inherit from the new base classes
• Adjusted tests and documentation to patch and reference beets.metadata_plugins instead of the old beets.plugins metadata methods

Reviewed Changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
test/test_ui.py	Adjusted plugin-loading assertion to use new API
test/test_importer.py	Patches updated to beets.metadata_plugins lookups
test/plugins/test_mbsync.py	References to plugins.track_for_id/album_for_id replaced
docs/dev/plugins.rst	Documentation revised for new metadata plugin interface
docs/dev/index.rst	Added Confuse library link
beets/util/id_extractors.py	Fixed key lookup to use lowercase source names
beets/plugins.py	Removed old metadata plugin base and helper methods
beets/metadata_plugins.py	New metadata plugin interface and loader functions
beetsplug/spotify.py	Migrated Spotify plugin to SearchApiMetadataSourcePluginNext
beetsplug/musicbrainz.py	Updated MusicBrainz plugin to MetadataSourcePluginNext
beetsplug/missing.py	Switched plugins.* calls to metadata_plugins.*
beetsplug/mbsync.py	Updated track/album lookup to new API
beetsplug/discogs.py	Refactored Discogs plugin to new metadata interface
beetsplug/deezer.py	Migrated Deezer plugin to SearchApiMetadataSourcePluginNext
beetsplug/chroma.py	Adapted Acoustid plugin to new base and import patterns
beetsplug/beatport.py	Refactored Beatport plugin to new base class and models

Comments suppressed due to low confidence (2)

test/test_ui.py:1062

• [nitpick] The variable name plugs is ambiguous; consider renaming to found_plugins or loaded_plugins for clarity.

plugs = plugins.find_plugins()

beetsplug/chroma.py:29

• TrackInfo is not exported by beets.metadata_plugins at runtime; import it from beets.autotag.hooks instead.

from beets.metadata_plugins import MetadataSourcePluginNext, TrackInfo

[snejus]

Good stuff! I added a couple of comments in the interfaces, however I'm unable to see how this affects the plugins because they have also been refactored.

Could you move optional changes to a separate PR to minimize any side effects this change may cause? I tried testing it and I found spotify stuck in some authentication exception loop, for example:

...done.
Success. Distance: 0.58
Sending event: albuminfo_received
Candidate: DRS - Mid Mic Crisis (6817507)
Computing track assignment...
...done.
Success. Distance: 0.58
spotify: Searching Spotify for 'album:Skeptical Answers artist:Vulkanski'
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.

[semohr]

Good stuff! I added a couple of comments in the interfaces, however I'm unable to see how this affects the plugins because they have also been refactored.

Could you move optional changes to a separate PR to minimize any side effects this change may cause? I tried testing it and I found spotify stuck in some authentication exception loop, for example:

...done.
Success. Distance: 0.58
Sending event: albuminfo_received
Candidate: DRS - Mid Mic Crisis (6817507)
Computing track assignment...
...done.
Success. Distance: 0.58
spotify: Searching Spotify for 'album:Skeptical Answers artist:Vulkanski'
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.
spotify: Spotify access token has expired. Reauthenticating.

That is a good point. How about we keep it for now (at least) until we are happy with the metadata_plugin.py file and after that I slowly start to create PRs to prepare the specific metadataplugins for adaption. Once all plugins are migrated we should than be able to merge this PR.

[snejus]

slowly start to create PRs to prepare the specific metadata plugins for adaption

If you undo MetadataSourcePlugin.get_artist -> artists_to_artist_str replacement and _get_id removal, you should be left with very little that needs adapting.

[snejus]

The annotation needs to be on line 231.

@semohr do you run mypy locally?

[snejus]

Can we keep this PR focused on what the title suggests?

Refactor of metadata plugin and opt in all metadata plugins to new baseclass

Could you move commits with plugins and documentation improvements to separate branches, so that we can review them separately? I'm keen to merge the changes that have been reviewed and would like to have a final look without any noise.