Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Importer: interactive option to show other releases from the same release group #3171

Open
msdos opened this issue Feb 28, 2019 · 17 comments
Open
Labels
feature features we would like to implement

Comments

@msdos
Copy link

msdos commented Feb 28, 2019

I have an album called Opeth - Garden Of The Titans. This comes from a blu-ray with 2 CDs. They are on musicbrainz as well:

https://musicbrainz.org/release/bf3f9954-c048-4353-ae57-6160ab56d4b5

The original tags from the mp3s have a discnumber tag:

     album: Garden Of The Titans
       art: False
    artist: Opeth
  channels: 2
      disc: 1
    format: MP3
     genre: Metal
    genres: Metal
samplerate: 44100
     title: Sorceress
     track: 1

When I try to import this album to beets, this comes from Musicbrainz:

Correcting tags from:
    Opeth - Garden Of The Titans
To:
    Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre
URL:
    https://musicbrainz.org/release/c29d1500-04c0-4b59-9744-38eb41905e10
(Similarity: 86.7%) (album, tracks) (Digital Media, 2018, XW, Nuclear Blast)
 * Ghost Of Perdition       -> Ghost of Perdition
 * Demon Of The Fall        -> Demon of the Fall
 * In My Time Of Need       -> In My Time of Need
 * The Devil's Orchard (#1) -> The Devil’s Orchard (#6) (index)
 * Cusp Of Eternity (#2)    -> Cusp of Eternity (#7) (index)
 * Heir Apparent (#3)       -> Heir Apparent (#8) (index)
 * Era (#4)                 -> Era (#9) (index)
 * Deliverance (#5)         -> Deliverance (#10) (index)
[A]pply, More candidates, Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort? M
Finding tags for album "Opeth - Garden Of The Titans".
Candidates:
1. Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre (86.7%) (album, tracks) (Digital Media, 2018, XW, Nuclear Blast)
2. LFO Demon / FFF - Clash of the Titans (25.5%) (tracks, artist, unmatched tracks, ...) (12" Vinyl, 2003, DE, Sprengstoff Recordings)
3. LFO Demon / FFF - Clash of the Titans (25.5%) (tracks, artist, unmatched tracks, ...) (Vinyl, 2003, DE, Mindbender Records)
4. Javier Navarrete - Wrath of the Titans (24.9%) (tracks, missing tracks, artist, ...) (CD, 2012, US, WaterTower Music)
5. Medusa - Clash of the Titans (23.0%) (tracks, artist, missing tracks, ...) (CD, 2005)

It's getting the digital release instead of the Blu-ray + 2 CDs.

If you search for "Garden of the Titans" in musicbrainz interface, you get 3 releases:

Garden of the Titans: Opeth Live at Red Rocks Amphitheatre | Opeth | Digital Media | 10 | 2018-11-02 | XW | Nuclear Blast | none | 727361435660 | eng / Latn | Album + Live | Official

Garden of the Titans: Opeth Live at Red Rocks Amphitheatre | Opeth | Blu-ray + 2×CD | 10 + 5 + 5 | 2018-11-02 | XE | Moderbolaget Records, Nuclear Blast Entertainment | 27361 43560, NE 4356-0 | 727361435608 | eng / Latn | Album + Live | Official

Garden of the Titans: Opeth Live at Red Rocks Amphitheatre | Opeth | Blu-ray + DVD-Video + 2×CD | 10 + 10 + 5 + 5 | 2018-11-02 | XW | Moderbolaget Records |   |   | eng / Latn | Album + Live | Official

But when searching like I did in the beginning of this issue, beets only get one release:

1. Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre (86.7%) (album, tracks) (Digital Media, 2018, XW, Nuclear Blast)

Do we have a difference in the website search and musicbrainz API? Is this a musicbrainz or beets issue? How it decided that the best option was the digital download instead of the 2xCD since I have the discnumber tag in my mp3s?

Can we add support in Musicbrainz for (see all versions of this release, X available) similar to what you do to More candidates? There's this option in the (wrong) selected digital release at https://musicbrainz.org/release/c29d1500-04c0-4b59-9744-38eb41905e10, and if I could select other release available in the beets dialog would solve my problem.

@sampsyo sampsyo added the needinfo We need more details or follow-up from the filer before this can be tagged "bug" or "feature." label Feb 28, 2019
@sampsyo
Copy link
Member

sampsyo commented Feb 28, 2019

Can you please include your configuration and verbose output, as requested by the issue template?

Beets should be including all the releases (it doesn't group them by release group). It's possible that your configuration is filtering out the video versions.

@msdos
Copy link
Author

msdos commented Mar 1, 2019

Can you please include your configuration and verbose output, as requested by the issue template?

I'm really sorry about that. Since my beets config is just directory, library and log I didn't think it would be useful and somehow missed the beet -vv command. Thanks for being polite. 😆

Checking the verbose output, I don't think there's a problem in this package: it found 10 tracks and the first release with 10 tracks it got from musicbrainz it used for matching... Makes perfect sense actually. Am I supposed to read https://beets.readthedocs.io/en/v1.4.7/reference/config.html#autotagger-matching-options and play with the parameters?

beet -vv import Opeth\ -\ \[2018\]\ Garden\ Of\ The\ Titans
user configuration: /HOME/.config/beets/config.yaml
data directory: /HOME/.config/beets
plugin paths:
Sending event: pluginload
library database: /HOME/beets/beets.db
library directory: /HOME/beets
Sending event: library_opened
Sending event: import_begin
Sending event: import_task_created
Sending event: import_task_start
Looking up: /HOME/beets/Opeth - [2018] Garden Of The Titans
Tagging Opeth - Garden of the titans
No album ID found.
Search terms: Opeth - Garden of the titans
Album might be VA: False
Searching for MusicBrainz releases with: {'release': 'garden of the titans', 'artist': 'opeth', 'tracks': '10'}
Requesting MusicBrainz release c29d1500-04c0-4b59-9744-38eb41905e10
primary MB release type: album
secondary MB release type(s): live
Sending event: albuminfo_received
Candidate: Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre (c29d1500-04c0-4b59-9744-38eb41905e10)
Computing track assignment...
...done.
Success. Distance: 0.42
Requesting MusicBrainz release 86969c0b-07e2-42c2-9725-6c7825b61b2b
primary MB release type: ep
Sending event: albuminfo_received
Candidate: LFO Demon / FFF - Clash of the Titans (86969c0b-07e2-42c2-9725-6c7825b61b2b)
Computing track assignment...
...done.
Success. Distance: 0.76
Requesting MusicBrainz release 5aedca0a-60b8-4629-b232-498b11a3af99
primary MB release type: album
secondary MB release type(s): compilation
Sending event: albuminfo_received
Candidate: Medusa - Clash of the Titans (5aedca0a-60b8-4629-b232-498b11a3af99)
Computing track assignment...
...done.
Success. Distance: 0.79
Requesting MusicBrainz release 900c6534-66b2-3147-a446-b5bc0acae937
primary MB release type: ep
Sending event: albuminfo_received
Candidate: LFO Demon / FFF - Clash of the Titans (900c6534-66b2-3147-a446-b5bc0acae937)
Computing track assignment...
...done.
Success. Distance: 0.76
Requesting MusicBrainz release 2c877fdb-322c-4759-865e-17b72b17ab3a
primary MB release type: album
secondary MB release type(s): soundtrack
Sending event: albuminfo_received
Candidate: Javier Navarrete - Wrath of the Titans (2c877fdb-322c-4759-865e-17b72b17ab3a)
Computing track assignment...
...done.
Success. Distance: 0.79
Evaluating 5 candidates.

/HOME/beets/Opeth - [2018] Garden Of The Titans (10 items)
Sending event: before_choose_candidate
Correcting tags from:
    Opeth - Garden of the titans
To:
    Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre
URL:
    https://musicbrainz.org/release/c29d1500-04c0-4b59-9744-38eb41905e10
(Similarity: 58.3%) (tracks, album, mediums) (Digital Media, 2018, XW, Nuclear Blast)
 * sorceress (0:05)               -> Sorceress (7:09) (length)
 * ghost_of_perdition (0:05)      -> Ghost of Perdition (12:08) (length)
 * demon_of_the_fall (0:05)       -> Demon of the Fall (9:55) (length)
 * the_wilde_flowers (0:05)       -> The Wilde Flowers (8:42) (length)
 * in_my_time_of_need (0:05)      -> In My Time of Need (5:44) (length)
 * the_devils_orchard (#1) (0:05) -> The Devil’s Orchard (#6) (7:10) (length, index)
 * cusp_of_eternity (#2) (0:05)   -> Cusp of Eternity (#7) (5:14) (length, index)
 * heir_apparent (#3) (0:05)      -> Heir Apparent (#8) (10:21) (length, index)
 * era (#4) (0:05)                -> Era (#9) (7:30) (length, index)
 * deliverance (#5) (0:05)        -> Deliverance (#10) (14:13) (length, index)
Apply, More candidates, Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort? b
Sending event: import
Sending event: cli_exit
>>> elapsed time 13s

I've created the album structure above, using random noise 5s mp3 generated from ffmpeg, I can attach here if you want (it's only 400kb).

I still think, if possible, a "(see all versions of this release, X available)" option to be useful.

@sampsyo
Copy link
Member

sampsyo commented Mar 1, 2019

I see! Thanks for investigating a little bit. It does seem like the issue might be that beets is searching for releases with 10 tracks, which is ordinarily a good idea but here is excluding the other releases that have non-audio tracks that won't be used anyway. That's a little strange! I wish it were possible to search by the number of audio tracks, but I don't think that's a thing.

Maybe it would be worth validating this hypothesis by looking into how the search results change if the track length is not included? If you're feeling particularly intrepid, the place to start hacking around with that would be modifying the beets.autotag.mb module.

You mentioned an option to show other releases from the same release group, which is interesting—we could consider adding that as one of the interactive options. That would launch a new search for all the releases in the same group as the current release.

@msdos
Copy link
Author

msdos commented Mar 1, 2019

Maybe it would be worth validating this hypothesis by looking into how the search results change if the track length is not included?

The example I'm showing you is the album with random mp3 I created to test the hypothesis, but it gives the same result as the original. The only difference is that (length) is not shown as a difference, only (index) in the last tracks.

When you say track length are you talking about 'tracks': '10' in the query above? I changed match_album in beets.autotag.mb changing the tracks variable:

  424     tracks = None
  425     if tracks is not None:

And the result was actually worse.

Searching for MusicBrainz releases with: {'release': 'garden of the titans', 'artist': 'opeth'}
Requesting MusicBrainz release 86969c0b-07e2-42c2-9725-6c7825b61b2b
primary MB release type: ep
Sending event: albuminfo_received
Candidate: LFO Demon / FFF - Clash of the Titans (86969c0b-07e2-42c2-9725-6c7825b61b2b)
Computing track assignment...
...done.
Success. Distance: 0.76
Requesting MusicBrainz release 5aedca0a-60b8-4629-b232-498b11a3af99
primary MB release type: album
secondary MB release type(s): compilation
Sending event: albuminfo_received
Candidate: Medusa - Clash of the Titans (5aedca0a-60b8-4629-b232-498b11a3af99)
Computing track assignment...
...done.
Success. Distance: 0.79
Requesting MusicBrainz release 900c6534-66b2-3147-a446-b5bc0acae937
primary MB release type: ep
Sending event: albuminfo_received
Candidate: LFO Demon / FFF - Clash of the Titans (900c6534-66b2-3147-a446-b5bc0acae937)
Computing track assignment...
...done.
Success. Distance: 0.76
Requesting MusicBrainz release 2c877fdb-322c-4759-865e-17b72b17ab3a
primary MB release type: album
secondary MB release type(s): soundtrack
Sending event: albuminfo_received
Candidate: Javier Navarrete - Wrath of the Titans (2c877fdb-322c-4759-865e-17b72b17ab3a)
Computing track assignment...
...done.
Success. Distance: 0.79
Requesting MusicBrainz release b79be3a1-7339-4087-903f-59351ebbf456
primary MB release type: album
Sending event: albuminfo_received
Candidate: Dave S-B - Revenge of the Titans (b79be3a1-7339-4087-903f-59351ebbf456)
Computing track assignment...
...done.
Success. Distance: 0.76
Evaluating 5 candidates.

HOME/beets/Opeth - [2018] Garden Of The Titans (10 items)
Sending event: before_choose_candidate
Finding tags for album "Opeth - Garden of the titans".
Candidates:
1. Dave S-B - Revenge of the Titans (24.1%) (tracks, artist, unmatched tracks, ...) (Digital Media, 2010, XW, Puppy Games)
2. LFO Demon / FFF - Clash of the Titans (24.1%) (tracks, artist, unmatched tracks, ...) (12" Vinyl, 2003, DE, Sprengstoff Recordings)
3. LFO Demon / FFF - Clash of the Titans (24.1%) (tracks, artist, unmatched tracks, ...) (Vinyl, 2003, DE, Mindbender Records)
4. Medusa - Clash of the Titans (21.1%) (tracks, artist, missing tracks, ...) (CD, 2005)
5. Javier Navarrete - Wrath of the Titans (20.8%) (tracks, missing tracks, artist, ...) (CD, 2012, US, WaterTower Music)
# selection (default 1), Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort?

So I started to think: is the search done in beets really the same as in the browser like in https://musicbrainz.org/search?query=garden+of+the+titans&type=release&method=indexed? Comparing this url and the beets query, Searching for MusicBrainz releases with: {'release': 'garden of the titans', 'artist': 'opeth'} I've noticed that in the browser a new parameter is added, type=release. Hacking beets.autotag.mb again, adding a new criteria:

  424     tracks = None
  425     criteria['type'] = 'release'
  426     if tracks is not None:

And keeping tracks out, now I get the something more similar to the browser search!

Searching for MusicBrainz releases with: {'release': 'garden of the titans', 'artist': 'opeth', 'type': 'release'}
Requesting MusicBrainz release c29d1500-04c0-4b59-9744-38eb41905e10
primary MB release type: album
secondary MB release type(s): live
Sending event: albuminfo_received
Candidate: Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre (c29d1500-04c0-4b59-9744-38eb41905e10)
Computing track assignment...
...done.
Success. Distance: 0.42
Requesting MusicBrainz release bf3f9954-c048-4353-ae57-6160ab56d4b5
primary MB release type: album
secondary MB release type(s): live
Sending event: albuminfo_received
Candidate: Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre (bf3f9954-c048-4353-ae57-6160ab56d4b5)
Computing track assignment...
...done.
Success. Distance: 0.36
Requesting MusicBrainz release 645a634c-02fd-4e53-88af-f0dba2f28068
primary MB release type: album
secondary MB release type(s): live
Sending event: albuminfo_received
Candidate: Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre (645a634c-02fd-4e53-88af-f0dba2f28068)
Computing track assignment...
...done.
Success. Distance: 0.38
Requesting MusicBrainz release 86969c0b-07e2-42c2-9725-6c7825b61b2b
primary MB release type: ep
Sending event: albuminfo_received
Candidate: LFO Demon / FFF - Clash of the Titans (86969c0b-07e2-42c2-9725-6c7825b61b2b)
Computing track assignment...
...done.
Success. Distance: 0.76
Requesting MusicBrainz release 5aedca0a-60b8-4629-b232-498b11a3af99
primary MB release type: album
secondary MB release type(s): compilation
Sending event: albuminfo_received
Candidate: Medusa - Clash of the Titans (5aedca0a-60b8-4629-b232-498b11a3af99)
Computing track assignment...
...done.
Success. Distance: 0.79
Evaluating 5 candidates.

/HOME/beets/Opeth - [2018] Garden Of The Titans (10 items)
Sending event: before_choose_candidate
Finding tags for album "Opeth - Garden of the titans".
Candidates:
1. Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre (64.4%) (tracks, album, mediums) (3xBlu-ray, 2018, XE, Moderbolaget Records)
2. Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre (62.1%) (tracks, mediums, album) (4xBlu-ray, 2018, XW, Moderbolaget Records)
3. Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre (58.3%) (tracks, album, mediums) (Digital Media, 2018, XW, Nuclear Blast)
4. LFO Demon / FFF - Clash of the Titans (24.1%) (tracks, artist, unmatched tracks, ...) (12" Vinyl, 2003, DE, Sprengstoff Recordings)
5. Medusa - Clash of the Titans (21.1%) (tracks, artist, missing tracks, ...) (CD, 2005)
# selection (default 1), Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort?

But if I add tracks again AND keep the new added type: 'release' attribute, I get something similar to my first search that motivated this issue:

Searching for MusicBrainz releases with: {'release': 'garden of the titans', 'artist': 'opeth', 'type': 'release', 'tracks': '10'}
Requesting MusicBrainz release c29d1500-04c0-4b59-9744-38eb41905e10
primary MB release type: album
secondary MB release type(s): live
Sending event: albuminfo_received
Candidate: Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre (c29d1500-04c0-4b59-9744-38eb41905e10)
Computing track assignment...
...done.
Success. Distance: 0.42
Requesting MusicBrainz release 86969c0b-07e2-42c2-9725-6c7825b61b2b
primary MB release type: ep
Sending event: albuminfo_received
Candidate: LFO Demon / FFF - Clash of the Titans (86969c0b-07e2-42c2-9725-6c7825b61b2b)
Computing track assignment...
...done.
Success. Distance: 0.76
Requesting MusicBrainz release 5aedca0a-60b8-4629-b232-498b11a3af99
primary MB release type: album
secondary MB release type(s): compilation
Sending event: albuminfo_received
Candidate: Medusa - Clash of the Titans (5aedca0a-60b8-4629-b232-498b11a3af99)
Computing track assignment...
...done.
Success. Distance: 0.79
Requesting MusicBrainz release 900c6534-66b2-3147-a446-b5bc0acae937
primary MB release type: ep
Sending event: albuminfo_received
Candidate: LFO Demon / FFF - Clash of the Titans (900c6534-66b2-3147-a446-b5bc0acae937)
Computing track assignment...
...done.
Success. Distance: 0.76
Requesting MusicBrainz release 2c877fdb-322c-4759-865e-17b72b17ab3a
primary MB release type: album
secondary MB release type(s): soundtrack
Sending event: albuminfo_received
Candidate: Javier Navarrete - Wrath of the Titans (2c877fdb-322c-4759-865e-17b72b17ab3a)
Computing track assignment...
...done.
Success. Distance: 0.79
Evaluating 5 candidates.

/HOME/beets/Opeth - [2018] Garden Of The Titans (10 items)
Sending event: before_choose_candidate
Correcting tags from:
    Opeth - Garden of the titans
To:
    Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre
URL:
    https://musicbrainz.org/release/c29d1500-04c0-4b59-9744-38eb41905e10
(Similarity: 58.3%) (tracks, album, mediums) (Digital Media, 2018, XW, Nuclear Blast)
 * sorceress (#1-1) (0:05)          -> Sorceress (#1) (7:09) (length)
 * ghost_of_perdition (#1-2) (0:05) -> Ghost of Perdition (#2) (12:08) (length)
 * demon_of_the_fall (#1-3) (0:05)  -> Demon of the Fall (#3) (9:55) (length)
 * the_wilde_flowers (#1-4) (0:05)  -> The Wilde Flowers (#4) (8:42) (length)
 * in_my_time_of_need (#1-5) (0:05) -> In My Time of Need (#5) (5:44) (length)
 * the_devils_orchard (#2-1) (0:05) -> The Devil’s Orchard (#6) (7:10) (length, index)
 * cusp_of_eternity (#2-2) (0:05)   -> Cusp of Eternity (#7) (5:14) (length, index)
 * heir_apparent (#2-3) (0:05)      -> Heir Apparent (#8) (10:21) (length, index)
 * era (#2-4) (0:05)                -> Era (#9) (7:30) (length, index)
 * deliverance (#2-5) (0:05)        -> Deliverance (#10) (14:13) (length, index)
Apply, More candidates, Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort?

I can't tell you exactly why this happens. I think it's a job for those that are more familiar with these libraries. That's why I'm adding this issue.

I've attached the files of this issue for testing:

Opeth - [2018] Garden Of The Titans.zip

These aren't the original copyrighted files. These are random noise mp3 I created using ffmpeg with the command: ffmpeg -f lavfi -i "sine=frequency=1000:duration=5" test.mp3 and the whole zip attachment is a < 400kb zip file. I just created the mp3 in the same album structure.

You mentioned an option to show other releases from the same release group, which is interesting—we could consider adding that as one of the interactive options. That would launch a new search for all the releases in the same group as the current release.

Exactly. I just don't know how difficult it would be or where it should be put.

@msdos
Copy link
Author

msdos commented Mar 1, 2019

Is the label needinfo still needed after all these logs? 😅

@msdos
Copy link
Author

msdos commented Mar 1, 2019

Ok. Now it's getting funny.

I've just tested the attached https://github.com/beetbox/beets/files/2920892/Opeth.-.2018.Garden.Of.The.Titans.zip file: beet import Opeth.-.2018.Garden.Of.The.Titans.zip. If you do it several times, you get different results... randomly. I don't have an explanation for this. Maybe a thread or concurrency issue?

beet import Opeth.-.2018.Garden.Of.The.Titans.zip                                                                                                                   [19:35:46]

/tmp/T/tmpz40ou501/Opeth - [2018] Garden Of The Titans (10 items)
Correcting tags from:
    Opeth - Garden of the titans
To:
    Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre
URL:
    https://musicbrainz.org/release/c29d1500-04c0-4b59-9744-38eb41905e10
(Similarity: 58.3%) (tracks, album, mediums) (Digital Media, 2018, XW, Nuclear Blast)
 * sorceress (#1-1) (0:05)          -> Sorceress (#1) (7:09) (length)
 * ghost_of_perdition (#1-2) (0:05) -> Ghost of Perdition (#2) (12:08) (length)
 * demon_of_the_fall (#1-3) (0:05)  -> Demon of the Fall (#3) (9:55) (length)
 * the_wilde_flowers (#1-4) (0:05)  -> The Wilde Flowers (#4) (8:42) (length)
 * in_my_time_of_need (#1-5) (0:05) -> In My Time of Need (#5) (5:44) (length)
 * the_devils_orchard (#2-1) (0:05) -> The Devil’s Orchard (#6) (7:10) (length, index)
 * cusp_of_eternity (#2-2) (0:05)   -> Cusp of Eternity (#7) (5:14) (length, index)
 * heir_apparent (#2-3) (0:05)      -> Heir Apparent (#8) (10:21) (length, index)
 * era (#2-4) (0:05)                -> Era (#9) (7:30) (length, index)
 * deliverance (#2-5) (0:05)        -> Deliverance (#10) (14:13) (length, index)
Apply, More candidates, Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort? B
>>> elapsed time 26s

Abort and run again beet import Opeth.-.2018.Garden.Of.The.Titans.zip...

beet import Opeth.-.2018.Garden.Of.The.Titans.zip                                                                                                                   [19:36:13]

/tmp/T/tmpccaeq6op/Opeth - [2018] Garden Of The Titans (10 items)
Finding tags for album "Opeth - Garden of the titans".
Candidates:
1. Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre (64.4%) (tracks, album, mediums) (3xBlu-ray, 2018, XE, Moderbolaget Records)
2. Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre (62.1%) (tracks, mediums, album) (4xBlu-ray, 2018, XW, Moderbolaget Records)
3. Opeth - Garden of the Titans: Opeth Live at Red Rocks Amphitheatre (58.3%) (tracks, album, mediums) (Digital Media, 2018, XW, Nuclear Blast)
4. LFO Demon / FFF - Clash of the Titans (24.1%) (tracks, artist, unmatched tracks, ...) (12" Vinyl, 2003, DE, Sprengstoff Recordings)
5. Medusa - Clash of the Titans (21.1%) (tracks, artist, missing tracks, ...) (CD, 2005)
# selection (default 1), Skip, Use as-is, as Tracks, Group albums,
Enter search, enter Id, aBort? B

I can't write a script trying to simulate this, since --pretend only shows the files on disk (you need to manually test like I did). Maybe a --pretend-matches option that just outputs musicbrainz matches would be a nice addition as well to this package, to help in debug situations like this?

@sampsyo
Copy link
Member

sampsyo commented Mar 2, 2019

That's very tricky! Thank you for all the extra digging! I'm surprised that including "type": "release" made a difference—I thought we were already only searching for releases by virtue of the API endpoint…

And I'm also mystified by the nondeterminism!!! I will absolutely need to look into that, but is there any chance the nondeterminism is on the MusicBrainz side? That is, that the search returns slightly different results each time?

I tend to leave the needinfo label until we decide exactly what to do (i.e., we need information from our own minds and from the universe 😃). In this case, we either need to decide there's a bug in the search functionality that we need to fix, or that we want to call this a feature request for the "search for other releases in this release group" interactive option, or both. Maybe both.

@msdos
Copy link
Author

msdos commented Mar 2, 2019

I suggest you to try the attached zip and see if you can simulate what I did. I really don't know if this is a musicbrainz issue or a beets one. Indeed, we have multiple stuff on this issue.

I tend to leave the needinfo label until we decide exactly what to do

Fair enough.

@sampsyo
Copy link
Member

sampsyo commented Mar 2, 2019

I can confirm that the MusicBrainz search server is acting nondeterministically. If you run this command:

$ curl 'http://musicbrainz.org/ws/2/release/?limit=5&query=release%3A%28garden+of+the+titans%29+artist%3A%28opeth%29+tracks%3A%2810%29'

And then wait a few moments and do it again, you get different results. In my experience, if you do it twice really fast, then you get the same results, but if you wait for 10 or 20 seconds, the next result is indeed different. That's certainly surprising!

I don't precisely know what to do next, but maybe we should see if this is a known issue for the MusicBrainz developers?

@msdos
Copy link
Author

msdos commented Mar 7, 2019

@sampsyo I suggest you creating it instead of me since you're more familiar with the code.

https://musicbrainz.org/doc/Bug_Tracker

@sampsyo
Copy link
Member

sampsyo commented Mar 7, 2019

I wish that were true. :) I don't have that much familiarity with the MusicBrainz server code.

Maybe the first thing to do—and it would be great to have your help with this too—would be to ask around on the MusicBrainz IRC channel to see if there are any search server developers there. They might be able to give you a quick "gut check" on whether nondeterminism seems like a normal/expected behavior.

@msdos
Copy link
Author

msdos commented Mar 7, 2019

After reading https://musicbrainz.org/doc/Communication/IRC just went to https://webchat.freenode.net/?channels=musicbrainz,metabrainz:

[17:58] <yvanzo> msdos: I cannot reproduce the example given in that comment, do you have output diff at hand?
[18:06] <msdos> yvanzo I will try to provide them.
[18:09] <@Freso> msdos: With a couple of minutes between queries, the only thing different between them seem to be the `<metadata created="">` timestamp.
[18:10] <@Freso> msdos: Did you try this multiple times? It could be that the data actually changed in MB between your first attempts (which gets reflected near-immediately in the search index now)?
[18:10] <yvanzo> I am still trying too
[18:14] <msdos> Freso: We tried multiple times, yes. I don't know if the data actually changed between the first attempts and now. Actually, I was using the musicbrainz call in beets when I found the issue, and someone from beets found this non deterministic problem just by doing curl calls.
[18:14] <msdos> I will try to create a bash script that exports a bunch of output

Here is the bash:

#!/usr/bin/env bash
url='http://musicbrainz.org/ws/2/release/?limit=5&query=release%3A%28garden+of+the+titans%29+artist%3A%28opeth%29+tracks%3A%2810%29'
for (( i = 0; i < 10; i++ )); do
    curl "$url"  > "$(date '+%Y-%m-%d-%H-%M-%S')-$RANDOM.xml"
done
sleep 10
curl "$url" > "$(date '+%Y-%m-%d-%H-%M-%S')-$RANDOM.xml"
sleep 20
curl "$url" > "$(date '+%Y-%m-%d-%H-%M-%S')-$RANDOM.xml"
sleep 30
curl "$url" > "$(date '+%Y-%m-%d-%H-%M-%S')-$RANDOM.xml"

Can you try to improve it to be able to simulate what's in #3171 (comment) so we can go back do IRC and ask the developers? Because I couldn't create different results...

@Freso
Copy link
Member

Freso commented Mar 7, 2019

wait a few moments and do it again, you get different results

Can you qualify what the different results are/were? As given in the IRC log snippet @msdos gave above, the only difference I can spot (with my eye) is different timestamps in the <metadata created="">.

@msdos
Copy link
Author

msdos commented Mar 8, 2019

@sampsyo can you help us here?

@sampsyo
Copy link
Member

sampsyo commented Mar 11, 2019

This is super weird, but I can no longer reproduce the nondeterminism! Neither in a beets search nor with isolated curl requests to the API. I swear I saw it happen before, but now I'm questioning my own sanity.

@msdos, are you still able to observe nondeterminism when you import that album?

@msdos
Copy link
Author

msdos commented Mar 22, 2019

I swear I saw it happen before, but now I'm questioning my own sanity.

Funny. I can't reproduce it anymore. But the suggestion of multiple releases still apply.

@sampsyo sampsyo changed the title Different results in musicbrainz search and musicbrainz in beets: can we have a "(see all versions of this release, X available)" feature? Importer: interactive option to show other releases from the same release group Mar 22, 2019
@sampsyo sampsyo added feature features we would like to implement and removed needinfo We need more details or follow-up from the filer before this can be tagged "bug" or "feature." labels Mar 22, 2019
@sampsyo
Copy link
Member

sampsyo commented Mar 22, 2019

Sounds good! Retitled this as a feature request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature features we would like to implement
Projects
None yet
Development

No branches or pull requests

3 participants