Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unicode troubles when using duplicates plugin #1551

Closed
willgrnr opened this issue Jul 28, 2015 · 5 comments
Closed

Unicode troubles when using duplicates plugin #1551

willgrnr opened this issue Jul 28, 2015 · 5 comments
Labels
needinfo We need more details or follow-up from the filer before this can be tagged "bug" or "feature."

Comments

@willgrnr
Copy link

When running beet duplicates beets outputs a list of songs that aren't duplicates, with the following error in the middle.

c:\users\100557855\beets\beetsplug\duplicates.py:253: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  if getattr(x, a, None) not in (None, '')])

Full output:

C:\Users\100557855\beets>beet duplicates
Watsky - Cardboard Castles - Moral Of The Story
Watsky - Cardboard Castles - Hey, Asshole
Macklemore & Ryan Lewis - Vs. [Redux] (The Vs. Re-Release) - Irish Celebration
Macklemore & Ryan Lewis - Vs. [Redux] (The Vs. Re-Release) - Otherside Feat. Fen
ces (Ryan Lewis Remix)
Nirvana - Nevermind - In Bloom
Nirvana - Nevermind - Lithium
c:\users\100557855\beets\beetsplug\duplicates.py:253: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  if getattr(x, a, None) not in (None, '')])
Die Atzen - Party Chaos - Party Chaos
Die Atzen - Party Chaos - Hasta La Atze
Die Atzen - Party Chaos - Supa Dupa
Die Atzen - Party Chaos - Machwasduwillst
Die Atzen - Party Chaos - Ex Ex Ex
Die Atzen - Party Chaos - Schwarze Katze
Die Atzen - Party Chaos - Atz Atz Atz
Die Atzen - Party Chaos - Lets Fetz
Die Atzen - Party Chaos - 1000 Geile Sachen
Die Atzen - Party Chaos - Attacke
Die Atzen - Party Chaos - Bombe
Die Atzen - Party Chaos - Scheiss Drauf
Die Atzen - Party Chaos - Disco Pogo II
Die Atzen - Party Chaos - Alles Erlaubt
Die Atzen - Party Chaos - Jump & Run
Die Atzen - Party Chaos - Schick Deine Butter
Die Atzen - Party Chaos - Das Geht Ab
Die Atzen - Party Chaos - Disco Pogo
Die Atzen - Party Chaos - Teufelswerk
Die Atzen - Party Chaos - Atzin
Die Atzen - Party Chaos - Florida Lady
Die Atzen - Party Chaos - Wer Nicht Springt
Die Atzen - Party Chaos - Anna Nass
Die Atzen - Party Chaos - Immernoch Die Alten
Die Atzen - Party Chaos - Auch Atzen Müssen Weinen
Die Atzen - Party Chaos - Polizei
Die Atzen - Party Chaos - Ein Ganz Normaler Atze
Die Atzen - Party Chaos - Kopf Hoch
Die Atzen - Party Chaos - Verrückt
Die Atzen - Party Chaos - Ihr Seid Atzen
Die Atzen - Party Chaos - Geschichte
Our Lady Peace - Spiritual Machines - Life
Nickelback - Greatest Hits - Hero
Nickelback - Greatest Hits - Photograph
Nickelback - Greatest Hits - Burn It To The Ground
Nickelback - Greatest Hits - If Everyone Cared
Nickelback - Greatest Hits - Because Of You
Nickelback - Greatest Hits - Animals
Nickelback - Greatest Hits - Gotta Be Somebody
Nickelback - Greatest Hits - Rockstar
Nickelback - Greatest Hits - Lullaby
Nickelback - Greatest Hits - If Today Was Your Last Day
Nickelback - Greatest Hits - Far Away
Croatia Squad - Back To Life - All The Girlz
Johnny Cash - The Legendary Johnny Cash - Thing Called Love
Johnny Cash - The Legendary Johnny Cash - I Walk The Line
George Strait - Icon 2 - All My Ex's Live In Texas
George Strait - Icon 2 - Check Yes Or No
George Strait - Icon 2 - Carried Away
George Strait - Icon 2 - Give It Away
@sampsyo sampsyo added the needinfo We need more details or follow-up from the filer before this can be tagged "bug" or "feature." label Jul 28, 2015
@sampsyo
Copy link
Member

sampsyo commented Jul 28, 2015

Hmm; interesting. What does your duplicates config look like (if any)? I'm surprised there would be any Unicode-to-bytes comparison for any normal fields, but path (the only bytes field I can think of) might cause it.

Is there anything interesting in the verbose output?

Is it possible to narrow this down to any specific files in your library?

@willgrnr
Copy link
Author

I don't currently have any config setup for duplicates. I ran it again using -v and I got the following output.

C:\Users\100557855>beet -v duplicates
user configuration: C:\Users\100557855\AppData\Roaming\beets\config.yaml
data directory: C:\Users\100557855\AppData\Roaming\beets
plugin paths:
Sending event: pluginload
library database: C:\Users\100557855\Desktop\Music\musiclibrary.blb
library directory: C:\Users\100557855\Desktop\Music
Sending event: library_opened
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Bang La Decks\Utopia (Radio Edit)\01 Utopia (Radio Edit).mp3 are nu
ll or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Non-Album\Dada Life\Rolling Stones T-Shirt (Steerner & Tjernberg's
Arena Remix).mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Dimitri Vegas, Like Mike, VINAI\Louder\01 Louder (Extended Mix).mp3
 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Hardwell & W&W\The Dance Floor Is Yours\01 The Dance Floor Is Yours
 (Original Mix).mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Hopsin\Hop Madness - Single\01 Hop Madness.mp3 are null or empty: s
kipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Hopsin\Ill Mind of Hopsin 4\01 Ill Mind of Hopsin 4.mp3 are null or
 empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Hopsin\Ill Mind of Hopsin 7\01 Ill Mind of Hopsin 7.mp3 are null or
 empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Hopsin\Ill Mind Six_ Old Friend - Single\01 Ill Mind Six_ Old Frien
d.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Jacob van Hage & Oliver Heldens\Thumper\01 Thumper (Original Mix).m
p3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Jacoo\00 Panama.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Jacoo\Hurt\01 Peace Of Mind.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Jacoo\Hurt\03 Hurt.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Jacoo\Hurt\04 Life Without You.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Jacoo\Hurt\05 Reincarnation.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Jacoo\Hurt\02 Hurt Feat. Oneira.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Non-Album\Kardinall Offishall\The Anthem.mp3 are null or empty: ski
pping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\Hump Days\01 Lion King.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\Hump Days\02 Scary Story.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\Hump Days\03 New Rap.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\Hump Days\04 Workaholics.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\Hump Days\05 Beef.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\Hump Days\06 Hit Me Up.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\Hump Days\07 The '90s.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\Hump Days\08 The Basement.mp3 are null or empty: skipping

duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\Hump Days\09 Back In Business.mp3 are null or empty: skip
ping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\Hump Days\10 A$AP DICKY.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\Hump Days\11 DEEZUS.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\Hump Days\12 Flames.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\Hump Days\13 Darwin.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\Lemme Freak\01 Lemme Freak.mp3 are null or empty: skippin
g
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Lil Dicky\White Crime\01 White Crime.mp3 are null or empty: skippin
g
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Mudvayne\Playlist_ The Very Best Of Mudvayne\01 Dull Boy.mp3 are nu
ll or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Mudvayne\Playlist_ The Very Best Of Mudvayne\02 Dig.mp3 are null or
 empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Mudvayne\Playlist_ The Very Best Of Mudvayne\03 Not Falling.mp3 are
 null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Mudvayne\Playlist_ The Very Best Of Mudvayne\04 Fish Out Of Water.m
p3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Mudvayne\Playlist_ The Very Best Of Mudvayne\05 Do What You Do.mp3
are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Mudvayne\Playlist_ The Very Best Of Mudvayne\06 Beautiful & Strange
.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Mudvayne\Playlist_ The Very Best Of Mudvayne\07 Determined.mp3 are
null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Mudvayne\Playlist_ The Very Best Of Mudvayne\08 Scream With Me.mp3
are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Mudvayne\Playlist_ The Very Best Of Mudvayne\09 World So Cold.mp3 a
re null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Mudvayne\Playlist_ The Very Best Of Mudvayne\10 Death Blooms.mp3 ar
e null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Mudvayne\Playlist_ The Very Best Of Mudvayne\11 Fall Into Sleep.mp3
 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Mudvayne\Playlist_ The Very Best Of Mudvayne\12 -1.mp3 are null or
empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Mudvayne\Playlist_ The Very Best Of Mudvayne\13 1000 Mile Journey.m
p3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Mudvayne\Playlist_ The Very Best Of Mudvayne\14 Happy_.mp3 are null
 or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Non-Album\Murdbrain\Trampoline (Radio Edit).mp3 are null or empty:
skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Nicky Romero\Tension (Vocal Mix)\01 Tension (Vocal Mix).mp3 are nul
l or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Non-Album\Old Dominion\Dirt on a Road.m4a are null or empty: skippi
ng
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Paris Blohm\EDMTunes.Com\00 Believe (Original Mix).mp3 are null or
empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Paris Blohm feat. Charles\Demons\01 Demons (Original Mix).mp3 are n
ull or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Paul Oakenfold\25 Years Of Perfecto Records (Mixed by Paul Oakenfol
d)\01 You Could Be Happy.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Pheeno\Open Up - Single\01 Open Up.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Quintino\Go Hard\00 Go Hard.mp3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Non-Album\Tony Junior & Marnik\Jump Around (Original Mix).mp3 are n
ull or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Vicetone feat. Barack Obama\Vicetone\01 Hope (Original Mix).mp3 are
 null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Vicetone feat. Kat Nestel\No Way Out\01 No Way Out (Extended Mix).m
p3 are null or empty: skipping
duplicates: all keys [u'mb_trackid', u'mb_albumid'] on item C:\Users\100557855\D
esktop\Music\Youngblood Hawke\We Come Running (Remixes)\05 We Come Running (Vice
tone Remix).mp3 are null or empty: skipping
Watsky - Cardboard Castles - Moral Of The Story
Watsky - Cardboard Castles - Hey, Asshole
Macklemore & Ryan Lewis - Vs. [Redux] (The Vs. Re-Release) - Irish Celebration
Macklemore & Ryan Lewis - Vs. [Redux] (The Vs. Re-Release) - Otherside Feat. Fen
ces (Ryan Lewis Remix)
Nirvana - Nevermind - In Bloom
Nirvana - Nevermind - Lithium
c:\users\100557855\beets\beetsplug\duplicates.py:253: UnicodeWarning: Unicode eq
ual comparison failed to convert both arguments to Unicode - interpreting them a
s being unequal
  if getattr(x, a, None) not in (None, '')])
Die Atzen - Party Chaos - Party Chaos
Die Atzen - Party Chaos - Hasta La Atze
Die Atzen - Party Chaos - Supa Dupa
Die Atzen - Party Chaos - Machwasduwillst
Die Atzen - Party Chaos - Ex Ex Ex
Die Atzen - Party Chaos - Schwarze Katze
Die Atzen - Party Chaos - Atz Atz Atz
Die Atzen - Party Chaos - Lets Fetz
Die Atzen - Party Chaos - 1000 Geile Sachen
Die Atzen - Party Chaos - Attacke
Die Atzen - Party Chaos - Bombe
Die Atzen - Party Chaos - Scheiss Drauf
Die Atzen - Party Chaos - Disco Pogo II
Die Atzen - Party Chaos - Alles Erlaubt
Die Atzen - Party Chaos - Jump & Run
Die Atzen - Party Chaos - Schick Deine Butter
Die Atzen - Party Chaos - Das Geht Ab
Die Atzen - Party Chaos - Disco Pogo
Die Atzen - Party Chaos - Teufelswerk
Die Atzen - Party Chaos - Atzin
Die Atzen - Party Chaos - Florida Lady
Die Atzen - Party Chaos - Wer Nicht Springt
Die Atzen - Party Chaos - Anna Nass
Die Atzen - Party Chaos - Immernoch Die Alten
Die Atzen - Party Chaos - Auch Atzen Müssen Weinen
Die Atzen - Party Chaos - Polizei
Die Atzen - Party Chaos - Ein Ganz Normaler Atze
Die Atzen - Party Chaos - Kopf Hoch
Die Atzen - Party Chaos - Verrückt
Die Atzen - Party Chaos - Ihr Seid Atzen
Die Atzen - Party Chaos - Geschichte
Our Lady Peace - Spiritual Machines - Life
Nickelback - Greatest Hits - Hero
Nickelback - Greatest Hits - Photograph
Nickelback - Greatest Hits - Burn It To The Ground
Nickelback - Greatest Hits - If Everyone Cared
Nickelback - Greatest Hits - Because Of You
Nickelback - Greatest Hits - Animals
Nickelback - Greatest Hits - Gotta Be Somebody
Nickelback - Greatest Hits - Rockstar
Nickelback - Greatest Hits - Lullaby
Nickelback - Greatest Hits - If Today Was Your Last Day
Nickelback - Greatest Hits - Far Away
Croatia Squad - Back To Life - All The Girlz
Johnny Cash - The Legendary Johnny Cash - Thing Called Love
Johnny Cash - The Legendary Johnny Cash - I Walk The Line
George Strait - Icon 2 - All My Ex's Live In Texas
George Strait - Icon 2 - Check Yes Or No
George Strait - Icon 2 - Carried Away
George Strait - Icon 2 - Give It Away
Sending event: cli_exit

Because the songs listed there are completely different from before, I ran it again without -v and it was the same as I originally posted.

From what I can tell, it has nothing to do with my files.

@sampsyo
Copy link
Member

sampsyo commented Jul 29, 2015

Very strange. Any chance you could share your library database file so I can reproduce this locally?

@willgrnr
Copy link
Author

musiclibrary.blb -https://www.dropbox.com/s/25n2bjy9lk9g9ef/musiclibrary.blb?dl=0

@sampsyo
Copy link
Member

sampsyo commented Jul 29, 2015

Thanks! I found the issue; the warning should be fixed in the above commit.

sampsyo added a commit that referenced this issue Jul 29, 2015
@sampsyo
Silly oversight in fix for #1551
f6ecbf6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
needinfo We need more details or follow-up from the filer before this can be tagged "bug" or "feature."
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sampsyo @willgrnr