Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unique constraint errors (genre) on scan #1564

Closed
fabpico opened this issue Jan 6, 2022 · 3 comments · Fixed by #1920
Closed

Unique constraint errors (genre) on scan #1564

fabpico opened this issue Jan 6, 2022 · 3 comments · Fixed by #1920
Labels

Comments

@fabpico
Copy link

fabpico commented Jan 6, 2022

Description

I am getting unique constraint errors in the scan log when scanning. I scan 2’855 Files in 239 Folders. Most of the scan log entries are infos.

Expected Behaviour

No contraint errors should happen / be logged.

Steps to reproduce

  1. Add files to music folder
  2. Open the terminal
  3. Run scan

Platform information

  • Navidrome version: 0.47.0 (8c7d95c1)
  • Operating System: Ubuntu 20.04.3 LTS
  • Docker image: deluan/navidrome:0.47.0

Additional information

  • Logs
    ERRO[0025] Error updating folder in the DB dir="/music/[path]" error="UNIQUE constraint failed: media_file.id"
    ERRO[0026] Error updating folder in the DB dir="/music/[path]" error="UNIQUE constraint failed: media_file_genres.media_file_id, media_file_genres.genre_id"
    ERRO[0096] Error writing albums to the DB error="UNIQUE constraint failed: album_genres.album_id, album_genres.genre_id"
    ERRO[0099] Error updating folder in the DB dir="/music/[path]" error="UNIQUE constraint failed: media_file_genres.media_file_id, media_file_genres.genre_id"
ERRO[0008] SQL: `INSERT INTO genre (id,name) VALUES (?,?)`  args="['3d8dd8ae-cb27-47e0-a55e-62dc1d500826','Minimal']" elapsedTime="767.806µs" error="UNIQUE constraint failed: genre.name" rowsAffected=0
ERRO[0008] SQL: `INSERT INTO genre (id,name) VALUES (?,?)`  args="['eb61e4dc-111d-4a93-9112-3b630a31e018','Minimal']" elapsedTime="364.883µs" error="UNIQUE constraint failed: genre.name" rowsAffected=0
ERRO[0008] SQL: `INSERT INTO genre (id,name) VALUES (?,?)`  args="['9b3bb53e-2b2b-4d75-af80-1792eb62605e','Minimal']" elapsedTime="405.682µs" error="UNIQUE constraint failed: genre.name" rowsAffected=0
ERRO[0008] SQL: `INSERT INTO genre (id,name) VALUES (?,?)`  args="['7846779b-f947-4428-ab45-b3af273fdb55','Minimal']" elapsedTime="376.545µs" error="UNIQUE constraint failed: genre.name" rowsAffected=0
ERRO[0008] SQL: `INSERT INTO genre (id,name) VALUES (?,?)`  args="['66a502e7-4b25-4551-ada9-8445722bf9df','Minimal']" elapsedTime="384.746µs" error="UNIQUE constraint failed: genre.name" rowsAffected=0
ERRO[0008] SQL: `INSERT INTO genre (id,name) VALUES (?,?)`  args="['653298d0-f844-4fd0-9023-2599ff0acc80','Minimal']" elapsedTime="925.026µs" error="UNIQUE constraint failed: genre.name" rowsAffected=0
ERRO[0008] SQL: `INSERT INTO genre (id,name) VALUES (?,?)`  args="['249ecd8d-dd03-4cea-ae3c-7e923da27df2','Minimal']" elapsedTime="382.917µs" error="UNIQUE constraint failed: genre.name" rowsAffected=0
ERRO[0008] SQL: `INSERT INTO genre (id,name) VALUES (?,?)`  args="['b8f04650-df05-4b5f-a205-44a0b6825a4e','Minimal']" elapsedTime="392.301µs" error="UNIQUE constraint failed: genre.name" rowsAffected=0
ERRO[0008] SQL: `INSERT INTO genre (id,name) VALUES (?,?)`  args="['405db359-2c39-497d-8772-c9ccac6e9d97','Minimal']" elapsedTime="418.633µs" error="UNIQUE constraint failed: genre.name" rowsAffected=0
@fabpico fabpico added the bug label Jan 6, 2022
@fabpico fabpico changed the title Unique constraint errors on scan Unique constraint errors (genre) on scan Jan 6, 2022
@vs49688
Copy link
Contributor

vs49688 commented Aug 27, 2022

Confirmed, I've just hit the same issue. Interestingly, it only happens when run in a Docker container. The exact same binary (with the same database and library path) works when run outside it.

EDIT: Scratch that, just tested it on another machine (same library, database, files, and docker image) and it works. Possibly an issue with that machine.

EDIT2: Have ruled out environment differences. The instances on both machines inside the container have the exact same environment variables (except HOSTNAME):

# busybox xargs -0 busybox echo < /proc/1/environ
HOSTNAME=75a78ae0d721
ND_PORT=4533
PWD=/
HOME=/
ND_LOGLEVEL=debug
SHLVL=0
PATH=/nix/store/s35c8dhrhwci7lanyfrsqhsiparq693x-ffmpeg-5.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ND_ADDRESS=0.0.0.0
ND_ENABLEEXTERNALSERVICES=false
ND_CONFIGFILE=/navidrome.json
ND_SCANNER_EXTRACTOR=ffmpeg

This rules out any C locale screwery affecting ffmpeg or taglib.

@vs49688
Copy link
Contributor

vs49688 commented Oct 6, 2022

Root cause: #1918

@deluan deluan linked a pull request Jan 13, 2023 that will close this issue
deluan added a commit that referenced this issue Jan 18, 2023
* persistence/genre: change Put() to upsert

Absolutely disgusting hack to work around [1]. Try to insert the genre,
but if it conflicts, ignore it and update the genre with the existing
ID.

[1]: #1918.

* scanner: remove cached genre repository

Not needed anytmore. And remember:

  "Many Small Queries Are Efficient In SQLite" [1].

[1]: https://www.sqlite.org/np1queryprob.html

* Revert "scanner: remove cached genre repository"

This reverts commit c5d900a.

* Use squirrel to build SQL, to reduce risk of SQL injection

Co-authored-by: Deluan <deluan@navidrome.org>
@github-actions
Copy link

github-actions bot commented Mar 7, 2023

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 7, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants