Permission issue? or something else? #57
Comments
|
Probably not a permission issue. Are your FFMPEG binaries actually working? Might be worth sharing info about your docker config and manually making sure ffprobe is functioning |
|
Ok, so strange. Apologies.. It seems between the time I tested the setup with a manual.py run and the time I actually tried to do a manual import in sonarr the docker image must have been updated by watchtower or SOMETHING. but it is actually working now. I had to redo the libssl gist steps from here: https://gist.github.com/joulgs/c8a85bb462f48ffc2044dd878ecaa786 Is it possible we could get these steps incorporated into the build process somehow? I'm a total noob at docker so, I don't know if I can simply just add it to the build context or if it requires a change in the dockerfile? |
|
If you put together a little summary of all the steps required to get nvidia working I can try and make a little script |
|
This is required for the build context in the compose: 3.0.10 is the last version on Ubuntu for v3 sonarr. The nvidia ffmpeg will change with time, but you can use whatever version, as long as nvidia2004 is used. In the container, this command needs to be run wget http://archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.0g-2ubuntu4_amd64.deb && dpkg -i libssl1.1_1.1.0g-2ubuntu4_amd64.debor a this The environment variable needs to be passed as well: environment:
- LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64And then reservations need to be made for docker to allow access to the GPUs. If you do it with reversations like this, you don't need to change the runtime, environment variables. deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: ['compute','graphics','utility','video']You can do it through another method if preferred. I prefer the other option as it's cleaner. runtime: nvidia # Expose NVIDIA GPUs
environment:
- NVIDIA_DRIVER_CAPABILITIES=compute,graphics,utility,video
- NVIDIA_VISIBLE_DEVICES=all
devices:
- /dev/dri:/dev/dri # VAAPI/NVDEC/NVENC render nodesAlso, the script download location isn't set with the proper PID or GID when it sets up. root@sonarr:/# ls -l /usr/local/sma/
total 172
drwxr-xr-x 2 root root 4096 Feb 26 21:23 autoprocess
drwxrwxr-x 2 abc abc 4096 Feb 26 22:17 config
drwxr-xr-x 2 root root 4096 Feb 26 21:23 converter
-rwxr-xr-x 1 root root 8089 Feb 26 21:23 delugePostProcess.py
-rw-r--r-- 1 root root 0 Feb 26 21:23 __init__.py
-rw-r--r-- 1 root root 1069 Feb 26 21:23 license.md
-rwxr-xr-x 1 root root 23096 Feb 26 21:23 manual.py
-rwxr-xr-x 1 root root 9535 Feb 26 21:23 NZBGetPostProcess.py
drwxr-xr-x 2 root root 4096 Feb 26 21:23 post_process
-rwxr-xr-x 1 root root 12443 Feb 26 21:23 postRadarr.py
-rwxr-xr-x 1 root root 2934 Feb 26 21:23 postSickbeard.py
-rwxr-xr-x 1 root root 13141 Feb 26 21:23 postSonarr.py
-rw-r--r-- 1 root root 72 Feb 5 18:32 postSonarr.sh
-rwxr-xr-x 1 root root 9382 Feb 26 21:23 qBittorrentPostProcess.py
-rw-r--r-- 1 root root 18338 Feb 26 21:23 README.md
drwxr-xr-x 2 root root 4096 Feb 26 21:23 resources
-rwxr-xr-x 1 root root 4204 Feb 26 21:23 SABPostProcess.py
drwxr-xr-x 3 root root 4096 Feb 26 21:23 setup
-rw-r--r-- 1 root root 2002 Feb 5 18:32 update.py
-rwxr-xr-x 1 root root 10534 Feb 26 21:23 uTorrentPostProcess.py
drwxr-xr-x 5 root root 4096 Feb 26 21:23 venvThis causes Sonarr to be unable to run the files needed to encode files inside the container. To fix this we can change the permissions, but I feel it should be included as part of the setup. because if the container is at all reset, these steps have to be taken each time. (Note: abc:abc is the lsio user name and group name that the process runs as.) As for the autoprocess.ini, not a clue what's strictly required... And there's a lot of confusion to be had. When I run ffmpeg -hwaccels, I only get |
|
Alright so for libssl, I should be able to add something like this to the startup script if [[ -n "${SMA_LIBSSL}"]]; then
filename=$(basename "${SMA_LIBSSL}")
wget -P /tmp "${SMA_LIBSSL}"
dpkg -i "/tmp/${filename}"
rm "/tmp/${filename}"
fiAnd then you would set whichever URL to libssl you wanted to use to SMA_LIBSSL The deploy variables will need to be set at the docker/docker-compose level so that's nothing that we need to bake in Similarly, the environment variables probably need to be set at the user level, though, basically these 3 extra for nvidia specifically environment:
- LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64
- NVIDIA_DRIVER_CAPABILITIES=compute,graphics,utility,video
- NVIDIA_VISIBLE_DEVICES=allWhat I could do to simplify this is also include this in the startup script if [ -f /usr/bin/apt ]; then
## Ubuntu
...
if [[ "${SMA_NVIDIA}" == "true" ]]; then
export NVIDIA_DRIVER_CAPABILITIES="compute,graphics,utility,video"
export NVIDIA_VISIBLE_DEVICES="all"
if [[ -z "${LD_LIBRARY_PATH}" ]]; then
export LD_LIBRARY_PATH="/usr/local/cuda-11.4/lib64"
fi
if [[ -z "${SMA_LIBSSL}" ]]; then
export SMA_LIBSSL="http://archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.0g-2ubuntu4_amd64.deb"
fi
fi
if [[ -n "${SMA_LIBSSL}"]]; then
filename=$(basename "${SMA_LIBSSL}")
wget -P /tmp "${SMA_LIBSSL}"
dpkg -i "/tmp/${filename}"
rm "/tmp/${filename}"
fi
...
fiThis will include everything and provide default values for SMA_LIBSSL but still allow that value to be overriden, and the same for the LD_LIBRARY_PATH Now, one thing I wanted to check, the permissions issue you bring up should not be the case, the last few lines of the start up script have been set for a long time with # permissions
chown -R abc:abc ${SMA_PATH}
chmod -R 775 ${SMA_PATH}/*.shMy concern is maybe the startup script isn't running at all? I wonder if the |
|
I did some digging myself and it looks like that's probably the case- the s6v3 update happened with the migration to alpine, so the startup scripts are probably not being executed, I'll probably have to make a separate build tag with the s6v2 directory structure to be used if you're doing it in conjunction with the old sonarr ubuntu builds |
|
Alright I put up a new build tag, build-s6v2 which includes the updates to the startup script, give it a try |
|
That sounds awesome! Thank you for all the work you put into this! So, forgive my ignorance. How do I use the new build tag with the build context? |
|
Just modify the context line build:
context: https://github.com/mdhiggins/sonarr-sma.git#build-s6v2
args:
- ffmpeg_tag=5.1-nvidia2004
- sonarr_tag=3.0.10 |
|
Will give it a try, it's encoding a backlog at the moment. Btw, are my settings set correctly here? I have 2 GPUs, but it's only using the second one, and I kinda thought maybe it would use both! But not sure if ffmpeg can do that though. |
|
Pretty sure it can only use one at a time |
This just gets resolved to a Python dictionary so you just overwrite the key when it's the same. You'll only end up with {cuda: 1} |
|
So, I gave it a try today. Took down the stack, removed the existing image, repulled and built the new one using the build-s6v2 tag... It doesn't seem like it's doing anything different. ffmpeg doesn't work without installing the libssl1.1 thing, and the /usr/local/sma is still owned by root:root I wanted to see if I could verify that the scripts were updated, but when I went to /etc/s6-overlay/s6-rc.d/ there wasn't any sma folder in there. Not sure if that's normal, docker is new to me, but this s6 overlay thing is alien. |
|
For s6v2, the startup script is going to be located at If you look at the docker logs when the container starts you should see things indicating it ran, though it looks like I had a syntax error I didn't catch, just fixed that now so try a fresh build and then look at the container logs and see if you see that the startup script executed; if it didn't libssl definitely won't be there You should see plus maybe a few more entries depending on what its doing Try again with the fixes I just pushed |
|
Ugh, docker logs. Can't believe I forgot about those lol! Here you go! |
|
You're using an old version, that's been fixed |
|
Ok, weird. I redeleted the image and did another pull and it's working now... Sorry about that! Thank you for all the help! :) |
|
New variables and everything worked ok? Going to update the readme if it's good to go and migrate the same changes to the other containers |
|
Hey it looked like it all worked. I would just not forget to mention that if you use the environment variables you'll also need to include the runtime portion. But other than that, I think this will work. Did lsio also do the same for radarr? I haven't even tried it yet. |
|
I think Radarr has been on alpine for over a year. I'll put together an updated readme next time I have off and link it back here for review. |
|
Merged these changes over to https://github.com/mdhiggins/radarr-sma and https://github.com/mdhiggins/sma-mod Also added a build-s6v2 tag for radarr-sma Any chance you would want to write up a wiki type page for the configuration stuff you used so I can add it for other people looking to do nvidia hwaccel? |
|
I might be able to help you with that, how do I contribute to it? |
|
You could just post it here and I'll format it and move it over |
Describe the bug
When running postSonarr.sh from docker, on an import, I get this in the logs.
Additional context
I've recently moved from a ubuntu hosted sonarr to a docker hosted sonarr, using your sonarr-sma image, and got it working with nvenc and all that. (Thanks for the help with that btw!)
This is the same NFS share that I used in the past, using the same user ID and group ID for sonarr that was used in the past as well. On the NFS share itself from truenas, I've mapped all users to root, and all groups to my shared group ID for all networked services. This usually allows any service to act and modify files as root without issue, however, I wonder if it's causing the issue shown above?
Other issues point to a permission issue, but looking at the permissions from the container, it shouldn't really be the cause... Here is an ls command from the sonarr container.
rootis the user all NFS actions are mapped too on the share side, andabcis the group I've set sonarr to run with in the docker container. As it shows, it's extremely permissive, 777 across the board for all files, but it's reporting an issue that in the past has been reported as a permission issue. Is that the case here as well, do you think? What's confusing to me is, it's able to copy the file there, but then it says nothing can be done to it.Any idea?
The text was updated successfully, but these errors were encountered: