Docker install errors

[maltokyo]

I get this error after the initial image build:

root@ssd:~/zDocker/docker-YoutubeDL-Tzahi12345# docker-compose up
Recreating ytdl_material ... done
Attaching to ytdl_material
ytdl_material    | internal/modules/cjs/loader.js:796
ytdl_material    |     throw err;
ytdl_material    |     ^
ytdl_material    | 
ytdl_material    | Error: Cannot find module '/app/app.js'
ytdl_material    |     at Function.Module._resolveFilename (internal/modules/cjs/loader.js:793:17)
ytdl_material    |     at Function.Module._load (internal/modules/cjs/loader.js:686:27)
ytdl_material    |     at Function.Module.runMain (internal/modules/cjs/loader.js:1043:10)
ytdl_material    |     at internal/main/run_main_module.js:17:11 {
ytdl_material    |   code: 'MODULE_NOT_FOUND',
ytdl_material    |   requireStack: []
ytdl_material    | }

[gerroon]

I have the same issue on Debian Testing

[Tzahi12345]

Interesting, are you using the latest docker-compose.yml file from the latest release? I'm having a hard time replicating this error.

[gerroon]

Here is my compose file

https://bin.privacytools.io/?dbcb4689efcf04fd#gcHvb0/cCkD2bjbvHBJFWajAHiGrmUuixBXCje1OYcA=

[maltokyo]

Here is mine:

version: "2"
services:
    ytdl_material:
        container_name: ytdl_material
        build: .
        environment: 
            # config items
            ytdl_url: http://localhost:8998
            ytdl_port: '17442'
            ytdl_use_encryption: 'false'
            ytdl_cert_file_path: /etc/letsencrypt/live/example.com/fullchain.pem
            ytdl_key_file_path: /etc/letsencrypt/live/example.com/privkey.pem
            ytdl_audio_folder_path: audio/
            ytdl_video_folder_path: video/
            ytdl_title_top: Web_Video_Downloader
            ytdl_file_manager_enabled: 'true'
            ytdl_allow_quality_select: 'true'
            ytdl_download_only_mode: 'false'
            ytdl_allow_multi_download_mode: 'true'
            ytdl_use_youtube_api: 'false'
            ytdl_youtube_api_key: 'false'
            ytdl_default_theme: default
            ytdl_allow_theme_change: 'true'
            ytdl_use_default_downloading_agent: 'true'
            ytdl_custom_downloading_agent: 'false'
            ytdl_allow_advanced_download: 'false'
            # do not touch this
            write_ytdl_config: 'true'
            ALLOW_CONFIG_MUTATIONS: 'true'
        restart: unless-stopped
        ports:
            - "8998:17442"
        image: tzahi12345/youtubedl-material:3.3
        volumes:
            - '/var/lib/docker/volumes/ytdl_material/_data_ytdl_audio:/audio'
            - '/var/lib/docker/volumes/ytdl_material/_data_ytdl_video:/video'
networks:
  default:
    external:
      name: webproxy

[maltokyo]

And yes, I am using latest version

[Tzahi12345]

The error above is basically saying that the container can't find the app.js file. This file gets copied in when the image gets built -- this is something I do on my end. It's weird that your containers can't find it, I even tried @gerroon's docker-compose and I still couldn't replicate.. There must be something else going on.

Just to clarify @maltokyo, other than the modifications you made to docker-compose (they seem harmless), the only commands you ran are:

docker-compose pull and docker-compose up?

[maltokyo]

correct @Tzahi12345 only those two.. and still get the error. I cannot work it out.

One hint, when I run docker ps I get the init command listed as "/bin/sh" as below:
(thats strange)

root@ssd:~/zDocker/docker-YoutubeDL-Tzahi12345# docker ps
CONTAINER ID        IMAGE                                      COMMAND                  CREATED             STATUS                         PORTS                                                                   NAMES
4281e19a325e        tzahi12345/youtubedl-material:3.3          "/bin/sh"                18 seconds ago      Restarting (0) 2 seconds ago                                                                           ytdl_material

[gerroon]

I was able to get it running after I pruned Docker images,containers and and networks ,

[maltokyo]

@gerroon that's strange.. you should not need to do that ;)

[Tzahi12345]

One hint, when I run docker ps I get the init command listed as "/bin/sh" as below:
(thats strange)

Interesting hint! When I run docker ps it shows this as my command:

CONTAINER ID        IMAGE                               COMMAND             CREATED             STATUS            PORTS                     NAMES
d19c19c03b8a        tzahi12345/youtubedl-material:3.3   "node app.js"       About an hour ago   Up About an hour    0.0.0.0:8998->17442/tcp   root_ytdl_material_1

I think there is something wrong with your image. Really weird that this is happening to multiple people, and making everyone who has this prune everything is unreasonable.

Let me know if pruning the image and re-downloading helps.

[maltokyo]

pruned everything and tried again, no luck and the same outcome. I just cant work out why it is doing it...

[Tzahi12345]

I put together a zip where you can quickly build the image yourself (hopefully this fixes your image):

https://github.com/Tzahi12345/YoutubeDL-Material/blob/master/releases/3.3/youtubedl-material-3.3-docker.zip

In the extracted directory, run:

docker image build -t tzahi12345/youtubedl-material:3.3 .

Then try running docker-compose up. If the error goes away, then the image is now fixed! You should be able to use your original docker-compose.yml as well.

[maltokyo]

This worked!! So, whatever is different in this zip file, now works. Thanks! Trying now.

[Tzahi12345]

Lit, I didn't actually change anything, just put together the folder structure that I build with. I suspect something went wrong (and kept going wrong) in the pulling of the docker image, but that's a guess.

I'll leave this issue open until I'm able to fix or find out more about this bug. Anyways, I appreciate the help!

[GlassedSilver]

I'm currently trying to generate an unRAID-ready docker for this (basically, it's an XML-based instruction for unRAID's Docker how to execute the Docker run command)

See:
https://selfhosters.net/docker/templating/templating/

I'm not having any luck though... It generates a container successfully, but then nothing happens...

Oddly enough when I run a shell in that container it has pulled Windows binaries????

Here's my docker create command for your debugging pleasure :P

/usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker create --name='youtubedl-material' --net='bridge' -e TZ="Europe/Berlin" -e HOST_OS="Unraid" -e 'ytdl_title_top'='YoutubeDL Material' -e 'ytdl_allow_advanced_download'='true' -p '17442:8998/tcp' -v '/mnt/user/Downloads/youtube-dl-test/audio/':'/audio/':'rw' -v '/mnt/user/Downloads/youtube-dl-test/video/':'/video/':'rw' 'tzahi12345/youtubedl-material' 

[Tzahi12345]

@GlassedSilver The Windows binary thing is my bad, accidentally included that in the container.. oops! :) will fix that in the next update

Perhaps the image you're using isn't the correct version? Try using the image tzahi12345/youtubedl-material:3.3 in your docker create.

Btw, the audio and video file paths are relative to the /app folder in the container (so I believe the volumes should point to /app/video/). Just something I noticed.

Anyways, let me know if that fixes things for you.

[GlassedSilver]

Something else I noticed... where are configs stored? Like preferences I set etc within the app?

Because I have mapped audio and video folders, but nothing for the configs. That would mean my settings get wiped everytime I reinstall or update the docker container... A docker image file should ONLY carry the programs, no settings or user data. Logs as well, but nothing more. At least that's my understanding, new to docker deployment beyond unRAID's glorious point and click system for easy docker installs.

Anyhow, I got it to work. I had to append a version, went with :latest though. Also I mixed up the port with the host port. So that helped as well.

[Tzahi12345]

Something else I noticed... where are configs stored? Like preferences I set etc within the app?

The settings are stored in the /app/config/default.json file. You should be able to map a volume to the /app/config directory.

That would mean my settings get wiped everytime I reinstall or update the docker container...

Your environment variables are checked on startup, so if you set the config through there it will update if you restart the container. But you are right in that without these environment variables, the original config is lost.

A docker image file should ONLY carry the programs, no settings or user data. Logs as well, but nothing more.

Right, I agree. It's why I made the environment variables write-through (meaning if you update them, the config will auto update accordingly). You can sort of use the docker-compose.yml as your config file in this way. Not sure if this is ideal (it probably isn't), but I'll keep looking to improve the Docker install.

Anyhow, I got it to work.

Cool, great to hear :) Lmk if you have any additional issues/questions

[GlassedSilver]

The settings are stored in the /app/config/default.json file. You should be able to map a volume to the /app/config directory.
Ooops, I thought the app actually perma-stores youtube-dl parameters. That's what I meant. I just noticed that it doesn't yet. Added it as trackable feature request: #19

Your environment variables are checked on startup, so if you set the config through there it will update if you restart the container. But you are right in that without these environment variables, the original config is lost.
Nevermind about that, see above, that's what I meant there as well. :)

A docker image file should ONLY carry the programs, no settings or user data. Logs as well, but nothing more.

Right, I agree. It's why I made the environment variables write-through (meaning if you update them, the config will auto update accordingly). You can sort of use the docker-compose.yml as your config file in this way. Not sure if this is ideal (it probably isn't), but I'll keep looking to improve the Docker install.
Well... and here as well. Sorry to cause confusion!

Anyhow, I got it to work.

Cool, great to hear :) Lmk if you have any additional issues/questions

See above seems to be the theme now. So far so good. I'll give it a test a little later and eventually release your docker as an unRAID-ready template to the Community Applications repo.

Maybe an official logo would do this project fine. Something that isn't black on transparent background. :P

Let me show you what I mean:

(dark backgrounds kill this icon haha)

[Tzahi12345]

I'll give it a test a little later and eventually release your docker as an unRAID-ready template to the Community Applications repo.

Sounds awesome!

Maybe an official logo would do this project fine. Something that isn't black on transparent background. :P

Yup, I see what you mean. I'll make a note to create one for the next update.

[Tzahi12345]

I was able to track down this bug! I had to install a Debian Testing on a virtual machine, and I was finally able to reproduce it.

Basically, on some systems, docker-compose pull failed to actually pull the image as the docker-compose.yml contained the line build: .. This convinced docker-compose that the image is supposed to be built in that folder rather than pulled. And when docker-compose up ran, it tried to build it but failed as it didn't have the necessary files (app.js, package.json, etc.).

Removing build: . from the docker-compose.yml fixes the issue. The reason why I had it in there in the first place is it made building the image on my end slightly easier. Anyways, you can check out the fix in this commit. It will make its way into the next update (3.6) which is coming up soon, so stay posted for that.

Anyways, thanks for the help @maltokyo!

[maltokyo]

Great news. Thanks!!