Add ARM64 builds for nightly tag again, if not feasible, add ref to DOCs that ARM32 is deprecated/outdated (at least for now)

[jm2003uk]

Running this on a Raspberry Pi 4 8gb and Raspberry Pi OS lite and using the following docker-compose file:

version: "2"
services:
    ytdl_material:
        environment:
            ALLOW_CONFIG_MUTATIONS: 'true'
        restart: always
        volumes:
            - ./appdata:/app/appdata
            - ./audio:/app/audio
            - ./video:/app/video
            - ./subscriptions:/app/subscriptions
            - ./users:/app/users
        ports:
            - "8998:17442"
        image: tzahi12345/youtubedl-material:latest

Everything seems to work fine, however if i change the tag to "nightly", i get the following appear:

Fatal error in , line 0
unreachable code
FailureMessage Object: 0xbebb21ec 

(this last part seems to be different every time)

Annoyingly, i had another instance running on a a seperate Pi, albeit the 4gb version and using portainer and that ran the nightly version fine. Not sure if something possibly updated between setting the two up?

Another quick thing i've noticed, I have 48 subscriptions with a check interval of 7200, however looking at pi-hole stats i'm getting a lot of hits to youtube...over 220000 in a 24 hour period. Is this expected behaviour?

[dejan995]

@jm2003uk This is an incompatibility between NodeJS (I think) and the architecture of the Pi4 and it's operating system, which is 32bit.
My suggestion is to use a 64bit OS on your Pi4, if it is possible. You can go with Ubuntu Server, it's pretty lightwight.
If not, the Docker image for armv7 will have to be rebuilt to somehow fix this.

@Tzahi12345 I am referencing this thread to provide the info above. Not sure if it is relevant for this issue. But the problems seem to be same. jesec/flood#390

Everything seems to work fine, however if i change the tag to "nightly", i get the following appear:

This is because the latest tag hasn't been update in about 8 months and is probably using an older version of NodeJS.
If you can't install Ubuntu on your Pi4 for some reason, you can use this image instead, at least until the nightly image gets patched. Only downside is that you might not have a couple of new features that were introduced in newer versions. Although I belive you can update the app from the GUI but don't know if it will be perssistent if you restart or recreate the container.

Annoyingly, i had another instance running on a a seperate Pi, albeit the 4gb version and using portainer and that ran the nightly version fine. Not sure if something possibly updated between setting the two up?

Again, probably the NodeJS version.
You can try with Portainer again, not sure but it might work.

Another quick thing i've noticed, I have 48 subscriptions with a check interval of 7200, however looking at pi-hole stats i'm getting a lot of hits to youtube...over 220000 in a 24 hour period. Is this expected behaviour?

I don't know if this is abnormal or expected. I only have a couple of subsriptions and they are for music videos and artists don't upload new music every single day. So I have mine set on a delay for a month. (2700000 sec.)
@Tzahi12345 Maybe you can clear this up?

[jm2003uk]

@dejan995 To confirm following from your suggestion, Ubuntu server 64bit works fine running nightly.

I also just wanted to provide some screen shots from pi-hole, seeing as i'm experiencing the same behaviour. You can see when i started up the container just before 2200 and the resulting traffic. Each bar represents 10mins.

I must have set the check interval to 7200 (2 hours), which does seem to match the overnight bar graph pattern and then extended it to 10800 (3 hours) during the day. So it could be that the scheduling is working....but it's going overboard sending requests?

I'm not sure if this is another issue, but after going to 'Advanced -> Restart server' and then checking the logs, it displays this:

2021-09-29T18:05:22.824Z VERBOSE: Watching *channel_name* with delay interval of 514285.71428571426

I could perhaps try removing all but one subscription and monitor the logs for when it actually is doing the video check. Interesting to see how just having one affects the domain hits too.

Is it worth either opening a new issue for this, posting into the concurrent checks thread or just carry on here?

[dejan995]

@jm2003uk Glad I could help with the first issue.

@Tzahi12345 It would be best to include on the README that 32bit armv7 or for that matter any 32bit will no longer be supported. I think it's better to drop support for this againg technology, that is already 90% phased out from use, then to try to find a way to support it.

@jm2003uk As for the second issue, until @Tzahi12345 chimes in on this, I really have no idea what is going on.
How many videos do you have in your library (total number from all 48 subsriptions)? These could be the metadata requests.

Is it worth either opening a new issue for this, posting into the concurrent checks thread or just carry on here?

It is best to open a new issue. We can keep the conversation about the 32bit issue going on here.

[Tzahi12345]

Realized I never addressed this issue, sorry about that!

Another quick thing i've noticed, I have 48 subscriptions with a check interval of 7200, however looking at pi-hole stats i'm getting a lot of hits to youtube...over 220000 in a 24 hour period. Is this expected behaviour?

The subscriptions check interval is the amount of time it takes for the server to go through all subscriptions, meaning it was checking one sub every 150 seconds (7200/48). 220k is a lot, it means every subscription check involved ~576 requests. The ytimg.com ones do indicate that a good portion (perhaps even a majority) of these requests were downloads. You can probably test this yourself by downloading a video and seeing how many requests it took for a video of that duration. But yeah as @dejan995 suggested let's fan this out into a separate issue.

@Tzahi12345 It would be best to include on the README that 32bit armv7 or for that matter any 32bit will no longer be supported. I think it's better to drop support for this againg technology, that is already 90% phased out from use, then to try to find a way to support it.

That's definitely a solution, and might be the best one. But this may have been due to #454 as apparently Alpine 3.13 doesn't support 32-bit OS's. So we can revert back to 3.12 and this may be resolved. The big caveat there is that support for it will drop in a year, and do we really want to be holding onto an outdated Alpine version for the sake of supporting 32-bit ARM. I'm down, but it's a decision we have to make.

[dejan995]

@Tzahi12345 A community poll might be a way to go for this. I belive GitHub has some apps for it.
Since Alpine will drop support for 3.12 in a year, we can have the poll running for about 6 month and let the community decide.
We will have to clearlly explain that this is not actually droping support for hardware but rather for outdated 32bit software. About 99.9% of CPUs and SoCs produced from 2010 onwards are 64bit or have some sort of 64bit compatibility layer on them and hence YoutubeDL-Material will be able to run on it, provided that the underlaying software (OS) is 64bit.

[yllekz]

Works fine on my pi in the same manner but with one exception: Switch your tag to nightly-arm perhaps?

[dejan995]

Works fine on my pi in the same manner but with one exception: Switch your tag to nightly-arm perhaps?

Currently the most up to date tag for youtubedl-material is the nightly tag.
If you go on Docker Hub you will see that the nightly-arm and latest tags were last built at around the same time, about a year ago. That is why it is working, they are using the same old versions of NodeJS and Alpine.
This is one way to go about this issue.
But I would recommend installing a 64bit OS on your pi and using the nightly tag.

Happy Holidays !!! 🥳

[GlassedSilver]

I've already been pondering whether we might want to go with Ubuntu as base image, this would entail testing before even making anything close to a promise of course, I'd be willing to take on that task sometime soonish.

So this will end up being either a DOCS issue to follow up on or a PR for the DOCKERFILE.

I would then consider bringing both x64 as well as ARM64 into one build and publish work file. That's the same thing that many docker builds do, to build and publish both in one work file.

I think ARM v7 (32bit) can be safely deprecated, because the typical ARM 32bit setup that you find these days is simply your average outdated Raspbian host on an ARM64 capable Pi. The 32bit-only Pis that are around likely lack the oomph for the sorts of things we want to accomplish with this project anyhow, since this has really developed into a lot more than just a downloader with A UI to view the files. :)

Very constructive discussion in here. 👍

[yllekz]

I'm kind of surprised at the sudden removal of 32-bit RPi OS support. I have never seen a 64-bit RPi OS in production, and the Raspberry Pi Imager itself does not yet have it as a default option. OpenMediaVault 5 requires the 32-bit OS as well, which a lot of folks use as the basis for their Pi. Can we pump the breaks on abandoning 32-bit RPi support for now?

[GlassedSilver]

I'm kind of surprised at the sudden removal of 32-bit RPi OS support. I have never seen a 64-bit RPi OS in production, and the Raspberry Pi Imager itself does not yet have it as a default option. OpenMediaVault 5 requires the 32-bit OS as well, which a lot of folks use as the basis for their Pi. Can we pump the breaks on abandoning 32-bit RPi support for now?

The Raspberry Pi Foundation have been offering 64bit capable Pis ever since the Pi 3 and only recently released 64bit Raspbian, that's why you never saw it.

That being said, Ubuntu Server for Raspberry Pi in 64bit has been available for longer and I'd be surprised if it was the only OS with 64bit ARM support for Raspberry Pi.

Remember, we're not talking 64bit desktop usage here, a proper thorough 64bit desktop has only recently been made available by Ubuntu as well with much speedier graphics, but we're talking about Pi as a server here, 64bit hasn't been as odd as you may imagine.

All that being said, if OMV indeed recommend to use Pi 4 and still require you to run it in ARM32 mode, I think it's fair to try keeping ARM32bit support! :)
I'll try my hand at it. :)

[GlassedSilver]

Build and push to repo is currently happening.

Any ARM32 and ARM64 users are welcome to switch to the nightly tag and report their findings once this is done. :)

[GlassedSilver]

ARM v7 (32bit), ARM v8 (64bit) and amd64 are now all back united in the YTDL-M family.

Simply use the nightly tag without any other prefix or suffix, and you'll be running the latest build for the platform of your choice! :)