500 error on some channels

[Code-Slave]

https://www.youtube.com/c/BlakeWeber
https://www.youtube.com/user/DrunkenWoodworker

cant add channel. there are a few others. All use the /c/channelname

try to add it and 500 error. if i add a video from the channel it adds it fine

[bbilly1]

Thanks for the feedback! The issue is, that a user can have multiple channels, so that is not a unique identifier. On youtube you don't subscribe to a user, only to a channel. That's the thinking behind using the same functionality in Tube Archivist. So for example /c/BlakeWeber is the user name, and UCLAhOL4OBkw42OkcNbqOgYQ the channel ID.
The Channel ID can be a bit hidden (on any video click on the channel name and you can see the channel ID), but for yt-dlp it's pretty easy to extract from a single video, hence you can also subscribe to a channel by providing a video link. I hope this makes sense?

[Code-Slave]

Yea makes sense. Just adds more steps. If i pass that url to yt-dl it geabs videos though

[nifoc]

Thank you for TA!

I'm only getting 500 errors, regardless of what I use (tried channel ID and video URL for quite a few channels).
Is there a way to turn on some kind of debug log?

[nifoc]

Looks like DJANGO_DEBUG=1 does the trick.

UnboundLocalError at /channel/
local variable 'script_content' referenced before assignment

---

It looks like this is because YT shows a consent form before actually displaying the channel page, which is why scraping does not work. I'm based in Germany, so it might be an EU thing?

[Code-Slave]

@nifoc did you put django debug as env var in the docker?

[nifoc]

@nifoc did you put django debug as env var in the docker?

Yes. That enabled the Django page that is usually shown when an exception is thrown during development.

---

I also looked a bit more into the consent page I mentioned. It is indeed something that YouTube/Google redirects you to when you visit for the first time (with certain cookies missing). Best I can tell you need an EU IP, otherwise they will not do the redirect.

[bbilly1]

Hmm, looks like that's a limitation I wasn't aware. Currently yt-dlp doesn't extract some key information from the channel page like:

• Channel Thumbnail URL
• Channel Description
• Channel Views

That's why I wrote my own scraper for that. Can you check what kind of cookie the consent screen will set? Maybe I can pass that to the scarper to avoid all that.

And then I'll have to look into a way on how to gracefully handle errors there.

[jwiren]

I am having the same issue with 500 error when trying to add Isaac Arthur's Science and Futurism channel.
I dug out channel name but it still behaves the same. I was able to add them to download list manually, and it finds 389 videos.
It will start downloading the first one ok, but once it downloads the parts it will just sit on downloading the video name forever. I can see the file under cache in filesystem. It seems to download another video if I press download now, but it will not show anything in the UI. I can see it appearing in the filesystem though. As far as I can tell, there should not be any permission issues, for example cache, redis and elasticsearch data folders do populate with data, but so far nothing comes in under media storage.

[bbilly1]

ok, regarding the cookie consent issue, German VPN to the rescue. I think I found the workaround for that:
f8b5b7c. Please, if you @nifoc @jwiren could test that by cloning the latest code from the testing branch and building the docker image from the Dockerfile, that would be very helpful.

If that solves it, I'll merge it and publish!

[bbilly1]

OK, as far as I can tell from afar, latest version v.0.0.1 (yes, we are doing versioning now) should solve the cookie consent issue.
Closing for now, please reopen if I'm mistaken.

[seriousm4x]

ran into the same issue. i've used bbilly1/tubearchivist:latest in docker-compose. changing it to bbilly1/tubearchivist:v0.0.1 works for me

[jwiren]

I'm still running the downloads and tests on the first channel, but I believe I can confirm it is working now. I was able to subscribe to the channel and I added it to the download queue, and everything seems to be downloading, and showing up in the GUI properly.

Might be worth mentioning that I run this on AlmaLinux 8 with podman and podman-compose, as a user.

[nifoc]

I can confirm that v0.0.1 works for me. Thank you!

(The latest tag seems to point an older/different version. It only works when I use the v0.0.1 tag.)

[bbilly1]

Man you are right, I could have swore i pushed both tags... Thanks for pointing it out!