-
-
Notifications
You must be signed in to change notification settings - Fork 216
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Help setting this up #26
Comments
I think that you aren't opening the Tube Archivist interface but the database. If you haven't changed anything from the defaults in the docker-comopse file, Tube Archivist serves the interface on port 8000. It looks like elasticsearch starts up fine, as the welcome json you posted in your questions is a good sign, despite the high disk usage message, this might not be a problem right now, but at some point it will. Tube Archivist also downloads artwork from youtube, that will need space, then the index itself too... |
Oh, that's exactly it! Thanks a lot! That makes a lot of sense, actually - Qnap's Container Station will pop up with a "link" icon to bring you to a container's page if it has one, and archivist-es links to port 9200 while archivist itself links to port 8000. Some more questions:
|
There was an issue with the "download now" button in a previous version. Make sure you are running the latest version, I've just published v0.0.3, you should be able to see the version number in the footer of the interface if you are running the latest version. Sometimes it is a good idea to delete old container versions, so things don't get mixed up. I'm not sure how QNAP handles that, I'm sure you'll find something in their documentation. If the download get stuck, you can delete the partial file in cache/download. But if you don't have an volume there, you'll have to access the container over the command line. I'm working on a more dynamic download queue that could handle that directly from the interface. When you create a volume in docker everything after the ':' is where the folder goes within the container, you can't change that. So that needs to be ':/youtube' and ':/cache' as described in the readme, otherwise things break. The part in front of the ':' is wherever the files will get stored on your host system, this you can choose however you like. So I think, If I were you I'd start over with this, set the volumes correctly, then things should work out as expected. |
Hey! Thanks for the help. I've changed up my docker-compose, but now I'm no longer able to get to the WebGUI. I'm having a problem with Elasticsearch now. I'll give you my docker-compose and what I got from Elasticsearch's terminal. Docker-compose: https://pastebin.com/keR0Jkse I'm still not sure if I have the folder mounting down-- What you said makes sense to me, but these errors make me wonder if I still messed it up somehow. Could I get a second opinion on the yml too? |
This looks like the access denied issue as described in the readme. You can see, it's the exact same message at the bottom there. |
Thank you! That was it-- I was so intimidated by the huge amount of logs that I didn't actually think about reading them... I've set everything back up, and it turned out I actually did miss the YAML anyway, even though that wasn't the problem. Turns out I had to add /share/CE_CACHEDEV1_DATA/ before the shared folders. So, I'm up and running at the GUI again! But, when trying to download the same test videos again, I got stuck on the same video. But, it's weird-- upon checking the cache > download folder, it seemed fine? There were three files, two webms and an mp4, and the mp4 looked right. Not sure if the sound was there, this video doesn't have any. Is there any way to just add them to the database then, or does it not work like that? For the meantime, I deleted the temporary files. But I tried redoing the download, and it just got stuck again. What should I do if the delete doesn't fix the problem? I am on version 0.0.3 now, if that helps- I can see it on the footer. Also, here's the video if that ends up meaning anything either - https://www.youtube.com/watch?v=0MttD7_8SMc |
The URL you have provided fails for me too. When you try to download the video outside of Tube Archivist, you can confirm that it fails there too. Most likely an issue with that the video doesn't have any audio stream so when yt-dlp and ffmpeg try to combine things together... That's a limitation of yt-dlp and ffmpeg and I have seen multiple issues raised on their platform. That's also what you see on your filesystem, the two webm files, one for video and one for audio and a broken mp4 file. So delete them from the cache folder and you should be able to continue, just not with that video. Then if you need to learn more about how docker volumes work, they have great documentation about that. |
Oh thank goodness-- if the video is out for you too, that means I didn't mess something up. I'm shocked that one of the test videos that I just randomly picked happened to be a lemon. I don't plan on downloading any more silent videos in the future, so it's not a big deal. Is that something that the yt-dlp devs would be able to fix? Or maybe like a timeout function so it knows to give up if its not going anywhere I think we're almost done here, I've got a few final questions before I start moving everything over to this:
[2021-09-25 14:57:29,680: WARNING/ForkPoolWorker-4] ERROR: [youtube] 2tdiKTSdE9Y: Requested format is not available And thankfully I think I can understand most of what this means, but- in terms of yt-dlp, did I mess something up? I've also tried removing one or both of the dashes at the beginning of --embed-thumbnail, but that hasn't worked out either
Thanks so much for all of the help you've been in addition to making this! Tubearchivist is a really cool idea and a lot of fun! |
For the 500 errors, that happens when the parsing of the links failes. For example if you are trying do download a YouTube username and not a YouTube Channel. In the latest release v0.0.4 I have added some more checks so it doesn't show the blank error page. The option for --embed-thumbnail is currently not implemented yet. But definitely useful, I have added it into the roadmap. The PyFilesystem is just something I'm looking into to provide additional options on how the media folder gets stored. That won't change anything of how it works now. |
Hey! I'm running trying to set this up on a QNAP NAS. It's running QTS's Docker Station (I'm not sure if any of this will be important information). Anyways, I pulled the Docker-compose file and just changed the locations for the files it'll pull. When I navigate to the URL for the server, the browser just displays this json:
{
"name" : "305f54c99168",
"cluster_name" : "docker-cluster",
"cluster_uuid" : "B270t1WITSWJ1m1ooeqA1Q",
"version" : {
"number" : "7.14.1",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "66b55ebfa59c92c15db3f69a335d500018b3331e",
"build_date" : "2021-08-26T09:01:05.390870785Z",
"build_snapshot" : false,
"lucene_version" : "8.9.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
I also got a TON of messages in the console terminal, I'm not sure what any of them mean. I think I might actually have more of this, the terminal might've cleared away some of it when I was checking on something else. It's attached here --> https://pastebin.com/kggPyasL Some of these are about my currently high disk usage - funnily enough, a lot of those are YouTube videos, and I'd love to move them onto Tubearchivist as soon as I can get it working.
The text was updated successfully, but these errors were encountered: