add *.info.json examples and info

[Styloy]

No description provided.

[bbilly1]

Very nice! Thanks for adding this here.

[dot-mike]

Funny this was just added and yet it's missing an important key named uploader which is used to create a channel that does not exist, yet. I came across this particular edge-case yesterday, but it was it was late so I did not get around to report it yet.

You can see that particular key is being used here:
https://github.com/tubearchivist/tubearchivist/blob/325bdf5cba9ce142c21425fcc8321ec5a55497c5/tubearchivist/home/src/index/channel.py#L62

[bbilly1]

I'm pretty sure that's only needed if the channel doesn't exist on YT anymore, no?

Otherwise this should have added the the channel from ES, or if not exist, it will index the channel from the channel ID.

In any case, please verify my pretty sure and make an other PR.

[dot-mike]

I'm pretty sure that's only needed if the channel doesn't exist on YT anymore, no?

Correct, TA will fail with "keyerror 'uploader'" or something similar in the web-gui, while the output of the program in (in docker logs) will contain the full backtrace as attached. Unsure if it's good include this in the example or better having the UI handle the error a bit more nicely when this edge-case occurs?

[2023-12-04 22:58:37,555: WARNING/ForkPoolWorker-3] 9f0d0d74-ffa2-4087-874c-4e1eea9b0a30 return callback
KeyError: 'uploader'
                    ~~~~~~~~^^^^^^^^^^^^
    \"channel_name\": fallback[\"uploader\"],
  File \"/app/home/src/index/channel.py\", line 116, in _video_fallback
   self._video_fallback(fallback)
  File \"/app/home/src/index/channel.py\", line 41, in build_json
    channel.build_json(upload=True, fallback=self.youtube_meta)
  File \"/app/home/src/index/video.py\", line 208, in _add_channel
   self._add_channel()
  File \"/app/home/src/index/video.py\", line 151, in build_json
   video.build_json(
  File \"/app/home/src/index/manual.py\", line 403, in index_metadata
               ^^^^^^^^^^^^^^^^^^^^^
   json_data = self.index_metadata()
  File \"/app/home/src/index/manual.py\", line 395, in run
    ManualImport(current_video, self.CONFIG).run()
  File \"/app/home/src/index/manual.py\", line 147, in process_videos
   self.process_videos()
  File \"/app/home/src/index/manual.py\", line 56, in scan
   ImportFolderScanner(task=self).scan()
  File \"/app/home/tasks.py\", line 274, in run_manual_import
          ^^^^^^^^^^^^^^^^^^^^^^^^^
    return self.run(*args, **kwargs)
  File \"/root/.local/lib/python3.11/site-packages/celery/app/trace.py\", line 760, in __protected_call__
                ^^^^^^^^^^^^^^^^^^^^
    R = retval = fun(*args, **kwargs)
  File \"/root/.local/lib/python3.11/site-packages/celery/app/trace.py\", line 477, in trace_task
Traceback (most recent call last):
[2023-12-04 22:58:37,537: ERROR/ForkPoolWorker-3] Task manual_import[9f0d0d74-ffa2-4087-874c-4e1eea9b0a30] raised unexpected: KeyError('uploader')
[2023-12-04 22:58:37,525: WARNING/ForkPoolWorker-3] 9f0d0d74-ffa2-4087-874c-4e1eea9b0a30 Failed callback
[2023-12-04 22:58:37,424: WARNING/ForkPoolWorker-3] UC9FM9sHrQT-WfiAtUP9lLxQ: fallback to video metadata
[2023-12-04 22:58:37,422: WARNING/ForkPoolWorker-3] https://www.youtube.com/channel/UC9FM9sHrQT-WfiAtUP9lLxQ: failed to get info from youtube with message ERROR: [youtube:tab] UC9FM9sHrQT-WfiAtUP9lLxQ: YouTube said: This account has been terminated because we received multiple third-party claims of copyright infringement regarding material the user posted.
[2023-12-04 22:58:37,421: WARNING/ForkPoolWorker-3] ERROR: [youtube:tab] UC9FM9sHrQT-WfiAtUP9lLxQ: YouTube said: This account has been terminated because we received multiple third-party claims of copyright infringement regarding material the user posted.
[2023-12-04 22:58:37,318: WARNING/ForkPoolWorker-3] WARNING: [youtube:tab] YouTube said: This account has been terminated because we received multiple third-party claims of copyright infringement regarding material the user posted.
[2023-12-04 22:58:36,096: WARNING/ForkPoolWorker-3] UC9FM9sHrQT-WfiAtUP9lLxQ: get metadata from youtube
[2023-12-04 22:58:36,096: WARNING/ForkPoolWorker-3] {\"_index\":\"ta_channel\",\"_id\":\"UC9FM9sHrQT-WfiAtUP9lLxQ\",\"found\":false}
[2023-12-04 22:58:36,084: WARNING/ForkPoolWorker-3] UC9FM9sHrQT-WfiAtUP9lLxQ: get metadata from es
[2023-12-04 22:58:36,081: WARNING/ForkPoolWorker-3] https://www.youtube.com/watch?v=xjrU3N8M4eo: failed to get info from youtube with message ERROR: [youtube] xjrU3N8M4eo: Video unavailable. This video is no longer available because the YouTube account associated with this video has been terminated.
[2023-12-04 22:58:36,081: WARNING/ForkPoolWorker-3] ERROR: [youtube] xjrU3N8M4eo: Video unavailable. This video is no longer available because the YouTube account associated with this video has been terminated.
[2023-12-04 22:58:33,766: WARNING/ForkPoolWorker-3] xjrU3N8M4eo: get metadata from youtube
[2023-12-04 22:58:33,735: WARNING/ForkPoolWorker-3] manual import: {'media': '/cache/import/[xjrU3N8M4eo].mp4', 'video_id': 'xjrU3N8M4eo', 'metadata': '/cache/import/[xjrU3N8M4eo].info.json', 'thumb': False, 'subtitle': []}
[2023-12-04 22:58:32,554: WARNING/ForkPoolWorker-3] manual import: {'media': '/cache/import/[xjrU3N8M4eo].mp4', 'video_id': False, 'metadata': '/cache/import/[xjrU3N8M4eo].info.json', 'thumb': False, 'subtitle': []}
[2023-12-04 22:58:32,284: WARNING/ForkPoolWorker-3] manual_import create callback
[2023-12-04 22:58:32,260: INFO/MainProcess] Task manual_import[9f0d0d74-ffa2-4087-874c-4e1eea9b0a30] received

[bbilly1]

Thanks for confirming. Note in the docs should be enough, too much of and edge case. Somehothing along the lines, if the channel doesn't exist anymore, you also have to add the uploaded key as the channel name...