Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only fetch videos with CC #86

Open
KevinColemanInc opened this issue Nov 4, 2023 · 2 comments
Open

Only fetch videos with CC #86

KevinColemanInc opened this issue Nov 4, 2023 · 2 comments
Labels
enhancement New feature or request

Comments

@KevinColemanInc
Copy link

KevinColemanInc commented Nov 4, 2023

I think yt-dlp fetches all the videos in a channel, then fetches the stats of each video (checking to see if there are captions).

Large channels with single-digit number number of videos with captions are slow to download (and hit api limits).

The (paid and official) YouTube API allows you to retrieve the video IDs with captions in a specific channel.

curl

curl \
  'https://youtube.googleapis.com/youtube/v3/search?channelId=[ChannelID]&part=id&type=video&videoCaption=closedCaption&key=[KEY]' \
  --header 'Accept: application/json' \
  --compressed

response

{
  "kind": "youtube#searchListResponse",
  "etag": "995jyKTI3Q_SpXkNvcBCDR77qP0",
  "nextPageToken": "CAUQAA",
  "regionCode": "",
  "pageInfo": {
    "totalResults": 141,
    "resultsPerPage": 5
  },
  "items": [
    {
      "kind": "youtube#searchResult",
      "etag": "",
      "id": {
        "kind": "youtube#video",
        "videoId": ""
      }
    },
    {
      "kind": "youtube#searchResult",
      "etag": "",
      "id": {
        "kind": "youtube#video",
        "videoId": ""
      }
    },
    {
      "kind": "youtube#searchResult",
      "etag": "",
      "id": {
        "kind": "youtube#video",
        "videoId": ""
      }
    },
    {
      "kind": "youtube#searchResult",
      "etag": "",
      "id": {
        "kind": "youtube#video",
        "videoId": ""
      }
    },
    {
      "kind": "youtube#searchResult",
      "etag": "",
      "id": {
        "kind": "youtube#video",
        "videoId": ""
      }
    }
  ]
}
@NotJoeMartinez
Copy link
Owner

Have you tried multi threaded downloading? It substantially decreases the time it takes to download a channel

yt-fts download --number-of-jobs 5 "https://www.youtube.com/@3blue1brown"

@NotJoeMartinez NotJoeMartinez added the enhancement New feature or request label Dec 19, 2023
@KevinColemanInc
Copy link
Author

@NotJoeMartinez I have not tried it, but I worry about hitting rate limits. I only tested a few channels, but I found at least one channel with over 1,000 videos with none of them having captions. Even with 5 jobs, that is still a lot of requests to waste time on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: No status
Development

No branches or pull requests

2 participants