Where is id taken from for archive [twitchClips]

[Bertaz]

Checklist

• I'm asking a question and not reporting a bug or requesting a feature
• I've looked through the README
• I've verified that I'm running yt-dlp version 2022.06.29 (update instructions) or later (specify commit)
• I've searched the bugtracker for similar questions including closed ones. DO NOT post duplicates
• I've read the guidelines for opening an issue

Please make sure the question is worded well enough to be understood

I have an old archive file created with youtube-dl, the format appears to still be the same, but the ids are different.
Excerpt of old archive file:

twitchclips 1203489334
twitchclips 1203454404
twitchclips 1096592193
twitchclips 1063641339
twitchclips 1063595332

New archive of same clips:

twitchclips 3706803745
twitchclips 849426066
twitchclips 2195718192
twitchclips 1861302963
twitchclips 2596157108

I don't understand where the new id are taken from. Old ids were the numeric code in the video file name, for example the first clip name is AT-cm|1203489334.mp4.
I can't find the new id anywhere in the clip web page or querying the api. Api response for first clip:

{
    "data": [
        {
            "id": "PlayfulCoweringGooseStoneLightning--WzJYPuOxk4i3Qi0",
            "url": "https://clips.twitch.tv/PlayfulCoweringGooseStoneLightning--WzJYPuOxk4i3Qi0",
            "embed_url": "https://clips.twitch.tv/embed?clip=PlayfulCoweringGooseStoneLightning--WzJYPuOxk4i3Qi0",
            "broadcaster_id": "15697166",
            "broadcaster_name": "Yotobi",
            "creator_id": "70997127",
            "creator_name": "Ber7az",
            "video_id": "",
            "game_id": "6675",
            "language": "it",
            "title": "GommaGatto",
            "view_count": 242,
            "created_at": "2021-06-02T13:31:12Z",
            "thumbnail_url": "https://clips-media-assets2.twitch.tv/AT-cm%7C1203489334-preview-480x272.jpg",
            "duration": 59.9
        }
    ],
    "pagination": {}
}

Provide verbose output that clearly demonstrates the problem

• Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
• Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-w', '-a', '..\\clipsLink.txt', '--download-archive', '..\\archive.txt', '--embed-thumbnail', '--cookies', '..\\twitch.tv_cookies.txt', '-vU']
[debug] Batch file urls: ['https://clips.twitch.tv/PlayfulCoweringGooseStoneLightning--WzJYPuOxk4i3Qi0?tt_medium=my_clips', {deleted}]
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.06.29 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg N-106634-g266cf23eb4 (fdk,setts), ffprobe N-106634-g266cf23eb4
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Loading archive file '..\\archive.txt'
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
[debug] Downloading _update_spec from https://github.com/yt-dlp/yt-dlp/releases/download/2022.06.29/_update_spec
Latest version: 2022.06.29, Current version: 2022.06.29
yt-dlp is up to date (2022.06.29)
[debug] [twitch:clips] Extracting URL: https://clips.twitch.tv/PlayfulCoweringGooseStoneLightning--WzJYPuOxk4i3Qi0?tt_medium=my_clips
[twitch:clips] PlayfulCoweringGooseStoneLightning--WzJYPuOxk4i3Qi0: Downloading clip access token GraphQL
[twitch:clips] PlayfulCoweringGooseStoneLightning--WzJYPuOxk4i3Qi0: Downloading clip GraphQL
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[download] GommaGatto has already been recorded in the archive
[debug] [twitch:clips] Extracting URL: {deleted}

[Bertaz]

I can view the id using the json option -j

> yt-dlp "https://clips.twitch.tv/PlayfulCoweringGooseStoneLightning--WzJYPuOxk4i3Qi0?tt_medium=my_clips" --cookies "twitch.tv_cookies.txt" -j
{"id": "3706803745", "display_id": "PlayfulCoweringGooseStoneLightning--WzJYPuOxk4i3Qi0", "title": "GommaGatto", "formats": [{"url": "https://production.ass{...}

I even took a look at the extractor github.com/yt-dlp/yt-dlp/blob/current/yt_dlp/extractor/twitch.py, but I still can't understand how the id is obtained

[dirkf]

Presumably you used https://api.twitch.tv, which is not used in the clips extractor (and, in the one place in the module where it is used, it's used explicitly instead of via the class variable _API_BASE).

The clips extractor makes two GraphQL queries to https://gql.twitch.tv whose results, if successful, are JSON objects. The id metadata parameter is the first of these that is available and not falsy:

1. the .data.clip.id member of the second query result;
2. the .id member of the first query result if no .data.clip member of the second query result was present, or it was empty or not an object;
3. the id group matched from the url (== display_id).

This raises the question of where the value served by Twitch in the first two cases originates, but that is a secret of the site.

[Bertaz]

Yes I used Helix, didn't think about GQL. Thank you, relooking at the extractor code now I understand.

This raises the question of where the value served by Twitch in the first two cases originates, but that is a secret of the site.

Also, if I understood correctly, with that value you have no way to lookup the clip or obtain any information about it.
While with the slug (what the extractor calls display_id) you can query either GQL or Helix and get that id

[pukkandan]

If the archive ids are different b/w ytdl and yt-dlp, it is a bug. I will look into what can be done

[dirkf]

The ID processing for TwitchClipsIE is the same in yt-dl, since ytdl-org/youtube-dl@232ed8e.

[gamer191]

yt-dlp https://clips.twitch.tv/PlayfulCoweringGooseStoneLightning--WzJYPuOxk4i3Qi0?tt_medium=my_clips --print id
3706803745

youtube-dl https://clips.twitch.tv/PlayfulCoweringGooseStoneLightning--WzJYPuOxk4i3Qi0?tt_medium=my_clips --get-id
3706803745

[pukkandan]

Now yt-dlp can recognize both the old and new archive ids as the same video