Skip to content

v0.5.0

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 12 Aug 04:52
· 3 commits to main since this release
v0.5.0
This tag was signed with the committer’s verified signature.
tamnd Tam Nguyen Duc
SSH Key Fingerprint: XWWNV0vI2y+8HNjgaL0Y++3thoCRaLa6Z2GV1Mymx7U
Verified
Learn about vigilant mode.
77475d1
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

This is a rewrite rather than a release with features in it.
The parts you can see are the transcript path, the download path, and a provenance block on every record.
The parts you cannot see are the reason those work: the tool now knows which of eleven public surfaces answered a question and says so on the record.

Read the breaking changes before you upgrade.
The store schema is new and old .db files are refused by name.

Nothing needs a key

There is no API key to get, no Google Cloud project to make, and no quota to run out of.
Every read goes to a surface a browser already reaches: the watch page, the InnerTube endpoints the page itself calls, the channel Atom feed, /oembed, the thumbnail CDN, the autocomplete endpoint, and music.youtube.com.

The InnerTube key is not shipped in the binary.
It is read off the first page of a run, because it identifies the web app rather than you, it is the same for every visitor, and it rotates, and a hardcoded one is a tool that breaks on a Tuesday for no reason you can see.
A policy test fails the build on any string literal shaped like a key, so this stays the only way one enters the program.

The one thing that is about you is a session cookie, and only if you hand one over with ytb auth import --cookies.
That is tier 1, it is the only tier 1 surface there is, and any record that touched it says tier: 1.

Transcripts need no yt-dlp

ytb transcript used to shell out.
It no longer does, and there is no fallback that quietly does.

ytb transcript dQw4w9WgXcQ
ytb transcript dQw4w9WgXcQ --format srt
ytb transcript dQw4w9WgXcQ --lang vi --translate en

No yt-dlp, no Deno, no ffmpeg, no JavaScript interpreter.
goja is gone from go.mod entirely, which is most of the binary size difference.

The caption XML is parsed by pkg/srv3 in pure Go, once, and text, srt, vtt and json all come off the same timeline, so the timings in the srt and the timings in the json are the same timings.
The caption URL comes from the ANDROID player response and not the web one, because the web baseUrl answers 200 with zero bytes.

ytb captions lists what a video has before you pick.

Downloads always range

Every request to googlevideo carries a Range header, in 1 MiB chunks.
This is not a tuning choice.
An un-ranged GET to googlevideo is throttled to roughly 32 KiB/s and the same read in ranges runs at around 4 MiB/s, so whether the header is there is the difference between a download and a wait.

There is no code path in the download engine that omits one, including the retries and the requests made after a stream URL is refreshed, and a policy test holds that line.

ytb download dQw4w9WgXcQ --audio
ytb download dQw4w9WgXcQ -vv

The total is known up front from contentLength, so progress is real rather than a spinner, and an interrupted download resumes by offset.
The expire parameter is parsed when the format is read and a download long enough to pass it re-reads the format rather than failing at 90%.
Audio-only and video-only need no ffmpeg at all, and --mux says plainly what is missing when ffmpeg is not installed.

-vv prints every request with its status, its timing and its Range, which is the fastest way to see what the tool actually did.

Every record says where it came from

Every record carries an envelope: kind, tier, surfaces, sources, client, via and missed.

A record without one is a claim with no evidence.
Two reads of the same video disagree in ways that used to be invisible: the watch page has the exact view count and no comment count, the Atom feed has an exact publication time for the newest fifteen uploads and nothing else, and a search lockup has a rounded "36K views" and no description.
With nothing on the record to say which surfaces answered, a listing row and a full read look identical and a rounded number looks like an exact one.

  • surfaces are the surface ids that answered, in read order.
  • sources are the URLs read, so every row can be traced back to a page you can open.
  • client is every InnerTube client claimed, because which client asked decides what comes back.
  • via names the surface behind a field two surfaces could have supplied, down to the block: duration_seconds is s1 microformat.lengthSeconds, because videoDetails on the same page says 213 where the microformat says 214.
  • missed is what the read did not see, in sentences rather than codes. Empty means the read believes it got everything its surfaces carry. It never means the thing has nothing.

The lists are not omitempty, on purpose.
An empty surfaces is a bug and should be visible as one, and an empty missed is a claim, which is a different statement from having said nothing.

Three new commands print the tables the envelope points into, and none of them makes a request:

ytb surfaces   # the eleven surfaces, with the host each one reads
ytb clients    # every InnerTube client this tool claims to be
ytb routes     # every read, with its HTTP route and its MCP tool name

A knowledge graph, not a pile of JSON

pkg/graph gives every node a yt: URI.
A handle is never a key, because handles move.

The vocabulary is closed: 21 predicates, each with a domain, a range and a parser that emits it, and ytb predicates prints the lot.
Every claim carries its source, surface, client and tier, and the client is part of the store's primary key, because WEB and ANDROID answering differently is data rather than a conflict to resolve.

ytb edges dQw4w9WgXcQ
ytb graph @RickAstleyYT --budget 50
ytb rdf dQw4w9WgXcQ --format turtle
ytb rdf dQw4w9WgXcQ --check

pkg/rdf writes n-triples, turtle and json-ld, byte-stable between runs.
The mapping starts from the schema.org terms YouTube already publishes on its own pages rather than from a vocabulary we invented, and ytb rdf --check compares our triples against the page's own microdata and reports per-predicate agreement.

Serving

ytb serve publishes one /v1/<verb> route per read, NDJSON for lists, plus /healthz and /v1/openapi.json.
ytb mcp speaks MCP over stdio.

Both are generated from the same op table the CLI is generated from, so neither can drift from the other.
ytb routes prints all three names side by side, so what the server offers can be read without starting it.

Breaking changes

The store schema is new.
videos, channels and a table per record type are gone, replaced by nodes, claims and reads.
An old file is refused when it is opened, by name, rather than failing three commands later with "no such table: nodes".
Delete it and crawl again.

youtube/ is now ytb/.
If you imported the package, the path changed.

Records have new fields.
Anything parsing ytb output with a strict schema will see kind, tier, surfaces, sources, client, via and missed on every record.

Go 1.26.5 is the toolchain, and govulncheck is clean at the tag.

Full notes with examples: https://ytb-cli.tamnd.com/release-notes/v0-5-0/

Commits

Changelog

Other

Contributors

tamnd
Assets 37
Loading