by2kb 0.7.5 — Separate platform search lists

by2kb 0.7.5 — Per-platform search recommendations

This release improves multi-platform search presentation and documents optional
platform authentication. Hermes Catalog pins are updated independently.

Separate provider lists

• Bilibili and YouTube no longer compete for one merged recommendation quota.
  Each enabled search source returns up to [search].max_candidates (3–5) relevant
  videos, ranked within that platform and shown in configured provider order.
• Hermes sends each platform's list separately, splitting long lists at video-card
  boundaries to fit IM message limits without mixing platforms. Global numbers stay
  unique: with three Bilibili results and three YouTube results, choose 2,5 to
  process one from each. Empty/failed sources retain their own labelled section;
  disabled sources are neither queried nor shown.
• CLI JSON adds provider_lists with references into the existing flat candidate
  array and separate display messages. The combined message remains available
  with platform sections for older hosts.
• Caption caches and selections support the full combined list, including numbers
  6–10 when both sources return five videos. Search previews remain caption-only;
  media download and ASR begin only after selection.

Upgrade compatibility

No configuration or database migration is required. Existing recommendation
numbers and cached captions remain valid; old sessions are not renumbered.
Existing ASR, Cookie options, disabled providers, knowledge-base paths and custom
Skills are preserved. New searches use the new per-source quotas automatically.

Caption previews can now cover up to ten candidates with both built-in sources,
instead of a maximum of five total. Existing size and shared time limits remain;
choose [search.preview] mode = "metadata_only" to avoid caption preview requests.

Upgrade CLI and Hermes plugin through their respective existing package managers
to obtain both structured lists and separate IM messages. Older plugins can still
read the combined display, but may need their host adapter to split long messages;
upgrade the plugin to obtain bounded per-platform messages. A newer plugin falls
back to the single message when paired with an older CLI. Do not run init --force
or replace personalized Skills to adopt this change.

Platform authentication is optional and explicit

Public videos are attempted anonymously by default. YouTube may require sign-in
or human verification on some networks even for public content. When required,
sign in to YouTube in a browser on the agent's host, then explicitly authorize the
agent to use that browser profile. A Cookie-export extension is not mandatory:
yt-dlp can read a supported browser profile, or an authorized agent can prepare a
YouTube-only Cookie file and configure sources.yt_dlp.cookie_file.

Browser access depends on the same OS user, profile location and operating-system
credential protection. by2kb's cookies_from_browser currently accepts a browser
name, not a custom profile selector. Use a private Cookie file for a custom profile;
never paste Cookie values into IM, logs or knowledge artifacts. Configure only one
of cookie_file and cookies_from_browser. Sessions can expire or rotate; login
does not guarantee access to every video and automatic session refresh is not a
new feature of this release. Existing Cookie settings are preserved on upgrade.

JS/EJS support remains necessary for supported YouTube extraction paths and is not
a replacement for login. Browsers, Playwright and PO Token services remain opt-in.
Bilibili likewise starts anonymously; its optional browser fallback may require a
manual login when the site requests it. Do not interpret every HTTP 412/403 as an
account problem or enable authenticated access without user permission.

by2kb 0.7.4 — Complete YouTube JS support

by2kb 0.7.4 — Complete YouTube JavaScript support

This patch fixes an incomplete YouTube dependency/runtime setup. It does not
claim to bypass YouTube login requirements or guarantee anonymous downloads.

Changes

• Install yt-dlp[default] so yt-dlp receives its matching EJS challenge scripts.
  The legacy youtube and source-ytdlp extras remain supported.
• Automatically select an installed Deno 2.3+ or Node.js 22+; explicitly enable
  the selected runtime in yt-dlp. Ingestion, topic search and caption previews
  share the same runtime settings.
• Support [sources.yt_dlp] js_runtime (auto, deno, node, none) and
  js_runtime_path, plus BY2KB_YTDLP_JS_RUNTIME and
  BY2KB_YTDLP_JS_RUNTIME_PATH overrides. An explicitly configured missing or
  outdated runtime fails clearly instead of silently selecting another.
• Add offline, bounded doctor checks for EJS version compatibility and JS runtime
  execution/version. Disabled yt-dlp and Bilibili-only configurations skip them.
• Update installation Skills and source/upgrade documentation. A missing OS
  runtime requires an explicit installation choice; processing does not silently
  install runtimes, browsers or services.

Upgrade and compatibility

Upgrade using the original package manager:

pipx upgrade by2kb
# For uv-managed installations instead:
uv tool upgrade by2kb

by2kb version
by2kb doctor --json

Existing configuration, ASR selection, knowledge-base paths, job history and
custom Skills are preserved. Missing JS settings inherit automatic selection;
no config migration or init --force is needed. Existing Cookie options remain
explicit opt-ins. Browser and PO Token providers are not installed or enabled by
this release. A supported runtime must be visible on the agent gateway's PATH,
or supplied through an explicit runtime path. See the
JS setup guide.

YouTube access limitations

Live anonymous tests on the Tokyo host still returned LOGIN_REQUIRED after
enabling Node 24 and matching EJS scripts. Separate isolated diagnostics with
Rust PO Token Provider 0.8.1 and upstream BgUtils Provider 2.0.2 also failed on two
public videos. The upstream provider returned a token, and request observation
confirmed its inclusion in player requests; stable yt-dlp and the tested nightly
still received LOGIN_REQUIRED, including with mweb, tv and web_safari.
No audio download or ASR succeeded in those tests.

These observations are consistent with platform risk-control restrictions, but
do not identify YouTube's exact rule or prove every network is blocked. Passing
doctor checks verifies dependencies, not permission to access a particular video.
See the upstream explanation.

Publication and US-host verification

• PR #53 merged; hosted Ubuntu and Windows CI each passed 396 tests.
• The official PyPI publishing workflow completed successfully. Wheel and source distribution are available on PyPI.
• An isolated install of by2kb[asr-doubao]==0.7.4 from PyPI on the US Vantar host passed EJS and Node 24 checks. Both tested public videos still returned LOGIN_REQUIRED; media download and ASR were not reached.
• Separate optional diagnostics with Rust Provider 0.8.1 and official Node Provider 2.0.2 generated PO Tokens and confirmed their inclusion in player requests, but both videos still required login. These providers are not bundled with by2kb.
• The 13-file isolated upgrade audit passed. Vantar configuration and environment-file hashes were unchanged. The host's production 0.7.3 installation and Hermes were left untouched; 0.7.4 testing used a separate temporary environment.

by2kb 0.7.3

by2kb 0.7.3

Bilibili topic search now initializes a normal anonymous web session by visiting the public homepage once per discovery HTTP client. Cookies remain in memory with their normal domain/path restrictions; no logged-in browser cookies are read and no cookie values are logged or persisted. A two-second bootstrap timeout is included within the existing attempt/discovery deadline; a failed homepage request does not suppress API search.

If search/type encounters HTTP 412/429 or a risk-control API code, the next service-controlled retry uses search/all/v2 and extracts only the video group. Backoff, the maximum three search attempts, deadline, normalized queries, intent-aware ranking and accurate partial-failure messages remain unchanged. Malformed group data is a source error rather than a misleading empty result.

Live investigation reproduced HTTP 412 on vantar without a session, followed by successful HTTP 200/API code 0 after homepage initialization. On unicorn all three API probes succeeded in that sample, including the baseline; that sample alone does not prove initialization caused its success. Persistent restrictions may still block both endpoints.

This patch follows 0.7.2; no config migration or additional credentials are needed. It does not enable disabled search providers. Configuration, ASR, learning-library and profile settings are preserved.

by2kb 0.7.2

by2kb 0.7.2

Conversational requests such as “我想查一下什么是supabase” and “我想了解下supabase” now produce a clean topic. Hermes uses its trusted host-model completion API to plan up to three constrained queries, falling back to local rules on timeout, invalid output, or unavailable host access. The original request remains attached to the selection and subsequent learning notes; no additional model credentials are required.

Discovery expands queries when results do not sufficiently match learning intent. Title-based introduction/tutorial/deployment/comparison cues break ties after topic relevance; short introductions receive a modest preference for “what is” requests. Candidate pools expand to 20 Bilibili and 10 YouTube entries per request. Views and likes remain display metadata, not quality scores. This is lexical and intent-aware ranking, not vector search or full-content review.

Each enabled provider may issue at most three requests across all queries and retries. Valid empty responses, connection failures, timeouts, HTTP 412/429 and selected server errors retry with bounded backoff (about 2/4 seconds, small jitter). Valid Retry-After values are respected only within the remaining budget. Permanent/authentication/schema failures do not blindly retry. A failed expansion preserves earlier successful candidates.

Source failure is distinct from zero matches. Structured JSON includes query plans and per-source attempts, sanitized outcomes/status codes/timing; partial lists include one concise source-availability note. Caption failures remain absent from individual cards. Search never downloads media or invokes ASR, and existing metadata survives preview failures.

Configuration and compatibility

Existing config files, search sessions, library paths, ASR credentials and enrichment settings remain readable; no reinitialization is needed. New optional [search] settings default to:

total_timeout_s = 28
attempt_timeout_s = 8
max_attempts = 3
retry_base_s = 2

timeout_s remains the search-phase cap (default 20 s), now including retries/waits; the overall deadline takes precedence. Concurrent caption previews are capped at 5 s or the smaller configured preview timeout / remaining budget. Hermes planning has an 8 s cap and consumes the same overall budget; unavailable planning falls back to rules. Startup, JSON serialization and local storage add some overhead, so these budgets are not a hard real-time service guarantee. Config validation rejects nonfinite settings. Providers remain independently enabled/disabled.

The Hermes hook reserves a search session before planning and reuses it through discovery, preventing an older slow planner from replacing a newer list. CLI supports optional --plan-file, --budget-s, --session-id; search begin reserves a pending session for hosts. Rule-only CLI search remains available.

Persistent platform access restrictions can still prevent retrieval; keyword planning and retry do not guarantee recovery from HTTP 412 or YouTube bot checks. Browser/login fallback is not part of this release.

Fixes #50.

v0.7.1 — Clearer video recommendations

0.7.1 — Clearer learning-video recommendations

Separated entries prominently label YouTube/Bilibili, show precise durations,
and retain public view/like counts when returned by the platform. Missing metrics
are omitted; a genuine zero is displayed. Repeated caption-preview failures and
partial-source warnings no longer clutter successful recommendation messages.
Preview status and provider warnings remain available in structured JSON.

YouTube search/detail metadata and Bilibili search/view metadata supply available
public metrics. Popularity does not change relevance ranking. Existing sessions
remain readable; optional fields need no migration. Configuration, credentials,
ASR, browser settings and knowledge-library paths are preserved.

Search still downloads no audio/video and never invokes ASR. This release does
not add browser-session fallback or resolve YouTube robot/login challenges.
A searchable candidate is not proof that selected-video ingestion can complete.

Upgrade

Upgrade the installed CLI with pipx upgrade by2kb. Legacy copy plugin installs
can use by2kb agent install hermes --force; catalog/Git-owned plugins must use
their original owner. Reload a running gateway after updating plugin code.
Verify the resulting configuration with by2kb doctor --json.

v0.7.0 — Learning-topic discovery and selection

0.7.0 — Find learning videos, then keep the knowledge

Send by2kb 我想了解一下 TiDB 向量检索 to Hermes to search for learning
videos. Receive up to 3–5 numbered candidates with links and recommendation evidence,
then reply 1 or 1,3 to enter the existing transcript, abstract and study-notes
workflow. Fewer results are returned when suitable matches are unavailable.

Changes

• Independently configurable Bilibili and YouTube search providers, concurrent
  searches with deadlines, topic-word screening, and canonical-URL deduplication.
• Configurable captions_only previews (default) or metadata_only. Search never
  downloads audio/video or invokes ASR. Captions fetched for displayed candidates
  are cached and reused when selected; unavailable captions leave metadata previews.
• User/chat/thread-scoped recommendation sessions with expiry, cancellation,
  supersession, validated numeric selections and repeat-safe ingestion.
• Learning-topic context accompanies source provenance and grounded summary prompts.
• Hermes explicit by2kb TOPIC activation and a matching manual Agent Skill.

Search settings are separate from ingestion settings. Disable discovery globally,
enable only selected providers, or disable caption previews. See
topic search configuration. Agent Reach inspired the discovery
and tool-routing design; no Agent Reach code or runtime dependency is included.

Upgrade

pipx upgrade by2kb
# Or: uv tool upgrade by2kb
by2kb version
by2kb doctor --json
hermes plugins update by2kb

Upgrade the CLI and plugin independently. The Hermes catalog update requires
upstream acceptance; until its 0.7.0 pin is accepted, a catalog update may still
install the prior plugin. Follow agent deployment for
the existing Git installation path. Reload a running gateway to use updated code.
Existing configuration is preserved; missing search settings inherit defaults.

Validation and limits

Local regression tests cover discovery, preview cost boundaries, cancellation,
scope isolation, cached-caption ingestion, learning context and Hermes routing.
A live search returned candidates from Bilibili and YouTube; accessible caption
previews were unavailable in that smoke run. No live gateway or paid ASR run was
performed. Recommendations use available metadata/captions, not a complete review
of every video's quality. Podcast and article ingestion remain future extensions.

OpenClaw/ClawHub Skill publication is not part of this release.

v0.6.1 — YouTube by default, configuration preserved

0.6.1 — YouTube works with default configuration

Fixes #45: submitting a YouTube URL could fail with
no configured source provider supports even when yt-dlp was installed. Missing
source configuration previously selected Bilibili only, before any download or ASR.

Changes

• Both Bilibili and YouTube now have default routes in runtime configuration,
  interactive initialization, and the agent-local preset.
• yt-dlp ships with the base package; existing youtube and source-ytdlp extras
  remain compatible. Browser support is still optional.
• YouTube uses available captions first, then audio with the user's configured
  ASR. No separate source selection is needed for the default workflow.
• Routing errors explain configuration overrides; missing-dependency errors explain
  installation repair. Doctor reports explicitly disabled yt-dlp without asking
  users to install unused dependencies.

Upgrade without resetting configuration

pipx upgrade by2kb
# Or, if installed with uv:
uv tool upgrade by2kb
by2kb version
by2kb doctor --json

No config migration writes are performed. Missing sources.providers inherits
both defaults, including existing configurations with only browser fallback options.
ASR, credentials, model choices, browser settings and knowledge-base paths are preserved.

Explicit provider lists and BY2KB_SOURCE_PROVIDERS overrides keep their existing
meaning and order. An older explicit Bilibili-only list is indistinguishable from
an intentional restriction: to adopt both defaults, remove just that list override
or change it to ["bilibili_native", "yt_dlp"]. An explicit
[sources.yt_dlp] enabled = false remains disabled. Do not run init --force.
See upgrade compatibility.

Hermes

by2kb is now listed in the Hermes Catalog. This release does not change the plugin
hook or CLI protocol; the 0.6.0 plugin can use the corrected 0.6.1 CLI. Updating
only the plugin does not upgrade the CLI. The 0.6.1 catalog pin is submitted as a
separate upstream PR; once accepted, catalog users can run
hermes plugins update by2kb. Do not overwrite catalog installs with the legacy
copy installer. Restart a running gateway only when ready to reload plugin code.

Validation and limits

• Local Windows regression suite: 309 passed, 1 platform-dependent skip.
• Fix PR regression CI passed on Windows and Linux.
• Tests cover config preservation, explicit lists/environment overrides, disabled
  sources, default caption ingestion, and audio fallback to the selected ASR.
• Wheel metadata confirms yt-dlp is a base dependency and compatibility extras remain.

No new live YouTube/Telegram, cloud-ASR, browser or model-download test was performed.
This fixes source selection, not YouTube authentication, rate limits, or access
restrictions. No production gateway or user configuration was changed for this release.

v0.6.0 — Hermes-native setup and safer upgrades

0.6.0 — Hermes-native installation and safer upgrades

This release prepares by2kb for the Hermes Plugin Catalog and improves first setup.
It does not add MCP or require a browser, and does not change the transcription or
staged enrichment protocol introduced in 0.5.3.

Changes

• The manifest now declares the registered pre_gateway_dispatch hook through
  provides_hooks. The old hooks field alone failed catalog admission checks.
• The plugin bundles both by2kb:install-by2kb and by2kb:video-to-knowledge.
  The setup Skill covers CLI installation, Python 3.12+, ffmpeg, local Whisper/model
  setup, a chosen Markdown library, and diagnosis of missing commands on Windows.
• The CLI remains separate from Hermes. It need not be installed into Hermes'
  Python environment. Missing CLI errors point to the bundled setup Skill.
• Local Whisper is the default; cloud ASR and browser fallback remain optional.
  Existing configuration takes precedence, with no automatic provider switching.
• The legacy copy installer refuses to overwrite Hermes Git/catalog-managed code,
  even with --force, and refuses to guess when ownership metadata is invalid.
• --hermes-home now also selects the profile used by the enable subprocess.

Upgrade

Use the manager that owns the CLI:

pipx upgrade by2kb
# Or, for a uv-managed installation:
uv tool upgrade by2kb

Then inspect hermes plugins list:

• Legacy copy adapter: by2kb agent install hermes --force refreshes plugin code.
• Catalog adapter: hermes plugins update by2kb follows the reviewed catalog pin.
• Git-pinned adapter: choose the new exact SHA using Hermes install with --ref.

Do not run init or delete state during an ordinary upgrade. Configuration, .env,
models, database, custom Skills and knowledge artifacts are preserved. Restart an
active gateway only with the user's agreement, then run by2kb doctor --json.

Catalog admission is a separate upstream review, not guaranteed by this release.
Until accepted, use the legacy installer or the custom Git subdirectory route in
Agent integration.

For first setup after native plugin installation, ask Hermes:

Load skill_view("by2kb:install-by2kb") and help me configure by2kb.
Preserve my existing configuration; for a new installation use local Whisper
and ask where to save the Markdown knowledge base.

Validation and limits

• Windows regression suite: 293 passed, 1 platform-dependent skip.
• Linux (unicorn Tokyo, Python 3.12): 294 passed.
• The real Hermes installer completed pinned-SHA subdirectory installation,
  enablement, both Skill registrations and hook registration in disposable profiles.
  The copy installer correctly refused to overwrite the resulting pin.
• Tested on unicorn's existing Hermes 866332bfb52c46e543143b2620a9aeee8bce9c77
  with its Python 3.11 runtime. The older scanner emitted two reviewed cautions:
  normal CLI subprocess execution and a false positive on the instruction prohibiting
  sending credentials in chat. Tests accepted these specific findings without disabling
  scanning. Users should review their own scanner output, not blindly force installation.
• Current catalog validation at Hermes 6005aa1fd9aac8b1024ace50fec8cd1c85a04bae
  passed all 11 checks, no warnings, security verdict safe, on Windows and Linux.
• A separately installed 0.6.0 CLI on Linux passed first-configuration and repeat-init
  preservation checks, and the plugin called that CLI's doctor through its real
  subprocess path. Missing optional runtime/model checks produced actionable results.
• Wheel resource inspection confirms both Skills and the manifest are packaged.

These are installation/configuration and regression tests, not a new live Telegram,
cloud-ASR, model-download, or macOS validation. No running gateway was restarted,
no real knowledge base was modified, and unicorn's production CLI stayed on 0.5.3.

v0.5.3 — Small task tickets, complete transcript content

by2kb 0.5.3 — Small task tickets, complete transcript content

Agent tool output could be truncated when large claim/next responses were printed
into a terminal. This release separates control information from prompt content:

• enrichment next --json returns a small schema-2 operation ticket with an immutable
  request-file reference, checksum and input size. The full prompt remains on disk.
• enrichment read pages system/user prompts with explicit offsets and EOF. The
  bundled Hermes adapter verifies and loads the file directly, without model-mediated
  copying of operation IDs. Do not concatenate claim and next output.
• Oversized ASR segments are split losslessly within their original time range.
  Program-controlled chunk summaries and recursive reduction remain responsible for
  long-input processing, not an Agent guessing how much of the transcript to read.
• A 24,000 estimated-token Agent input guard rejects excessive prompts without
  silently truncating them. It is a heuristic safeguard, not a context guarantee for
  every host model. Request-file transport has a separate 32 MiB safety limit.
• Includes unpublished 0.5.2 fixes: idempotent submissions, local process locks,
  bounded resynchronization and completed-state/error-notification protection.

Upgrade

After this version is available on PyPI:

pipx upgrade by2kb
by2kb agent install hermes --force
by2kb version

Restart the Hermes gateway after refreshing the plugin. Back up custom edits inside
the managed plugin directory before --force; it replaces that code copy. User
configuration, personalized Skills, checkpoints and knowledge-base files are not
automatically overwritten. Personalized Skills need the updated paging instructions
applied explicitly. Pinned Git installations must first switch to a PyPI source if
they are intended to follow PyPI upgrades.

Custom CLI adapters must support the new file ticket, or request --inline-prompts
for legacy output. Python service APIs remain inline-compatible. Request files must
be accessible to the Agent and may contain private transcript text. See the
Agent protocol for paging and submission rules.
Pipeline 1.1 invalidates reuse of older intermediate cache keys without deleting
their files; in-progress jobs may regenerate intermediate notes after upgrading.

Validation scope

Deterministic tests cover a million-character request, million-character single-
segment lossless planning, Unicode page reassembly, tamper detection, oversized-input
rejection and a complete file-backed staged CLI flow with intermediate summaries.
Linux/Windows CI exercises the full suite. No paid ASR/LLM call or live Telegram
production replay is implied by these tests.

v0.5.1 — Readable title fallback

by2kb 0.5.1 — readable missing-title fallback

Before summaries, the existing API or external Agent enrichment client generates
one concise title from transcript evidence when the source title is blank, a bare
BVID, or an exact generic placeholder (Untitled, Unknown title, Video,
无标题, etc.). Ordinary readable titles are never rewritten just to improve style.
There is one effective source.title, not an alternative display-title field.

Provenance and artifacts

title_source is original or generated. Missing, null or whitespace-only legacy values
mean original; they do not exempt a BVID/blank title from checking. Generated
provenance is propagated into source.json (top-level and nested source metadata),
transcript.json, and Markdown frontmatter. The generated title is used for the
raw heading, raw/short/long filenames, summary metadata, prompts and chunk plans.
The raw transcript text itself is not changed. Existing unrelated source metadata
is retained. Summary bodies remain governed by the configured Skills.

Title generation requests use a bounded 12,000-character transcript excerpt. The
model must return JSON {"title":"...","evidence":"exact transcript quote"},
with a plain title of 3–100 characters. Invalid JSON, placeholders, multiline or
oversized titles and non-matching evidence are rejected, not silently substituted.
An exact quote validates the evidence's presence, not semantic correctness of the
model's title; review generated titles where correctness is critical. The model
can abstain with {"title":null,"evidence":""}. Empty/very short or failed-quality
transcripts do not trigger title generation. There is no first-sentence truncation
fallback and no invented title when evidence is insufficient.

External Agent compatibility

The staged enrichment next / enrichment submit schema remains version 1.
For missing titles, the first live operation asks for the title JSON; normal
chunk/reduce/summary operations follow with the resolved title. Invalid title
submissions are rejected before caching so the host can resubmit the same pending
operation. API enrichment uses exactly the same generation and validation path.

enrichment claim returns title_required: true, a title prompt, and empty outputs
with pipeline: null when a title operation must precede summary planning. Use
enrichment next and enrichment submit to finish such jobs. Legacy whole-summary
completion remains supported for readable titles; it cannot bypass a required
title operation. Existing hosts using the staged protocol need no schema change.

Upgrade and raw-only behavior

Upgrade the package normally; do not reinitialize the home or database. Upgrading,
listing and duplicate ingestion never scan or rewrite archived jobs. Explicit
--re-enrich checks only the selected stored transcript and republishes coherent
raw/short/long artifacts after enrichment succeeds. Previous filenames for that
job are retired by the existing filesystem sink at publication, not at upgrade.

With enrichment disabled, raw-only ingestion remains raw-only: it does not need
an API key, a new provider or a host-model call. A missing title remains the source
placeholder/BVID until the user explicitly re-enriches with API or external Agent.
During pending Agent enrichment the initial raw artifact also retains that title;
the final artifact set is published only after the staged enrichment completes.

Validation

Regression tests use local transcript fixtures and fake model responses. No live
ASR, browser, production pipx installation or private configuration is required.