Repository navigation
Changes in 0.12.0
Added
- The DB holds the CLI Reference for every manual version MikroTik serves. It
stores the 7.24 and 7.25 trees plus the default tree (7.26 today), each under the
version itsllms.txtdeclares, in a newcliref_treestable.cliref_pages
gainsmanual_version(schema 12). The tree list comes from MikroTik's
cli-compare.json. A tree MikroTik stops serving is skipped with a warning, while
a served tree that's incomplete or declares the wrong version fails the build.
MCP and TUI answers read the default tree only, so they are unchanged. This is
slice 1 of #188, and it adds no per-version answers. - The DB stores each manual version's CLI Reference landing page. That page
documents what every menu page leaves out: argument types, standard list entry
properties (comment,disabled,copy-from,place-before), standard print
modifiers and address flags. Each tree keeps the page verbatim incliref_trees.
Its four sections are parsed intocliref_standard, one row per item, with
descriptions kept word for word. A tree that's served without the page, or whose
sections change shape, fails extraction. The export writes them to
cli-reference/standard.tsv; MCP answers don't use them yet (#193; #188 slice 2). - Each release publishes the CLI Reference dataset as its own asset, and keeps trees
MikroTik stops serving.rosetta-cli-reference.tar.gz(thecli-reference/
directory of export format 2, about 3.5 MB) and its.sha256sit next to
ros-help.db.gz, and the release notes repeat the digest; pin the tag and the digest.
The release build downloads the newest earlier release's copy and passes it to the new
extract-cliref --archive=DIR. A tree whose/docs/<v>/llms.txtnow returns 404 is
re-parsed from that archive with its originalfetched_atandretained = 1, instead
of disappearing. A former default tree is stored under/docs/<v>/(#188 slice 4). - Each build checks the CLI Reference against MikroTik's own version diffs. It
compares the changes between trees it computes from its own rows with
cli-compare.jsonand writescli-compare-reconcile.tsv. A mismatch doesn't fail
the release: it opens or updates one tracking issue, and the first clean release
closes it. Each tree's count is stored incliref_trees.cli_compare_mismatches. - Command-version answers say which arches a version's data came from (#91). 38 of
the 67 versions with command data come from x86-onlyinspect.jsonbuilds, so they lack
every arm64-only path: 7.24rc2 → 7.24rc3 diffed as 1,387 removals that never happened.
routeros_command_diffnow returnsfrom_arches/to_archesand says when they differ
(arch=x86gives that diff 0 changes).routeros_command_version_checkreturns
absent_in_single_arch_versionswith a note, and its "earliest tracked version" note
now fires, since the range it checks is the one with command data.routeros_stats
groups versions incommand_version_arches, and the TUI shows the same. Extracting one
version's arches in separate runs now merges their paths instead of keeping only the
last run's. routeros_statsnames the manual version the docs came from. A new
manual_versionblock (docs,cli_reference) reports the version each
manual.mikrotik.com tree declares in itsllms.txtheader. The browse TUI's
stats view and therosetta exportmanifest show it too. The site's default
tree is not the stable manual (it declared 7.26 on 2026-10-20, while stable
was 7.24.5), so treat this, not the stable channel, as the docs' version.
Extraction now fails when the header is missing, or when the prose and CLI
Reference were read from different trees.--helpdocuments the pipeable JSON path and--db. Top-level
--helppoints atbrowse --once .helpwith JSON examples, and the working
--db <path>override (same asDB_PATH) is now listed and mirrored in the
MANUAL CLI Flags table.
Changed
-
Release binaries and OCI images compile with current Bun, not a pinned 1.3.14.
The #142 pin hid a real incompatibility: Bun 1.4's--bytecodecompile turned the
defaultbun:sqliteimport into "Object is not a constructor" at HTTP startup. The
code now uses the namedDatabaseimport, and the release workflow and
Dockerfile.releasetrack current Bun, so a future compiler regression fails the
release smoke instead of hiding behind a pin. -
A Release OCI image that fails its smoke no longer moves
:nextor:latest.
The publish job used to push the version and floating tags in the same build that
the smoke later tested. It now pushes onlysha-<short>, smokes that, and then
promotes the same multi-arch manifest tov<version>and any floating tag. -
rosetta exportformat 2 carries every CLI Reference tree (breaking for
cli-reference/readers). Everycli-reference/*.tsvgains a leading
manual_versioncolumn and rows for each stored tree, oldest first. New
trees.tsvlists the trees, each with arows_digestthat ignores row ids and
the inspect crosswalk, so a new release with unchanged content keeps its digest.
Newstandard.tsvholds the landing-page facts. Raw source moves to
source/<manual_version>/<slug>.md, next to each tree's_llms.txtand landing
pagecli-reference.md. A newcli-reference/manifest.tomldescribes that
directory on its own: provenance, contract, a[[trees]]table with counts and
cli_compare_mismatches, and the dataset's six contract properties.
format_versionis 2 in both manifests; the other datasets are unchanged
(#188 slice 3). -
The CLI Reference cache has one directory per manual version. Each tree is cached
inmanual/cli-reference/<version>/, with a_tree.jsonthat lets--from-cache
rebuild it. A complete live run removes pre-#188 flat-layout pages and any tree no
longer served. The release build's CLI-Reference step fetches about three times as
many pages. -
Product matrix refreshed to the 2026-10-29 snapshot.
devicesand the hardware
overlay now readmatrix/2026-10-29/matrix.csv(154 products). MikroTik no longer
lists CubeSA 60Pro ac (CubeG-5ac60ay-SA), so it leavesdevices, but it stays in
the hardware catalog as discontinued (see Fixed). It corrects CRS312-4C+8XG-RM,
CRS504-4XQ-IN and CRS518-16XS-2XQ-RM to 128 MB RAM and 32 MB storage (previously 64 MB
and 16 MB). The hardware and product-page assessments were re-scraped the same day;
device-map.tsvchanges only by the dropped row.
Fixed
--db <path>works before the subcommand.rosetta --db X browse --once …started
the stdio MCP server instead of running the command, printed nothing and exited 0, and
rosetta --db X export <dir>did the same.rosetta export --db X <dir>used the DB path
as the output directory.--dbnow selects the DB wherever it appears, as--helpsays.- A device that leaves the product matrix stays findable as discontinued, even when
its only /hardware page is shared. The hardware catalog gave a row only to matrix
devices and /hardware pages, so a former matrix device documented only on a combined
page or on a series page that matched other devices lost its row. Its product page
then reached only the drop ledger. Groove 52 and SXTsq 5 ac had been lost this way
since the 2026-10-03 matrix, and CubeSA 60Pro ac would have followed. A new
hardware-www-map.tomlfield,www_rows, gives each such product a catalog row
under its old id, with name, specs and discontinued flag from its product page.
routeros_device_lookupresolves all three by name or product code. Extraction fails
if a listed product makes no row. - Changelogs include the current stable and development releases again. The
channel-head lookup tookupgrade.mikrotik.com's whole answer (7.25 1792678320,
version plus build time) as the version, so each head 404'd and a build could ship
without the newest stable or development changelogs (7.25 and 7.26beta4 at the
time). A local run now picks up both (6,950 entries). browse --once .<tool>stdout is now exactly one JSON document. The args
echo banner moved to stderr and excerpt highlighting no longer injects ANSI
escapes into the JSON, so shell consumers can pipe straight intojqor
JSON.parse. Unknown dot-commands, missing required args, and tool errors
now exit non-zero with the diagnostic on stderr instead of exiting 0.- Piped
browseoutput is plain text;NO_COLORis honored. Human-rendered
commands (browse --once stats,search, …) no longer emit ANSI escapes or
OSC 8 hyperlinks when stdout is not a TTY, so piped output needs no stripping
(links render astext (url)). SettingNO_COLORforces plain output even on
a terminal. Interactive TTY output is unchanged, and--oncedot-command JSON
was already escape-free. - Manual extraction works with manual.mikrotik.com's new Markdown layout. On
2026-10-06 the site moved category pages from<dir>/index.mdto<dir>.md,
CLI Reference menu pages from<dir>/<dir>.mdto<dir>.md, and dropped the MDX
importpreamble from CLI Reference pages. The release build failed on all three
(74 prose pages 404'd; the CLI Reference discovery gate tripped). The extractors
now accept both layouts, so the corpus picks up the new CMR pages (RouterOS
7.26beta1) with no other entries lost. - CLI Reference extraction skips MikroTik's new CLI-compare tool page. On
2026-10-20 the site added/docs/cli-reference/compare-versions, listed in
llms.txtand in the sitemap (with a trailing slash). It documents no CLI
path, but discovery took it for a menu, which failed every live extraction.
It is now excluded by name, with the oldindexlanding page. Each excluded
page thatllms.txtlists is still fetched, and extraction fails if it ever
gains a**Type:**entry, so a page that becomes a real menu can't be
dropped silently. - Version ordering understands
alphaprereleases.compareVersionshandled
onlybetaandrc, so a label like7.26alpha1compared asNaNand kept
whatever position it arrived in. It now sortsalpha<beta<rc< release.
The CLI Reference extractor acceptsalphatree labels, so the export's tree
order depends on it. - Changelog extraction fails loudly instead of shipping an empty table (#231).
Fetches retry like the test-results scrape (#218); the run exits non-zero, with the
previous table kept, on 0 entries, over 10% fetch errors, over 15% missing versions,
or 5 consecutive errors. Previously a download.mikrotik.com outage wiped the table
and exited 0. - Device-benchmark extraction fails the release instead of shipping an empty
or gapped table (#218). A prerelease shipped 0device_test_results
rows after mikrotik.com 503'd the whole scrape, and nothing caught it:
routeros_search_testsanswered nothing while its description still claimed
thousands of rows. The extractor now retries 5xx, 429 (honoring Retry-After
up to 30s), network, and timeout failures — other statuses, including 403,
fail at once — sends an identifying User-Agent, distinguishes gone pages
(404) from host failures, and treats a 200 page with no benchmark rows and
no block diagram as a host failure rather than a quiet zero-row success. It
exits non-zero on an empty result, over 10% hard fetch errors, over 15%
missing pages, or a 15-minute global fetch deadline
(ROSETTA_EXTRACT_DEADLINE_MSoverrides it); a failed run leaves the
previous table untouched instead of wiped or partial. The release db-content
gate additionally floorsdevice_test_resultsat 2,000 rows, and the tool
description reads its counts from the served DB so the claim can't drift
again. - Release DB floors cover every shipped corpus (#231). The release content gate
now also fails on fewer than 80 block diagrams, 2,500 sections, 600 callouts, 4,000
changelog entries, 600 videos, 50 Dude pages, or 10 skills.make extractand
make extract-fullnow import the committed video cache, matching the release build. - Release smoke waits on the install metadata Bun reads, not
npm view
(#202).bunx-smokepolled the full packument, which can serve a
just-published version before the abbreviated install manifest Bun resolves
from does — twice the wait passed and the exact-versionbunxthen failed
with "No version matching". A newscripts/wait-for-registry.tsgate polls
the abbreviated manifest for the exact version plus a usable tarball
reference on the same 900s budget, and the--versionprobe now asserts the
exact executed version instead of a substring match. The upgrade path follows
its own contract (see #226). - The release upgrade smoke gates the channel tag and invokes the explicit-tag
form (#226).bunx-upgrade-smokewaited onnpm view(full packument) and
invoked the bare client form — but a warm barebunxserves its cache with zero
registry requests, so no metadata probe could gate it, and the full-packument wait
carried the same lag class #202 fixed on the fresh-install path. The wait now
requires the channel tag (latest/next) to resolve to the just-published version
with a servable tarball through the abbreviated install manifest Bun reads, and the
smoke invokes@latest/@nextwith no--refresh, still asserting the new-release
DB banner. Bare-form staleness is accepted, documented behavior (MANUAL "How Updates
Work"), not something CI gates. - Clarify first-run MCP pre-download with the same package version and DB path,
including the need to preserve pins and overrides when adapting setup snippets.
Correct obsolete release-planning, manual-version, and bare-bunxauto-update
guidance; a cached bare invocation need not see a just-published version. - Correct contributor setup docs that still described the retired Confluence-first
build: the default pipeline is live Docusaurus (make db-syncis the fast path),
rebuilds requiremake cleanfirst, and skill extraction needsGITHUB_TOKEN. - README install links point at the current manual, and MANUAL documents the CLI
Reference tables. The/appand Container links had 404'd since MikroTik's URL
change; they now go to/docs/containers/apps/and/docs/containers/. MANUAL's table
summary and logical schema reference gain thecliref_*tables (schema 12), and a
"Refreshing the product matrix" procedure replaces the stale "update the extractor
default" step.
Database Stats
- Pages: 372
- Commands: 5370 (command entries;
routeros_statsalso counts menus and arguments) - RouterOS versions with command data: 67
- Devices: 154
- Properties: 2516
- Videos: 746 (2374 transcript segments)
CLI Reference dataset
- Asset:
rosetta-cli-reference.tar.gz(export format 2, every stored manual tree) - sha256:
7a246bfb8d926521b9e45ea5d66d786b1174a812759dd6f9879d9ee8ecba7907(pin the tag and this digest)
Build Info
- Commit: 6b6397d
- Docs date: extracted live at build time
- Release version source: package.json
- Full version extraction: true
- Workflow run: https://github.com/tikoci/rosetta/actions/runs/41129834224