Skip to content

OpenThumb 1.1.0

Choose a tag to compare

@github-actions github-actions released this 29 Jul 18:23

Use 1.1.1 instead. This build binds the abstract socket
native-offload, which is device-global rather than per-app, so if an
earlier OpenThumb is still installed — which the note below tells you to
expect — whichever starts first claims the name and the other dies in
Application.onCreate before drawing anything. Fixed in 1.1.1 by naming
the socket after the application id. On a device with no earlier install
this build runs, but there is no reason to pick it over 1.1.1.

The application ID changes to com.fug.openthumb, so 1.1.0 installs beside
1.0.x rather than over it.
Android treats a different application ID as a
different app entirely. Uninstall the old one after checking anything you want
to keep — settings and provider credentials do not carry across. Doing this now
is deliberate: the ID is fixed forever once the app is on Google Play, and the
product is published under FUG. The app is still called OpenThumb.

Added

  • openthumb-fetch in the sandbox — fetch a page and be told whether what came
    back is actually the page. A WAF challenge, a consent wall and a bot
    interstitial all arrive as HTTP 200 with an HTML body, and an agent that
    trusts the status code summarises the interstitial as if it were the article.
    Seven checks decide: status semantics, structural WAF markers, caller-supplied
    stub sizes, JSON awareness, CSS selectors as positive proof, soft markers with
    the Akamai sensor cookie, and content density.
  • A failed fetch never returns a bare failure. It names untried_routes — the
    escalation steps the fetcher structurally could not take — so "the site blocks
    us" cannot be reported while the phone's own WebView, which holds the real TLS
    stack and cookie jar, has not been tried. 429 is classified as transient
    rather than a wall, which is the single most common cause of a premature
    give-up. An empty list is the permission to fail honestly.
  • The ladder is inverted from the upstream desktop tool: here the browser is the
    first escalation and cheap HTTP is the probe under it, because on a phone the
    WebView is the real client.
  • The fetcher returns readable text, not markup. A measured Threads profile
    is 861KB of HTML carrying 1.6KB of words — 516x. Returning the raw body would
    spend an on-device model's whole context on one page, and truncating instead
    drops whatever was past the cut, usually the article. Three sources compete
    and the most complete wins: schema.org articleBody (which carries the text
    even when the visible DOM is a JavaScript shell), the <article>/<main>
    block with navigation and footers removed, or all visible text. --raw still
    returns the HTML for the cases that need it.
  • A fetch interrupted by Doze resumes instead of restarting. Android
    suspends background work, and a fetch that walks several URL variants is long
    enough to be killed partway — with no record of what was already tried, the
    next run starts over and dies at the same place forever. Each attempt is now
    appended to a journal as it finishes, and a resumed run skips what the journal
    covers. Measured: 2.68s for a first pass, 0.14s to resume it. The journal
    expires after ten minutes so a lifted block is retried, and --forget starts
    clean.
  • Cookies are kept per host, so clearing a challenge once pays off. Hand the
    WebView's cookies in with --cookies after browser_use gets past a
    challenge and later fetches to that host start already solved. Storage is
    keyed by host and stores the host inside the file as well, so a cookie can
    never be attached to a request for a different site; the files are 0600
    because session cookies are credentials.
  • It degrades honestly when it cannot start. The Alpine rootfs ships 15
    packages and python3 is not one of them, and on a Galaxy Note8 apk cannot
    install it — network is fine (busybox wget fetches the same mirror over
    HTTPS), but apk's own fetch fails with IO ERROR over http and https alike and
    with both IPv6 and IPv4 nameservers. The wrapper tries apk add, and when
    that fails prints a JSON object saying so and escalating to browser_use,
    stating outright that this is a runtime problem on the phone rather than the
    site refusing the request. Verified on the device: valid JSON, exit 2. The
    same limitation applies to the inherited minis-mcp-cli, which installs
    itself the same way.
  • python3 is baked into the rootfs at build time, since it cannot be added
    later. scripts/rootfs_add_packages.py resolves an Alpine package's full
    dependency closure and unpacks it into the image using nothing but curl and
    the standard library — not apk, which is a Linux binary a macOS build host
    cannot run, and not Docker, which would make the shipped image depend on
    whether a daemon happened to be running. Measured: 3.7MB to 14.6MB for
    python3 3.12.13 and its 17 dependencies, taking the APK from 33MB to about
    44MB. Verified on a Note8: python3 runs, all 69 checks pass on the phone, and
    openthumb-fetch https://example.com returns the page with the text
    extracted. Details in docs/sandbox-python.md.
  • Ported from insane-search (MIT, github.com/fivetaku/insane-search). Pure
    standard library — CSS selectors are matched with html.parser rather than
    BeautifulSoup so nothing has to be installed on the phone. curl_cffi is used
    for TLS impersonation when importable; its absence costs capability, never
    correctness. scripts/test-fetch.sh runs 69 offline checks, wired into CI
    ahead of the build so a broken verdict fails in seconds rather than after the
    NDK compile.