Skip to content

EYAS v0.8.17-beta — Hands that make things

Choose a tag to compare

View all tags
@eyssen eyssen released this 29 Aug 21:14
· 1 commit to main since this release
v0.8.17-beta
bf5a6bc

An agent that can remember still cannot show you the thing. This wave is the
hands: generate a still or a clip through a vendor you already pay, render a
title card on this machine, cut footage from a transcript, and fill a form in
a browser that is not the one you use every day.

Media is SaaS prompt-to-pixel. Studio is local production. The browser is
EYAS's own Chromium, plus optional sidecars when the work needs the Chrome
you already logged into. Recordly records the screen; it is AGPL, so it is a
catalogue card, never a bundled engine.

Every lane fail-closes with a remedy. Missing Node, missing FFmpeg, missing
CLI, missing Chromium: the tool says so. None of them silently disable the
sandbox. None of them vendor a third-party LLM. The model stays EYAS's.

Media is SaaS, and none of it is default

  • Agents generate, upscale, and wait through five media_* tools.
    media_catalog, media_generate, media_wait, media_balance,
    media_history. The vendor is a routing choice, not a tool dump.
  • Magnific, Higgsfield, and fal are optional backends. None is default;
    several can run at once. Zero connected providers is an empty, fail-closed
    state — never mock pixels. Magnific and Higgsfield sign in with OAuth; fal
    takes an API key. The Media page and the handbook compare them on the
    criteria that actually matter here: strength, sign-in, credits, and file
    lifetime.
  • Completed files land in Documents and on the producing turn. Vendor CDN
    URLs expire — Higgsfield's in about seven days — so ingest copies the bytes
    locally (up to 200 MB, no JPEG recompress) rather than leaving a link that
    will 404.
  • Raw vendor MCP tools stay off. Turning them on dumps mcp_magnific_* /
    mcp_higgsfield_* / mcp_fal_* onto the agent and skips ingest. Leave them
    off unless you are debugging.
  • Routing is per kind, with an optional budget. Default / fallback cover
    an outage; "also run on" fans the same prompt out and doubles credits.
    Daily and monthly caps fail before the vendor is called.

Studio is local production

  • Studio is a new category (Content → Studio, /studio), not Media.
    Local engines live here; Media stays hosted prompt-to-pixel.
  • First engine: Hyperframes. The agent authors HTML compositions and
    renders deterministic MP4s through six hyperframes_* tools. Node.js 22+,
    FFmpeg, and the Hyperframes CLI are required; missing any of them fail
    closed with a remedy. Hyperframes downloads its own chrome-headless-shell
    — never EYAS_CHROMIUM_PATH, never --no-sandbox. Output lands in
    Documents and on the producing turn. The CLI is Apache-2.0 and is not
    vendored.
  • Second engine: Video Use. Transcript-first footage cuts (videouse_*).
    EYAS reimplements the open-source hard rules in TypeScript (MIT) rather
    than vendoring librosa or Manim. FFmpeg on this machine; ElevenLabs Scribe
    is optional for transcription. Confirm a cut strategy before writing
    ranges. Overlays can be Hyperframes renders. Not Media, not Hyperframes.

Recordly is a companion, not a product

  • Recordly is an AGPL desktop screen recorder. It is not a Studio engine
    and it is not bundled. Catalogue card: Extensions → Third-Party
    (recordly). Manual install only (GitHub / Setup guide);
    POST /extensions/recordly/install is refused. Export MP4/GIF in Recordly,
    then attach in Documents. Skill: config/skills/integrations/recordly.md.

A browser of its own

  • Headless browser_* tools share the design-print Chromium. Numbered
    interactive indexes from browser_snapshot (click/fill by index; CSS is
    the fallback). Indexes and snapshotId die on navigation — snapshot
    again. Same SSRF, same 5-minute process.
  • The session is a real browser, not a one-shot page. Tabs, back, wait,
    hover, select, dialog, file upload, page evaluate, download → Documents,
    Playwright storageState, and an EYAS-owned userDataDir
    (data/browser/profile / EYAS_BROWSER_USER_DATA_DIR). The daily Chrome
    profile is rejected first (Chrome 136+ blocks Default-profile CDP).
  • Action cache without Stagehand. A successful browser_click /
    browser_fill with intent stores a durable CSS/role locator in vault
    JSON (projects/<id>/ or procedural/browser-action-cache.json).
    browser_replay reuses it on the same origin without an LLM or a snapshot
    index. Fill values and TOTP seeds are never cached.
  • browser_totp (yellow) reads the seed from Secrets or macOS Keychain
    and returns only the 6-digit code for browser_fill. The seed never
    leaves that call.
  • Snapshot and locator scripts run as IIFEs. Playwright evaluate of a
    string does not call () => sources, so indexes and cached locators would
    otherwise stay empty. That was a live miss, not a design choice.

Sidecars for the Chrome you already logged into

  • Agent Browser is the recommended sidecar (Vercel, Apache-2.0).
    EYAS_AGENT_BROWSER_BIN → PATH, fail-closed doctor
    (doctor --offline --quick --json). Tools agent_browser_status /
    agent_browser_run (argv or batch JSON, @e1 refs). MCP catalog
    agent-browser mcp --tools core,state → mcp_agent_browser_*. EYAS-owned
    --profile (data/browser/agent-browser/profile). Daily Chrome /
    --profile Default / --auto-connect / chat / --tools all refused.
    Rust is not vendored. AI_GATEWAY_* is stripped on spawn. The LLM stays
    the EYAS model module.
  • Python Browser Use remains as a legacy sidecar. Extra module, MIT CLI
    wrapper, telemetry off, Cloud API key stripped unless turned on.
    browser_use_status / browser_use_exec. UI /browser-use. Prefer Agent
    Browser when that card is Ready.
  • Playwright MCP is a Connections catalog row (playwright-mcp) plus
    MCP catalog sidecar (npx @playwright/mcp@latest --isolated). Agent tools
    arrive through the existing MCP bridge (mcp_playwright_*). Doctor is
    fail-closed (Node 18+, npx). Telemetry off. --no-sandbox is stripped and
    refused. The Python browser-use MCP is rejected (it wants an LLM key and
    retry_with_browser_use_agent). Live tab: Playwright MCP Bridge extension
    (--extension).
  • Chrome DevTools MCP (Google, Apache-2.0) is a separate coding/debug
    lane — Connections type chrome-devtools-mcp plus MCP catalog
    (npx chrome-devtools-mcp@latest --isolated). Console, network,
    Lighthouse, WebMCP. Not form-filling (browser_* stays the form
    lane). Tools arrive as mcp_chrome-devtools_*. WebMCP
    (list_webmcp_tools / execute_webmcp_tool) only if the sidecar
    advertises them. --autoConnect and the daily Chrome profile refused.
    --no-sandbox stripped. Doctor fail-closed.

MCP that can reach a hosted server

  • The MCP client speaks Streamable HTTP and OAuth. Hosted creative
    servers connect without a custom adapter per vendor. That is what made
    Magnific and Higgsfield possible as Media backends rather than one-off
    integrations.

The handbook

  • The Starlight docs are rewritten around a first-hour path, with a
    purpose opening on every chapter, the missing admin surfaces filled in,
    and in-app ? help wired on every remaining product page. Six languages.

A skill proposal can turn the skill off

  • The third button is global. "Not this time" still only covers this
    conversation. "Turn it off" declines here and disables the skill, so it
    will not match again until someone turns it back on in Skills. The turn
    then resumes the same way as a decline. Only owner and admin see the
    button — a user who can talk but cannot manage skills still has yes and
    no.

Fixed along the way

  • The kanban context stripe used the conversation's lifetime token total
    against a hardcoded 128k window.     Opening the same Grok card painted
    green (composed size / 500k) while the board painted red (cumulative /
    128k). Occupancy inputs now come from one function
    (loadConversationContext): latest composition estimated_tokens over
    the model's real window. The board, the conversation GET, and the
    end-of-turn frame all attach those fields; both stripes only paint them.
    A card with no composition stays blank rather than inventing a reading
    from tokensUsed.
  • The open chat overflowed the viewport by about a centimetre. The page
    sized itself with 100vh minus the top bar and ignored the status bar
    plus leftover main padding. New messages then called scrollIntoView,
    which scrolled every overflow ancestor and hid the header. The pane now
    fills the chrome remainder, and only the message list scrolls.
  • The template picker is opaque. It already sits above the page (the
    header is a stacking context), but it was still a glass-card — 3%
    white in dark mode — so the page bled through the names. It now uses
    the same solid popover surface as the notification panel and the
    user menu.

Known issues

  • The skill matcher still scores badly. The third button means a bad
    match now costs a click and then stops being offered, but the scoring
    itself is untouched — google-drive-integration can still light up for
    "what time is it".
  • Action-cache locators are CSS/role, not visual. A redesigned page on
    the same origin will miss. That is the Stagehand idea without the
    library, and a restyle is a cache miss rather than a silent click on
    the wrong control.
  • No sidecar binary is in the tree. Agent Browser, Playwright MCP,
    Chrome DevTools MCP, Hyperframes CLI, FFmpeg, and the Chromium used by
    browser_* are resolved or they say they are missing. A VPS without
    them is a working EYAS that cannot click or render until they are
    installed.
Assets 2
Loading