Let's build play.observal.io, a zero-setup way to feel Observal in 60 seconds #437
Apoorvgarg-creator
started this conversation in
Ideas
Replies: 1 comment
-
|
this seems nice |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hey folks 👋
Floating this idea for community input before I spike anything. TL;DR: a hosted, resettable playground at
play.observal.ioso people can understand Observal without running 8 Docker services on their laptop.The use case we're trying to kill
Today the "can I try this?" path looks like:
That's fine for someone who already wants Observal. It's brutal for someone who just saw a tweet and has 5 minutes. Most of them bounce before they ever see a trace, a scorecard, or an agent page.
Problem: we're losing people at "can I see it work?" before we ever get to "should I self-host it?"
What I want play.observal.io to be:
pip install observal && observal auth login --playground→ CLI just works, points at the playground, no server neededobserval ops traces, seeing a scorecard - end-to-end, in a browser tabThe constraint that makes this spicy
Observal is self-hosted by design. The whole point is that your keys, your traces, your agents live on your infrastructure. A playground contradicts that since we'd be running the one managed Observal instance that exists.
So the rule I'd like us to hold: zero forks, zero
if playground: ...branches in the core code. Anything we need should be configurable via env vars or feature flags we already have, or a thin wrapper service in front. If we start special-casing "playground mode" inside the API, it becomes a maintenance tax on every future PR.🌱 A small starting sketch (so we have something to pick apart)
Here's the simplest thing I think could work - please tear it apart:
1. One shared server, ephemeral per-visitor "workspace"
2. CLI
--playgroundflag issues a scoped demo keyobserval auth login --playground→ opens browser → backend mints a short-lived API key tied to that same ephemeral workspaceplay.observal.ioexactly like it would to a self-hosted server. No CLI code changes beyond reading a flag3. Pre-seeded fixtures, not real telemetry
playground-seed.yaml- 5 agents, 3 MCP servers, 2 sandboxes, a few hundred fake traces with realistic-looking spans and scorecards4. Eval engine gets a mock provider in playground mode only
EVAL_MODEL_PROVIDER=mock→ returns canned scorecards instead of hitting Bedrock/OpenAI5. Nightly reset via
docker compose down -v && up💸 Where to host without going broke
Rough cost napkin - open to better suggestions:
My gut: we can run this for <$15/mo if we're disciplined about retention and mocking the eval engine. The second we let real LLM calls through, costs become unbounded.
🔒 Keeping user data safe without touching core code
This is the part I want the most opinions on. Things I'd lean on:
/api/v1/admin/*entirely for public trafficDEPLOYMENT_MODE=localturned off, registration disabled, SSO off, bootstrap off. All accounts are minted server-side by the session provisionersandboxcomponent type stays disabled or heavily restrictedDATA_RETENTION_DAYS, just turn it down to 1If there's data worth leaking on
play.observal.io, we've already done something wrong upstream.❓ What I'd love the community to weigh in on
--playgroundor should we just document the URL? First is smoother UX, second is less couplingDrop thoughts below 👇 - I'd rather merge the community's ideas than ship my first draft.
Sources used while drafting:
Beta Was this translation helpful? Give feedback.
All reactions