From a sentence to a stakeholder-ready prototype. Eight stages, eight integrations, three minutes.
A Fortune 500 innovation manager has an idea. By the time they've corralled a brand strategist, a product designer, a 3D artist, a video editor, a research panel moderator, and a brief writer, the moment has passed. Their idea dies in committee.
Auctopus is the missing director. Drop in a brand and an idea. We compose the entire product journey — research, spec, mesh, reveal, audience, AR, press kit, distribution — into a single guided run. Eight tools running in concert behind one shareable link. Three minutes from a sentence to something a CEO can hold in AR on their iPhone.
The demo is Red Bull → Caffeinated Chewing Gum, but the platform is brand-agnostic. Type any company, any product idea, and Auctopus takes it the rest of the way.
AUCTOPUS · innovation-lab OS
┌──────────────────────────────────────────────────────────────────────┐
│ │
│ 01 Brand research 02 Idea mapping 03 3D model │
│ ───────────────── ───────────── ───────── │
│ Gemini Gemini Meshy │
│ reads docs + raw idea → text → │
│ web research structured spec GLB mesh │
│ │
│ ▼ ▼ ▼ │
│ │
│ 04 Reveal video 05 THE PANEL 06 AR Quick │
│ ───────────────── ───────────── ───────── │
│ Veo MiroShark Cloudinary │
│ one shot, live focus group USDZ + QR │
│ 6 seconds in 3D + Slack on iPhone │
│ │
│ ▼ ▼ ▼ │
│ │
│ 07 Press kit 08 Distribution 09 Campaign │
│ ───────────── ───────────── ───────── │
│ Cloudinary Composio The brief │
│ hero stills + Notion / Slack / one URL │
│ 360° spin Google Calendar ships it │
│ │
└──────────────────────────────────────────────────────────────────────┘
Each stage is a real API call. Nothing is mocked. Nothing is staged. The orchestrator streams progress to the browser via Server-Sent Events; the user watches each stage materialize in real time.
Eight integrations, six sponsor partners, one engine. We didn't pick these to check boxes — every choice is doing real work in the pipeline.
| # | Stage | Tool | What it does | Where it lives |
|---|---|---|---|---|
| 01 | Brand research | Gemini | Reads uploaded PDFs + web context, distills voice, palette, target segments, positioning. The fingerprint for every downstream stage. | pipeline/01-brand-research.ts |
| 02 | Idea mapping | Gemini | Turns "caffeinated gum" into a structured product spec — category, form factor, materials, dimensions, competitor map, rationale quote. | pipeline/02-idea-mapping.ts |
| 03 | 3D model | Meshy | Text-to-3D job → GLB mesh. Topology suitable for both <model-viewer> and AR Quick Look. |
pipeline/03-3d-model.ts |
| 04 | Reveal video | Veo | Six-second cinematic product reveal animated from the hero render. On-demand to keep cost predictable. | pipeline/04-video.ts |
| 05 | The Panel | MiroShark | Synthetic focus group — Neo4j-backed personas argue about your product live in a 3D room and mirror to a Slack channel. The centerpiece of the demo. | pipeline/05-simulation.ts |
| 06 | AR Quick Look | Cloudinary | GLB → USDZ conversion + QR code generation. iPhone scans, product appears on the conference table at real scale. | pipeline/06-ar-quick-look.ts |
| 07 | Press kit | Cloudinary | e_camera 3D transforms generate ten hero stills + a 360° spin video from one mesh. Drop straight into a deck. |
pipeline/07-cloudinary-upload.ts |
| 08 | Distribution | Composio | Files a Google Doc brief, posts a Slack ping in #sim-…, books a Google Calendar review with the team. |
pipeline/08-distribute.ts |
Plus the supporting cast:
| Layer | Tool | Doing what |
|---|---|---|
| LLM router | OpenRouter | One key powers every Gemini call + every MiroShark agent. Keeps the bill on one card. |
| Voice (optional) | ElevenLabs | Gives each Panel persona a distinct voice. Hash-derived voiceId per persona; gracefully off when the API key isn't set. |
| Outbound briefs | Pingram | Email + phone-call delivery of the final convergence report to a stakeholder. |
| Investor view | Backboard | Sponsor track — explored as a presentation surface for the press kit. |
| Code review | Polarity | Sponsor track — end-of-build evaluation pass. |
| Secrets | CyStack Locker | Prefetches API secrets from a vault before next dev boots. Keys never sit in plaintext alongside the code. |
| 3D reference | Claw3D | MIT-licensed inspiration for The Panel's multi-character scene. Pattern adopted: Billboard + Text nameplates, status-driven blinks, lerp-toward-target scaling. |
"The room is filling with synthetic consumers. Listen closely — objections are usually where the truth hides."
Stage 5 is what separates Auctopus from "a wrapper around a few APIs." The Panel is a live 3D focus group. Each persona generated by MiroShark appears as a stylized character in a half-circle around the product. As the swarm debates the idea, the matching character lights up, scales forward, captions appear, and (with ElevenLabs configured) the persona speaks in their own voice.
What's wired:
- Deterministic seating. The same persona handle always lands in the same seat. Stable across refresh, share-page revisits, and SSE reconnects.
- Per-disposition silhouettes. Skeptics get cone-topped heads, enthusiasts have a fluffy crown, pragmatists are clean spheres. You can read the room's stance distribution at a glance.
- One-by-one drip. New SSE messages arrive on a 700ms beat — no PowerPoint-style burst dumps. Live runs catch up to the tail; replays drip from the start.
- Mirrored to Slack. Every persona utterance also posts to a per-run
#sim-…channel via Composio withusername+icon_urloverrides. The audience can watch the focus group on their own laptop. - Convergence report. When the debate closes, a Gemini-distilled verdict appears: buy-intent percentage, headline pull-quote, top objections, unexpected wins, strongest segments, full memo on demand.
Open /run/[id], navigate to chapter 05, and watch eight to fifteen synthetic consumers tear into your idea in real time.
┌───────────────┐
│ Cloudinary │
│ Meshy · Veo │
│ ElevenLabs │
└───────▲───────┘
│
┌───────────────────┐ POST /api/pipeline ┌─────────────┴─────────┐
│ Landing page │ ─────────────────────▶ │ Orchestrator │
│ ComposeFirstIdea│ │ (linear async chain │
└─────────┬─────────┘ │ over 8 stages) │
│ └─────────┬──────────────┘
│ runId │
▼ │ stage events
┌───────────────────┐ SSE: /stream ┌──────────┴──────────────┐
│ /run/[id] │ ◀─────────────────── │ PipelineEvent emitter │
│ Chapters deck │ │ (run store + sse.ts) │
└─────────┬─────────┘ └──────────┬──────────────┘
│ │
│ usePanelStream + SSE │
▼ ▼
┌───────────────────┐ ┌──────────────────┐
│ The Panel │ │ MiroShark │
│ (3D + captions │ ◀─ persona events ─ │ (Docker, Neo4j │
│ + voice) │ │ + OpenRouter) │
└───────────────────┘ └──────────────────┘
│
│ persona Slack relay
▼
┌──────────────────┐
│ Composio │
│ #sim-<runId> │
└──────────────────┘
- One Next.js 15 app at the root. App Router, TypeScript, single-app-at-root (no monorepo). React 19. Tailwind for layout, drei + R3F for 3D scenes, three.js underneath.
- A real fire-and-forget orchestrator. Each stage is an
async function run(input, ctx)insrc/pipeline/0X-*.ts. Inputs and outputs arezod-validated. Failures throw a typedPipelineErrorwith stage attribution and an actionable hint that surfaces in the UI. - State streamed live via SSE. No polling treadmill. The browser opens an
EventSourceto/api/pipeline/<runId>/streamand receivesstage:start/stage:progress/stage:complete/stage:error/sim:message/pipeline:completeevents. The Panel's transcript merges live SSE messages with the persisted server copy via a stable composite-key dedupe so reconnects don't double-add. - Run state persisted to disk at
.cache/runs/<runId>.jsonwith atomic writes. Refresh, kill the dev server, come back the next morning — the share URL still works. - Manual stage-to-stage navigation. The pipeline runs ahead in the background; the user controls when they advance between chapters via keyboard arrows, the rail, or touch swipe. Stage 5 completing does not auto-advance to Stage 6. The "auto" you see is the intro reveal within a chapter, never between chapters.
- Reduced motion respected. Every keyframe in
globals.cssis gated byprefers-reduced-motion: reduce. Animation-heavy polish (3D scenes, ambient drift, ken burns zoom) drops to zero-duration; textual polish (typing indicators, finale checklist) stays visible.
# 1. Clone, install
git clone https://github.com/Jeremyliu-621/soon.git
cd soon
npm install
# 2. Configure
cp .env.example .env.local
# Fill in: GOOGLE_GENERATIVE_AI_API_KEY, MESHY_API_KEY, CLOUDINARY_*,
# COMPOSIO_API_KEY, OPENROUTER_API_KEY, ELEVENLABS_API_KEY (optional),
# LOCKER_ACCESS_KEY_ID + LOCKER_SECRET_ACCESS_KEY (optional fallback vault)
# 3. Start MiroShark — the audience-simulation engine
cd services/miroshark
cp .env.example .env
# Paste your OpenRouter key into the LLM_API_KEY / SMART_API_KEY /
# NER_API_KEY / OPENAI_API_KEY / EMBEDDING_API_KEY slots (same key, 5 places)
docker compose up -d
# 4. Back to the root and boot the app
cd ../..
npm run dev
# → http://localhost:3000predev runs scripts/locker-prefetch.mjs (CyStack vault → .env.local fallback) and then scripts/check-env.mjs (fail-fast key validation).
For the live demo: open http://localhost:3000, click Try it on Red Bull, and watch the eight stages roll. Use arrow keys to step through chapters, Home/End to jump to the beginning or end, Space to advance.
src/
├── app/ Next.js App Router
│ ├── page.tsx Landing — hero, pipeline, intake, sponsor strip
│ ├── api/
│ │ ├── pipeline/ Run creation + SSE stream
│ │ ├── share/ Public run dossier JSON
│ │ ├── panel/voice/ ElevenLabs TTS proxy (optional)
│ │ └── pingram/ Outbound email + phone briefs
│ ├── run/[id]/ Live run experience (renders Chapters)
│ └── share/[id]/ Final dossier (renders Chapters)
│ ├── Chapters.tsx Master orchestrator — SSE + posters + nav
│ ├── PersonaPanelScene.tsx 3D Panel scene
│ ├── Crowd.tsx Persona crowd visual
│ └── … One poster file per stage
├── components/ React UI
│ ├── BrandIntake.tsx "Compose your first idea" form
│ ├── BuiltWithSection.tsx Sponsor / partner credits
│ ├── Pipeline.tsx Landing pipeline strip
│ ├── mascot/ Aucto, your lab guide
│ └── arena3d/ Walking-crowd 3D scene
├── pipeline/ Numbered stage modules (01-…-08)
├── integrations/ One folder per vendor
│ ├── gemini/, meshy/, veo/, cloudinary/, composio/, miroshark/,
│ ├── elevenlabs/, pingram/, backboard/
├── lib/ Types, store, sse, orchestrator, parsers
└── prompts/ Prompts as .md files
services/miroshark/ Forked MiroShark — Docker, Neo4j, OpenRouter
public/mascot/ Mascot pose PNGs (Aucto in seven states)
research/ Sponsor-tool deep research (read before integrating)
docs/ Architecture, pipeline, demo runbook, slack bridge
Meet Aucto — the lab assistant, brand whisperer, and product strategist who walks you through each chapter. First-person voice, slightly mysterious, never says "processing…". Eight CSS-driven tentacle tips drift around the avatar (because of course an Auctopus mascot has tentacles).
- Chapter-aware dialogue (running / complete / failed states for all eight stages, plus prologue and finale)
- Collapsible bubble that respects your reading flow
- Localstorage persistence for the collapse state
- Status-tinted halo that pulses with the active stage
- Reduced-motion users get a static orb with the same dialogue
Edit src/lib/mascotDialogue.ts to retune Aucto's tone before a demo.
Six commits, twenty integrations, one ambitious deadline.
| Who | Shipped | |
|---|---|---|
| Jeremy Liu | The pipeline scaffold, MiroShark wiring, Composio Slack + Docs distribution, the Story Mode → Chapters refactor that became the new UI. The orchestrator is his fingerprint. | |
| Chloe Houvardas | The Panel as a 3D focus group. Persona avatars, the Crowd scene, the brief intake → draft → run flow, Veo image-to-video, structured convergence reports. The audience-simulation showpiece is hers. | |
| Darsh Shah | Story Mode UX. The launch dossier. Auctopus rebrand from soon → Palette → Auctopus. The mascot system. Live-update fixes for The Panel after a major UI rebase. The chapter-deck transitions and demo polish. | |
| Caellum Yip Hoi-Lee | Backend hardening — error model, run-state persistence, Composio v3 migration, MiroShark hybrid mode. The "doesn't crash mid-demo" person. |
Plus a code-quality review pass from Polarity Paragon (the bot, but it caught real bugs).
The branches tell the story: main is the production-shaped trunk, palette-rename was the deep-feature spike, panel was the 3D Panel R&D, reworkPanel is the live demo target. We left old branches in place because hackathon git history is the changelog.
- Polarity Paragon — code-quality pass beyond the one already merged
- Multi-tenant deploy — single-user demo for now
- Auth — single-user demo for now
- Database — disk-persistent JSON store; the
store.tsinterface was designed for a five-line swap to Vercel KV / Supabase / Redis when this becomes a real product
TBD — pre-submission. All rights reserved by the project authors.
Built for the Aucctus AI Hackathon · May 2026
Drop in a brand. Walk out with a launch packet.
auctopus.demo