Skip to content

v0.1.0 — initial release

Latest

Choose a tag to compare

@amitkssolanki amitkssolanki released this 17 Aug 19:01
· 1 commit to main since this release

Initial development. M1 (foundation) — SpreeMenuChat::Credential (encrypted per-store Gemini +
Voyage AI API keys), SpreeMenuChat::LlmClient (Gemini generateContent REST calls — no official
Ruby SDK exists for the Gemini API, same "plain REST" situation spree_doordash already handles
for DoorDash's Drive API), and a plain credential-entry admin form.

One real bug caught only by spree_menu_chat:verify_connection against a genuinely fresh Gemini
API key, not by the spec suite (which stubs the request and never touches Google's model
availability): the pinned model gemini-2.5-flash-lite returned a real 404 "no longer available to new users" — despite still being listed by the models.list endpoint, which apparently doesn't
reflect real per-account availability. Fixed by switching the default generation_model from a
pinned dated model string to the gemini-flash-lite-latest alias (currently resolves to
gemini-3.5-flash-lite, confirmed live), which should avoid this exact rot going forward.

M2 (pgvector + embeddings pipeline) — SpreeMenuChat::Embedding (polymorphic, has_neighbors),
SpreeMenuChat::EmbeddingClient (Voyage embeddings REST calls), ContentBuilder, Embedder,
ReembedProductJob/ReembedCatalogJob, the Spree::Product re-embed hook, and
spree_menu_chat:reembed_all.

Three real bugs found only by running reembed_all against spree_host's actual, live,
Square-synced catalog (39 products, 4 policies) — none of them visible to this gem's own isolated,
SQLite-backed dummy-app spec suite:

  • neighbor's t.vector migration helper raised NoMethodError — it's only a gemspec
    (transitive) dependency of this gem, and Bundler.require doesn't auto-require those. Fixed with
    an explicit require 'neighbor' in lib/spree_menu_chat.rb.
  • SpreeMenuChat::EmbeddingClient::RequestError raised NameError: uninitialized constant under
    rails runner's lazy autoloading, despite passing every spec (Zeitwerk's eager_load_all, which
    spec/zeitwerk_spec.rb exercises, loads every file regardless of order and masked the bug). Root
    cause: it was defined as a second class inside llm_client.rb instead of its own file. Fixed by
    giving it its own app/services/spree_menu_chat/request_error.rb.
  • ContentBuilder#modifier_summary called SpreeSquare::Modifier#display_price, a method that
    doesn't exist on that model (only a JSON key of the same name in SpreeSquare::ProductSerializer)
    — raised NoMethodError on every product with a real Square modifier list attached. Invisible to
    this gem's own specs because the dummy app never has spree_square installed at all (the
    defined? soft-dependency guard short-circuits before reaching this code). Fixed to call the
    model's real #price method instead, formatted as a plain +$X.XX string; added a regression
    spec that stubs the SpreeSquare constants so this exact path is exercised going forward without
    taking a hard gemspec dependency on spree_square.

Also discovered live, not a code bug but a real operating constraint worth documenting plainly:
Voyage AI accounts with no payment method on file are capped at 3 requests/minute, 10K
tokens/minute — separate from, and far tighter than, the genuinely generous 200M free-token
allowance. A tight per-record loop over even a ~40-item menu hits this within the first few calls.
EmbeddingClient now self-throttles every request to that limit (tracking this process's own
recent request timestamps, sleeping proactively before the next call would exceed it) rather than
firing calls as fast as the caller loops and hoping ReembedCatalogJob's job-level retry_on
eventually catches up — see the class comment on SpreeMenuChat::EmbeddingClient for the full
reasoning, including why an earlier reactive-retry-only version of this fix undershot in practice.

M3 (retrieval + generation, no streaming) — SpreeMenuChat::Retriever (embeds the question with
Voyage's query input type, pgvector cosine-distance search scoped to the store, filters below the
configured similarity floor, caps at retrieval_top_k), SpreeMenuChat::AnswerGenerator (builds a
menu/FAQ-scoped system prompt, injects retrieved context, calls Gemini; returns a canned
"contact us" fallback — using the store's real customer_support_email/contact_phone — whenever
retrieval finds nothing above the floor or Gemini fails, rather than a raw error or a hallucinated
answer), and SpreeMenuChat::ChatController (POST /menu_chat/chat, authenticated with the store's
Storefront publishable key — the same pk_... key type spree_api's Store API accepts).

The read-only guardrail is enforced structurally, not just described in the system prompt:
AnswerGenerator never builds or passes a tools/function-declarations argument to
LlmClient#generate, and no write-capable Spree model is referenced anywhere in the request path —
covered by a spec that inspects the real outgoing Gemini request body for the absence of a tools
key.

One real calibration bug, again only found by curling real questions against the real synced
catalog: the originally-planned similarity_floor default of 0.7 (max cosine distance 0.3) was
picked speculatively, before ever querying real embeddings, and was badly miscalibrated. Real
Voyage cosine distances for genuinely correct, on-topic matches ranged 0.32-0.63 across a set of
real test questions — a 0.7 floor rejected nearly all of them, meaning the assistant would have
given the canned fallback for almost every real, answerable question ("what comes with the buffalo
wings?" initially returned "I don't have information about that," despite Buffalo Wings being a
real menu item with a real matching description). Fixed by lowering the default to 0.35, based on
that live calibration data; a genuinely off-topic/unanswerable question in the same test batch
landed at 0.76 distance, comfortably excluded with margin to spare. See the class comment on
SpreeMenuChat::Configuration for the full data.

Manually verified against the real dev app: real curl'd questions ("what comes with the buffalo
wings?", "do you have vegetarian options?", "is the chicken parmesan spicy?") return accurate,
grounded answers; a request to place an order is correctly refused while still answering the
underlying menu question; a genuinely off-topic question ("what is the capital of France?")
triggers the no-context fallback rather than a hallucinated answer.

M4 (streaming) — SpreeMenuChat::LlmClient#generate_stream (Gemini's streamGenerateContent?alt=sse
endpoint, via Faraday's on_data streaming callback), SpreeMenuChat::AnswerGenerator.stream (yields
each real fragment as Gemini produces it, or the same canned fallback as a single fragment if there's
no context/generation fails), and SpreeMenuChat::ChatController now streams SSE
(ActionController::Live) instead of returning one JSON blob — the storefront widget's Next.js proxy
streams the response straight through unmodified.

One serious real bug, found only by curling the live streaming endpoint against the real Gemini
API
— every spec passed and it looked completely broken in practice. LlmClient#generate_stream's
SSE frame parser searched for a literal "\n\n" to split data: <json> frames — matching this gem's
own hand-written WebMock fixtures (which used bare \n, the natural-looking assumption) but not
Gemini's real wire format, which uses \r\n line endings. The parser never found a match, so every
real streaming request silently yielded zero chunks and zero errors — no exception, nothing logged —
and AnswerGenerator.stream's "Gemini never yielded any text" guard quietly substituted the canned
"I don't have that" fallback for every single real question, including ones with a perfect context
match. Because the WebMock fixtures shared the same wrong assumption as the code under test, the spec
suite was self-consistently green throughout — this is exactly the failure mode this project's
"verify against real infrastructure" standard exists to catch, and did. Fixed by normalizing \r\n to
\n before splitting; the fixtures were also corrected to use \r\n so they now test the real wire
format instead of re-testing the same wrong assumption.

Verified: 75/75 specs green, Brakeman clean. Live-verified end to end after the fix: curl -N against
the real streaming endpoint for "what desserts do you have?" (previously silently falling back)
correctly streams three real SSE fragments naming actual real menu items, followed by a done event;
a genuinely off-topic question still correctly streams the no-context fallback as a single fragment.

M5 (guardrails) — SpreeMenuChat::RateLimiter (Postgres-backed fixed-window counter,
rate_limit_per_hour requests per (store, IP) — no Redis anywhere in this stack, same rationale as
Solid Queue running inside Postgres) and SpreeMenuChat::TokenBudget (Postgres-backed daily cap on
Gemini generation tokens per store, using Gemini's own reported usageMetadata.totalTokenCount, not
an estimate). ChatController checks the rate limit before opening the SSE stream (a real 429 JSON
error, not a raw exception or a broken stream); AnswerGenerator checks the token budget before
even retrieval, and skips straight to a distinct "we've reached today's limit" fallback once
exhausted, same as the existing no-context fallback but honestly worded for this different reason.
LlmClient#generate_stream now returns Gemini's final reported token total (each SSE frame's
usageMetadata carries a running total; the last one seen is the true total for the turn) so
AnswerGenerator can record real spend without estimating.

Deliberately scoped to generation tokens only, not embedding tokens — see the class comment on
SpreeMenuChat::TokenBudget for why folding in Voyage's query-embedding cost too was left as a
follow-up rather than blocking this guardrail.

Verified live against the real dev app, not just specs: a real chat question recorded its real
Gemini-reported token count (393 tokens) against today's budget; simulating the rate limit already
being at its configured cap made the next real request return a genuine 429 with a plain JSON
error body (confirmed Content-Type: application/json, no SSE stream opened) instead of a raw
error; simulating the daily token budget already being exhausted made the next real request stream
the budget-exceeded fallback as a single SSE fragment with zero further Gemini spend (confirmed the
stored token total was unchanged after). Also grepped the full app//lib/ tree to confirm neither
a write-capable Spree model (Spree::Order, Spree::LineItem, Spree::Cart, ...) nor a
tools/function-declarations argument is referenced anywhere outside this file's own comments —
the read-only guardrail described since M3 is still structurally true, not just documented.

M6 (admin visibility) — confirmed with the user before building: persist real conversation content
(not just operational counters), since that's a data-retention decision the plan explicitly flagged
rather than a default to build silently. SpreeMenuChat::Conversation/SpreeMenuChat::Message
(anonymous, grouped by a 30-minute time-window heuristic on (store, IP) rather than an explicit
client-sent session id — the widget has no session of its own, see Conversation::SESSION_WINDOW's
comment for the real tradeoff this makes), logged by ChatController after each SSE stream closes
(best-effort — a logging failure is captured, never allowed to affect the response already sent).
Admin nav position 72 (right after the M1 credential page at 71) and a read-only
Spree::Admin::MenuChatConversationsController index — identifier, first question, message count,
started/last-message timestamps — scoped to the current store via a for_store scope
(Admin::ResourceController#scope picks this up automatically, no controller override needed).

new_resource: false set on the table registration up front, not found the hard way a third time —
both spree_square and spree_doordash hit and fixed the identical real bug (new_admin_..._url
routing error on an empty read-only, index-only table) on their own first read-only admin pages.

Verified against the real dev app, not the gem's own SQLite dummy-app suite (M6's own stated
standard, and precisely where both sibling extensions' changelogs record real Postgres-only bugs
invisible to specs): logged into the real admin, loaded the Conversations page while genuinely
empty
and confirmed the empty state renders cleanly (no routing error); sent a real chat question
through the live endpoint, reloaded, and confirmed the real row appeared with the real question text,
a correct message count of 2, and correct relative timestamps; confirmed clicking the row does
nothing (no :show route exists, matching the index-only design) rather than erroring.

Two real gaps found immediately after, by the user testing the shipped feature live rather than by
a spec:

  • M6 shipped index-only, matching the sibling extensions' own read-only admin tables — but that
    left a conversation genuinely unreadable beyond an 80-char truncated first-question preview.
    Added a real :show route/page (link_to_action: :show on the table registration — the
    framework defaults to :edit, which doesn't exist here, so rows silently rendered as plain
    unclickable text instead of erroring) showing the full ordered message thread.
  • Gemini's real answers regularly come back with markdown (bold ingredient names, bullet lists of
    menu items — see the real examples above), but the admin conversation view showed that as literal
    asterisks. Added SpreeMenuChat::Message#rendered_content (Kramdown, sanitized after conversion —
    content ultimately derives from the customer's question and retrieved catalog/policy text,
    neither of which this extension trusts blindly elsewhere, so unsanitized HTML output could let a
    crafted message smuggle a <script>/<img onerror> tag into the admin's browser). Required
    adding kramdown as a real gemspec dependency, and — the same gotcha neighbor already taught
    this gem in M2 — an explicit require 'kramdown' in lib/spree_menu_chat.rb, since a
    gemspec-only dependency isn't auto-required by Bundler.require.

Verified: a spec confirms sanitization actually strips a real <script> tag, not just that it
doesn't error. Live-verified in the real admin: clicked into a real 8-message conversation from
live testing, confirmed every message renders in order with correct roles and timestamps, and that
bold now renders as real bold text instead of literal asterisks.

117/117 specs green, Brakeman clean.