The kirana ledger that listens to the beep and to you.
A Paytm Soundbox announces every payment and then forgets it. Vyapar Vaani sits behind that beep and turns it into bookkeeping. Cash sales, which never reach the Soundbox at all, can simply be spoken out loud in Hindi, Telugu or English.
Built by Code Vyapar for the Build for India AI Hackathon, hosted by Paytm, Hyderabad.
- The Problem
- What It Does
- The Walkthrough
- The 90 Second Demo
- How a Sale Flows
- Tech Stack
- Getting Started
- Environment Variables
- Scripts
- Project Structure
- Data Model
- Design System
- What Is Real and What Is Simulated
- Resilience
- Known Limits
- Path to Production
- Credits
The person this is built for is not running a business on a laptop.
A kirana owner shuts the shutter at ten at night and rebuilds the whole day from memory, a cash box and a pile of loose notes. Stock gets counted when something runs out, which is always one customer too late. The Soundbox on the counter announced every single digital payment that day and did nothing with any of them.
Two facts shape the entire design:
- A payment knows an amount, not an item. A static UPI QR lets the customer type any figure, so the webhook carries a number and a timestamp. In this catalogue, milk and Tata Salt are both priced at twenty eight rupees. The beep alone can never tell you what was sold.
- Most kirana sales are still cash. Those never touch the Soundbox in the first place.
So the product takes input from two directions. The beep says how much. The owner says what.
| Capability | Behaviour |
|---|---|
| Sale capture | A payment books the sale, decrements stock and splits the day's takings by payment mode, all in one atomic database transaction |
| Voice ledger | A hands free conversation. Press once, speak, and the mic closes itself on your pause, books the sale, replies out loud with the amount, then reopens for the next one |
| Spoken answers | Ask the ledger a question in the same conversation and hear the answer back in the language you asked it |
| Live dashboard | Stock counts and takings move on Postgres change events, not polling. No refresh, and every figure that changes animates |
| Customer messaging | A thank you is written per customer in their own language and queued to WhatsApp, ready to send in one tap |
| Supplier reorders | When stock crosses its reorder point a supplier message drafts itself and waits for human approval |
| GST invoicing | One tap produces the day's tax invoice as a PDF, with tax backed out of MRP rather than added on top |
| Themes | Light by default, with a dark theme that carries the full palette including WhatsApp's own dark colours |
A recorded run is embedded on the landing page at /, and the file itself is at
public/demo.mp4. One minute six seconds, covering a Soundbox sale, a sale
spoken out loud, and the invoice at the end.
Run npm run reset first. It does not empty the shop, it fills it. A judge opening the dashboard
should see a working business rather than a blank slate they have to populate themselves, so the
reset writes a morning of trading: eight sales totalling 1,650 rupees split across UPI, cash and
card, the customer messages those sales generated, and one supplier reorder already approved.
Two products are held back deliberately. Fortune Sunflower Oil sits at five units against a threshold of four, so a single sale trips the reorder agent live on stage. Surf Excel is left below its threshold, so the low stock state is visible on arrival.
| Step | Action | What to watch for |
|---|---|---|
| 1 | Landing page, then Open the dashboard | No sign up, and nothing to set up. The day is already part traded, so every feature is populated before anyone clicks anything |
| 2 | Simulate payment, pick Aashirvaad Atta, UPI, customer Priya Sharma, confirm | The stock count and the UPI tile both count up and flash. Nothing was refreshed. Postgres pushed it |
| 3 | Voice, press play once, say "do packet Parle-G cash mein bech diya" | It replies out loud with the amount, then reopens the mic on its own. Both sides of the conversation stay on screen with a receipt card beside them |
| 4 | Keep talking: "aaj kitna kamaya" | No tapping between turns. Same conversation, but it answers instead of booking |
| 5 | Messages | The customer thank you is already written and waiting |
| 6 | Voice: "ek Fortune oil bech diya" | The receipt card turns amber and a badge appears on Messages. Stock hit the reorder point |
| 7 | Messages, then Approve and send | The human gate. The agent writes, a person sends |
| 8 | Invoice, then Generate today's invoice | A GST ready PDF downloads with CGST and SGST split out |
The supplier gate is deliberate. The agent notices the shortfall, works out the reorder quantity and writes the message, but it does not place the order. An autonomous agent committing a shop's money to a real supplier is not a feature. The approval step is on screen rather than buried in a setting.
GST is backed out, not added on. Indian kirana prices are MRP and already include tax. Adding five percent on top would overstate both the day's takings and the tax owed, so the invoice divides by 1.05 and splits the result into CGST and SGST.
Both entry points converge on one write path, so a spoken sale and a Soundbox sale are literally the same code.
Simulate payment Voice conversation
(Soundbox stand in) (Sarvam speech to text)
| |
| Gemini extraction
| item, quantity, mode, amount
| |
+------------------+------------------+
|
recordSale() server action
|
record_sale() Postgres function
......................................
: insert transaction :
: decrement product stock : one transaction
: update customer last purchase : all or nothing
: report whether stock is now low :
......................................
|
+--------------+---------------+
| |
Postgres change events after() response hook
| |
Realtime to the browser Gemini writes the customer
stock and totals animate message and, if stock is low,
the supplier reorder draft
Two design decisions are load bearing here:
- Atomicity lives in the database.
record_sale()is a single Postgres function, so the ledger row and the stock movement can never half apply. There is no state where a sale is recorded but the shelf count did not move. - Side effects follow the commit. The customer message and the reorder draft each cost a
Gemini call, so they run inside Next's
after()hook once the response has already been sent. A slow or failing model can never cost you a sale, and the browser picks the outbox rows up over Realtime whenever they land. This took a voice turn from about eleven seconds to 1.4.
| Layer | Choice | Why |
|---|---|---|
| Framework | Next.js 16, App Router | Server actions give one write path shared by every entry point |
| Language | TypeScript 5 | Strict throughout, no any in the source |
| Styling | Tailwind v4 | CSS first @theme, so themes are token overrides rather than class rewrites |
| Database | Supabase Postgres | Atomic sales in a database function, and Realtime without building a socket layer |
| Speech | Sarvam AI | saarika:v2.5 for speech to text and bulbul for text to speech, across Hindi, Telugu and English |
| Language model | Gemini 3.6 Flash | Transcript to structured entry, spoken answers, and message copy |
| jsPDF and jspdf-autotable | Client side invoice generation with no server round trip | |
| Icons | Lucide | Real icons rather than emoji standing in for them |
Node 20 or newer, and a Supabase project.
npm installcp .env.local.example .env.localFill in the values described in Environment Variables.
Open the Supabase SQL editor and run supabase/schema.sql. This creates
the tables, the record_sale() function, the Realtime publication and the demo access policies.
npm run seedLoads eleven real kirana products and one demo customer.
npm run devThe landing page is at / and the dashboard at /dashboard. There is no authentication, so
there is no sign up step.
| Variable | Required | Purpose |
|---|---|---|
NEXT_PUBLIC_SUPABASE_URL |
Yes | Supabase project URL |
NEXT_PUBLIC_SUPABASE_ANON_KEY |
Yes | Browser client, used for reads and Realtime |
SUPABASE_SERVICE_ROLE_KEY |
Yes | Server only. Never reaches the browser bundle |
GEMINI_API_KEY |
Yes | Transcript extraction, spoken answers, message copy |
SARVAM_API_KEY |
Yes | Speech to text and text to speech |
WHATSAPP_PHONE_NUMBER_ID |
No | Cloud API automatic sending. Without it, messages stay a preview |
WHATSAPP_ACCESS_TOKEN |
No | As above |
.env*.local is gitignored. The service role key is read only in
src/lib/supabase/server.ts.
Note on Gemini quotas. The free tier caps at roughly twenty requests before returning
429 RESOURCE_EXHAUSTED, which is not enough for a rehearsal. Enable billing on the key's project. The app degrades gracefully when it happens, but the phrasing gets plainer.
| Command | Does |
|---|---|
npm run dev |
Development server |
npm run build |
Production build |
npm run start |
Serve the production build |
npm run seed |
Load the catalogue and demo customer from scratch |
npm run reset |
Restore the shop to a normal trading afternoon: eight sales, the messages they generated, and stock reduced to match. Run between demo takes |
npm run typecheck |
tsc --noEmit |
npm run lint |
ESLint |
src/
app/
page.tsx Landing page, fully static, no database connection
layout.tsx Fonts, theme resolution before first paint
actions.ts recordSale() and approveDraft(), the only write paths
(app)/
layout.tsx App shell and the single Realtime subscription
dashboard/page.tsx Stock on hand and today's ledger
voice/page.tsx Live conversation, turns, receipt cards, typed fallback
messages/page.tsx WhatsApp preview and supplier reorder queue
invoice/page.tsx GST invoice preview and PDF download
api/voice/route.ts Speech to text, extraction, then book a sale or answer aloud
components/
ui/ Primitives, custom select, skeletons, animated numbers
voice/ Mic orb, conversation turns, result cards
simulate-payment.tsx Soundbox stand in with a real UPI intent QR
whatsapp-preview.tsx Outbound message queue styled as WhatsApp
supplier-drafts.tsx Reorder queue with the human approval gate
lib/
llm.ts The four Gemini prompts, each with a fallback
fallback.ts Deterministic reader used when Gemini is unavailable
sarvam.ts Speech to text and text to speech
use-voice-loop.ts Hands free turn loop with silence detection
audio.ts WAV re-encoding and microphone level metering
use-shop-data.ts One Realtime subscription for the whole app
invoice.ts GST arithmetic and PDF generation
links.ts WhatsApp click to chat and UPI intent links
supabase/
schema.sql Tables, record_sale(), Realtime publication, policies
scripts/
seed.ts Catalogue and customer seeding
reset.ts Restore the shop to opening time
| Column | Type | Notes |
|---|---|---|
id |
uuid | |
name, sku |
text | sku is unique |
price |
numeric | MRP, inclusive of GST |
quantity_on_hand |
integer | Never allowed below zero |
low_stock_threshold |
integer | Crossing it triggers a reorder draft |
| Column | Type | Notes |
|---|---|---|
product_id |
uuid, nullable | A spoken sale may name an item not in the catalogue |
quantity, amount |
integer, numeric | |
payment_mode |
text | upi, cash or card |
source |
text | soundbox, voice or manual |
raw_voice_text |
text, nullable | The original transcript, kept for transparency |
whatsapp_messages and supplier_drafts are outbox tables. Both are written in the same flow as
the sale, and both are read straight back into the UI over Realtime.
supplier_drafts carries a unique partial index allowing one open draft per product, so a
run of sales past the threshold cannot spam the supplier queue with duplicates.
The single atomic write. Inserts the ledger row, decrements stock, updates the customer, and returns the remaining quantity along with a flag for whether that sale crossed the reorder point.
The palette is defined once as Tailwind v4 tokens in
src/app/globals.css. Dark mode overrides token values rather than
rewriting components, so almost nothing needs a dark: variant.
| Group | Intent |
|---|---|
| Neutrals | Warm paper tones rather than default grey, so the app does not read as a template |
| Brand | Deep petrol green, distinct from the default blue every dashboard reaches for |
| Payment modes | UPI violet, cash green, card orange, chosen to stay distinguishable from the back of a room |
| Semantic | Money, accent, warn and danger, each with a matched tint background |
Conventions worth knowing:
- Fonts are self hosted. Inter and JetBrains Mono are bundled rather than fetched from Google Fonts, because a demo on conference wifi should not depend on a font request resolving.
- Numbers animate only when they change. The count up deliberately skips first paint, otherwise every figure animates on load and the real live change stops reading as special.
- Tabular figures everywhere. Every number in the app is money or a count, so columns align.
- Navigation is two components, not one compromise. A rail on laptops and up, a bottom tab bar below that. The real user is holding a phone one handed while serving somebody, so navigation belongs under the thumb rather than in a drawer behind a hamburger.
- The dark theme has its own flash keyframe. The light one washes green over paper, which on a dark surface reads as the row going darker when it changes, which is backwards.
Being precise about this is deliberate, since a judge will find out during the demo anyway.
| Piece | Status |
|---|---|
| Speech recognition and speech output | Real, Sarvam AI |
| Understanding what was said | Real, Gemini |
| Ledger, stock, atomicity | Real, Postgres |
| Live updating | Real, Postgres Realtime |
| WhatsApp message text | Real, written by Gemini per customer |
| WhatsApp sending | Real, via wa.me click to chat |
| Payment QR | Real NPCI UPI intent code carrying the live amount |
| GST invoice | Real |
| Receiving the payment webhook | Simulated |
Exactly one thing is simulated: receiving the transaction webhook back from Paytm. That needs a merchant account, and item level order detail needs enterprise access that is not open to a one day build. So the trigger is a button rather than a beep, but it fires the identical code path a real webhook would, and everything downstream of it is real.
WhatsApp sending goes through click to chat rather than the Business API, which needs Meta verification and per template approval. For a one person shop that is arguably the better design anyway, since the owner sees exactly what is about to go out before it goes. Automatic sending is implemented and dormant behind the two optional environment variables.
The demo cannot be allowed to die on a rate limit, so every model call has a floor beneath it.
| Failure | What happens instead |
|---|---|
| Gemini rate limited or down | fallback.ts reads the sentence deterministically. It handles quantities in digits, English, romanised Hindi and Telugu, payment mode keywords, and catalogue matching by word scoring |
| Gemini cannot write a message | A templated message is used, so no bubble is ever blank |
| Speech not understood | A spoken apology in the same language, rather than silence |
| Supabase not configured | A setup screen listing the exact remaining steps |
| Data still loading | Skeletons shaped like the content, so nothing briefly reads as empty |
The fallback returns nothing when it is unsure, so the real model is always preferred and the deterministic reader only catches the fall.
This is a one day hackathon build. The following are deliberate omissions rather than oversights.
- No authentication. RLS policies are permissive and the shop is a single hardcoded record. Do not point this schema at real data.
- No multi tenancy. There is no
shop_idcolumn anywhere. - No offline support. Every sale needs a live round trip, which real kirana connectivity will not always provide.
- No idempotency key. A replayed webhook would double book.
- Flat five percent GST. Real invoicing needs per product HSN codes and the full slab range.
- Demo data is invented. The shop identity in
src/lib/shop.tsis fictional.
Ordered by what would actually block a first real shop.
- Confirm Paytm webhook access. Whether a third party app can receive Soundbox transaction events at all is the largest unknown, and it is outside the team's control.
- Rebuild the flow around beep triggers a voice confirmation. This sidesteps the fact that a payment carries no item, and it needs no new integrations.
- Put one real shop on it, manually onboarded, before writing any more code.
- Then add authentication, multi tenancy and offline queueing.
Voice first is shippable today with no partnership at all. The Soundbox is the story, and the voice system is the product.
Built by Code Vyapar for the Build for India AI Hackathon hosted by Paytm in Hyderabad.
Theme: AI Powered Fintech Innovation. Problem statement: AI for Small Businesses.
Speech by Sarvam AI. Language understanding by Gemini. Database and Realtime by Supabase.