Skip to content

Latest commit

 

History

94 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 

Repository files navigation

BayesPilot — Two-Level Deterministic Offline Probabilistic Agent for Conversational Discovery

TikTok TechJam 2026 — Track 4: Conversational E-Commerce Search Challenge

  • Team Name: Algo Lover
  • Team Members: Ewen Cheung, Alvin Saw, Daeren Kim

Team Member Contributions

Member Contributions
Ewen Cheung (Team Lead) Led the technical direction and system architecture. Owned the core algorithmic work: the two-level Bayesian formulation (Level 1 category posterior with tau-mass pooling, Level 2 bounded evidence fusion), the expected-utility depth policy $U(k)$, the session state and decay model, and the hyperparameter search design. Responsible for the central ideas behind the approach and the majority of the inference and ranking code.
Alvin Saw Drove exploration, experimentation, and validation. Ran the ablation studies that justified pruning BM25, the popularity prior, dense embeddings, and GBDT rerankers; built and checked the evaluation datasets and stress scenarios; verified benchmark reproducibility and bootstrap confidence intervals; and led documentation and correctness review across the codebase.
Daeren Kim Supported both the algorithmic and validation workstreams and drove productisation of the submission. Contributed to implementation across the pipeline, built the interactive replay visualiser (submission/demo/index.html), produced the demo video walkthrough, and assembled the Devpost write-up and submission packaging.

Quick Start

cd submission
python3 -m venv .venv && source .venv/bin/activate    # On Windows: .venv\Scripts\activate
pip install -r requirements.txt
python3 scripts/evaluation/evaluate.py

Executive Summary

BayesPilot is a deterministic, offline, probabilistic multi-turn shopping agent that locates a hidden target product in a frozen 50,000-item Amazon catalog within 10 turns.

The evaluated BayesPilot runtime is built purely on NumPy and the Python standard library. It operates with zero LLM calls, zero neural network weights, zero token costs, and an average inference latency of 7.8–16.9 ms per session. Optuna is used only to reproduce the offline hyperparameter-fitting process.

TechnicalScore = 0.50 × Hit@10 + 0.30 × MRR + 0.20 × Efficiency
Efficiency     = clip((11 − MTTC) / 10, 0, 1)      # MTTC counts a miss as turn 11

Benchmark Results

Evaluated with the official unmodified evaluation harness:

Dataset Sessions Hit@10 MRR MTTC TechnicalScore Total Time Latency / Session LLM Calls Cost
public_set 200 1.0000 0.9942 2.19 0.9744 3.4s 16.9 ms 0 $0.00
generated_template_set 2,800 0.9911 0.9783 2.64 0.9562 21.7s 7.8 ms 0 $0.00
freeform_set 800 0.9912 0.9801 2.62 0.9572 10.1s 12.7 ms 0 $0.00
  • The complete output from running the agent on the public set, including per-session results, is available in submission/results.json.
  • Official Baseline Comparison: Official starter agent = 0.1067 · BayesPilot = 0.9744

Key Highlights & Reviewer Alignment

Here is a quick snapshot of what BayesPilot delivers and key aspects of the system that are helpful to keep in mind during review:

Dimension What BayesPilot Delivers
Technical Execution 0.9744 TechnicalScore on the official 200-session public set (1.0000 Hit@10, 0.9942 MRR, 2.19 MTTC).
High Generalization on Unseen Data: Achieves 0.9562 across 2,800 ASIN-disjoint sessions (generated_template_set) and 0.9572 across 800 non-template natural language sessions (freeform_set), demonstrating robust performance beyond the public split.
• Modular, typed Python codebase with isolated session state and safe popularity fallback.
Innovation & Problem Insight • Reframes conversational product discovery as two-level sequential Bayesian inference rather than ungrounded dialogue generation.
Level 1 Tau-Mass Pooling: Uses category posteriors to prune 50,000 items to a median of 182 candidates in <1 ms with 100% recall.
Level 2 Bounded Evidence Fusion: Combines SoftCard Jaccard token matching ($L_{\min}=0.02$) with temporal decay ($\gamma=0.9$) to track evolving preferences and Turn 3/4 intent overrides.
Expected-Utility Depth Policy: Dynamically determines recommendation depth $k^*$ by maximizing expected utility $U(k)$ to balance MRR and MTTC.
Impact & Real-World Relevance Solves Real-World E-Commerce Bottlenecks: Overcomes the latency (2–5s) and cost ($0.02+/query) barriers of LLM chatbots for high-QPS search and live-stream shopping.
Instant & Free Inference: 7.8–16.9 ms per full session (sub-millisecond per turn) and $0.00 marginal model cost (0 tokens, 0 GPUs).
Handles Realistic Shopper Dynamics: Seamlessly adapts to vague initial browsing queries, shifting customer requirements, noisy catalog metadata, and negative feedback from rejected items.
Feasibility & Practicality Zero Weights & Model-Free Runtime: Pure Python + NumPy. No downloaded neural weights, no vector DB, no external network APIs, and no API key management.
100% Deterministic & Auditable: Runs offline on any standard CPU, provides transparent probability scores, and reproduces with a single command (evaluate.py).
Presentation & Transparency Interactive Replay Visualizer: Standalone web visualizer (submission/demo/index.html) to inspect turn-by-turn belief updates, candidate rankings, and utility curves.
Demo Video Walkthrough: Accompanied by a clear video presentation detailing live discovery, architectural decomposition, and the mathematics behind our approach.
Mathematical Rigor & Clear Disclosures: Step-by-step LaTeX formulas, explicit limitations disclosures, and complete per-session JSON results.

High-Level Architecture

BayesPilot replaces opaque neural architectures with a mathematically grounded, two-level probabilistic discovery pipeline:

---
config:
  layout: fixed
---
flowchart LR
    U["Customer message"] --> P["Cascaded parser<br/>(Templates → Ontology)"]
    P --> S["Session state<br/>(Constraints, decay, overrides)"]
    S --> C["Level 1<br/>Category posterior"]
    C --> I["Level 2<br/>Item log-posterior"]
    I --> D["Expected-utility<br/>depth policy"]
    D --> O["Ranked ASINs<br/>+ Next question"]

    U:::input
    P:::input
    S:::state
    C:::inference
    I:::inference
    D:::inference
    O:::output

    classDef input fill:#dbeafe,stroke:#2563eb,color:#0f172a
    classDef state fill:#ffedd5,stroke:#f97316,color:#0f172a
    classDef inference fill:#ede9fe,stroke:#7c3aed,color:#0f172a
    classDef output fill:#dcfce7,stroke:#16a34a,color:#0f172a
Loading

Detailed Architecture

flowchart LR
    subgraph S1["1. Deterministic NLP & State Tracking Tier"]
        T{"Exact regex<br/>template match?"}
        U["Customer utterance at turn t"]
        SS["Session State Tracker<br/>• Slot constraints C_t<br/>• Age decay γ = 0.9<br/>• Override demotion factor = 0.35"]
        ON["Ontology Normalizer<br/>(Fuzzy attribute-value extraction:<br/>brand, color, size, specs)"]
    end

    subgraph S2["2. Level 1: Bayesian Category Belief (1,115 Categories)"]
        C1["Category scoring:<br/>s_c(x) = W_c(x) · coverage_c(x)<br/>+ 3.0 · 1[quoted] · W_c(x)"]
        C2["Softmax with prior (T = 2.0, catalog share π_c):<br/>P(c|x) = softmax(s_c(x)/2.0 + 0.25 · log π_c)"]
        C3["Prefix mass pruning (τ = 0.85):<br/>Retain the smallest set where ΣP(c|x) ≥ 0.85<br/>(50,000 items → median 182 candidates)"]
    end

    subgraph S3["3. Level 2: Bounded Item Likelihood Fusion"]
        L1["Multi-source evidence accumulator:<br/>• Exact card match (g_exact = 3.2)<br/>• Soft-card Jaccard (J ≥ 0.34, g_soft = 1.5)<br/>• Lexical token overlap"]
        L2["Bounded log-likelihood floor:<br/>log L_r = log(max(0.02, exp(g_r(s−1))))"]
        L3["Temporal item log-posterior:<br/>log P_t(i) ∼ Σ 0.9^(t−turn)<br/>[log L_main + log L_soft]"]
        L4["Hard rejection masking:<br/>Proven-wrong shipped ASINs → log P(i) = −∞"]
    end

    subgraph S4["4. Decision-Theoretic Recommendation Depth Policy"]
        K1["Dynamic expected-utility maximizer:<br/>k* = argmax [Σ(p_j/j) + (1 − Σp_j) · V]"]
        K2["Continuation value:<br/>V = max(0, 0.75 · d^s − 0.0667)<br/>d ∈ {0.8 understood, 0.2 unreadable}"]
        OUT["Emit top k* ASIN recommendations or ask for evidence<br/>(k* ∈ 0..10, dynamic turn by turn)"]
    end

    U --> T
    T -- No --> ON
    ON --> SS
    SS --> C1
    C1 --> C2
    C2 --> C3
    C3 --> L1
    L1 --> L2
    L2 --> L3
    L3 --> L4
    L4 --> K1
    K1 --> K2
    K2 --> OUT
    T -- "Yes (deterministic fast path)" --> SS

    T:::stage1
    U:::stage1
    SS:::stage1
    ON:::stage1
    C1:::stage2
    C2:::stage2
    C3:::stage2
    L1:::stage3
    L2:::stage3
    L3:::stage3
    L4:::stage3
    K1:::stage4
    K2:::stage4
    OUT:::stage4

    classDef stage1 fill:#dbeafe,stroke:#2563eb,color:#0f172a
    classDef stage2 fill:#ede9fe,stroke:#7c3aed,color:#0f172a
    classDef stage3 fill:#fef3c7,stroke:#d97706,color:#0f172a
    classDef stage4 fill:#dcfce7,stroke:#16a34a,color:#0f172a
Loading

1. Level 1 — Category Posterior (Search Space Reduction)

  • Inspiration: Narrow down product candidates using category as an early high-precision signal, slashing the search space before performing fine-grained item scoring.
  • Mechanism: Computes a posterior distribution $P(c \mid \text{opener})$ over 1,115 product categories using category-IDF weighting, stemmed token overlap, and verbatim quote bonuses ($\text{bonus} = 3.0$).
  • Tau-Mass Pooling: Selects the minimal set of categories covering 85% of posterior mass ($\tau = 0.85$), pruning 50,000 catalog items down to a median of 182 candidates with 100% target recall on public_set.

2. Level 2 — Bounded Item Likelihood Fusion

  • Bounded Likelihood ($L_{\min} = 0.02$): Bounding evidence terms from below prevents soft mismatches from erroneously eliminating the true target.
  • Multi-Channel Evidence: $$\log P(\text{item}) = \sum_t w_t \cdot \log L(e_t \mid \text{item})$$
    • Exact constraint strings (gain: 3.2)
    • Normalized attribute-value pairs (gain: 1.5)
    • Token overlap (gain: 0.9)
    • SoftCard Jaccard token matching against product intent-card strings (gain: 1.5, floor: 0.34)
  • Aging Decay & Intent Override: Older preferences decay geometrically over turns, allowing changed customer constraints (Turn 3/4 Intent Overrides) to seamlessly override earlier statements.
  • Survival Evidence: Evaluator stops immediately upon a target hit. A surviving session proves all previously recommended items are incorrect, setting their log-posterior to $-\infty$.

3. Optimal K — Expected Utility Formula $U(k)$

  • Mathematical Insight: From the Technical Score formula: $$\text{Score}(\text{Turn 2, Rank 1}) = 0.50(1) + 0.30(1) + 0.20\left(\frac{11 - 2}{10}\right) = 0.980$$ $$\text{Score}(\text{Turn 1, Rank 2}) = 0.50(1) + 0.30(0.5) + 0.20\left(\frac{11 - 1}{10}\right) = 0.850$$ $$\implies (\text{Turn 2, Rank 1}) &gt; (\text{Turn 1, Rank 2})$$ MRR weight (0.30) outweighs early turn speed (0.20). Prematurely guessing with low confidence damages MRR more than asking another clarifying question.
  • Expected Utility Equation: $$U(k) = \sum_{i=1}^k \frac{p_i}{i} + \left(1 - \sum_{i=1}^k p_i\right) \left(V_{\text{continue}} \cdot \text{hope} - \text{cost}_{\text{turn}}\right)$$ The agent derives list length $k$ dynamically by choosing the largest $k$ where marginal value $1/k &gt; V$.

4. Hyperparameter Tuning via TPE (Tree-structured Parzen Estimator)

  • Rather than manual trial-and-error or blind grid search, all 8 hyperparameter constants (evidence gains, category temperature, depth policy thresholds) were fitted offline using Optuna's Tree-structured Parzen Estimator (TPE) on the training split with noise-gated bootstrap confirmation.

5. Pruned / Rejected Components (Empirical Negative Results)

  • Dense / Semantic Embeddings (BLaIR, SVD): Evaluated extensively and deleted. Catalog intent matching is exact and keyword-grounded; semantic embeddings introduced noise and reduced Hit@10.
  • Heavy LLM Tier / LLM Router: Removed from the active path. Evaluator messages follow structured patterns; deterministic ontology extraction achieves higher accuracy at $0$ token cost and $100\times$ lower latency. A thin escalation hook remains available behind the llm_extract flag so the pipeline can be extended, but it is off by default and unused in all reported results.
  • GBDT / LightGBM Rerankers: Overhead in runtime and complexity without statistically significant gains over bounded Bayesian fusion.

Limitations

  • Distribution dependence: The deterministic parser is strongest on the published evaluator's structured customer-message patterns. Novel phrasing, misspellings, or implicit constraints outside the catalog ontology can reduce extraction and ranking quality.
  • Fully deterministic submitted configuration: The agent ships with every semantic-model path switched off. An optional escalation hook is retained behind the llm_extract flag purely for flexibility and future experimentation — it is disabled by default, is not required by any reported result, and every benchmark in this README was produced with zero external calls. The trade-off is deliberate: open-ended language coverage is exchanged for zero network, credential, quota, and cost risk.
  • Catalog dependence: Rankings and category statistics are built from the frozen 50,000-product catalog. A materially changed catalog requires restarting the agent so its in-memory indexes are rebuilt.
  • Metadata ambiguity: Products with sparse or near-identical catalog metadata may remain difficult to distinguish; the depth policy can return several candidates or spend an additional turn clarifying.
  • Evaluation scope: The reported public and generated-set scores demonstrate performance on the supplied simulator and derived stress sets. They do not guarantee the same performance on unreleased final sessions or unconstrained real-world conversations.
  • Runtime scope: Measurements use the official sequential evaluator. Concurrent throughput, peak memory, and behavior on substantially larger catalogs were not benchmarked.

What We Would Improve Given More Time

Each item below maps directly to a limitation listed above.

  1. Broader paraphrase and typo robustness (addresses distribution dependence) — Extend the deterministic ontology with edit-distance matching and a learned-but-offline synonym table, so novel phrasing degrades gracefully instead of falling back to popularity ordering. We would grow the freeform_set with adversarial paraphrases and misspellings and tune against that harder split.
  2. Multi-modal image likelihood fusion (addresses metadata ambiguity) — Products with near-identical text metadata are the main residual failure mode. Precomputed visual feature vectors (texture, pattern, silhouette) could enter Level 2 as an additional bounded evidence channel $\log L_{\text{visual}}$ without adding any runtime model.
  3. Incremental catalog indexing (addresses catalog dependence) — Replace the full in-memory rebuild with incremental index updates so the agent can absorb catalog changes without a restart.
  4. Concurrency and scale benchmarking (addresses runtime scope) — Profile peak memory, multi-worker throughput, and behaviour on catalogs an order of magnitude larger, then publish a proper QPS envelope rather than single-session latency alone.
  5. Dynamic client-side personalisation — Allow the category prior $\log \pi_c$ to adapt to on-device browsing history, keeping personalisation entirely local with no privacy leakage.
  6. WebAssembly / edge deployment — Compile the deterministic engine to WebAssembly and C++ for sub-millisecond execution directly inside mobile apps and browser tabs, which the zero-dependency design already makes feasible.

Project Structure

README.md                               # Project documentation & architecture report
submission/                             # Standalone submission directory
  agent.py                              # Entry point exporting Agent
  requirements.txt                      # NumPy runtime dependency
  results.json                          # Public-set output with per-session results
  data/
    catalog.jsonl                       # Frozen 50,000-product catalog
    public_set.jsonl                    # 200 official public evaluation sessions
    freeform_set/                       # Free-form natural-language dataset
    generated_template_set/             # ASIN-disjoint 60/20/20 template dataset
  demo/
    index.html                          # Interactive multi-turn replay visualizer
  participation_kit/                    # Official competition kit
    evaluator/local_evaluator.py        # Official local evaluator engine
    starter/agent.py                    # Starter agent baseline
    docs/                               # Specification & API contracts
    data/public_set.jsonl               # Official dataset copy
  scripts/
    evaluation/evaluate.py              # Multi-dataset evaluation CLI
    training/hyperparameter_tuning.py   # Bayesian hyperparameter fitting (Optuna)
    earlyhit.py                         # EarlyHit@k curve analysis
    llm_tier.py                         # Diagnostic tool
  src/
    simulator.py                        # Customer simulator
    copilot/
      agent.py                          # Core Agent class (reset, respond, fallback)
      flags.py                          # All submission hyperparameters & defaults
    retrieve/
      bm25.py                           # Okapi BM25 implementation (ships disabled)
      category.py                       # Level 1: Category posterior distribution
      index.py                          # 50K catalog vocabulary index
    rank/
      belief.py                         # Level 2: Item log-posterior & depth policy
      likelihood.py                     # Bounded log-likelihood evidence fusion
      softcard.py                       # Paraphrase-tolerant SoftCard matching
    state/
      session.py                        # Conversational state & aging decay
    understand/
      attributes.py                     # Attribute extraction from prose
      extract.py                        # Attribute extraction helpers
      intent.py                         # Intent pipeline & catalog resolution
      parse.py                          # Deterministic ontology parsing cascade
      tokens.py                         # Numeric-preserving tokenizer
      llm.py                            # Optional escalation hook (off by default; see `llm_extract`)
    eval/
      harness.py                        # Non-invasive evaluator harness
      stress.py                         # Paraphrase stress engine
      ablations.py                      # Ablation test suite
      compare.py                        # TechnicalScore bootstrap CI calculator
      datasets.py                       # Dataset loader utilities
      measure.py                        # Metric computation (Hit@10, MRR, MTTC)
      freeform.py                       # Free-form dataset evaluation
      holdout.py                        # Held-out split evaluation
      instrument.py                     # Latency & call-count instrumentation

Development Tools, APIs, Libraries & Datasets

Category Used
Development tools VS Code · Git & GitHub · Python 3.11.9 virtual environments (venv) · macOS / Linux terminal · Mermaid (architecture diagrams) · Optuna dashboard for inspecting tuning trials
APIs used None. BayesPilot makes zero external network calls — no LLM API, no search API, no vector database, no cloud endpoint, no API keys. The agent runs fully offline.
Libraries & frameworks Runtime: NumPy 2.3.3 and the Python standard library — that is the complete runtime dependency set (submission/requirements.txt). Offline tuning only: Optuna 4.9.0 (TPE hyperparameter search), not imported at inference time. No PyTorch, TensorFlow, Hugging Face Transformers, scikit-learn, or LangChain.
Datasets & assets Provided: the frozen 50,000-product Amazon catalog (data/catalog.jsonl) and the 200-session official public_set, both from the TechJam Track 4 participation kit. Self-generated: a 2,800-session ASIN-disjoint generated_template_set (60/20/20 split) and an 800-session non-template freeform_set, both produced by our own simulator (src/simulator.py) for generalisation testing. No manually labelled data, pretrained weights, embeddings, or third-party assets are used.

Setup & Reproduction Instructions

Prerequisites

  • Python: 3.11.9 used for the reported public-set result; Python 3.11+ supported
  • OS: macOS / Linux / Windows

1. Environment Setup

cd submission

python3 -m venv .venv
source .venv/bin/activate    # On Windows: .venv\Scripts\activate

pip install -r requirements.txt

2. Run Evaluation

# Evaluate on public_set.jsonl (200 sessions)
python3 scripts/evaluation/evaluate.py \
    --agent agent:Agent \
    --catalog data/catalog.jsonl \
    --dataset data/public_set.jsonl \
    --offline \
    --output results.json

# Run full evaluation across all benchmark datasets
python3 scripts/evaluation/evaluate.py --all --ci --scenarios --output runs/all_results.json

3. Reproduce Hyperparameter Tuning

The production agent has no runtime dependency beyond NumPy. Install the pinned Optuna version only when reproducing the offline TPE fitting process:

python3 -m pip install optuna==4.9.0

Quick Smoke Test

Use this small run to verify that dataset loading, evaluation, Optuna, checkpointing, and result writing all work. With only 200 sessions and 3 trials, it is a pipeline check—not a statistically meaningful tuning result:

python3 scripts/training/hyperparameter_tuning.py \
    --dataset data/generated_template_set/train.jsonl \
    --catalog data/catalog.jsonl \
    --n 200 \
    --levels 0,2,3 \
    --trials 3 \
    --seed 0 \
    --resume runs/tuning_smoke.db \
    --output runs/refit_smoke.json

Full Reproduction Run

python3 scripts/training/hyperparameter_tuning.py \
    --dataset data/generated_template_set/train.jsonl \
    --catalog data/catalog.jsonl \
    --n 3000 \
    --levels 0,2,3 \
    --trials 60 \
    --seed 0 \
    --resume runs/tuning.db \
    --output runs/refit.json

The search jointly fits eight constants, stores resumable trials in runs/tuning.db, applies a paired-bootstrap noise gate, and writes the reproducible result to runs/refit.json. It is a long-running, CPU-only experiment and does not call an LLM or modify the shipped defaults.


Cost, Latency & Resource Disclosures

Metric Measured Value
Average Inference Latency 7.8 – 16.9 ms per multi-turn session
Prompt Tokens 0
Completion Tokens 0
Model Cost $0.00
GPU / MPS Requirements None (Runs purely on standard CPU)
External Network APIs None (100% offline & reproducible)
Runtime Fallback Safe popularity-ordered fallback on any unexpected exception
Measured Environment Python 3.11.9 · Darwin 25.5.0 · arm64 CPU; exact CPU model and RAM were not captured

Safe Submission Packaging

Commit the frozen submission files first, then build the archive from Git-tracked content only:

git archive --format=zip --output bayespilot-submission.zip HEAD:submission

This excludes ignored local files such as submission/.env, .venv, caches, and editor metadata. Never ZIP the working directory directly, and never include real credential values in the archive.

For the frozen final run, first commit the solution and confirm the worktree is clean. Run the released evaluator without changing the Agent or configuration, retain its per-session submission/results.json, and verify that its provenance reports the submitted commit, "dirty": false, and "kit_pristine": true before packaging.


Interactive Demo & Video Presentation

1. Interactive Multi-Turn Replay Visualizer

A standalone interactive session visualizer is provided in submission/demo/index.html.

  • Open in Browser: Simply double-click or open submission/demo/index.html in any web browser (no local web server or internet connection required).
  • Features:
    • Turn-by-Turn Replay: Step through customer utterances, slot extraction, and slot decay across multi-turn sessions.
    • Belief Distribution Visualizer: Inspect Level 1 category posterior masses and Level 2 item log-posterior rankings in real time.
    • Decision Policy Inspection: Visualize continuation value $V_{\text{continue}}$ and dynamic recommendation depth ($k^*$) calculations at each conversational step.

2. Video Demonstration & Walkthrough

▶️ Watch the demo: https://youtu.be/3jJaCy7OI4o

The video presentation covers:

  1. Live Discovery Walkthrough: Demonstrating end-to-end sessions across Buying, Browsing, and Intent Override scenarios.
  2. Architecture Breakdown: Explaining the two-level coarse-to-fine Bayesian pipeline and deterministic NLP tier.
  3. Mathematical Foundations: Step-by-step exposition of Tau-Mass pruning, bounded log-likelihood evidence fusion ($L_{\min}=0.02$), and expected-utility depth optimization ($U(k)$).
  4. Real-World Impact & Feasibility: Demonstrating sub-17ms execution, zero model cost, and high resilience on commodity CPUs.

References & Academic Citations

The mathematical foundations, Bayesian belief models, and decision-theoretic rules in BayesPilot are grounded in the following academic research:

1. Conversational Decision Policy & Expected Utility

  • Ahsan-Ul-Haque, A. S. M., & Wang, H. (2022). Rethinking Conversational Recommendations: Is Decision Tree All You Need? In Proceedings of CIKM '22 (pp. 686–695). (Core motivation for lightweight information-gain decision structures, asking strategies, and early stopping rules in conversational recommendation).
  • Fuhr, N. (2008). A probability ranking principle for interactive information retrieval. Information Retrieval, 11(3), 251–265. (Decision-theoretic basis for ranking interactive actions using their probability of success and associated conversational turn costs).
  • Chapelle, O., Metlzer, D., Zhang, Y., & Grinspan, P. (2009). Expected reciprocal rank for graded relevance. In Proceedings of CIKM '09 (pp. 621–630). (Inspiration for rank-sensitive expected utility, where discovery at higher ranks receives substantially greater reward).

2. Multi-Evidence Probabilistic Retrieval & Negative Feedback

  • Turtle, H., & Croft, W. B. (1991). Evaluation of an inference network-based retrieval model. ACM Transactions on Information Systems (TOIS), 9(3), 187–222. (Probabilistic foundation for combining heterogeneous retrieval evidence—exact match, normalized attributes, lexical overlap, and SoftCard Jaccard—into a unified relevance belief).
  • Bi, K., Ai, Q., Zhang, Y., & Croft, W. B. (2019). Conversational Product Search Based on Negative Feedback. In Proceedings of CIKM '19 (pp. 359–368). (Theoretical precedent for incorporating implicit negative feedback on unselected products directly into subsequent conversational product search).

3. Hyperparameter Optimization & Baseline Discipline

  • Bergstra, J., Bardenet, R., Bengio, Y., & Kégl, B. (2011). Algorithms for hyper-parameter optimization. In Advances in Neural Information Processing Systems (NeurIPS 24). (Core formulation of the Tree-structured Parzen Estimator (TPE) algorithm used for joint offline constant tuning).
  • Akiba, T., Sano, S., Yanase, T., Ohta, T., & Koyama, M. (2019). Optuna: A next-generation hyperparameter optimization framework. In Proceedings of KDD '19 (pp. 2623–2631). (Practical framework utilized for executing the offline Bayesian tuning pipeline).
  • Dacrema, M. F., Cremonesi, P., & Jannach, D. (2019). Are we really making much progress? A worrying analysis of recent neural recommendation approaches. In Proceedings of RecSys '19 (pp. 101–109). (Empirical motivation for rigorous evaluation of simple, well-tuned non-neural baselines before introducing unnecessary neural complexity).

License

Competition submission for TikTok TechJam 2026, Track 4.

About

Interactive demo for BayesPilot — TikTok TechJam Track 4. Deterministic shopping copilot: 1 product out of 50,000 in under 3 turns, zero LLM calls.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages