Skip to content

fix(categories): resolve namespaced entities for HANA backfill + seed classifier descriptions - #2095

Merged
jung-thomas merged 2 commits into
DEVfrom
fix/backfill-categories-hana
Sep 1, 2026
Merged

fix(categories): resolve namespaced entities for HANA backfill + seed classifier descriptions#2095
jung-thomas merged 2 commits into
DEVfrom
fix/backfill-categories-hana

Conversation

@jung-thomas

Copy link
Copy Markdown
Contributor

Problem

Two independent bugs kept the Admin UI Categories facet empty on DEV/PROD:

  1. Backfill dies on HANA. scripts/backfill-categories.cjs and the classifier persist() queried bare short-name entity strings (Tutorials, TutorialCategories). Those resolve on local SQLite but emit unqualified TUTORIALS / TUTORIALCATEGORIES SQL against HANA (real tables are COM_SAP_DEVELOPERS_IMS_*). Running the sanctioned backfill via cds bind --exec failed on its first query (Could not find table/view TUTORILS). Only the SQLite-based unit test ever passed, so the bug was invisible. In-service self-heal worked (handler namespace context resolves bare names) — hence the lone stray junction row.

  2. seedDescription unseeded. Categories.csv ships only ID/slug/label/sortOrder, so all 8 category rows had empty seedDescription → the embedding classifier degraded to LLM-only everywhere (~1400 AI Core LLM calls, lower quality).

Fix

  • Backfill: resolve ims entity objects via a linked model (resolveImsEntities, falling back to cds.linked(cds.load('*')) when cds.model is unset in the standalone runner) and SELECT with explicit .columns('ID').
  • Classifier persist(): resolve the junction via cds.entities(ns)[junction] instead of a bare string (loadItemText already did this).
  • Seed descriptions: baseline texts in category-seed-descriptions-defaults.js + an idempotent, non-destructive boot-seed (seed-category-descriptions.js, wired into cds.on('served'), VITEST-gated, fills only empty rows). Deliberately not in CSV — a CSV column would full-replace the admin-editable field on every deploy.

Tests

  • Unit (test/unit/seed-category-descriptions.test.js): defaults integrity (keys match shipped slugs), idempotency, non-destructive (never clobbers admin edits), self-heal.
  • Hybrid (test/hybrid/backfill-categories-hana-resolution.test.js, guarded by HYBRID_TESTS=true + isSafeForWrites()): the load+link resolution path reaches the namespaced HANA tables (.name === com.sap.developers.ims.<Name>, resolved-object SELECT executes) — the exact standalone branch the bare-string code broke on; and the seed runs against real HANA.

Verified against DEV HANA

Targets DEV per branching policy.

…A + seed classifier descriptions

The category backfill (scripts/backfill-categories.cjs) and classifier
persist() queried bare short-name entity strings ('Tutorials',
'TutorialCategories'). Those resolve on local SQLite but emit unqualified
TUTORIALS / TUTORIALCATEGORIES SQL against HANA (real tables are
COM_SAP_DEVELOPERS_IMS_*), so the backfill died on its first query via
`cds bind --exec` and only its SQLite unit test ever passed.

- backfill: resolve ims entity OBJECTS via a linked model (resolveImsEntities,
  falling back to cds.linked(cds.load('*')) when cds.model is unset in the
  standalone runner), and SELECT with explicit .columns('ID').
- classifier persist(): resolve the junction via cds.entities(ns)[junction]
  instead of a bare string (loadItemText already resolved this way).

Second, independent blocker: Category.seedDescription was unseeded (CSV ships
only ID/slug/label/sortOrder), so the embedding classifier degraded to
LLM-only. Add baseline seed texts (category-seed-descriptions-defaults.js) and
an idempotent, non-destructive boot-seed (seed-category-descriptions.js, wired
into cds.on('served'), VITEST-gated, fills only empty rows). Kept out of CSV so
deploys can't full-replace the admin-editable column.

Tests: unit (defaults integrity + idempotency + non-destructive) and hybrid
(load+link resolution reaches the namespaced HANA tables; seed runs on HANA).
…eddings

category-seed-embeddings.js called embed() with no model arg in loadAll,
recomputeStale, and embedAdHoc, so AzureOpenAiEmbeddingClient(undefined) threw
"Cannot read properties of undefined (reading 'modelName')" on every classify
— the embedding path was dead and every item fell back to LLM-only, even with
seedDescriptions populated. Same #2001 class of bug as the freshness pipeline.

Resolve the model via resolveEmbeddingSettings() like every other embed() caller
(relevance-seed-embeddings.js is the direct analog), memoized at module level so
a bulk backfill doesn't re-read ChatSettings per item. Verified against DEV:
path=embedding now fires. Both category-seed-embeddings.js and
chat-settings-resolver.js are already in the srv-qa cp list, and the resolver
imports only @sap/cds → no srv-qa boot-crash risk.
@jung-thomas

Copy link
Copy Markdown
Contributor Author

DEV backfill — verified against real HANA

Ran the fixed backfill via cds bind --exec against DEV HANA. Junction rows now populate (was fatal on query #1 before the entity-qualification fix). Both fixes in this PR are exercised end-to-end: the resolved-entity SELECT/DELETE/INSERT reach the namespaced COM_SAP_DEVELOPERS_IMS_* tables, and the embedding path fires (seed descriptions seeded → cosine classification, no LLM needed for the confident majority).

Coverage on DEV:

Kind Distinct items classified Junction rows
Missions 858 / 890 (96%) 2011
Groups 356 / 362 (98%) 966
Tutorials 2293 / 2922 (78%) 5503

Missions/groups classified almost entirely via the embedding path (0 LLM calls for missions). The remaining gap is items whose cosine score falls below HIGH_THRESHOLD/ambiguity gap and fall through to the LLM fallback, which hit sustained AI Core 429 (rate-limit) — those items are safely skipped, not wiped (pickEmbeddingResult returns null, so persist() is never called with an empty set). They self-heal incrementally: content-publish-session.js re-runs classifyAndPersist on every tutorial republish, and a re-run on a fresh AI Core quota window will close the rest.

Not a code defect — the fix is correct; the tutorial tail is bounded by AI Core LLM throughput.

@jung-thomas
jung-thomas merged commit e0053ee into DEV Sep 1, 2026
4 checks passed
@jung-thomas
jung-thomas deleted the fix/backfill-categories-hana branch September 1, 2026 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant