fix: renders dejan de fallar (17% -> 0%), generative UI en los dos chats - #56
Merged
Conversation
Reported as "el chat genera markdown y ni siquiera lo renderiza bien, debería
usar OpenUI". Half of that turned out to be already working and the other half
was worse than described.
The OpenUI path was fine. Proved three ways before touching anything: stubbing the
`ui` event with the exact program bytes from `chat_messages.metadata` painted the
`StepSequence`; a live instrumented run showed `ui` arriving at t=1354 ms with the
kit in the final bubble; and the deployed bundle already contained the branch.
The prose fallback was the defect. It rendered as
`<p className="whitespace-pre-line">{content}</p>` — `react-markdown` is a
dependency of this app and nothing in the chat used it. The learner saw
`1. **Escucha la pregunta**`, `| Turno | Responsable |` and backticks as literal
characters.
And that is the **majority** path, not an edge case: of the ten most recent
assistant turns, two had a program and eight did not. The admin assistant never
lays out, answers under `MIN_LAYOUT_CHARS` never do, and the gate rejects some. So
"the chat produces markdown" was an accurate description of what most answers are,
and it was being shown raw.
`ChatMarkdown` is bubble-scale rather than `LessonContent`: colour inherited,
tighter rhythm, `last:mb-0` so the common one-paragraph answer is pixel-identical
to the `<p>` it replaces, tables scrolling inside their own container so a bubble
can never widen the log. Images are disallowed — an image is the one construct
that fires an unattended request to an arbitrary host, and this text is downstream
of a document anyone can upload. No `rehype-raw`, so HTML stays inert.
`ChatAnswer` renders blocks **or** prose, never both, and gates the program before
committing to it. That closes a real hole: `UiSpecRenderer` answers "may this be
painted?" with `null`, and `null` in a chat bubble is a blank bubble with the prose
already thrown away. The browser gate is deliberately stricter than the server's,
so a server-valid program can still be refused here — now it falls back instead of
vanishing.
The streaming caret is now a glyph appended to the markdown source, so it sits at
the true end of the text whatever element the parser is mid-building. A `<span>`
cannot: the last node changes every token. It also stops animating unconditionally
and honours reduced motion.
The old tests passed while the screen was wrong, because they asserted
`message.program` was set. The new ones assert
`getByText('Escucha la pregunta').tagName === 'STRONG'` and that the prose is gone
once blocks exist. 427 tests.
… could change Three things the owner caught by using the app. **The internal scrollbar.** `<main>` carried `overflow-y-auto`, which made it a scroll region instead of an element that grows. A long list scrolled inside the panel while the window stayed still — most obvious on Empleados, but it was every screen. Nothing needed it: the sidebar is `fixed left-0 top-0 bottom-0` and the header is `fixed top-0`, so both hold their position under ordinary page scroll on their own. Removed from both layouts; `overflow-x-hidden` stays, because that one does a job. **Scrollbars, once, globally.** There are 18 scroll containers in this app and the only thing worse than an unstyled scrollbar is half of them being styled. Thin, rounded, in the border colour, inset so the thumb floats in the track. Plus `scrollbar-gutter: stable` on scrollable elements, which reserves the track so content does not jump sideways by ~12px the moment a list outgrows its box. **Ajustes.** The AI provider card is gone. It became read-only when the provider moved to the environment, and read-only was the problem: an admin could look at `groq/llama-3.1-8b-instant` and do nothing about it, because the person who can change it already has the `.env` open. The argument for keeping it — "is the AI working?" — died when generation failures started saying why in the moment they happen, which is better diagnosis than a page you have to remember to visit. One line survives, and only when it earns itself: a warning when no model is configured at all. That is the single case where this screen tells the admin something they could not otherwise find out. What is left is the one switch they can actually act on, out of its card, full width, with a real toggle and copy cut from three paragraphs to one.
Asked "como van mis empleados", it replied with four bullets of management advice — revisa el parte de incidencias, habla con el encargado, organiza una reunión. Generic model output, while the actual answer sat in the database. The cause: `stream_admin` delegated to the same `_stream` as the employee tutor, so the admin assistant was a *document* assistant. It retrieved from training material, found nothing about people, and fell through to general knowledge. `org_snapshot.py` assembles the facts in eight aggregate queries and renders them into the turn the way a document already is. Chosen over tool-calling with numbers, not taste: measured on the live database it is 3.1 kB for five employees, growing ~45 tokens per employee, so forty employees is still cheaper than one tool round trip costs in latency. Past forty it switches to totals plus the people who need attention, and **says in the block that the list is partial** rather than truncating silently. What made the answer good was not the snapshot. The first live run came back as five name-by-name paragraphs with no headline, because "state only what you were given" forbids the model from computing the total it needs for one. So the block carries a pre-counted summary — headcount, assignments by status, who has started nothing, who is overdue, with names. Every figure an admin would open the dashboard for is a literal string in the prompt, which is what makes the no-arithmetic rule usable instead of a gag. The privacy line holds by three independent mechanisms, not by care: no query touches the private columns, `EmployeeFact` is a frozen dataclass with seven fields and nowhere to put one, and a test parses the module's AST and fails on an attribute access named `preset`, `experience_level`, `format_vector`, `tutor_notes`, `goal`, `learning_profile` or `accessibility`. Verified live: asked for Aitana's learning profile, it answers that the platform does not show it. The prompt says withheld, not missing, so it answers as policy. Greetings no longer get "No tengo suficiente información". `small_talk.py` matches the whole message, accent-folded, against a closed set, four words maximum — so "hola, como van mis empleados" goes down the real path. Zero tokens, no provider call. New SSE event `org_data` on admin turns, deliberately not folded into `grounding`: that label is about documents, and an admin answer can be grounded on platform data and no document at all. The frontend ignores unknown events, so this ships safely ahead of it. 2573 unit tests (+75), 16 integration, ruff unchanged. v1 regression untouched.
The admin assistant was excluded from generative UI. That was my call and it was wrong: exploring generative UI is the point of this project, not something to hand out where it happens to feel natural — and the strongest case in the product is the one that was excluded. "¿Cómo van mis empleados?" is tabular data being flattened into five paragraphs. The admin bubble now renders through `ChatAnswer`, the same component and the same rules as the employee chat: blocks or prose never both, the browser gate consulted before committing to blocks, prose kept when a program is refused. `ChatAnswer` needed no new prop — it was already endpoint-agnostic, and the admin page was the only thing standing between it and the stream. `api/chat.ts` was **not** dropping the event: there is one hook, one SSE parser, and the endpoint is a URL fragment, so `/chat/admin` had been parsing `ui` correctly all along and landing `program` on the message. The bubble threw it away. Verified live: a real admin turn streams prose, shows "Dando formato…", and repaints as a two-column five-row table, one row per employee. Also here, from the owner using the app: **The chat had a scroll box inside a page that now scrolls.** Both chats were a fixed `h-[calc(100vh-50px-48px)]` with an `overflow-y-auto` log — two scrollbars for one conversation. The log grows and the page scrolls; the composer is sticky so removing the inner scroll does not bury it at the bottom of a long thread. `endRef.scrollIntoView` keeps working, it just moves the page. **And a CSS trap I left behind this morning.** `<main>` kept `overflow-x-hidden` after losing `overflow-y-auto`, and per spec `overflow-x: hidden` forces the other axis from `visible` to `auto` — which would have quietly recreated the scroll container I had just removed. It is `overflow-x-clip` now, which does not. 431 frontend tests.
Reported as "mientras carga sigue habiendo scroll, luego se pone bien". The
symptom as described could not happen: on Empleados the skeleton is 55px
**shorter** than the settled content, so the page can only gain height when data
lands, never lose it. Measured at seven viewport heights, cold load and
client-side nav, plus the pre-React boot phase.
What is real is a horizontal jump, and it is mine. This morning I moved the scroll
container from `<main>` to the page. `scrollbar-gutter: stable` in `index.css` is
scoped to `.overflow-y-auto` / `.overflow-x-auto` / `.overflow-auto` — which was
`<main>` while `<main>` was the scroller. Once the page became the scroller, `html`
matched nothing, so the gutter stopped being reserved exactly when it started to
matter.
Measured on /admin/contenido at 1440×920:
| | skeleton | data lands |
|---|---|---|
| before | `main` 1192px | `main` 1182px — 10px jump |
| after | 1182px | 1182px |
And it compounded: 10px narrower re-wraps the cards, so the settled document was
960px with the bar up against 916px without — about 44px of extra height caused by
the scrollbar that the height caused.
`html { scrollbar-gutter: stable }` fixes it everywhere at once.
One measurement note worth keeping, because it invalidated a whole first pass:
**headless Chromium uses overlay scrollbars**, so `innerWidth - clientWidth` reads
0 headless and 15 headed on the same 2000px page. Every headless number was blind
to this entire class of bug.
Separate hole found on the way: below `md`, Employees hides the desktop table and
gated the mobile list on `!isLoading`, so on a phone the loading state reserved
zero height and showed nothing at all. Three placeholder cards now.
Left alone deliberately: the generic 3×`SkeletonRow` used by four pages matches
nothing it replaces, which is the residual 55px vertical delta. It cannot be
eliminated without knowing the row count before the fetch, and the admin Dashboard
— the only page whose skeletons are sized to their content — shows what fixing it
properly would take.
431 tests.
… lays out
Two changes that turned out to be the same change.
**Classify, then populate.** The layout call used to ask the model to author an
OpenUI Lang program. It now returns one JSON object — a shape from a five-value
enum plus that shape's fields — and the server writes the program.
The evidence is ours, not borrowed. Over 22 renders in `bench_out/`: 63.6% clean,
13.6% repaired, 22.7% fell back. And every validator error in the failure dumps,
classified: **10 of 10 are markup-authoring errors** — an accent inside a bare
identifier, `{` for an object, named arguments, twice the line cap, a missing
answer-key block. Not one is "picked the wrong block" or "got the content wrong".
The model's judgement about shape was never what failed. Its typing was.
So every one of those failure classes becomes unrepresentable rather than
rejected. The gate still runs on the emitted text, because the strings inside are
still the model's bytes. The layout prompt also shrank 72% — 7988 to 2176
characters — which on a 6000 tokens-per-minute key is the difference between the
admin assistant laying out and not.
One call rather than Curio's two: it splits classify from populate because it runs
1–4B models locally, and doubling latency and rate-limit exposure to shrink a
five-branch enum is the wrong trade here. Splitting it later touches the prompt
and the emitter, nothing else.
**And the admin assistant lays out now**, which is what the owner asked for: a
question about five employees is tabular data, and it was being flattened into
five paragraphs. `routes/chat.py` never passed `generative_ui` to `stream_admin`,
so it defaulted to `False` — invisible while the exclusion existed, and the whole
feature the moment it was lifted.
The three defects he found by using it:
- **"quien eres"** returned "No consta la informacion de identidad del
administrador" — it searched the org snapshot for the admin instead of saying
who *it* is. Now answered like a greeting: no provider call, no snapshot query.
- **"usa openui para esta respuesta"** was a dead end. The first fix made it
worse: it answered by pasting the entire platform data block and both source
documents, four kilobytes with five people's training records in it. Trading a
refusal for a context dump is the same non-answer, larger. The persona now
refuses to reproduce the block, and treats a format instruction with no subject
as a question to ask back.
- **The closing action fired everywhere.** A question about allergens ended with
"Escribe a Aitana, que no ha abierto ninguno de sus tres cursos." The prompt now
sorts the question into management / content / about-the-assistant before the
usage rules, and the close is prohibited in the last two.
Three more found while sweeping: a canned answer was long enough to trigger a
layout call, so a question guaranteed never to reach a provider was paying for
one; numeric table cells silently blanked the whole table; one unusable step
silently shortened a procedure.
`invented_figures` refuses any program carrying a digit run the prose did not
have. These blocks name real people, so "no invented figures" needed to be
mechanical rather than a prompt rule — it immediately caught three fixtures where
the table disagreed with the answer it came from.
2665 unit tests, 16 integration, ruff unchanged.
The owner opened the first node of the seeded allergens course and got fourteen
mandatory allergens as a comma-separated paragraph, in 25.7 seconds, using three
of the nine available blocks. His verdict: "no parece que tengamos mucho recurso,
incluso en Curio me va mejor."
Measured over the same 10 briefs against real Groq:
| | before (30) | after (20) |
|---|---|---|
| first try | 46.7% | **95.0%** |
| fallback | 16.7% | **0.0%** |
| `Table` share | 4.1% | 8.5% |
| `genera_ui` calls needed | 46 | 21 |
**The 25.7 seconds was never the model.** `llm_usage_log` holds that render as two
rows — 1039 ms then 24715 ms — so it was repaired, not first-try, and roughly 23.7
of those seconds were spent asleep. Two calls at 6491 input tokens against a
6000-tokens-per-minute key means the second is rate-limited, and the wait happens
inside the span `duration_ms` measures. Across the baseline: 63 waits, 1191
seconds slept over 30 renders. p50 tracked `tokens_in` and never `tokens_out`.
Latency here is a token-budget problem, so the fix is fewer calls.
**Three blocks of nine was not a palette restriction.** The only worked example in
the prompt was prose-shaped, and rule 15 offered "un solo TextContent cuyo texto
lleve la enumeracion" as a peer of Table and StepSequence — the cheapest option,
so it won. The same gap has a second face: elsewhere the model emitted nineteen
components, one per allergen, and was refused by rule 4. A paragraph and nineteen
blocks are the same defect: nothing connected "fourteen allergens" to a shape.
**And the largest single failure class was in the prompt's own framing.** Eight of
about thirty baseline rejections were `prop 'tone' must be one of: info, warn,
success (got 'critical')`. The prompt printed `- Criticidad: critical` four lines
above a catalogue entry whose first argument is a tone enum. A rule forbidding it
in words lost; deleting the token won.
The allergen node now, first try, 1156 ms — down from 25754:
root = Stack([intro, lista], "md")
intro = TextContent("Conocer los catorce alergenos…", "lead")
lista = Table(["Alergeno"], [["Gluten"], ["Crustaceos"], …])
Getting there took two failed attempts and both are now regression tests. The
second is a real gate hole it exposed: told to use one column, the model emitted
one row of fourteen cells, and `STRING_MATRIX` never checked row width against
headers, so the gate served it happily.
`agents/runtime/shape.py` reads the node's own section for enumerations, labelled
lists, ordered procedures and numeric series. `refine_format` corrects only the
two cases where the declared format is *impossible* — the screen the owner
rejected was a correct `explanation` made of the wrong blocks, so format is not
the knob. Confirmed by a seeded node declared `chart` whose every figure is
written as a word: no digit exists to plot, and nothing caught it because nothing
read the source.
`load_source_context`'s `full_text` branch is now scoped to `source_headings` like
the chunked branch already was; without it all three allergen nodes were handed
the same document.
The bench gained block-type coverage — the metric that made "three of nine"
visible and that nothing reported.
**Verdict against the classify-then-populate rework, on evidence.** The three
named-argument rejections in the old dumps came from the offline scripted fixture,
not the model. The real classes were a copied enum token, a missing answer key,
too many blocks and duplicate ids — none of which a JSON populate step fixes
better than naming the block up front, and a second constrained call spends the
quota that is the actual latency constraint. 95% first-try with zero fallbacks
says the free half of Curio's idea was enough.
2671 unit tests, 16 integration, ruff unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Siete commits, todos salidos de probar la aplicación a mano y encontrar que lo que se veía en pantalla no era lo que los tests decían.
Lo que más importa: los renders dejaron de fallar
El nodo de alérgenos salía como catorce alérgenos en un párrafo con comas, en 25,7 segundos, usando tres de los nueve bloques disponibles. Medido sobre los mismos 10 encargos contra Groq real:
genera_uinecesariasTableY ese nodo, ahora, a la primera y en 1.156 ms en vez de 25.754:
Tres causas, y ninguna era «el modelo es malo»:
llm_usage_logguarda ese render como dos llamadas, 1.039 ms y 24.715 ms, de los cuales unos 23.700 durmiendo. Las dos juntas son 6.491 tokens de entrada contra un límite de 6.000 por minuto, así que la segunda recibe un 429 y la espera cae dentro del tramo queduration_msmide. En toda la línea base: 63 esperas, 1.191 segundos dormidos en 30 renders. El p50 seguía atokens_iny nunca atokens_out— la latencia aquí es presupuesto de tokens, no velocidad de modelo.prop 'tone' must be one of: info, warn, success (got 'critical'), porque el prompt imprimía- Criticidad: criticalcuatro líneas encima de un componente cuyo primer argumento es un enum de tono. Prohibirlo por escrito perdió; borrar la palabra ganó.De paso, un agujero real en la puerta:
STRING_MATRIXnunca comprobaba que cada fila tuviera tantas celdas como cabeceras, así que servía tan tranquila una tabla de una fila con catorce celdas.El chat deja de escribir marcado
La llamada de maquetado pedía al modelo que escribiera un programa OpenUI Lang. Ahora devuelve un objeto JSON — una forma de un enum de cinco valores más sus campos — y el servidor escribe el programa.
La evidencia es nuestra, no prestada. Clasificados todos los errores del validador de los volcados: diez de diez son errores de escribir marcado — una tilde dentro de un identificador,
{para un objeto, argumentos con nombre, el doble del tope de líneas. Ni uno es «eligió el bloque equivocado» ni «se equivocó en el contenido». El juicio del modelo sobre la forma nunca fue lo que falló; su mecanografía sí.Efecto lateral que decide si esto es viable: el prompt de maquetado encogió un 72 %, de 7.988 a 2.176 caracteres. Con una clave de 6.000 tokens por minuto, eso es la diferencia entre que el asistente del admin pueda maquetar y que no.
Una llamada y no las dos de Curio: parte clasificar de rellenar porque corre modelos de 1-4B en local, y aquí duplicar latencia y exposición al límite de cuota para reducir un enum de cinco ramas es el intercambio equivocado.
Verdicto en contra de rehacer el motor de nodos igual, y sobre evidencia: los rechazos por argumentos con nombre de los volcados antiguos venían del fixture de pruebas, no del modelo. 95 % a la primera con cero fallbacks dice que la mitad barata de la idea de Curio bastaba.
Generative UI en los dos chats
El asistente del admin estaba excluido. Era una decisión mía y era mala: explorar generative UI es el objetivo del proyecto, y el caso más fuerte del producto era justo el excluido — una pregunta sobre cinco empleados son datos tabulares aplanados en cinco párrafos.
El motivo real de que no funcionara era más tonto de lo que parecía:
routes/chat.pynunca pasabagenerative_uial servicio, así que estaba enFalsepor defecto. Invisible mientras existía la exclusión, y el fallo entero en cuanto se levantó.El asistente del admin responde con datos reales
Preguntado «cómo van mis empleados», contestaba con cuatro viñetas de consejos de gestión — revisa el parte de incidencias, habla con el encargado — mientras la respuesta estaba en la base de datos.
org_snapshot.pymonta los hechos en ocho consultas agregadas. Elegido sobre llamada a herramientas con números: 3,1 kB para cinco empleados, ~45 tokens más por empleado, así que cuarenta empleados siguen costando menos que una ida y vuelta de herramienta en latencia. Pasados cuarenta cambia a totales más las personas que necesitan atención, y lo dice en el bloque en vez de truncar en silencio.La raya de privacidad se sostiene por tres mecanismos independientes, no por cuidado: ninguna consulta toca las columnas privadas,
EmployeeFactes un dataclass congelado de siete campos sin sitio donde meter una, y un test parsea el AST del módulo y falla si aparece un acceso apreset,experience_level,format_vector,tutor_notes,goal,learning_profileoaccessibility.Tres defectos que salieron al usarlo:
invented_figuresrechaza cualquier programa con una tirada de dígitos que la prosa no tuviera. Estos bloques nombran personas reales, así que «no inventes cifras» tenía que ser mecánico y no una regla de prompt; pilló tres fixtures propios donde la tabla no cuadraba con su respuesta.El markdown del chat nunca se renderizaba como markdown
Reportado como «el chat genera markdown y ni siquiera lo renderiza bien». La mitad de OpenUI ya funcionaba —comprobado de tres formas antes de tocar nada— y la otra mitad era peor de lo descrito: la prosa se pintaba con un
<p className="whitespace-pre-line">, sinreact-markdown, que es dependencia de esta app y el chat no usaba.Y es el camino mayoritario: de las diez últimas respuestas del asistente, dos con programa y ocho sin. El asistente del admin no maquetaba nunca, las respuestas cortas tampoco, y el validador rechaza algunas.
ChatAnswervalida el programa antes de comprometerse a los bloques, lo que cierra un agujero real:UiSpecRendererresponde «¿se puede pintar esto?» connull, y unnullen una burbuja de chat es una burbuja vacía con la prosa ya descartada.Y tres cosas de layout, todas encontradas usando la app
<main>era una caja con scroll propio (overflow-y-auto) en vez de crecer, así que una lista larga scrolleaba dentro del panel mientras la ventana se quedaba quieta. Nada lo necesitaba: la barra lateral y la cabecera sonfixed.overflow-x: hiddenobliga al eje vertical devisibleaauto, así que iba a recrear el contenedor de scroll que acababa de quitar. Ahora esclip.scrollbar-gutter: stableestaba en.overflow-y-auto— que eramainmientrasmainscrolleaba. Al pasar el scroll a la página,htmlno encajaba con nada y la reserva dejó de existir justo cuando empezaba a importar: 10 px de salto horizontal al aparecer la barra, que recolocaban las tarjetas y añadían 44 px de altura. Un detalle de método que invalidó toda una primera tanda de medidas: Chromium sin ventana usa barras superpuestas, así queinnerWidth - clientHeightda 0 sin ventana y 15 con ella.En Ajustes desaparece la tarjeta del proveedor de IA: se volvió de solo lectura cuando el proveedor pasó al
.env, y solo lectura era el problema — el admin podía mirargroq/llama-3.1-8b-instanty no hacer nada, porque quien puede cambiarlo ya tiene el fichero abierto. Sobrevive una línea, y solo cuando se la gana: un aviso si no hay ningún modelo configurado.Pruebas
tsc -blimpio, lint limpio, ruff en las 5 violaciones heredadas demain. Sigue rojotests/test_grading.py::test_grade_open_answer_fallback, roto desdec68d045— el desfasado es el test, no el código.Lo que queda dicho y no hecho
List.Tablecon una columna es un apaño para un hueco real del catálogo: no hay forma de expresar «una enumeración de N cosas», que es la forma más común de este dominio. Propuesta medida: añadirList, quitarCodeBlock(cero usos en 171 bloques, y ni uno de los diez encargos ni de los cuatro documentos contiene código), total de emitibles sin subir. No se ha aplicado porque el navegador resuelve el componente por nombre: unListservido sin su bloque de React no pinta nada, y eso es peor que el fallo de partida. Va en un solo parche coordinado.Chartno se elige nunca (cero usos), pero el bloque no sobra: hay un documento sembrado con cinco temperaturas representables. Lo que falla es quedecide_formatonunca selecciona ese formato. Quitarlo tocaría el enum, una migración, el router y el sembrado — cambio aparte.docs/design/chat-agents.mdsigue describiendo el chat que escribe el programa.