Idea: built-in UI focus/attention awareness for copilots

[vamgan]

Problem

CopilotKit has useCopilotReadable for passing application state to the copilot, but there's no built-in way to tell the copilot what the user is currently looking at in the UI. Developers must manually track and serialize this themselves.

This matters because the most natural copilot interactions are contextual — "explain this chart", "why is this metric down?", "what does this row mean?" — and the copilot needs to know which element the user is referring to.

Proposed pattern

A first-class way to declare which UI regions are "askable" and automatically feed the focused element's context into the copilot:

// 1. Annotate elements
<div data-askable='{ "widget": "revenue", "value": "$2.3M" }'>
  <RevenueChart />
</div>

// 2. Context updates automatically on click/hover/focus
useCopilotReadable({
  description: 'What the user is focused on',
  value: focusContext, // auto-maintained
});

What this enables

• "Explain this" — copilot knows which chart/card/row the user means
• Multi-step reasoning — history of recent interactions gives the copilot browsing context
• Source awareness — distinguish passive browsing (hover) from explicit asks (button click)
• Third-party libraries — AG Grid, TanStack Table, chart libraries that own their DOM can push context programmatically

Existing solution

askable-ui implements this pattern today as a standalone library (~1kb, zero deps). It works with CopilotKit via a 3-line useCopilotReadable bridge:

const { ctx } = useAskable();
useCopilotReadable({
  description: 'UI element the user is focused on',
  value: ctx.toContext({ history: 5 }),
});

Would love to hear if this is something CopilotKit would consider supporting natively or recommending as a complementary library.