Skip to content

Screen Highlights: tighten the prompt to extract atomic values #2294

@OwenMcGirr

Description

@OwenMcGirr

Summary

The Screen Highlights feature shipped in v2.26.0 sometimes returns whole sentences instead of just the URL, or groups a label with its value (e.g. OTHER: Order number: ABC-9876 instead of OTHER: ABC-9876). The user taps an item to copy it, so the value needs to be the only thing in the result.

Rewrite the prompt in service/llm/ScreenHighlightsTask.kt to:

  • Lead with atomicity ("each entry is a single atomic value, not a sentence")
  • Add a worked few-shot example per HighlightType — the cheapest way to steer small on-device models
  • Add an explicit "strip surrounding prose / labels" rule with the Tracking: ABC123OTHER: ABC123 example
  • Reinforce that multiple URLs on one screen are multiple URL: lines
  • Tighten OTHER to "atomic identifiers" so the model doesn't dump prose there

No parser, type, cap, backend, UI, or test changes — only the prompt string.

Acceptance criteria

  • Atomic extraction: URL inside a sentence becomes just the URL; "Tracking: ABC123" becomes OTHER: ABC123; two emails on one screen are two separate entries
  • ./gradlew compileDebugKotlin testDebugUnitTest passes (12 existing parser tests unchanged)
  • Manual verification on AICore (S26) and MediaPipe fallback paths

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions