Add context fields to history and logbook APIs

[loganrosen]

Describe your core improvement

Add optional context fields (user_id, parent_id) to the history and logbook APIs. The context data already exists in the database and shows up in the current state API (/api/states/<entity>), but it's currently stripped from all historical endpoints like /api/history/period and /api/logbook.

With an optional include_context=true parameter, each historical state change would include who or what triggered it - whether it was a user action, an automation, a physical device, or a service call.

Current limitations

The history APIs return state changes but don't include any attribution data. If I see that my kitchen light turned on 60 times last week, I can't tell if those were manual actions (which I should automate) or already triggered by automations (no action needed).

This makes several use cases impossible or unreliable:

• Pattern analysis for automation opportunities - I'm trying to analyze my usage patterns to figure out what to automate, but I have to guess based on timing patterns which is unreliable
• Debugging past issues - When something weird happened yesterday, I can't tell what triggered it
• Measuring automation effectiveness - Can't tell if users are constantly overriding an automation (indicating it needs adjustment)
• Attribution in energy/usage analytics - Can't distinguish user behavior from automated behavior in energy tracking

The data exists in the database (states.context_user_id_bin, states.context_parent_id_bin) and is accessible in real-time through /api/states/<entity>, but there's no way to get it for historical analysis without direct database access (which requires filesystem access and isn't suitable for integrations).

Technical benefits

• Better analytics and insights - Third-party tools and integrations could analyze patterns and provide actionable automation suggestions based on actual user behavior vs existing automations
• Improved debugging - When troubleshooting, users could trace back exactly what triggered state changes, not just guess based on timing
• Backward compatible - Default to include_context=false to maintain current API behavior and performance
• Simple implementation - The data already exists in the database, just needs to be included in the API response when requested
• Enables smarter integrations - Tools could adapt behavior based on whether changes were user-initiated vs automated

Example API addition:

/api/history/period/2024-01-01?filter_entity_id=light.kitchen&include_context=true

Would return:

{
  "entity_id": "light.kitchen",
  "state": "on",
  "last_changed": "2024-01-01T10:00:00+00:00",
  "context": {
    "id": "abc123",
    "parent_id": null,
    "user_id": "def456"
  }
}

Additional context

I know context tracking has some reliability issues (#90669, #1888), but even imperfect data is more useful than no data. For pattern analysis, I can combine context with heuristics to handle gaps.

Related but different requests:

• Ensure reliable context.entity_id propagation for state change events following service calls. #1888 - fixing context reliability in real-time (for automation logic)
• Logging of Entity / attribute values changed (and by what/with old/new value logged) #2150 - logging state changes with attribution to files
• Audit log #2447 - audit logging for configuration changes
• Add device_id to context architecture#978 - adding device_id to context

This request is specifically about exposing existing context data through historical APIs.

Alternatives I've tried:

• Direct database queries (requires filesystem access)
• Pattern detection heuristics (unreliable - can't distinguish manual 10pm light turn-ons from automation-triggered ones)
• Real-time event stream logging (misses historical data)

The data is already in the database. This would just make it accessible through the standard APIs.

[karwosts]

Are you asking about just REST APIs? The websocket APIs for logbook return user_id with the events, that's how they can display the user in the Activity panel.