🗺️ Session Tracking #2619

mikeldking · 2024-03-18T20:49:20Z

As a user of phoenix, I want to be able to associate multiple traces (e.g. conversational flows) under a single session_id. This way I can track the back and forth between a user and a chat completion endpoint (e.g. visualize the back and forth).

In addition to tracking a session.id, we might also want to track session metadata such as user_id etc.

As a developer building a chat or agent application, I want to be able to track user interactions with my application. Notably if my application keeps track of user interactions under a single “session”, I want to be able to view the user interactions at a higher level rather than a trace.

OpenInference Semantic Convention changes

session
- .id the application provided correlation id

OpenInference instrumentation changes

New “openinference-instrumentation” package
- contains utilities around correlating spans to sessions

from openinference.instrumentation.session import using_session, set_session, clear_session

# Using a context manager
With using_session(id: “my-unique-id”):
       // Invoke application code here
       
# implicit
set_session(id: "my-session-id")
// run code here
clear_session(id: "my-session-id")

Proposed solution is to leverage trace context to set “inheritable” attributes that would be added to each span that is nested below a session context. This means that the setting of the session

Open questions:

Threading @axiomofjoy
What if a person starts another session (session collision) @axiomofjoy

Custom Instrumentation

 Custom instrumentation would also have to inherit the attributes in a simple way. We need to provide convenience to pull these attributes and attach them to the span.

from openinference.instrumentation import get_context_attributes

def do_work():
    with tracer.start_as_current_span("span-name", get_context_attributes()) as span:
        # do some work that 'span' will track
        print("doing some work...")
        # When the 'with' block goes out of scope, 'span' is closed for you

Architecture

Under the above, session.id would correlate to a trace (not a span) to avoid the danger of spans that get captured in the absence of a session.id. This must be the assumption when querying for sessions.

Open Questions:

How to inherit sessions from decorators @axiomofjoy
[sessions] Tech spec on sessions #2680