Store arbitrary values in ALS/Context

[mbleigh]

Within a flow there are a variety of reasons I might need to store out-of-band context. An example is let's say I have state stored according to a session id. I want tools to be used in the context of this session id, but I don't want the LLM to be forced to forward the session id through.

There should be an easy mechanism to do something like:

defineFlow({
  name: 'doThing',
  inputSchema: z.object({sessionId: z.string()}),
}, async input => {
  storeContext('sessionId', input.sessionId); // saved into ALS
  await generate({
    prompt: 'do stuff with tools',
    tools: ['getSessionHistory'],
  });
});

defineTool({
  name: 'getSessionHistory',
}, async () => {
  const sessionId = loadContext('sessionId');
  return await doSomeServerThing(sessionId);
});

In Go we may not need to do this because ctx.Context already gives people ways to do this for themselves, but I don't think folks are familiar enough with ALS to do this for themselves.

[romechenko]

I'm facing this same thing. I don't want the flow to pass the sessionId or uid because I can't trust that. There needs to be a way get the validated value that I set in the flow.

[mbleigh]

@romechenko we have a solution for this that will be released very soon! There's a new "action context" that lets you provide this kind of side-channel metadata. See the draft docs for more on how it will work 😄

[romechenko]

@mbleigh, the design looks great. It’s thorough, I couldn’t think of any suggestions. I’ll be happy to ditch my current solution for this once it’s available.