How do you deal with turn state?

[masylum]

So this is a pattern that we are struggling a little bit. Each time a user prompts a Think agent, we need certain information to be on the state. Things like the user_id that prompted the agent, the workspace id, and so on. We need this information to do things like observability, fetch mcps from the db, etc...

Right now, this requires saving things on the state of the agent (or using dynamic configuration) and then hoping for the best. Type-wise is frustrating because both agent state and dynamic configuration are nullable to acknowledge the stateful nature of the agent.

How do you deal with this? Are we holding it wrong?

[phpnode]

@masylum can't you use per-message metadata for this? https://ai-sdk.dev/docs/ai-sdk-ui/message-metadata

[threepointone]

Can you be more specific? Like with some pseudo code? Trying to understand what you want, when and where.

[masylum]

Sure, here is some pseudo code illustrating the pattern.

Let's say we extract user id from connection and we store it on state (from the cross-domain authentication docs). This state, will be nullable since it may not be setup (although we know that it will always be). The same happens when sending metadata during a turn, for instance the budget or anything like that. Since Think is a Template method API, we can't pass this metadata around, so we need to store it on the class itself. Either as an instance variable (ephemeral) or using state or dynamic configuration (not clear to me when to use one or another). In either case, that will also be nullable, despite us knowing that certain methods will always have the value, because we know the order of the lifecycle methods. But that's not great, you don't want to program with the knowledge of order, that's time coupling.

So that's not something I want, just wondering how are you work around it.

export class MyAgent extends Think<Env> {
  async onConnect(connection: Connection, ctx: ConnectionContext) {
    //...
    const userId = url.searchParams.get("userId");
    // ...

    connection.setState({ userId, authenticated: true });
  }

  private async beforeTurn(ctx) {
    this.budget = ctx.body.budget // ephemeral state, could be this.setState or dynamic configuration.
  }

  getModel() {
    const budget = this.budget
    if (!budget) throw Error('this will never happen')

    return createWorkersAI({ binding: this.env.AI })(
      budget === 'high'
        ? "@cf/moonshotai/kimi-k2.6"
        : "@cf/google/gemma-4-26b-a4b-it"
    );
  }

  async onChatResponse(result: ChatResponseResult) {
    const { userId } = this.state
    const budget = this.budget
    if (!userId) throw Error('this will never happen')
    if (!budget) throw Error('neither will this')

    await saveResult({ userId, budget })
  }
}

[cjol]

Perhaps you want some getters to transparently check values are set before reading?

export class MyAgent extends Think<Env, MyState> {
  private _budget: 'high' | 'low' | undefined;

  get userId(): string {
    if (!this.state.userId) throw new Error('userId not set');
    return this.state.userId;
  }

  get budget(): 'high' | 'low' {
    if (!this._budget) throw new Error('budget not set');
    return this._budget;
  }

  private async beforeTurn(ctx) {
    this._budget = ctx.body.budget;
  }

  getModel() {
    return createWorkersAI({ binding: this.env.AI })(
      this.budget === 'high' ? '@cf/moonshotai/kimi-k2.6' : '@cf/google/gemma-4-26b-a4b-it'
    );
  }

  async onChatResponse(result: ChatResponseResult) {
    await saveResult({ userId: this.userId, budget: this.budget });
  }
}

[threepointone]

connection.setState is not the same as this.setState, one is per connection, the other is per agent