Dynamically returning a component from a tool call

[nakedkeegan]

Hi there!

It seems from the doc that you have to define upfront what component your tool will return like

server.registerTool(
  "get_zoo_animals",
  {
    title: "get_zoo_animals",
    description: "Lists zoo animals and facts about them",
    inputSchema: { count: z.number().int().min(1).max(20).optional() },
    annotations: {
      readOnlyHint: true,
    },
    _meta: {
      "openai/outputTemplate": "ui://widget/widget.html",
    },
  },
  async ({ count = 10 }, _extra) => {
    const animals = generateZooAnimals(count);
    return {
      content: [],
      structuredContent: { animals },
    };
  }
);

But in our use case we would like to be able to decide at runtime which component this tool should return. Is this possible at all currently?

Something like

server.registerTool(
  "get_zoo_animals",
  {
    title: "get_zoo_animals",
    description: "Lists zoo animals and facts about them",
    inputSchema: { count: z.number().int().min(1).max(20).optional() },
    annotations: {
      readOnlyHint: true,
    },
  },
  async ({ count = 10 }, _extra) => {
    const animals = generateZooAnimals(count);
    return {
      content: [],
      structuredContent: { animals },
    _meta: {
      "openai/outputTemplate": "ui://widget/widget.html",
    },
    };
  }
);

[rinormaloku]

Another similar feature that is required, is the ability to use resource templates.

For example tool call has:

"openai/outputTemplate": "ui://widget/{dynamicValue}",

And that would call the resource template "ui://widget/{dynamicValue}", where it would generate the view on the fly, otherwise, how do we support dynamic data?

This doesn't seem to be supported. and only the resources pointed to by tools are discovered

[katia-openai]

Hi both, just to make sure I understand, you want to invoke a different component based on the tool params?
What's preventing you at the moment from handling that logic inside the component directly?

[nakedkeegan]

Hi @katia-openai

you want to invoke a different component based on the tool params?

yes, or based on some conditional logic inside the tool's logic

What's preventing you at the moment from handling that logic inside the component directly?

I tried that and worked well but sometimes I want to return no component, just text, and couldn't find a way to gracefully do that