Exit early from a tool call

[TheBinaryGuy]

Hi, when using generateText with tools and maxAutomaticRoundtrips, is there a way to exit early from a tool call without feeding the result back to the LLM?

Currently, I'm using abortSignal to abort the request and move on, is that the right way? Or is there a better way to handle this?

[lgrammel]

Currently there isn't. What's your use case? Why would you want to exit early?

[TheBinaryGuy]

Thanks for replying so fast!

I am building a Telegram agent for appointment scheduling, check ins and check outs. The app's architecture sends the final response from generateText to the user.

One of the requirement is requesting location before checking in / out, which in telegram translates to this:

const keyboard = new Keyboard().requestLocation('Check In');
await sendMessage('Please provide your location by clicking the button below.', ctx, {
    reply_markup: {
        keyboard: keyboard.build(),
        force_reply: true,
        one_time_keyboard: true,
    },
    parse_mode: 'Markdown',
});

The above code runs inside one of the tools (appointment check in tool), line 2 already sends a message with the button. If I return from the tool, text gets generated and sent back again to the user. Also because I only want the user to click the button once, I have specified the one_time_keyboard flag, so when the AI sends the second text, the above one disappears.

I am currently working around this using the AbortController, which aborts the request after the sendMessage above.

[lgrammel]

you could have a tool without an execute method. then when the finishReason is tool call, you can check the tool calls to see if there are any for that specific tool, and execute them separately

[TheBinaryGuy]

Will empty execute bypass maxAutomaticRoundtrips and not make further roundtrips?

[TheBinaryGuy]

Just tested this, it does bypass maxAutomaticRoundtrips, thanks!

[khanakia]

@TheBinaryGuy Do you have some demo code how you achieved this ?

[TheBinaryGuy]

@khanakia Hey, not anything in particular, it's all closed source and I've not worked on it for a while, but here's the gist of it, hope this helps.

We have some tools that we pass to generateText, two of the tools, check in and out, requires the user to take an action (i.e. provide their current location). On both of these tools we have an empty execute method, which exits out of the generateText call when used by the LLM, then we check if the finishReason was tool-calls and if the tool name matches the expected name, if so we send out a location request to their telegram and when they click it we handle the rest on telegrams location callback.

Note: We discontinued our product a while ago, so there might be updates that broke this or there might be even better ways to handle this now, so consult the docs.