PreCalls

PreCalls

Sometimes there's no point in relying on the LLM to call functions you already know must be called for a certain task. It's pointless to rely on the LLM decision, it wastes tokens, time, and the innate unreliability of LLMs can be entirely avoided. For this reasons, pre-calls exist. Pre-calls are a way to invoke functions before the LLM gets involved.

Additionally, if a scripting engine is enabled, it's also possible to run a code snippet instead of invoking a pre-built function. As a default, the CLI has a JavaScript engine enabled, but it really depends on the integrating application.

There are two level of preCalls:

• Global: they are invoked at the runner start, before any session is created.
• Session: they belong to a specific session and are invoked before the prePrompt phase

YAML

It's as simple as:

sessions:
  s1:
    preCalls:
      - name: postgres_query
        description: list all users
        args:
          query: select * from users
    prompt: fit the results in the schema

The results will be injected in the LLM context before the prompt happens.

Go

There's nothing specific for the Go implementation. Simply recreate the preCalls structure with code.

Routing

As a default, preCall outputs are routed to the LLM context. However, this may not necessarily be the best solution, so it's possible to route them to a different destination by:

1. Providing a value to the in attribute. Possible values are:
   1. vars: routing the output to the session variables
   2. context: routing the output to the a root variable in the context object
2. Providing a value to the var attribute, indicating the name of the variable to use as destination.

In the following example we select users first name from a database and route the result to the session vars so that we can the iterate on them and ask the LLM for each one individually.

sessions:
  s1:
    preCalls:
      - name: postgres_query
        description: list all users
        args:
          query: select first_name from users
        in: vars
        var: user_names
    prompt: Who is this person {{ .it }}?
    iterateOn: vars.user_names

Additionally, it's important to know that preCalls can optionally be intercepted by transformers.

IMPORTANT: global preCalls require in to be set to vars or context. Omitting in will result in no preCalls being executed as there's no global LLM context to target.

Notes

Notice how there's no tools section. PreCalls have access to the entire function scope and do not require explicit enablement in the session. The tools simply controls the visibility of the functions for the LLM.