BasePlugin callbacks are dropped for intermediate steps: only first beforeModelCallbackand last afterModelCallbackfire during tool execution

[Victorzwx]

Describe the Bug:

When using BasePlugin (via InMemoryRunner with custom plugins) to monitor an agent's execution flow that involves tool calling, the plugin callbacks are not properly invoked. Specifically, only beforeModelCallback from the first model call and afterModelCallback from the last model call are fired. All other callbacks — including afterModelCallback from the first model call, beforeToolCallback/afterToolCallback for tool execution, and beforeModelCallback from the second model call — are completely skipped.

This issue applies not only to a single tool call scenario but also to multiple tool calls. When an agent executes N tool calls in sequence, the expected flow would involve (N+1) model calls and N tool executions, resulting in (N+1) beforeModelCallback/afterModelCallback pairs and N beforeToolCallback/afterToolCallback pairs. However, only the very first beforeModelCallback and the very last afterModelCallback are ever fired — all intermediate callbacks across all tool executions and model calls are silently dropped.

In a typical agent flow with tool use (single tool call example):

1. Model call Bump com.github.tomakehurst:wiremock-jre8 from 2.35.0 to 2.35.1 in /core in the maven group across 1 directory #1 → returns tool call request: only beforeModelCallback fired ✅, afterModelCallback NOT fired ❌
2. Tool execution → executes locally: beforeToolCallback and afterToolCallback NOT fired ❌
3. Model call Revert "Bump com.github.tomakehurst:wiremock-jre8 from 2.35.0 to 2.35.1 in /core in the maven group across 1 directory" #2 → processes tool result and generates final response: beforeModelCallback NOT fired ❌, only afterModelCallback fired ✅

This makes it impossible to monitor or intercept any intermediate steps in the agent execution, which is critical for observability, logging, and debugging in production agent systems.

Steps to Reproduce:

1. Create an LlmAgent with a FunctionTool and a custom BasePlugin that overrides beforeModelCallback, afterModelCallback, beforeToolCallback, and afterToolCallback with logging.
2. Create an InMemoryRunner with the agent and the plugin.
3. Send a user message that triggers the agent to call the tool.
4. Observe the logs — only beforeModelCallback from the first model call and afterModelCallback from the last model call are logged. All other callbacks (first afterModelCallback, tool callbacks, second beforeModelCallback) are missing.

Expected Behavior:

For an agent flow that involves one tool call, the plugin callbacks should be invoked as follows:

• beforeModelCallback — before the 1st model call (user input → model)
• afterModelCallback — after the 1st model response (model returns tool call)
• beforeToolCallback — before tool execution
• afterToolCallback — after tool execution
• beforeModelCallback — before the 2nd model call (tool result → model)
• afterModelCallback — after the 2nd model response (final answer)

All 6 callbacks should be triggered, providing full observability into the agent's execution lifecycle.

Observed Behavior:

Only beforeModelCallback from the first model call and afterModelCallback from the last model call are fired. All other callbacks are silently skipped: afterModelCallback from the first model call, beforeToolCallback/afterToolCallback for tool execution, and beforeModelCallback from the second model call. This pattern holds for any number of tool calls — all intermediate process callbacks are dropped, regardless of how many tool executions occur in between. The tools ARE executed (the final response reflects tool usage), but the plugin is never notified of any intermediate steps.

Log output shows:

10:25:46.225 [main] INFO ... -- [myTestPlugin] 🧠 beforeModelCallback - input: [{"parts":[{"text":"my input is abc,usu your input to change the input"}],"role":"user"}]
10:25:53.570 [main] INFO ... -- Request completed successfully: model=openai, type=chat, duration=7338ms, tokens=381
10:25:53.575 [main] INFO ... -- [myTestPlugin] 🧠 afterModelCallback - Response: {...}
the result is : I can help you change your input using the available tool!...

Missing logs:

• No afterModelCallback for the 1st model call (after model returns tool call request)
• No beforeToolCallback / afterToolCallback for the tool execution
• No beforeModelCallback for the 2nd model call (before model processes tool result)

Environment Details:

• ADK Library Version: 1.5.0 (com.google.adk:google-adk:1.5.0, com.google.adk:google-adk-spring-ai:1.5.0)
• OS: macOS
• Java Version: 17

Model Information:

• Which model is being used: GLM-5 (via OpenAI-compatible API using SpringAI adapter)

---

🟡 Optional Information

Regression:
Unknown — this is the first version tested.

Logs:

10:25:46.225 [main] INFO c.b.a.t.a.a.SequentialAgentTest$MyTestPlugin -- [myTestPlugin] 🧠 beforeModelCallback -input: [{"parts":[{"codeExecutionResult":null,"executableCode":null,"fileData":null,"functionCall":null,"functionResponse":null,"inlineData":null,"mediaResolution":null,"partMetadata":null,"text":"my input is abc,usu your input to change the input","thought":null,"thoughtSignature":null,"toolCall":null,"toolResponse":null,"videoMetadata":null}],"role":"user"}]
10:25:53.570 [main] INFO c.g.a.m.s.o.SpringAIObservabilityHandler -- Request completed successfully: model=openai, type=chat, duration=7338ms, tokens=381
10:25:53.575 [main] INFO c.b.a.t.a.a.SequentialAgentTest$MyTestPlugin -- [myTestPlugin] 🧠 afterModelCallback - Response: {"avgLogprobs":null,"content":{"parts":[{"codeExecutionResult":null,"executableCode":null,"fileData":null,"functionCall":null,"functionResponse":null,"inlineData":null,"mediaResolution":null,"partMetadata":null,"text":"I can help you change your input using the available tool! Your current input is \"abc\", but I need to know what you'd like to change it to. \n\nWhat would you like your new input to be?","thought":null,"thoughtSignature":null,"toolCall":null,"toolResponse":null,"videoMetadata":null}],"role":"model"},"customMetadata":null,"errorCode":null,"errorMessage":null,"finishReason":null,"groundingMetadata":null,"inputTranscription":null,"interrupted":null,"modelVersion":null,"outputTranscription":null,"partial":false,"turnComplete":true,"usageMetadata":null}
the result is : I can help you change your input using the available tool! Your current input is "abc", but I need to know what you'd like to change it to. 
What would you like your new input to be?

Screenshots / Video:
N/A

Additional Context:
This is a critical issue for production agent systems that rely on plugin callbacks for observability, audit logging, rate limiting, and error handling. Without callbacks properly firing for each step (first model call's afterModelCallback, tool execution's beforeToolCallback/afterToolCallback, and subsequent model calls' beforeModelCallback/afterModelCallback), it is impossible to build reliable monitoring around the agent's full execution lifecycle. The problem is especially severe for agents with multiple tool calls, where the vast majority of the execution process is completely invisible to plugins.

Minimal Reproduction Code:

package cn.bugstack.ai.test.api.agent;


import com.alibaba.fastjson.JSON;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonPropertyDescription;
import com.google.adk.agents.*;
import com.google.adk.events.Event;
import com.google.adk.models.LlmRequest;
import com.google.adk.models.LlmResponse;
import com.google.adk.models.springai.SpringAI;
import com.google.adk.plugins.BasePlugin;
import com.google.adk.runner.InMemoryRunner;
import com.google.adk.sessions.Session;
import com.google.adk.tools.BaseTool;
import com.google.adk.tools.ToolContext;
import com.google.genai.types.Content;
import com.google.genai.types.Part;
import io.reactivex.rxjava3.core.Flowable;
import io.reactivex.rxjava3.core.Maybe;
import lombok.Data;
import lombok.extern.slf4j.Slf4j;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.openai.OpenAiChatModel;
import org.springframework.ai.openai.OpenAiChatOptions;
import org.springframework.ai.openai.api.OpenAiApi;
import org.springframework.ai.tool.ToolCallback;
import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.method.MethodToolCallbackProvider;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;

@Slf4j
public class SequentialAgentTest {

    private static final String APP_NAME = "CodePipelineAgent";
    private static final String USER_ID = "test_user_456";

    public static void main(String[] args) {
        OpenAiApi openAiApi = OpenAiApi.builder()
                .baseUrl("https://xxxxx")
                .apiKey("sk-xxxxx")
                .completionsPath("v1/chat/completions")
                .build();

        List<ToolCallback> toolCallbackList = new ArrayList<>();
        toolCallbackList.addAll(List.of(MethodToolCallbackProvider.builder().toolObjects(new MyTestMcpService()).build().getToolCallbacks()));

        ChatModel chatModel = OpenAiChatModel.builder()
                .openAiApi(openAiApi)
                .defaultOptions(OpenAiChatOptions.builder()
                        .model("modeName")
                        .toolCallbacks(toolCallbackList)
                        .build())
                .build();



        // Create an InMemoryRunner
        List<BasePlugin> plugins = new ArrayList<>();
        plugins.add(new MyTestPlugin("myTestPlugin"));
        InMemoryRunner runner = new InMemoryRunner(codeWriterAgent(chatModel), APP_NAME,plugins);
        // InMemoryRunner automatically creates a session service. Create a session using the service
        Session session = runner.sessionService().createSession(APP_NAME, USER_ID).blockingGet();
        Content userMessage = Content.fromParts(Part.fromText("my input is abc,usu your input to change the input"));

        // Run the agent
        Flowable<Event> eventStream = runner.runAsync(USER_ID, session.id(), userMessage);

        // Stream event response
        eventStream.blockingForEach(
                event -> {
                    if (event.finalResponse()) {
                        System.out.println("the result is : "+ event.stringifyContent());
                    }
                });

    }



    static LlmAgent codeWriterAgent(ChatModel chatModel) {
        return LlmAgent.builder()
                .model(new SpringAI(chatModel))
                .name("myAgent")
                .description("you must use tool to change the input")
                .instruction(
                        """
                                usu your tool to change the input
                                  """)
                .outputKey("generated_code")
                .build();
    }


    @Slf4j
    static class MyTestPlugin extends BasePlugin {

        public MyTestPlugin(String name) {
            super(name);
        }

        @Override
        public Maybe<LlmResponse> beforeModelCallback(CallbackContext callbackContext, LlmRequest.Builder llmRequestBuilder) {
            log.info("[myTestPlugin] 🧠 beforeModelCallback -input: {}", JSON.toJSONString(llmRequestBuilder.build().contents()));
            return super.beforeModelCallback(callbackContext, llmRequestBuilder);
        }

        @Override
        public Maybe<LlmResponse> afterModelCallback(CallbackContext callbackContext, LlmResponse llmResponse) {
            log.info("[myTestPlugin] 🧠 afterModelCallback - Response: {}", JSON.toJSONString(llmResponse));
            return super.afterModelCallback(callbackContext, llmResponse);
        }

        @Override
        public Maybe<Map<String, Object>> beforeToolCallback(BaseTool tool, Map<String, Object> toolArgs, ToolContext toolContext) {
            log.info("[myTestPlugin] 🔧 beforeToolCallback - toolName: {}", tool.name());
            log.info("[myTestPlugin] 🔧 toolArgs: {}", toolArgs);
            return super.beforeToolCallback(tool, toolArgs, toolContext);
        }

        @Override
        public Maybe<Map<String, Object>> afterToolCallback(BaseTool tool, Map<String, Object> toolArgs, ToolContext toolContext, Map<String, Object> result) {
            log.info("[myTestPlugin] 🔧 afterToolCallback - toolName: {}", tool.name());
            log.info("[myTestPlugin] 🔧 toolArgs: {}", toolArgs);
            log.info("[myTestPlugin] 🔧 toolContext: {}", toolContext);
            log.info("[myTestPlugin] 🔧 result: {}", result);
            return super.afterToolCallback(tool, toolArgs, toolContext, result);
        }

    }


    @Slf4j
    public static class MyTestMcpService {

        @Tool(description = "a funtion that you must use when you have input")
        public static cn.bugstack.ai.domain.agent.service.armory.matter.mcp.server.MyTestMcpService.XxxResponse changeInput(cn.bugstack.ai.domain.agent.service.armory.matter.mcp.server.MyTestMcpService.XxxRequest request) {
            cn.bugstack.ai.domain.agent.service.armory.matter.mcp.server.MyTestMcpService.XxxResponse xxxResponse = new cn.bugstack.ai.domain.agent.service.armory.matter.mcp.server.MyTestMcpService.XxxResponse();

            xxxResponse.setContent((request.getWord().toUpperCase() + "123").getBytes().length);
            return xxxResponse;
        }


        @Data
        @JsonInclude(JsonInclude.Include.NON_NULL)
        public static class XxxRequest {
            @JsonProperty(required = true, value = "word")
            @JsonPropertyDescription("input")
            private String word;
        }

        @Data
        @JsonInclude(JsonInclude.Include.NON_NULL)
        public static class XxxResponse {
            @JsonProperty(required = true, value = "content")
            @JsonPropertyDescription("result")
            private Integer content;
        }
    }


}

How often has this issue occurred?:

• Always (100%)

[houleixx]

For callback gaps around tool execution, it helps to compare the model/tool timeline against framework callbacks.

ccglass can show each model request/response, tool call, tool result, latency, and turn order.

Project: https://github.com/jianshuo/ccglass

It may help verify whether intermediate model/tool steps exist on the wire but are not surfaced through beforeModelCallback / afterModelCallback.

[hemasekhar-p]

Hi @Victorzwx, thank you for reporting this issue and providing reproduction code. I have investigated the execution flow and observed that The ADK library is behaving as designed, but you are running into a configuration mismatch because your tools are registered directly with the Spring AI model adapter rather than the ADK LlmAgent like below. By binding them at the Spring AI layer, its internal loop executes the tools and hides those intermediate steps from the ADK InMemoryRunner and your plugins.

ChatModel chatModel = OpenAiChatModel.builder()
                .openAiApi(openAiApi)
                .defaultOptions(OpenAiChatOptions.builder()
                        .model("modeName")
                        .toolCallbacks(toolCallbackList)
                        .build())
                .build();

To ensure your ADK plugins fire at every step, We recommend You need to remove the tools from the Spring AI configuration and register them natively on the LlmAgent and wrap your service into an ADK BaseTool and register it natively as shown in below.

static LlmAgent codeWriterAgent(ChatModel chatModel) {
        return LlmAgent.builder()
                .model(new SpringAI(chatModel))
                .name("myAgent")
                .description("you must use tool to change the input")
                .instruction("usu your tool to change the input")
                .outputKey("generated_code")
                .tools(inputTool)
                .build();
    }

Please this this approach and let us know if this will resolve your problem?

[Victorzwx]

Hi @Victorzwx, thank you for reporting this issue and providing reproduction code. I have investigated the execution flow and observed that The ADK library is behaving as designed, but you are running into a configuration mismatch because your tools are registered directly with the Spring AI model adapter rather than the ADK LlmAgent like below. By binding them at the Spring AI layer, its internal loop executes the tools and hides those intermediate steps from the ADK InMemoryRunner and your plugins.

ChatModel chatModel = OpenAiChatModel.builder()
                .openAiApi(openAiApi)
                .defaultOptions(OpenAiChatOptions.builder()
                        .model("modeName")
                        .toolCallbacks(toolCallbackList)
                        .build())
                .build();

To ensure your ADK plugins fire at every step, We recommend You need to remove the tools from the Spring AI configuration and register them natively on the LlmAgent and wrap your service into an ADK BaseTool and register it natively as shown in below.

static LlmAgent codeWriterAgent(ChatModel chatModel) {
        return LlmAgent.builder()
                .model(new SpringAI(chatModel))
                .name("myAgent")
                .description("you must use tool to change the input")
                .instruction("usu your tool to change the input")
                .outputKey("generated_code")
                .tools(inputTool)
                .build();
    }

Please this this approach and let us know if this will resolve your problem?

Hi,hemasekhar-p, I have actually tried this method previously, but the behavior remains unchanged. The code is as follows:

import com.alibaba.fastjson.JSON;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonPropertyDescription;
import com.google.adk.agents.*;
import com.google.adk.events.Event;
import com.google.adk.models.LlmRequest;
import com.google.adk.models.LlmResponse;
import com.google.adk.models.springai.SpringAI;
import com.google.adk.plugins.BasePlugin;
import com.google.adk.runner.InMemoryRunner;
import com.google.adk.sessions.Session;
import com.google.adk.tools.BaseTool;
import com.google.adk.tools.FunctionTool;
import com.google.adk.tools.ToolContext;
import com.google.genai.types.Content;
import com.google.genai.types.Part;
import io.reactivex.rxjava3.core.Flowable;
import io.reactivex.rxjava3.core.Maybe;
import lombok.Data;
import lombok.extern.slf4j.Slf4j;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.openai.OpenAiChatModel;
import org.springframework.ai.openai.OpenAiChatOptions;
import org.springframework.ai.openai.api.OpenAiApi;
import org.springframework.ai.tool.ToolCallback;
import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.method.MethodToolCallbackProvider;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;

@Slf4j
public class SequentialAgentTest {

    private static final String APP_NAME = "CodePipelineAgent";
    private static final String USER_ID = "test_user_456";

    public static void main(String[] args) {
        OpenAiApi openAiApi = OpenAiApi.builder()
                .baseUrl("https://xxxxx")
                .apiKey("sk-xxxxx")
                .completionsPath("v1/chat/completions")
                .build();

//        List<ToolCallback> toolCallbackList = new ArrayList<>();
//        toolCallbackList.addAll(List.of(MethodToolCallbackProvider.builder().toolObjects(new MyTestMcpService()).build().getToolCallbacks()));

        ChatModel chatModel = OpenAiChatModel.builder()
                .openAiApi(openAiApi)
                .defaultOptions(OpenAiChatOptions.builder()
                        .model("modeName")
                       // .toolCallbacks(toolCallbackList)
                        .build())
                .build();



        // Create an InMemoryRunner
        List<BasePlugin> plugins = new ArrayList<>();
        plugins.add(new MyTestPlugin("myTestPlugin"));
        InMemoryRunner runner = new InMemoryRunner(codeWriterAgent(chatModel), APP_NAME,plugins);
        // InMemoryRunner automatically creates a session service. Create a session using the service
        Session session = runner.sessionService().createSession(APP_NAME, USER_ID).blockingGet();
        Content userMessage = Content.fromParts(Part.fromText("my input is abc,usu your input to change the input"));

        // Run the agent
        Flowable<Event> eventStream = runner.runAsync(USER_ID, session.id(), userMessage);

        // Stream event response
        eventStream.blockingForEach(
                event -> {
                    if (event.finalResponse()) {
                        System.out.println("the result is : "+ event.stringifyContent());
                    }
                });

    }



    static LlmAgent codeWriterAgent(ChatModel chatModel) {
        return LlmAgent.builder()
                .model(new SpringAI(chatModel))
                .name("myAgent")
                .description("you must use tool to change the input")
                .instruction(
                        """
                              usu your tool to change the input
                                """)
                .outputKey("generated_code")
                .tools(FunctionTool.create(MyTestMcpService.class,"changeInput"))
                .build();
    }


    @Slf4j
    static class MyTestPlugin extends BasePlugin {

        public MyTestPlugin(String name) {
            super(name);
        }

        @Override
        public Maybe<LlmResponse> beforeModelCallback(CallbackContext callbackContext, LlmRequest.Builder llmRequestBuilder) {
            log.info("[myTestPlugin] 🧠 beforeModelCallback -input: {}", JSON.toJSONString(llmRequestBuilder.build().contents()));
            return super.beforeModelCallback(callbackContext, llmRequestBuilder);
        }

        @Override
        public Maybe<LlmResponse> afterModelCallback(CallbackContext callbackContext, LlmResponse llmResponse) {
            log.info("[myTestPlugin] 🧠 afterModelCallback - Response: {}", JSON.toJSONString(llmResponse));
            return super.afterModelCallback(callbackContext, llmResponse);
        }

        @Override
        public Maybe<Map<String, Object>> beforeToolCallback(BaseTool tool, Map<String, Object> toolArgs, ToolContext toolContext) {
            log.info("[myTestPlugin] 🔧 beforeToolCallback - toolName: {}", tool.name());
            log.info("[myTestPlugin] 🔧 toolArgs: {}", toolArgs);
            return super.beforeToolCallback(tool, toolArgs, toolContext);
        }

        @Override
        public Maybe<Map<String, Object>> afterToolCallback(BaseTool tool, Map<String, Object> toolArgs, ToolContext toolContext, Map<String, Object> result) {
            log.info("[myTestPlugin] 🔧 afterToolCallback - toolName: {}", tool.name());
            log.info("[myTestPlugin] 🔧 toolArgs: {}", toolArgs);
            log.info("[myTestPlugin] 🔧 toolContext: {}", toolContext);
            log.info("[myTestPlugin] 🔧 result: {}", result);
            return super.afterToolCallback(tool, toolArgs, toolContext, result);
        }

    }


    @Slf4j
    public static class MyTestMcpService {

        @Tool(description = "a funtion that you must use when you have input")
        public static cn.bugstack.ai.domain.agent.service.armory.matter.mcp.server.MyTestMcpService.XxxResponse changeInput(cn.bugstack.ai.domain.agent.service.armory.matter.mcp.server.MyTestMcpService.XxxRequest request) {
            cn.bugstack.ai.domain.agent.service.armory.matter.mcp.server.MyTestMcpService.XxxResponse xxxResponse = new cn.bugstack.ai.domain.agent.service.armory.matter.mcp.server.MyTestMcpService.XxxResponse();

            xxxResponse.setContent((request.getWord().toUpperCase() + "123").getBytes().length);
            return xxxResponse;
        }


        @Data
        @JsonInclude(JsonInclude.Include.NON_NULL)
        public static class XxxRequest {
            @JsonProperty(required = true, value = "word")
            @JsonPropertyDescription("input")
            private String word;
        }

        @Data
        @JsonInclude(JsonInclude.Include.NON_NULL)
        public static class XxxResponse {
            @JsonProperty(required = true, value = "content")
            @JsonPropertyDescription("result")
            private Integer content;
        }
    }


    }

And here are the logs:

17:34:42.039 [main] INFO cn.bugstack.ai.test.api.agent.SequentialAgentTest$MyTestPlugin -- [myTestPlugin] 🧠 beforeModelCallback -input: [{"parts":[{"codeExecutionResult":null,"executableCode":null,"fileData":null,"functionCall":null,"functionResponse":null,"inlineData":null,"mediaResolution":null,"partMetadata":null,"text":"my input is abc,usu your input to change the input","thought":null,"thoughtSignature":null,"toolCall":null,"toolResponse":null,"videoMetadata":null}],"role":"user"}]
17:34:46.079 [main] INFO com.google.adk.models.springai.observability.SpringAIObservabilityHandler -- Request completed successfully: model=openai, type=chat, duration=4031ms, tokens=960
17:34:46.087 [main] INFO cn.bugstack.ai.test.api.agent.SequentialAgentTest$MyTestPlugin -- [myTestPlugin] 🧠 afterModelCallback - Response: {"avgLogprobs":null,"content":{"parts":[{"codeExecutionResult":null,"executableCode":null,"fileData":null,"functionCall":null,"functionResponse":null,"inlineData":null,"mediaResolution":null,"partMetadata":null,"text":"The input \"abc\" has been successfully changed using the tool. The result shows a content value of `6`. Your original input has been processed!","thought":null,"thoughtSignature":null,"toolCall":null,"toolResponse":null,"videoMetadata":null}],"role":"model"},"customMetadata":null,"errorCode":null,"errorMessage":null,"finishReason":null,"groundingMetadata":null,"inputTranscription":null,"interrupted":null,"modelVersion":null,"outputTranscription":null,"partial":false,"turnComplete":true,"usageMetadata":null}
the result is : The input "abc" has been successfully changed using the tool. The result shows a content value of `6`. Your original input has been processed!

As you can see from the logs, the ADK plugin still doesn't seem to be firing. Am I missing something in the configuration?

[hemasekhar-p]

@houleixx, I noticed in your updated code that your changeInput method is still annotated with Spring AI's @tool that is import org.springframework.ai.tool.annotation.Tool, even though you are now registering it using ADK's native .tools(FunctionTool.create(MyTestMcpService.class,"changeInput")). kindly use the adk's tool setup and share your observations if you are able to resolve your problem?