Skip to content

Conversation

@cabljac
Copy link
Contributor

@cabljac cabljac commented Jul 9, 2024

Fixes #452

Only manually tested so far in a couple of specific cases.

Checklist (if applicable):

  • Tested (manually, unit tested, etc.)
  • Changelog updated
  • Docs updated

@cabljac cabljac requested a review from huangjeff5 July 10, 2024 08:23
@cabljac cabljac marked this pull request as ready for review July 10, 2024 08:23
@cabljac cabljac force-pushed the @invertase/vertexai-gemini-tool-response branch 3 times, most recently from d9318f5 to d3e3c97 Compare July 11, 2024 11:07
@cabljac cabljac force-pushed the @invertase/vertexai-gemini-tool-response branch from d3e3c97 to b8e1e41 Compare July 11, 2024 11:13
@MichaelDoyle
Copy link
Member

MichaelDoyle commented Jul 11, 2024

Discussing on Discord, but I am of the opinion this should be fixed further upstream. That could be in the middleware. But I am not 💯 about this interaction.

  1. The issue applies to both Google AI and Vertex AI
  2. [Prior to this fix] the context appears to "move" from the original user message down into the toolResponse during the tool calling loop. This is definitely not what I would expect. I would expect the original history to be maintained, and a toolResponse simply tacked on. IOW, there should be no need to "move it back".

Reproduced in a trace for Google AI:
https://gist.github.com/MichaelDoyle/a5e4336d92e4353b2cd5237f97200690

@cabljac
Copy link
Contributor Author

cabljac commented Jul 12, 2024

closing in favour of #591

@cabljac cabljac closed this Jul 12, 2024
@cabljac cabljac deleted the @invertase/vertexai-gemini-tool-response branch July 16, 2024 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[JS] VertexAI.ClientError when providing context from retriever and Gemini calling a provided tool

3 participants