Say when a model never calls tools - #76
Merged
Merged
Conversation
로컬 모델 붙이고 나서 생긴 문제. tool use 못 하는 작은 모델은 도구 정의를 받고도 그냥 말로만 답해서, 작업하기를 눌러도 파일이 하나도 안 바뀐다. 앱이 고장난 줄 알기 딱 좋다. 턴마다 경고하면 시끄럽다 — 도구 필요 없는 질문이 더 많으니까. 그래서 한 번이라도 도구를 부른 적 있으면 그냥 넘어가고, 그런 적 없으면서 연속 3턴 비었을 때만 한 번 알린다.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Leftover from the provider work in #72.
Small local models either don't support tool calling or accept the tool definitions and answer in prose anyway. When that happens the agent talks and changes nothing, which looks like the app being broken rather than the model being unable.
Telling the user on every tool-less turn would be noise — most questions ("what does this function do?") need no tools. So two things have to hold: the provider has never once made a tool call, and three consecutive turns went out with tools and came back with none. Then it says so, once per provider. Turns sent without tools aren't counted at all, since they prove nothing either way.
The decision lives in
src/ai/toolSupport.tsas pure functions — 10 tests, including the case where a model starts using tools later and the streak resets.Checked against a stub server that receives the tool definitions and never calls anything: silent on turns 1 and 2, one notice on turn 3, silent again on 4 and 5.