How to use RIG without automatically executing tool calls? #1634
qianxiaoming
started this conversation in
General
Replies: 2 comments 2 replies
|
There’s a full example here: It shows the manual pattern: use |
2 replies
|
Thank you. I'm sorry I overlooked the example. |
|
You didn't overlook it, I added it specifically for your question :) Happy hacking |
|
@gold-silver-copper Hi, You provided a good example, but when I changed it to streaming mode and used other models compatible with OpenAI, I found that the reasoning content must be removed from the history, otherwise the following error occurs: My code in loop just like this: ......
let mut tool_calls = Vec::<ToolCall>::new();
let mut total_tokens = 0u64;
let mut response = request.stream().await?;
while let Some(item) = response.next().await {
let content = item?;
match content {
StreamedAssistantContent::Text(text) => {
if state != StreamingState::Text {
println!("\nText:");
state = StreamingState::Text;
}
print!("{}", text.text());
io::stdout().flush()?;
}
StreamedAssistantContent::ReasoningDelta { reasoning, .. } => {
if state != StreamingState::Reasoning {
println!("Thinking:");
state = StreamingState::Reasoning;
}
print!("{}", reasoning);
io::stdout().flush()?;
}
StreamedAssistantContent::ToolCall { tool_call, .. } => {
tool_calls.push(tool_call);
}
StreamedAssistantContent::Final(resp) => {
total_tokens = resp.usage.total_tokens;
},
_ => {}
}
}
history.push(current_prompt.clone());
history.push(Message::Assistant {
id: response.message_id.clone(),
content: response.choice.clone()
});So I have to add this function: fn filter_assistant_content(content: &OneOrMany<AssistantContent>) -> OneOrMany<AssistantContent> {
let filtered: Vec<_> = content
.iter()
.filter_map(|c| match c {
AssistantContent::Text(_) => Some(c.clone()),
AssistantContent::ToolCall(_) => Some(c.clone()),
_ => None,
})
.collect();
OneOrMany::many(filtered).unwrap()
}and change the last history.push(Message::Assistant {
id: response.message_id.clone(),
content: filter_assistant_content(&response.choice),
});It is work, but I don't need to do this in non-streaming mode. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I hope to get one or multiple tool calls from the LLM output, without automatic execution. How can I set this up? Thanks.
All reactions