function calling

[DmitriyG228]

Hey, I have been using function calling through llamaindex and langchain to delivere structured output. Can you give a hint on how to implement this in dspy?

[DmitriyG228]

I am tying to set function call with lm set in dspy.settings.
``
turbo = dspy.OpenAI(model='gpt-3.5-turbo-16k',
max_tokens=7000,
functions=[function],
function_call={"name": "client_meeting_insights"}, ) #gpt-3.5-turbo-16k
dspy.settings.configure(lm=turbo)`

the function:
function = { "name": "client_meeting_insights", "description": "A function that takes in a list of arguments related to client meetings and processes the insights.", "parameters": { "type": "object", "properties": { "start_time": { "type": "string", "description": "The start time of the insight in the format HH:mm." }, "tags": { "type": "array", "description": "Tags categorizing the type of insights.", "items": { "type": "string" } }, "description": { "type": "string", "description": "The description of the insight gathered from the client meeting." } } }, "required": ["start_time", "tags", "description"] }

here is the error:

`---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
/app/nbs/call_annotated_2610/evaluate.ipynb Cell 16 line 4
2 fail = []
3 for chunk in chunks:
----> 4 r = compiled.predict(chunk=chunk)
5 try:
6 success.append((chunk,eval(r.answer)))

File /usr/local/lib/python3.9/site-packages/dspy/predict/predict.py:57, in Predict.call(self, **kwargs)
56 def call(self, **kwargs):
---> 57 return self.forward(**kwargs)

File /usr/local/lib/python3.9/site-packages/dspy/predict/predict.py:81, in Predict.forward(self, **kwargs)
78 x = dsp.Example(demos=demos, **kwargs)
80 if self.lm is None:
---> 81 x, C = dsp.generate(signature, **config)(x, stage=self.stage)
82 else:
83 with dsp.settings.context(lm=self.lm, query_only=True):
84 # print(f"using lm = {self.lm} !")

File /usr/local/lib/python3.9/site-packages/dsp/primitives/predict.py:79, in _generate..do_generate(example, stage, max_depth, original_example)
77 prompt = template(example)
78 completions: list[dict[str, Any]] = generator(prompt, **kwargs)
---> 79 completions: list[Example] = [template.extract(example, p) for p in completions]
81 # Find the completions that are most complete.
82 field_names: list[str] = [field.input_variable for field in template.fields]

File /usr/local/lib/python3.9/site-packages/dsp/primitives/predict.py:79, in (.0)
77 prompt = template(example)
78 completions: list[dict[str, Any]] = generator(prompt, **kwargs)
---> 79 completions: list[Example] = [template.extract(example, p) for p in completions]
81 # Find the completions that are most complete.
82 field_names: list[str] = [field.input_variable for field in template.fields]

File /usr/local/lib/python3.9/site-packages/dsp/templates/template_v2.py:146, in TemplateV2.extract(self, example, raw_pred)
135 """Extracts the answer from the LM raw prediction using the template structure
136
137 Args:
(...)
142 Example: The example with the output variables filled in
143 """
144 example = dsp.Example(example)
--> 146 raw_pred = raw_pred.strip()
148 idx = 0
149 while idx < len(self.fields):

AttributeError: 'NoneType' object has no attribute 'strip'`

Any better ideas on how to achieve this?

[haddyadnan]

@DmitriyG228 Have you been able to achieve this?

[ctyler9]

having a similar issue. can't even call the simple RAG pipeline in a flask server. probably some threading issue

[jamesschinnerplxs]

I managed to get this to work by modifying _get_choice_text like so:

def _get_choice_text(self, choice: dict[str, Any]) -> str:
        if self.model_type == "chat":
            message = choice["message"]
            if content := message['content']:
                return content
            elif tool_calls := message.get('tool_calls', None):
                return json.dumps([tool['function']['arguments'] for tool in tool_calls])
        return choice["text"]

NOTE: I'm using tools and tool_choice https://platform.openai.com/docs/api-reference/chat/create#chat-create-tools

Basically the GPT3 object will only look for text response, and expects a str be returned (hence why I json encode the returned tool calls).

Diff: https://github.com/stanfordnlp/dspy/compare/main...jamesschinnerplxs:dspy:handle-tool-calls?expand=1

PS. I don't think this is a good solution, but it's letting me hack at it ;)