cannot use Azure OpenAI any more

[behrica]

Using this in v 0.3.6

(def template "
Answer the following question by outputting the letters A, B, C, D, and E in order of the most likely to be correct to the to least likely to be correct.

{{question}}

A) {{option-a}}

{% llm-generate model=text-davinci-003 var-name=play %}
")


(bg/complete-template template
             {:question "Which of the following statements accurately describes the impact of Modified Newtonian Dynamics (MOND) on the observed \"missing baryonic mass\" discrepancy in galaxy clusters?"
              :option-a "MOND is a theory that reduces the observed missing baryonic mass in galaxy clusters by postulating the existence of a new form of matter called \"fuzzy dark matter.\""
             {:play :llm/azure})

config.edn

{:azure-openai-api-key      <correct key>
 :azure-openai-api-endpoint <correct endpoint>}

I am not able to switch to use azure open AI. I get exception on missing/wrong OpenAI key.

I debugged this intensively but could not figure out, if my configuration or teh code is teh problem.

[behrica]

Log output:

2023-09-09T14:56:15.842Z nixos INFO [bosquet.llm.openai:70] - Calling OAI with:
2023-09-09T14:56:15.843Z nixos INFO [bosquet.llm.openai:71] - 	Params: '{:model "text-davinci-003", :temperature 0.2, :max_tokens 250, :presence_penalty 0.4, :frequency_penalty 0.2, :n 1, :top_p 1}'
2023-09-09T14:56:15.844Z nixos INFO [bosquet.llm.openai:72] - 	Config: '{:impl :openai, :api-endpoint "https://api.openai.com/v1"}'

[behrica]

I think I do the same as this code example:
https://github.com/zmedelis/bosquet#generation

[behrica]

From my debuugin it seems that it fails in this line allreday:

bosquet/src/bosquet/complete.clj

Line 23 in 95d410a

(let [llm (system/llm-service (get-in opts [system/llm-service-key gen-key]))]

It does not take the passed in options into account to construct the llm record.

[behrica]

Even if this syntax would work:

(bg/complete-template template
   ...
             {:play :llm/azure})

how should we be able to configure the options for "llm/azure" ?

[behrica]

The issue applies to complete-template only.
generate does work with Azure.

[zmedelis]

I do not have MS Azure account (tried going through the OAI application form and it requires a company email etc).

Is this still the case with v.0.3.7? If you go to bosquet.llm.generator comment section at the bottom, and change config to point to Azure service like that

  (complete-template
   "You are a playwright. Given the play's title and it's genre write a synopsis for that play.
     Title: {{title}}
     Genre: {{genre}}
     Playwright: This is a synopsis for the above play: {% gen var-name=text %}"
   {:title "Mr. X" :genre "crime"}
   {:text {:bosquet.llm/service [:llm/openai :provider/azure]
              :bosquet.llm/model-parameters {:temperature 0 :model "gpt-4"}}})

Does it work? For both complete-template and generate?

[behrica]

yes, both work.
We can close the issue.