Extend .WithSource to Support APIs Requiring API Keys

[Kyadda99]

Extend .WithSource to Support APIs Requiring API Keys

This feature extends the existing .WithSource functionality to support integration with external APIs that require authentication via API keys. It enables developers to define custom API interactions and inject the response as context for LLM processing.

---

Implementation Overview

• Allows detailed configuration of API call parameters through AgentApiSourceDetails, including:
  • Authentication via AuthorisationToken passed as a header.
  • Flexible request definition, either by:
    • Supplying a raw curl command string.
    • Or optionally providing a structured Payload object (JSON).
• API responses are fetched and used as context alongside the LLM prompt.
• No assumptions are made about the structure of the request payload (e.g., "q" vs. "question"), ensuring compatibility with various external services.
• Added parsing for double-serialized JSON in text passed to the model, improving performance for smaller LLMs.

---

Example Usage

var context = AIHub.Agent()
    .WithModel("gemma2-2b")
    .WithInitialPrompt("Provide info on request recieid")
    .WithSource(new AgentApiSourceDetails()
    {
        Method = "Post",
        Url = "https://api.tavily.com/search",
        ResponseType = "JSON",
        ChunkLimit = 10,
        AuthorisationToken = token,
        Curl = @"curl --request POST \
            --url https://api.tavily.com/search \
            --header 'Authorization: Bearer tavily-dev-addwda' \
            --header 'Content-Type: application/json' \
            --data '{
                      ""query"": ""who is Leo Messi?""
                    }'"
    }, AgentSourceType.API)
    .WithSteps(StepBuilder.Instance.FetchData().Build())
    .Create();

var result = await context
    .ProcessAsync("Summarise me this");

Authorization Header Override

Note:
If both the AuthorisationToken property and an Authorization header are specified in the Curl command string,
the value provided in AuthorisationToken will override the Authorization header in the Curl string.

This ensures that the API key or token set via the AuthorisationToken property is always used for authentication.