Enabling certs for Cognee LLM calls

[LStromann]

=Hi <@508682252595626015>,

I'm trying to use cognee with a custom endpoint, but the SSL cert isn't picked up. All cognee requests are being blocked, I've tried a few patches to the HTTP client but it isn't working.

I've also tried setting LiteLLM config via env variables and python code too, the internal Cognee calls / LLM gateway doesn't pick up this config.

Is there a standard way of doing this?

Really appreciate it!

This discussion was automatically pulled from Discord.

[LStromann]

Just to confirm you've tried setting the LiteLLM related settings specified here: https://docs.litellm.ai/docs/guides/security_settings

And these settings were not read by the LiteLLM in Cognee?

Btw Cognee reads environment variables only once on starting of Cognee, maybe restrating it would help?

[LStromann]

Tried,

import litellm litellm.ssl_verify = "client.pem"

and

import litellm litellm.ssl_certificate = "/path/to/certificate.pem"

These two didn't work. The request still get blocked at Cloudflare. I've tried a simple curl command with curls, they reached the worker.

[LStromann]

Yes, I'm using a debugging script so it runs fresh everytime. I can share it if you want to take a look.

[LStromann]

Sure might help

[LStromann]

I will try and see why this isn't working in a few hours

[LStromann]

import asyncio
import os
from pathlib import Path

from pydantic import BaseModel
PEM_FILE = Path("~/Desktop/certs/client.pem").expanduser().resolve(strict=True)

import litellm

litellm.ssl_certificate = PEM_FILE
litellm.ssl_verify = PEM_FILE

from cognee import config
from cognee.infrastructure.llm import LLMGateway

class Echo(BaseModel):
text: str

async def main():
base = os.getenv("LLM_BASE").rstrip("/")
endpoint = f"{base}/ai-gateway/openai/v1"
key = os.getenv("KEY")

config.set_llm_config({
    "llm_provider": "openai",
    "llm_model": "gpt-4o-mini",
    "llm_endpoint": endpoint,
    "llm_api_key": key,
})

try:
    out = await LLMGateway.acreate_structured_output(
        text_input='Return JSON: {"text":"pong"}',
        system_prompt="You are a tester.",
        response_model=Echo,
    )
    print("Result:", out)
except Exception as e:
    print("LLMGateway error:", repr(e))

if name == "main":
asyncio.run(main())