How to review actual API calls made to LLMs?

[oooskarrr]

Can anyone tell me where I can review the complete API calls that were made to the LLMs during the automatic tag creation?

From the logs I can only see the total amount of tokens that were used:

info: [inference][49] Starting an inference job for bookmark with id "k6rxaazoag309jz594130xud"
info: [inference][49] Inferring tag for bookmark "k6rxaazoag309jz594130xud" used 1831 tokens and inferred: Artificial Intelligence,Large Language Models,Model Benchmarks,AI Model Pricing

I'd like to see the full API call, including the system/user prompt and what exactly Karakeep decided to send along with it so the LLM has sufficient context to generate these tags.

Are these full API calls logged/stored somewhere? If yes, where? And would you ever consider an option in the admin settings to make them easily accessible in order to fine-tune the prompts over time?

Cheers :)

[redge76]

If you seulhost, I can recommend LiteLLM. It's a bit of work but really rewarding once configured.
https://docs.litellm.ai/docs/proxy/ui_logs

You will have to use the MAX_STRING_LENGTH_PROMPT_IN_DB configuration option to capture the whole prompt.(otherwise truncated to a few characters)
https://docs.litellm.ai/docs/proxy/config_settings