Skip to content

feat(litellm): improve GenAI util instrumentation#191

Closed
sipercai wants to merge 3 commits into
alibaba:mainfrom
sipercai:feat/litellm-genai-util
Closed

feat(litellm): improve GenAI util instrumentation#191
sipercai wants to merge 3 commits into
alibaba:mainfrom
sipercai:feat/litellm-genai-util

Conversation

@sipercai
Copy link
Copy Markdown
Collaborator

Description

This PR improves the LoongSuite LiteLLM instrumentation to populate GenAI util invocation data consistently for chat completions, streaming completions, and embeddings.

It adds positional argument normalization, richer LiteLLM response mapping, streaming output aggregation for multi-choice and tool-call deltas, streaming time-to-first-token reporting, provider inference for OpenAI-compatible endpoints, and a real smoke example for non-streaming, streaming, and concurrent calls.

Fixes # (N/A)

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

  • tox -e precommit
  • tox -c tox-loongsuite.ini -e py310-test-loongsuite-instrumentation-litellm -- instrumentation-loongsuite/loongsuite-instrumentation-litellm/tests/test_genai_util_wrapper.py instrumentation-loongsuite/loongsuite-instrumentation-litellm/tests/test_utils.py instrumentation-loongsuite/loongsuite-instrumentation-litellm/tests/test_stream_completion.py instrumentation-loongsuite/loongsuite-instrumentation-litellm/tests/test_embedding.py -q

Does This PR Require a Core Repo Change?

  • Yes. - Link to PR:
  • No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

@sipercai
Copy link
Copy Markdown
Collaborator Author

Migrated to upstream repository branch PR for Copilot review and organization-side review flow: #196

@sipercai sipercai closed this May 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants