Skip to content

Conversation

aabmass
Copy link
Member

@aabmass aabmass commented Sep 16, 2025

Description

Follow up to #3752. Implements an uploader with fsspec (optional dependency) to gen AI utils. fsspec generically supports many cloud and local backends like /path/on/disk, gs://, s3://..., etc. This PR contains the entry point, environment variable for configuring, queueing, and uploading. Next PR will add refs to telemetry.

Part of #3065, #3753

Example usage

export OTEL_INSTRUMENTATION_GENAI_UPLOAD_HOOK=fsspec

# s3 with configuration https://filesystem-spec.readthedocs.io/en/latest/features.html#configuration
export OTEL_INSTRUMENTATION_GENAI_UPLOAD_BASE_PATH="s3://foo/bar"
export FSSPEC_S3='{"anon": false, "key": "YOUR_ACCESS_KEY", "secret": "YOUR_SECRET_KEY", "client_kwargs": {"endpoint_url": "http://localhost:9000"}}'

# local dir
export OTEL_INSTRUMENTATION_GENAI_UPLOAD_BASE_PATH="/tmp/prompts"

opentelemetry-instrument my-agent.py

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • Added tests for the entry point, optional dependency handling, and queueing.

Does This PR Require a Core Repo Change?

  • No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

  • Followed the style guidelines of this project
  • Changelogs have been updated - will add once the feature is complete
  • Unit tests have been added
  • Documentation has been updated

@aabmass aabmass added the Skip Changelog PRs that do not require a CHANGELOG.md entry label Sep 16, 2025
@aabmass aabmass marked this pull request as ready for review September 16, 2025 04:05
@aabmass aabmass requested a review from a team as a code owner September 16, 2025 04:05
@aabmass aabmass requested a review from xrmx September 17, 2025 03:09
@aabmass aabmass merged commit 7819be1 into open-telemetry:main Sep 17, 2025
632 checks passed
@aabmass aabmass deleted the fsspec-hook branch September 17, 2025 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Skip Changelog PRs that do not require a CHANGELOG.md entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants