Skip to content

Adding RubyLLM Support #18

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Adding RubyLLM Support #18

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
14bb1e6
adding gen_ai
aniamisiorek Nov 11, 2025
987d252
adding halting
aniamisiorek Nov 12, 2025
6b2bc5f
coderabbit comments
aniamisiorek Nov 14, 2025
3e0b4c5
adding conversation id
aniamisiorek Nov 17, 2025
da3f2b4
adding conversation id -- fix
aniamisiorek Nov 17, 2025
6509a5d
guarding rubyLLM usage
aniamisiorek Nov 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions semantic_conventions_ai/lib/opentelemetry/semantic_conventions.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,17 @@ module SpanAttributes

# Deprecated
TRACELOOP_CORRELATION_ID = "traceloop.correlation.id"

# Gen AI
Copy link
Member

@nirga nirga Nov 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just replace the existing ones

Copy link
Author

@aniamisiorek aniamisiorek Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and remove references to llm semantics?

GEN_AI_REQUEST_MODEL = "gen_ai.request.model"
GEN_AI_RESPONSE_MODEL = "gen_ai.response.model"
GEN_AI_USAGE_OUTPUT_TOKENS = "gen_ai.usage.output_tokens"
GEN_AI_USAGE_INPUT_TOKENS = "gen_ai.usage.input_tokens"
GEN_AI_COMPLETIONS = "gen_ai.completion"
GEN_AI_PROMPTS = "gen_ai.prompt"
GEN_AI_SYSTEM = "gen_ai.system"
GEN_AI_PROVIDER = "gen_ai.provider.name"
GEN_AI_CONVERSATION_ID = "gen_ai.conversation.id"
end

module LLMRequestTypeValues
Expand Down
82 changes: 62 additions & 20 deletions traceloop-sdk/lib/traceloop/sdk.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,21 @@ module Traceloop
module SDK
class Traceloop
def initialize
api_key = ENV["TRACELOOP_API_KEY"]
raise "TRACELOOP_API_KEY environment variable is required" if api_key.nil? || api_key.empty?

OpenTelemetry::SDK.configure do |c|
c.add_span_processor(
OpenTelemetry::SDK::Trace::Export::SimpleSpanProcessor.new(
OpenTelemetry::SDK::Trace::Export::BatchSpanProcessor.new(
OpenTelemetry::Exporter::OTLP::Exporter.new(
endpoint: "#{ENV.fetch("TRACELOOP_BASE_URL", "https://api.traceloop.com")}/v1/traces",
headers: { "Authorization" => "Bearer #{ENV.fetch("TRACELOOP_API_KEY")}" }
headers: {
"Authorization" => "#{ENV.fetch("TRACELOOP_AUTH_SCHEME", "Bearer")} #{ENV.fetch("TRACELOOP_API_KEY")}"
Copy link
Member

@nirga nirga Nov 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is that?

Copy link
Author

@aniamisiorek aniamisiorek Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

authorization headers may vary in type. i.e. for our dynatrace exporter, they use Api-Token.

}
)
)
)
puts "Traceloop exporting traces to #{ENV.fetch("TRACELOOP_BASE", "https://api.traceloop.com")}"
puts "Traceloop exporting traces to #{ENV.fetch("TRACELOOP_BASE_URL", "https://api.traceloop.com")}"
end

@tracer = OpenTelemetry.tracer_provider.tracer("Traceloop")
Expand All @@ -41,25 +46,30 @@ def log_messages(messages)
def log_prompt(system_prompt="", user_prompt)
unless system_prompt.empty?
@span.add_attributes({
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_PROMPTS}.0.role" => "system",
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_PROMPTS}.0.content" => system_prompt,
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_PROMPTS}.1.role" => "user",
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_PROMPTS}.1.content" => user_prompt
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_PROMPTS}.0.role" => "system",
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_PROMPTS}.0.content" => system_prompt,
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_PROMPTS}.1.role" => "user",
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_PROMPTS}.1.content" => user_prompt
})
else
@span.add_attributes({
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_PROMPTS}.0.role" => "user",
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_PROMPTS}.0.content" => user_prompt
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_PROMPTS}.0.role" => "user",
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_PROMPTS}.0.content" => user_prompt
})
end
end

def log_response(response)
if response.respond_to?(:body)
log_bedrock_response(response)
# Check for RubyLLM::Message objects
elsif defined?(::RubyLLM::Message) && response.is_a?(::RubyLLM::Message)
log_ruby_llm_message(response)
elsif defined?(::RubyLLM::Tool::Halt) && response.is_a?(::RubyLLM::Tool::Halt)
log_ruby_llm_halt(response)
# This is Gemini specific, see -
# https://github.com/gbaptista/gemini-ai?tab=readme-ov-file#generate_content
elsif response.has_key?("candidates")
elsif response.respond_to?(:has_key?) && response.has_key?("candidates")
log_gemini_response(response)
else
log_openai_response(response)
Expand All @@ -73,10 +83,29 @@ def log_gemini_response(response)

@span.add_attributes({
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_COMPLETIONS}.0.role" => "assistant",
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_COMPLETIONS}.0.content" => response.dig("candidates", 0, "content", "parts", 0, "text")
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_COMPLETIONS}.0.content" => response.dig(
"candidates", 0, "content", "parts", 0, "text")
})
end

def log_ruby_llm_message(response)
@span.add_attributes({
OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_RESPONSE_MODEL => response.model_id,
OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_USAGE_OUTPUT_TOKENS => response.output_tokens || 0,
OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_USAGE_INPUT_TOKENS => response.input_tokens || 0,
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_COMPLETIONS}.0.role" => response.role.to_s,
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_COMPLETIONS}.0.content" => response.content
})
end

def log_ruby_llm_halt(response)
@span.add_attributes({
OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_RESPONSE_MODEL => @model,
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_COMPLETIONS}.0.role" => "tool",
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_COMPLETIONS}.0.content" => response.content
})
end

def log_bedrock_response(response)
body = JSON.parse(response.body.read())

Expand Down Expand Up @@ -109,25 +138,38 @@ def log_openai_response(response)
})
if response.has_key?("usage")
@span.add_attributes({
OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_USAGE_TOTAL_TOKENS => response.dig("usage", "total_tokens"),
OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_USAGE_COMPLETION_TOKENS => response.dig("usage", "completion_tokens"),
OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_USAGE_PROMPT_TOKENS => response.dig("usage", "prompt_tokens"),
OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_USAGE_TOTAL_TOKENS => response.dig("usage",
"total_tokens"),
OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_USAGE_COMPLETION_TOKENS => response.dig(
"usage", "completion_tokens"),
OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_USAGE_PROMPT_TOKENS => response.dig("usage",
"prompt_tokens"),
})
end
if response.has_key?("choices")
@span.add_attributes({
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_COMPLETIONS}.0.role" => response.dig("choices", 0, "message", "role"),
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_COMPLETIONS}.0.content" => response.dig("choices", 0, "message", "content")
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_COMPLETIONS}.0.role" => response.dig(
"choices", 0, "message", "role"),
"#{OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_COMPLETIONS}.0.content" => response.dig(
"choices", 0, "message", "content")
})
end
end
end

def llm_call(provider, model)
def llm_call(provider, model, conversation_id: nil)
@tracer.in_span("#{provider}.chat") do |span|
span.add_attributes({
OpenTelemetry::SemanticConventionsAi::SpanAttributes::LLM_REQUEST_MODEL => model,
})
attributes = {
OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_REQUEST_MODEL => model,
OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_SYSTEM => provider,
OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_PROVIDER => provider,
}

if conversation_id
attributes[OpenTelemetry::SemanticConventionsAi::SpanAttributes::GEN_AI_CONVERSATION_ID] = conversation_id
end

span.add_attributes(attributes)
yield Tracer.new(span, provider, model)
end
end
Expand Down
4 changes: 2 additions & 2 deletions traceloop-sdk/traceloop-sdk.gemspec
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ Gem::Specification.new do |spec|

spec.add_dependency 'opentelemetry-semantic_conventions_ai', '~> 0.0.3'

spec.add_dependency 'opentelemetry-sdk', '~> 1.3.1'
spec.add_dependency 'opentelemetry-exporter-otlp', '~> 0.26.1'
spec.add_dependency 'opentelemetry-exporter-otlp', '~> 0.31.1'
spec.add_dependency 'opentelemetry-sdk', '~> 1.10.0'

if spec.respond_to?(:metadata)
spec.metadata['source_code_uri'] = 'https://github.com/traceloop/openllmetry-ruby/tree/main/traceloop-sdk'
Expand Down