Skip to content

httpx instrumentation doesn't work for httpx.get, httpx.post, etc. #1742

Closed
@macieyng

Description

@macieyng

Describe your environment Describe any aspect of your environment relevant to the problem, including your Python version, platform, version numbers of installed dependencies, information about your cloud hosting provider, etc. If you're reporting a problem with a specific version of a library in this repo, please check whether the problem has been fixed on main.

#Python
Python 3.8.16

# OTEL
opentelemetry-api==1.15.0 ; python_version >= '3.7'
opentelemetry-instrumentation==0.36b0 ; python_version >= '3.7'
opentelemetry-instrumentation-httpx==0.36b0
opentelemetry-instrumentation-requests==0.36b0
opentelemetry-sdk==1.15.0 ; python_version >= '3.7'

# 3rd Party
httpx==0.23.3
requests==2.28.2 ; 
python_version >= '3.6'

Azure Function run locally.

Steps to reproduce
Describe exactly how to reproduce the error. Include a code sample if applicable.

  1. Install requests and httpx libs and instrumentations
  2. Initialize instrumentors
  3. Make a get/post/etc request to some api without initializing a client.
  4. Make a get/post/etc request to some api with initializing a client.
import httpx 
import requests

from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
from opentelemetry.instrumentation.requests import RequestsInstrumentor

HTTPXClientInstrumentor().instrument()
RequestsInstrumentor().instrument()


requests.get("www.google.com")
httpx.get("www.google.com")


with httpx.Client() as client:
    response = client.get("https://www.google.com")
with requests.Session() as session:
    response = session.get("https://www.google.com")

What is the expected behavior?
What did you expect to see?

Whenever I use httpx.get (or similar) a new dependency/span should be created and exported - in my case to Azure Application Insights.

What is the actual behavior?
What did you see instead?

I don't see expected dependencies/spans/traces exported.

Zrzut ekranu 2023-04-3 o 12 53 01

Additional context
Full example of the function.

import httpx
import logging
import requests

import azure.functions as func
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
from opentelemetry.instrumentation.requests import RequestsInstrumentor

from tracing import OpenTelemetryExtension, TracedClass


OpenTelemetryExtension.configure(instrumentors=[
    HTTPXClientInstrumentor,
    RequestsInstrumentor
])


def main(req: func.HttpRequest, context: func.Context):
    with context.span():
        return _main(req)


class Greeter(TracedClass):
    trace_private_methods = True
    trace_exclude_methods = ["_private"]
        
    def __init__(self) -> None:
        super().__init__()
        
    def _private(self):
        pass
        
    def greet(self):
        self._private()
        response = httpx.get("https://api.namefake.com/")  # doesn't work with httpx
        response = requests.get("https://api.namefake.com/")  # works with requests
        body = response.json()
        return f"Hello {body['name']}"


def _main(req: func.HttpRequest) -> None:
    logger = logging.getLogger(__name__)
    logger.info("Hello world!")
    g = Greeter()
    greeting = g.greet()
    logger.info("%s", greeting)
    with httpx.Client() as client:
        response = client.get("https://www.google.com")  # works correctly
    with requests.Session() as session:
        response = session.get("https://www.google.com")  # works correctly
    return func.HttpResponse(response.content, headers={"Content-Type": "text/html"})

It uses OpenTelemetryExtension POC described here and TracedClass implemented as follows:

class MetaTracer(type):
    def __new__(cls, name, bases, attrs):
        for key, value in attrs.items():
            if any([
                type(value) is not FunctionType,
                key in attrs.get("trace_exclude_methods", [])
            ]):
                continue
            method_type = cls._get_method_type(key)
            trace_setting = f"trace_{method_type.value}_methods"
            should_be_traced = attrs.get(trace_setting, getattr(bases[0], trace_setting, False))
            if not should_be_traced:
                continue
            setattr(value, "__trace_name__", (
                f"{value.__module__}::{name}::{value.__name__}"
            ))
            attrs[key] = trace()(value)
        return super().__new__(cls, name, bases, attrs)
    
    class _MethodType(str, Enum):
        PUBLIC = "public"
        PRIVATE = "private"
        MAGIC = "magic"
    
    @staticmethod
    def _get_method_type(name: str) -> _MethodType:
        if name.startswith("__") and name.endswith("__"):
            return MetaTracer._MethodType.MAGIC
        if name.startswith("_"):
            return MetaTracer._MethodType.PRIVATE
        return MetaTracer._MethodType.PUBLIC


class TracedClass(metaclass=MetaTracer):
    trace_magic_methods = False
    trace_private_methods = False
    trace_public_methods = True
    trace_exclude_methods = []

While I understand why we should use httpx.Client over httpx.get there are reasons why people are using latter more often than former, but current implementation doesn't allow to trace simple requests. In the OpenCensus httpx extensions there was an opposite issue: httpx.Client was not traced instead - census-instrumentation/opencensus-python#1186.

I'm up for working on this issue and would love to hear your comments and tips.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions