httpx instrumentation doesn't work for httpx.get, httpx.post, etc.

[macieyng]

Describe your environment Describe any aspect of your environment relevant to the problem, including your Python version, platform, version numbers of installed dependencies, information about your cloud hosting provider, etc. If you're reporting a problem with a specific version of a library in this repo, please check whether the problem has been fixed on main.

#Python
Python 3.8.16

# OTEL
opentelemetry-api==1.15.0 ; python_version >= '3.7'
opentelemetry-instrumentation==0.36b0 ; python_version >= '3.7'
opentelemetry-instrumentation-httpx==0.36b0
opentelemetry-instrumentation-requests==0.36b0
opentelemetry-sdk==1.15.0 ; python_version >= '3.7'

# 3rd Party
httpx==0.23.3
requests==2.28.2 ; 
python_version >= '3.6'

Azure Function run locally.

Steps to reproduce
Describe exactly how to reproduce the error. Include a code sample if applicable.

1. Install requests and httpx libs and instrumentations
2. Initialize instrumentors
3. Make a get/post/etc request to some api without initializing a client.
4. Make a get/post/etc request to some api with initializing a client.

import httpx 
import requests

from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
from opentelemetry.instrumentation.requests import RequestsInstrumentor

HTTPXClientInstrumentor().instrument()
RequestsInstrumentor().instrument()


requests.get("www.google.com")
httpx.get("www.google.com")


with httpx.Client() as client:
    response = client.get("https://www.google.com")
with requests.Session() as session:
    response = session.get("https://www.google.com")

What is the expected behavior?
What did you expect to see?

Whenever I use httpx.get (or similar) a new dependency/span should be created and exported - in my case to Azure Application Insights.

What is the actual behavior?
What did you see instead?

I don't see expected dependencies/spans/traces exported.

Additional context
Full example of the function.

import httpx
import logging
import requests

import azure.functions as func
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
from opentelemetry.instrumentation.requests import RequestsInstrumentor

from tracing import OpenTelemetryExtension, TracedClass


OpenTelemetryExtension.configure(instrumentors=[
    HTTPXClientInstrumentor,
    RequestsInstrumentor
])


def main(req: func.HttpRequest, context: func.Context):
    with context.span():
        return _main(req)


class Greeter(TracedClass):
    trace_private_methods = True
    trace_exclude_methods = ["_private"]
        
    def __init__(self) -> None:
        super().__init__()
        
    def _private(self):
        pass
        
    def greet(self):
        self._private()
        response = httpx.get("https://api.namefake.com/")  # doesn't work with httpx
        response = requests.get("https://api.namefake.com/")  # works with requests
        body = response.json()
        return f"Hello {body['name']}"


def _main(req: func.HttpRequest) -> None:
    logger = logging.getLogger(__name__)
    logger.info("Hello world!")
    g = Greeter()
    greeting = g.greet()
    logger.info("%s", greeting)
    with httpx.Client() as client:
        response = client.get("https://www.google.com")  # works correctly
    with requests.Session() as session:
        response = session.get("https://www.google.com")  # works correctly
    return func.HttpResponse(response.content, headers={"Content-Type": "text/html"})

It uses OpenTelemetryExtension POC described here and TracedClass implemented as follows:

class MetaTracer(type):
    def __new__(cls, name, bases, attrs):
        for key, value in attrs.items():
            if any([
                type(value) is not FunctionType,
                key in attrs.get("trace_exclude_methods", [])
            ]):
                continue
            method_type = cls._get_method_type(key)
            trace_setting = f"trace_{method_type.value}_methods"
            should_be_traced = attrs.get(trace_setting, getattr(bases[0], trace_setting, False))
            if not should_be_traced:
                continue
            setattr(value, "__trace_name__", (
                f"{value.__module__}::{name}::{value.__name__}"
            ))
            attrs[key] = trace()(value)
        return super().__new__(cls, name, bases, attrs)
    
    class _MethodType(str, Enum):
        PUBLIC = "public"
        PRIVATE = "private"
        MAGIC = "magic"
    
    @staticmethod
    def _get_method_type(name: str) -> _MethodType:
        if name.startswith("__") and name.endswith("__"):
            return MetaTracer._MethodType.MAGIC
        if name.startswith("_"):
            return MetaTracer._MethodType.PRIVATE
        return MetaTracer._MethodType.PUBLIC


class TracedClass(metaclass=MetaTracer):
    trace_magic_methods = False
    trace_private_methods = False
    trace_public_methods = True
    trace_exclude_methods = []

While I understand why we should use httpx.Client over httpx.get there are reasons why people are using latter more often than former, but current implementation doesn't allow to trace simple requests. In the OpenCensus httpx extensions there was an opposite issue: httpx.Client was not traced instead - census-instrumentation/opencensus-python#1186.

I'm up for working on this issue and would love to hear your comments and tips.

[macieyng]

@lzchen @jeremydvoss

[phillipuniverse]

@macieyng funny that you opened this, I just hit the same thing where no httpx spans showed up. If you turn on debug logging in opentelemetry my guess is maybe you have a dependency conflict? I noticed you've got httpx 0.23:

[2023-04-06T15:58:29.753796-05:00] | DEBUG    | [service=tempus env=localdev version=no-rel-identifier-found trace_id=0 span_id=0] [opentelemetry.instrumentation.auto_instrumentation.sitecustomize] [sitecustomize.py:_load_instrumentors:76] - Skipping instrumentation httpx: DependencyConflict: requested: "httpx<=0.23.0,>=0.18.0" but found: "httpx 0.23.3"

This was caused by the changes at #1460

[macieyng]

@phillipuniverse I haven't check that, but seems like you're right about it. Thank you for linking the issues and making the PR!
There were similar issues with OpenCensus in the past that, some versions compatibility was too strict.