Instrument FastAPI app with Sentry and OTel and check differences.

[antonpirker]

Instrument a Django application once with Sentry and once with Sentry using OpenTelementry for span creation. We want to compare the span waterfalls, to see the differences and what needs to be done to have feature parity.

Test setup:

Application: https://github.com/getsentry/demo-movie-search/tree/main/backend/fastapi
A request to the /api/shows/{pk}/ API endpoint will do:

• Run a FastAPI view with a custom middleware
• Runs a database query
• In the middleware a Redis counter is increased
• Start a FastAPI background task

Scenario 1) Sentry:

sentry_sdk.init(
    dsn=sentry_dsn,
    release=sentry_release,
    environment=sentry_environment,
    traces_sample_rate=sentry_traces_sample_rate,
    send_default_pii=sentry_default_pii,
    debug=sentry_debug,
)

This will load Sentries integrations (like FastAPI, Starlette, ...) by default and those Sentry integrations will create spans. Opentelementry is not used.

Scenario 2) Sentry using OpenTelementry:

sentry_sdk.init(
    dsn=sentry_dsn,
    release=sentry_release,
    environment=sentry_environment,
    traces_sample_rate=sentry_traces_sample_rate,
    send_default_pii=sentry_default_pii,
    debug=sentry_debug,
    _experiments={
        "otel_powered_performance": True
    },
)

This will disable all Sentry span creation and spans will only be created by the OpenTelemetry SDK.

[antonpirker]

Example traces for both scenarios:

1. Sentry:
   https://sentry-sdks.sentry.io/performance/trace/468c8e612cf04fff85a6d7a08643dac9/?node=ag-bee52f45864e1065&node=txn-d1a91910d62640f3af1a7f9098bf42f7&project=6618415&query=http.method%3AGET&referrer=performance-transaction-summary&showTransactions=recent&source=performance_transaction_summary&statsPeriod=1h&timestamp=1719555666&transaction=%2Fapi%2Fshows%2F%7Bshow_id%7D&unselectedSeries=p100%28%29&unselectedSeries=avg%28%29

2. Sentry using OTel:
   https://sentry-sdks.sentry.io/performance/trace/a159065d04eeb490536d391007264ed1/?node=txn-b691b33f2c0e45c686331b138915ed0b&project=6618415&query=http.method%3AGET&referrer=performance-transaction-summary&showTransactions=recent&source=performance_transaction_summary&statsPeriod=1h&timestamp=1719555842&transaction=%2Fapi%2Fshows%2F%7Bshow_id%7D&unselectedSeries=p100%28%29&unselectedSeries=avg%28%29

[antonpirker]

1. Sentry:

Trace in Sentry.io

[antonpirker]

2. Sentry using OpenTelemetry

Trace in Sentry.io

[antonpirker]

Things I noticed:

• db connect is missing in Sentry instrumentation
• OTel does not instrument middleware
• None of the above show that a FastAPI background task has been run.

[antonpirker]

OTel db spans also include information about the http server:

[antonpirker]

can be closed.