Please add Ray framework integration for sentry/apm.

[dPeS]

Problem Statement

https://www.ray.io/

I want to annotate that kind of code with spans:

import ray
import os
import time
import sentry_sdk
import random
import asyncio
from sentry_sdk.integrations.asyncio import AsyncioIntegration

logging.basicConfig(level=logging.DEBUG)

def step1():
    with sentry_sdk.start_span(description="step1"):
        time.sleep(random.randint(200,500)/1000.)
def step2():
    with sentry_sdk.start_span(description="step2"):
        time.sleep(random.randint(200,500)/1000.)
def step3():
    with sentry_sdk.start_span(description="step3"):
        time.sleep(random.randint(200,500)/1000.)

@ray.remote
def retrieve_task(item):
    step1()
    step2()
    step3()
    return 1

async def main():
    sentry_sdk.init(
        dsn=os.getenv('SENTRY_DSN'),
        traces_sample_rate=1.0,
        integrations=[
            AsyncioIntegration(),
        ],
    )
    #---
    ray.init()
    with sentry_sdk.start_transaction(op="task", name="example ray task transaction"):
        object_references = [
            retrieve_task.remote(item) for item in range(8)
        ]
        data = ray.get(object_references)
        print(data)

if __name__ ==  '__main__':
    asyncio.run(main())

But I only see one big transaction trace without those 3 spans.

Solution Brainstorm

I think new integration needs to be done.

[antonpirker]

Hey @dPeS !

Thanks for bringing this up!

I guess the problem is that the step*() functions are run in another process or something like this (don't know how Ray works) And that the tracing data are not propagated to the other process thus the spans can not be attached to the transaction you create...

[antonpirker]

Lets test how much demand there is:

If you want to see us support Ray, please "Thumb up" this issue!

[dPeS]

I guess the problem is that the step*() functions are run in another process or something like this (don't know how Ray works) And that the tracing data are not propagated to the other process thus the spans can not be attached to the transaction you create...

@antonpirker not the only problem - in those "remote" ray workers sentry isn't initialized - that's for starter.

[glowskir]

@szokeasaurusrex Hey I made PR with Ray integration, it handles tracing propagation to ray cluster, feedback is more than welcome! #2444

[antonpirker]

Thanks @glowskir for the integration! We will have a look sometime soon. Can not promise any ETA yet.

[leokster]

@antonpirker is it planned to merge this PR? ray is becoming more and more popular, would be great to use sentry with it.

[szokeasaurusrex]

@leokster Thanks for reaching out, I will try to look into this later this week