Remove path prefix from filename in profile

[mrocklin]

Currently in profiles we include the full filename in the profile information. These filenames look like this:

/srv/conda/evs/notebook/libs/python3.8/site-packages/fsspec/asyn.py

We might consider chopping off everything before site-packages so that we have the following:

fsspec/asyn.py

The path prefix is useful in that it helps us to immediately identify a file if we're on the same machine, but it also leaks slightly personal information, and is generally not helpful especially if we're looking at these results on a different system. Aesthetically this is also often the longest part of the printed result, and results in wonky display issues.

I propose that we strip out everything up-to-and-including "site-packages"

(example performance report for reference)

[mrocklin]

This work would be done in the distributed/profile.py file (search for filename to see most occurrences).

[mrocklin]

One can see these filenames in any of the "profile" tabs by hovering over any of the rectangles in the flame graph (the main chart on the page). One can view these live at /profile in the dashboard and one can inspect them by calling client.profile() for use in tests. A test might look something like the following:

@gen_cluster(client=True)
def test_cleaned_filenames_in_profile(c, s, a, b):
    # do some work with the client
    profile = await c.profile()
    # todo: make sure that we got a non-empty profile

    path_prefix = ... # strip some things off of sys.executable to get the equivalent of /srv/conda/envs/notebook/libs/python3.8 above
    assert path_prefix not in str(profile)

[ncclementi]

I'll be working on this one. Just a heads up.

[jrbourbeau]

Great, thanks @ncclementi!

[ncclementi]

I'm trying to find a good logic to either strip the part we don't want to show or to keep what we want. But can't seem to be able to generalize.
In profile.py the place where we are getting the filename that we want to strip out is in the function info_frame(frame) where we do "filename": co.co_filename.
Now this won't necessarily always be of the form /srv/conda/evs/notebook/libs/python3.8/site-packages/fsspec/asyn.py. For example if we are running a test from test_profile.py the path will look more like /Users/username/Documents/distributed/distributed/utils_test.py.

So the question is what do we want to actually display and how to set up the logic behind? Any ideas?