Scalene: internal error: unable to find Python allocator functions

[didimelli]

Describe the bug
When starting scalene (with poetry run python -m scalene my_script.py), I get the error Scalene: internal error: unable to find Python allocator functions. This happens both with poetry run python -m scalene my_script.py or if started within the venv (python -m scalene my_script.py or just scalene my_script.py).

Some seconds after this output, i get a long traceback (reported afterwards).

I don't really think this is something about my program, since it always worked without scalene, but I think it is something related to Trio.

To Reproduce
Steps to reproduce the behavior:

Snippet that cause the error:

#asd.py

import time
import trio
time.sleep(3)

and then poetry run python -m scalene asd.py. This gives this traceback:

Error in program being profiled:
 string index out of range
Traceback (most recent call last):
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/scalene/scalene_profiler.py", line 1326, in profile_code
    exec(code, the_globals, the_locals)
  File "asd.py", line 2, in <module>
    import trio
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/trio/__init__.py", line 48, in <module>
    from ._sync import (
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/trio/_sync.py", line 475, in <module>
    @attr.s(eq=False, hash=False, repr=False)
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/attr/_make.py", line 1312, in wrap
    field_transformer,
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/attr/_make.py", line 610, in __init__
    field_transformer,
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/attr/_make.py", line 503, in _transform_attrs
    AttrsClass = _make_attr_tuple_class(cls.__name__, attr_names)
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/attr/_make.py", line 299, in _make_attr_tuple_class
    eval(compile("\n".join(attr_class_template), "", "exec"), globs)
  File "", line 1, in <module>
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/scalene/scalene_profiler.py", line 231, in invalidate_lines
    if f.f_code.co_filename[0] == "<" or "scalene" in f.f_code.co_filename:
IndexError: string index out of range

Commenting out the line import trio, makes everything works.

However, the line Scalene: internal error: unable to find Python allocator functions is always present.

Desktop (please complete the following information):

• OS: WSL2
• Python version: 3.7.10, installed with pyenv
• Scalene version: 1.3.15

[emeryberger]

Thanks for the report. I've fixed the case triggered by import trio; looking into the allocator function issue now.

[didimelli]

Awesome! Let me know if you need any info about my environment for the allocator function issue!

[sternj]

Running on the current HEAD of scalene's master branch I am not seeing any issues, please try downloading Scalene's source and adding it to pyproject.toml in the tool.poetry.dependencies section with scalene = { path = "/path/to/scalene/", develop = true } and post here if the error continues.

[didimelli]

Hi! Done what you proposed about the local dependency, and i still get the allocator function issue. The trio one is gone though!

Let me know if I can do (and how) some debugging on my side (logging, ...)

[sternj]

Replicated on python 3.7.10, does not arise on 3.8.10. Attempting on 3.7.12

[sternj]

Error arises in 3.7.12

[sternj]

This seems to be a python3.7 issue instead of a Poetry issue

[didimelli]

I made a 3.8 environment using the same tools (pyenv, poetry, scalene as a local dep, ...) and the allocator function issue does not appear. So no pyenv nor poetry seem to be related to the issue.

[sternj]

I need to verify this by doing a binary install of Python 3.7, but it appears to be either a python 3.7 issue or an issue with the interaction between Pyenv and Python 3.7. Pyenv starts up a bunch of child processes and for some reason one of the children doesn't load libpython

[LvJC]

Same error when importing mediapipe package even you updated

Scalene: internal error: unable to find Python allocator functions
Error in program being profiled:
 string index out of range
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/scalene/scalene_profiler.py", line 1326, in profile_code
    exec(code, the_globals, the_locals)
  File "src/long_to_short_video.py", line 18, in <module>
    import content_relevance
  File "/workspace/src/content_relevance.py", line 10, in <module>
    from det_hub.detection import large_data
  File "/workspace/src/mimic-octopus/det_hub/detection.py", line 17, in <module>
    from pose import MediapipPoseClient as PoseClient
  File "/workspace/src/mimic-octopus/det_hub/pose/__init__.py", line 1, in <module>
    from .mediapip import InferenceClient as MediapipPoseClient
  File "/workspace/src/mimic-octopus/det_hub/pose/mediapip/__init__.py", line 1, in <module>
    from .client import InferenceClient
  File "/workspace/src/mimic-octopus/det_hub/pose/mediapip/client.py", line 1, in <module>
    import mediapipe as mp
  File "/usr/local/lib/python3.7/dist-packages/mediapipe/__init__.py", line 17, in <module>
    import mediapipe.python.solutions as solutions
  File "/usr/local/lib/python3.7/dist-packages/mediapipe/python/solutions/__init__.py", line 25, in <module>
    import mediapipe.python.solutions.objectron
  File "/usr/local/lib/python3.7/dist-packages/mediapipe/python/solutions/objectron.py", line 96, in <module>
    class ObjectronModel(object):
  File "/usr/local/lib/python3.7/dist-packages/attr/_make.py", line 1489, in wrap
    field_transformer,
  File "/usr/local/lib/python3.7/dist-packages/attr/_make.py", line 684, in __init__
    field_transformer,
  File "/usr/local/lib/python3.7/dist-packages/attr/_make.py", line 576, in _transform_attrs
    AttrsClass = _make_attr_tuple_class(cls.__name__, attr_names)
  File "/usr/local/lib/python3.7/dist-packages/attr/_make.py", line 367, in _make_attr_tuple_class
    _compile_and_eval("\n".join(attr_class_template), globs)
  File "/usr/local/lib/python3.7/dist-packages/attr/_make.py", line 319, in _compile_and_eval
    eval(bytecode, globs, locs)
  File "", line 1, in <module>
  File "/usr/local/lib/python3.7/dist-packages/scalene/scalene_profiler.py", line 231, in invalidate_lines
    if f.f_code.co_filename[0] == "<" or "scalene" in f.f_code.co_filename:
IndexError: string index out of range

Thanks for the report. I've fixed the case triggered by import trio; looking into the allocator function issue now.

[tungdam]

I have the same issue with python 3.7.3

[root@tungdam10-virt.dev:/tmp]# cat test.py
import time
print("hello")
time.sleep(3)

[root@tungdam10-virt.dev:/tmp]# scalene test.py
Scalene: internal error: unable to find Python allocator functions
hello
....

First time i try scalene so not sure if i did anything wrong. Advice please

[ijpulidos]

I can confirm the error message appearing with python 3.8.12 version.

[emeryberger]

With conda or without? We are trying to figure out what on earth is going on here, since naturally it works fine on our end and in our CI environment. Thanks!

[ijpulidos]

With conda or without? We are trying to figure out what on earth is going on here, since naturally it works fine on our end and in our CI environment. Thanks!

Using conda and python 3.8.12 from conda-forge, and I do see the allocators exist, as in

❯ for fn in $(find /home/user/miniconda3/envs/env-dev-profiling/lib/ -name "libpython*.so"); do echo $fn; readelf -s $fn | grep -i "pymem_[gs]etallocator"; done
/home/user/miniconda3/envs/env-dev-profiling/lib/libpython3.8.so
   626: 00000000001adf00    64 FUNC    GLOBAL DEFAULT   11 PyMem_GetAllocator
  1450: 00000000001adf40    39 FUNC    GLOBAL DEFAULT   11 PyMem_SetAllocator
 34188: 00000000001adf40    39 FUNC    GLOBAL DEFAULT   11 PyMem_SetAllocator
 34998: 00000000001adf00    64 FUNC    GLOBAL DEFAULT   11 PyMem_GetAllocator

EDIT: It works fine with simple code as the one in #278 (comment) and also in other simpler codes of mine that use CUDA.

[michellehoog]

Getting the same error on WSL2, running a conda environment with python 3.7.11. I still get profiling information from scalene, but the message is always present.

[emeryberger]

Update: we believe we have finally figured out what's going on and hope to have a fix out soon!

[MS-AvivG]

Downgrading to Scalene version 1.3.12 fixes the issue for me

[jaltmayerpizzorno]

This should be fixed now (in yesterday's release). Please re-open if it's not.

[SnoopJ]

If anybody else has been following this issue and wants to know a bit more about the details of the fix, I believe these are the most important changes between v1.3.15 and v1.3.16

• c9e2bf2 - handle empty filename edge case
• F20211019 cant find allocator functions #286 - only call PyMem_*() if all symbols are found
• F20211021 preload on non python #287 - introduces check for Python