Tests run when called by name but seg fault when part of a collection #3672

mvahowe · 2018-07-09T12:32:37Z

Hi,

I'm running the python3-pytest package of Ubuntu 18.04, v3.3.2-2, with cov-2.5.1 from a package and faulthandler-1.5.0 from pip. (I'm not using a virtual environment.) The application I am testing makes extensive use of gevent. This has worked well until now but today, when refactoring my tests to move the startup/teardown code to a module, I'm getting seg faults.

I've tried to isolate the issue, and it seems to be something around pytest teardown after the last test in a collection, ie

running one test by name works
running all the tests in a file seg faults (after successful completion of all tests)
running one test by mark also seg faults (after successful completion of that test).

ie

$ pytest-3 test_app_cancel_upload.py -m crash_when_marked -v
===================================================================== test session starts ======================================================================
platform linux -- Python 3.6.5, pytest-3.3.2, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python3
cachedir: ../../.cache
rootdir: /home/mark/dbl-uploader-clients/dbl_dot_local, inifile: pytest.ini
plugins: faulthandler-1.5.0, cov-2.5.1
collected 5 items                                                                                                                                              

test_app_cancel_upload.py::test_cancel_upload_jobs PASSED                                                                                                [100%]

====================================================================== 4 tests deselected ======================================================================
=========================================================== 1 passed, 4 deselected in 10.47 seconds ============================================================
Fatal Python error: Segmentation fault

Current thread 0x00007fe73c7eb740 (most recent call first):
Erreur de segmentation (core dumped)
$ pytest-3 test_app_cancel_upload.py::test_cancel_upload_jobs -v
===================================================================== test session starts ======================================================================
platform linux -- Python 3.6.5, pytest-3.3.2, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python3
cachedir: ../../.cache
rootdir: /home/mark/dbl-uploader-clients/dbl_dot_local, inifile: pytest.ini
plugins: faulthandler-1.5.0, cov-2.5.1
collected 1 item                                                                                                                                               

test_app_cancel_upload.py::test_cancel_upload_jobs PASSED                                                                                                [100%]

=================================================================== 1 passed in 8.83 seconds ===================================================================
$

The test above looks like this:

@pytest.mark.crash_when_marked
@pytest.mark.cancelUploadJobs
def test_cancel_upload_jobs(temp_config_path):
    """
    Test cancelUploadJobs for host bundle
    """
    config_ob = Config(prepend_real(temp_config_path))
    du_app = DownUpApp(config_ob, verbose=True)
    # Find an entry to download
    entry_id, entry_revision = first_text_entry_ids(du_app.dbl_endpoints)
    # task to download entry
    tasks_xml = """<tasks>
    <downloadMetadata>
        <entry>{0}</entry>
        <revision>{1}</revision>
        <label>downloadedBundle</label>
        <tasks>
            <createUploadJob/>
        </tasks>
    </downloadMetadata>
</tasks>""".format(entry_id, entry_revision)
    tasks_dom = etree.fromstring(tasks_xml)
    first_bundle_id = du_app.storer.new_bundle("text", tasks_dom)
    du_app.tick_while_active(60)
    jobs = du_app.dbl_endpoints.entry_upload_jobs(entry_id)[3]
    assert len(jobs) > 0
    second_bundle_id = du_app.storer.label(first_bundle_id, "downloadedBundle")
    tasks_xml = """<tasks>
    <cancelUploadJobs/>
</tasks>"""
    du_app.storer.add_tasks_xml(second_bundle_id, etree.fromstring(tasks_xml))
    du_app.tick_while_active(60)
    assert not du_app.is_active()
    jobs = du_app.dbl_endpoints.entry_upload_jobs(entry_id)[3]
    assert len(jobs) == 0

and two of the fixtures look like this:

@pytest.fixture()
def temp_bundle_dir():
    """
    Set up a temporary bundle dir with matching settings file, delete it after use.
    """
    temp_dir = tempfile.mkdtemp()
    if platform.system() == "Windows":
        temp_dir = re.sub(
                "\\\\",
                "/",
                temp_dir
            )
    with open(prepend_test_data_dir("valid_settings.xml.template"), "r") as t_in:
        template = t_in.read()
    template = re.sub("%%TMP_DIR%%", temp_dir, template)
    temp_bundle_dir_path = os.path.join(temp_dir, "bundles")
    os.mkdir(temp_bundle_dir_path)
    temp_session_bundle_dir_path = os.path.join(temp_dir, "sessions")
    os.mkdir(temp_session_bundle_dir_path)
    print('built {0}'.format(temp_dir))
    yield temp_dir
    print('cleanup {0}'.format(temp_dir))
    shutil.rmtree(temp_dir)

@pytest.fixture
def temp_config_path(temp_bundle_dir):
    temp_dir = temp_bundle_dir
    with open(prepend_test_data_dir("valid_settings.xml.template"), "r") as t_in:
        template = t_in.read()
    template = re.sub("%%TMP_DIR%%", temp_dir, template)
    temp_config_path = os.path.join(temp_dir, "valid_settings.xml")
    with open(temp_config_path, "w") as t_out:
        t_out.write(template)
    print('config path: {0}'.format(temp_config_path))
    return temp_config_path

This looks to me like a bug in pytest, since the test behaviour varies according to how it is invoked. But I'm quite willing to believe that there's also something evil in my code - I just have no clue how to find it.

All suggestions welcome.

The text was updated successfully, but these errors were encountered:

pytestbot · 2018-07-09T12:37:03Z

GitMate.io thinks possibly related issues are #3251 (Warnings not captured during test collection), #2188 (Test runner hangs during collection with doctest-modules and mock.call in module), #1521 (High memory usage if tests called by test case name), #337 (there is no warning when tests have the same name), and #1337 (Segmentation fault in tests).

nicoddemus · 2018-07-10T11:49:43Z

Hi @mvahowe,

Unfortunately it is very hard to suggest something at this point because we can't really see all the code. It might be a bug in pytest, but I would actually guess it is some bad interaction with your code.

The only hint I can give is to try to isolate the problem in a minimal reproducible problem; that will help you find the problem in your code or isolate the bug in pytest so we can fix it.

mvahowe · 2018-07-10T14:44:30Z

Thanks for this. A colleague of mine got this, from OS X. (Another colleague failed to reproduce the seg fault with Windows):

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   etree.cpython-36m-darwin.so       0x0000000107f53001 __pyx_tp_dealloc_4lxml_5etree__ParserContext + 113
1   etree.cpython-36m-darwin.so       0x0000000107ef1393 __pyx_tp_dealloc_4lxml_5etree__BaseParser + 131
2   org.python.python                 0x00000001066adad2 dict_dealloc + 120
3   org.python.python                 0x00000001066cd26c subtype_clear + 142
4   org.python.python                 0x0000000106750df7 collect + 1830
5   org.python.python                 0x0000000106750641 collect_with_callback + 58
6   org.python.python                 0x0000000106750694 _PyGC_CollectIfEnabled + 44
7   org.python.python                 0x00000001067393bf Py_FinalizeEx + 98
8   org.python.python                 0x00000001067399cd Py_Exit + 13
9   org.python.python                 0x000000010673c003 handle_system_exit + 317
10  org.python.python                 0x000000010673bc54 PyErr_PrintEx + 54
11  org.python.python                 0x000000010673b464 PyRun_SimpleFileExFlags + 966
12  org.python.python                 0x000000010674ff9a Py_Main + 3466
13  org.python.python                 0x000000010666ee1d 0x10666d000 + 7709
14  libdyld.dylib                     0x00007fff65b77015 start + 1

From this it looks to us like the issue is something along the lines of double deallocation within the lxml module, which uses the libxml2 library under the hood. libxml2 internals are famously murky. I'll try to come up with a simpler test case but I'm not optimistic.

What I would say is that whatever is happening depends, in a reproducible way, on how the test is called. It doesn't happen if the test is called directly in isolation, it doesn't happen if the file containing the failing test is called with other files, it does occur if all the tests in the file are run, of if the test is called in isolation via a mark decorator. This suggests to me that there's something different about how cleanup occurs. Given that there are lots of persistent gevent threads in our code, could it be something to do with how threads are shut down in different cases?

mvahowe · 2018-07-10T14:46:50Z

Also, I don't think sharing all our sprawling codebase here is sensible, but if someone has time to look into the problem we could give them access to our repo. We're getting identical behaviour with Linux and OS X so I'm confident the problem can be reproduced.

nicoddemus · 2018-07-10T15:09:27Z

This suggests to me that there's something different about how cleanup occurs.

Seems like a good venue for investigation. Keep in mind that pytest itself doesn't know about gevent or attempts to close threads in anyway, so if there's threads being shutdown they are not being shutdown by pytest.

Also, I don't think sharing all our sprawling codebase here is sensible, but if someone has time to look into the problem we could give them access to our repo.

I agree, probably people won't be able to allocate the necessary time to understand the entire code base.

TvoroG · 2018-07-19T15:59:16Z

I had segfault too and I think it was due to updating of python 3.6.6 to 3.7.0, because application worked just fine with python 3.6.6.

m3nu · 2018-11-04T05:32:02Z

Same here. Python version has no influence though. Running individual files is OK. Running them together segfaults.

nicoddemus · 2019-05-01T14:21:29Z

Closing this for now as it seems this is not related to pytest per-se. Will be happy to reopen if more information comes to light. 👍

daveliu-riviera · 2019-08-08T18:23:02Z

Getting same issue. I was recently running a job in Python and ctrl+c killed it. I don't know if maybe some memory allocation issue arose because of that abrupt termination? Just throwing hypotheses out there.

mvahowe · 2019-08-09T07:53:07Z

FWIW this seems to me to be a pytest issue because it happens in the pytest context but not in other contexts. If there are certain types of code that seg fault with pytest but not elsewhere, that's either a bug that should be fixed or a constraint that should be documented.

AvnerCohen · 2019-10-28T07:31:44Z

This code, when running under python works just fine:

from lxml import html

def test_():
    pass

if __name__ == "__main__":
    test_()

Run this with pytest, and it segfaults.

I can't say this is pytest issue per se, but It does feels like it.

stracing the segafault:

stat("/opt/app/venv3.7.4_try/lib/python3.7/site-packages/lxml", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/opt/app/venv3.7.4_try/lib/python3.7/site-packages/lxml/etree.cpython-37m-x86_64-linux-gnu.so", {st_mode=S_IFREG|0755, st_size=6880975, ...}) = 0
open("/opt/app/venv3.7.4_try/lib/python3.7/site-packages/lxml/etree.cpython-37m-x86_64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 9
read(9, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\216\6\0\0\0\0\0"..., 832) = 832
fstat(9, {st_mode=S_IFREG|0755, st_size=6880975, ...}) = 0
mmap(NULL, 7710904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 9, 0) = 0x7f7082b9b000
mprotect(0x7f70830a7000, 2097152, PROT_NONE) = 0
mmap(0x7f70832a7000, 278528, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 9, 0x50c000) = 0x7f70832a7000
mmap(0x7f70832eb000, 43192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f70832eb000
close(9)                                = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x2380} ---
+++ killed by SIGSEGV +++
Segmentation fault

Anyone has any insights into that ?

mvahowe · 2019-10-28T08:15:26Z

FWIW, my crashing code above made heavy use of lxml.

AvnerCohen · 2019-10-28T08:28:02Z

Thanks.
I have removed all my dependencies one by one, and ion my case, seems like the culprit is https://github.com/syrusakbary/snapshottest , not 100% how and why, but dropping this dependency fixed the issue for me.

I guess for future readers, while pytest might indeed be the issue, it's better to start off with a clean venv, and see if there is any specific dependency that is the issue.

blueyed · 2019-10-28T09:56:34Z

Shouldn't faulthandler provide more inside? (i.e. a Python traceback).
Please also provide Python and pytest versions at least.

RonnyPfannschmidt · 2019-10-28T10:27:49Z

@AvnerCohen its possible that your segfault is only hit when certain preconditions are met,

as far as i i can tell, snapshot-test does setup/clean out sys.modules details,

its thinkable that a unexpected gc happens due to loosing/reimporting a module, thus its possible that it needs a blacklist for toplevels not to touch

AvnerCohen · 2019-10-28T15:29:03Z

I'll try to provide all the info I can here, let me know if anything is missing.

Given this basic test file:

from lxml import html

def test_():
    pass

if __name__ == "__main__":
    test_()

If I run this with pytest, here is the output:

=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.7.4, pytest-5.2.2, py-1.8.0, pluggy-0.13.0
rootdir: /opt/app
collected 1 item

moshe.py .                                                                                                                                                                                           [100%]

============================================================================================ 1 passed in 0.02s =============================================================================================

Once I also install snapshottest (pip install snapshottest ), the result I see is:

$ pytest moshe.py
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.7.4, pytest-5.2.2, py-1.8.0, pluggy-0.13.0
rootdir: /opt/app
plugins: snapshottest-0.5.1
collecting ... Fatal Python error: Segmentation fault

Current thread 0x00007f933f78f740 (most recent call first):
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1043 in create_module
  File "<frozen importlib._bootstrap>", line 583 in module_from_spec
  File "<frozen importlib._bootstrap>", line 670 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 967 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 983 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1035 in _handle_fromlist
  File "/opt/app/moshe/lib/python3.7/site-packages/lxml/html/__init__.py", line 53 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 728 in exec_module
  File "<frozen importlib._bootstrap>", line 677 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 967 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 983 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1035 in _handle_fromlist
  File "/opt/app/moshe.py", line 1 in <module>
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/assertion/rewrite.py", line 142 in exec_module
  File "<frozen importlib._bootstrap>", line 677 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 967 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 983 in _find_and_load
  File "/opt/app/moshe/lib/python3.7/site-packages/py/_path/local.py", line 701 in pyimport
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/python.py", line 501 in _importtestmodule
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/python.py", line 433 in _getobj
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/python.py", line 256 in obj
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/python.py", line 449 in _inject_setup_module_fixture
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/python.py", line 436 in collect
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/runner.py", line 256 in <lambda>
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/runner.py", line 229 in from_call
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/runner.py", line 256 in pytest_make_collect_report
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/manager.py", line 86 in <lambda>
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/runner.py", line 375 in collect_one_node
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/main.py", line 703 in genitems
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/main.py", line 478 in _perform_collect
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/main.py", line 439 in perform_collect
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/main.py", line 246 in pytest_collection
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/manager.py", line 86 in <lambda>
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/main.py", line 236 in _main
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/main.py", line 193 in wrap_session
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/main.py", line 230 in pytest_cmdline_main
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/manager.py", line 86 in <lambda>
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/config/__init__.py", line 90 in main
  File "/opt/app/moshe/bin/pytest", line 8 in <module>
Segmentation fault

Happy to provide anything else if this can be of help.

blueyed · 2019-10-28T15:54:29Z

File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/assertion/rewrite.py", line 142 in exec_module

Please try if it also crashes with --assert=plain.

AvnerCohen · 2019-10-28T15:56:52Z

It does.

$ pytest  --assert=plain moshe.py
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.7.4, pytest-5.2.2, py-1.8.0, pluggy-0.13.0
rootdir: /opt/app
plugins: snapshottest-0.5.1
collecting ... Fatal Python error: Segmentation fault

Current thread 0x00007f6e9e135740 (most recent call first):
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1043 in create_module
  File "<frozen importlib._bootstrap>", line 583 in module_from_spec
  File "<frozen importlib._bootstrap>", line 670 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 967 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 983 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1035 in _handle_fromlist
  File "/opt/app/moshe/lib/python3.7/site-packages/lxml/html/__init__.py", line 53 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 728 in exec_module
  File "<frozen importlib._bootstrap>", line 677 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 967 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 983 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1035 in _handle_fromlist
  File "/opt/app/moshe.py", line 1 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 728 in exec_module
  File "<frozen importlib._bootstrap>", line 677 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 967 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 983 in _find_and_load
  File "/opt/app/moshe/lib/python3.7/site-packages/py/_path/local.py", line 701 in pyimport
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/python.py", line 501 in _importtestmodule
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/python.py", line 433 in _getobj
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/python.py", line 256 in obj
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/python.py", line 449 in _inject_setup_module_fixture
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/python.py", line 436 in collect
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/runner.py", line 256 in <lambda>
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/runner.py", line 229 in from_call
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/runner.py", line 256 in pytest_make_collect_report
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/manager.py", line 86 in <lambda>
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/runner.py", line 375 in collect_one_node
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/main.py", line 703 in genitems
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/main.py", line 478 in _perform_collect
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/main.py", line 439 in perform_collect
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/main.py", line 246 in pytest_collection
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/manager.py", line 86 in <lambda>
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/main.py", line 236 in _main
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/main.py", line 193 in wrap_session
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/main.py", line 230 in pytest_cmdline_main
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/manager.py", line 86 in <lambda>
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
  File "/opt/app/moshe/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/opt/app/moshe/lib/python3.7/site-packages/_pytest/config/__init__.py", line 90 in main
  File "/opt/app/moshe/bin/pytest", line 8 in <module>
Segmentation fault

blueyed · 2019-10-28T16:00:05Z

File "/opt/app/moshe/lib/python3.7/site-packages/py/_path/local.py", line 701 in pyimport

Then try what py does there manually: https://github.com/pytest-dev/py/blob/34f716fe1fb4df9a5257b5f640ebb8a71e10aa88/py/_path/local.py#L689-L701 (to get pytest / your tests out of the equation / try to reproduce it without).

AvnerCohen · 2019-10-31T09:06:25Z

@blueyed Sorry, not sure I follow your exact suggested test here.

The recreation of this is very simple.

virtualenev with:
pip install -U lxml pytest pip snapshottest
Run py.test :

from lxml import html

def test_():
    pass

if __name__ == "__main__":
    test_()

blueyed · 2019-10-31T09:12:51Z

ERROR: Could not find a version that satisfies the requirement snapshottest (from versions: none)

Not listed for Python 3.7. (have only later seen that you have it with py37)
With 3.6 I see:

platform linux -- Python 3.6.8, pytest-5.2.2, py-1.8.0, pluggy-0.13.0
rootdir: /tmp
plugins: snapshottest-0.5.1
collected 1 item                                                                            

t.py .                                                                                [100%]

Is the if __name__ == "__main__": part relevant for you? Is it triggered through this somehow?

AvnerCohen · 2019-10-31T11:56:51Z

Thanks for the help @blueyed
I've get a repository with a docker container that recreates this, you can see it's pretty straight forward:
https://github.com/AvnerCohen/repro-pytest-segafault

I did notice this did not occur on an alpine-python, but is present on this centos based docker.

pytestbot added the type: bug problem that needs to be addressed label Jul 9, 2018

nicoddemus added the status: needs information reporter needs to provide more information; can be closed after 2 or more weeks of inactivity label Jul 10, 2018

nicoddemus closed this as completed May 1, 2019

AvnerCohen mentioned this issue Oct 28, 2019

Segmentation fault in Python 3.7 syrusakbary/snapshottest#103

Open

Borda mentioned this issue Dec 2, 2020

CI: skip hanging Lightning-AI/pytorch-lightning#4943

Merged

11 tasks

nzw0301 mentioned this issue May 25, 2021

tests/integration_tests/test_integration.py is failing on macOS optuna/optuna#2695

Closed

Garyfallidis mentioned this issue Jan 4, 2022

Changing how we do things with our test suite. fury-gl/fury#515

Merged

paulthebaker mentioned this issue Feb 6, 2023

Segmentation fault in test suite nanograv/enterprise#339

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tests run when called by name but seg fault when part of a collection #3672

Tests run when called by name but seg fault when part of a collection #3672

mvahowe commented Jul 9, 2018 •

edited

Loading

pytestbot commented Jul 9, 2018

nicoddemus commented Jul 10, 2018

mvahowe commented Jul 10, 2018

mvahowe commented Jul 10, 2018

nicoddemus commented Jul 10, 2018

TvoroG commented Jul 19, 2018

m3nu commented Nov 4, 2018

nicoddemus commented May 1, 2019

daveliu-riviera commented Aug 8, 2019

mvahowe commented Aug 9, 2019

AvnerCohen commented Oct 28, 2019 •

edited

Loading

mvahowe commented Oct 28, 2019

AvnerCohen commented Oct 28, 2019

blueyed commented Oct 28, 2019

RonnyPfannschmidt commented Oct 28, 2019

AvnerCohen commented Oct 28, 2019

blueyed commented Oct 28, 2019

AvnerCohen commented Oct 28, 2019

blueyed commented Oct 28, 2019

AvnerCohen commented Oct 31, 2019

blueyed commented Oct 31, 2019

AvnerCohen commented Oct 31, 2019

Tests run when called by name but seg fault when part of a collection #3672

Tests run when called by name but seg fault when part of a collection #3672

Comments

mvahowe commented Jul 9, 2018 • edited Loading

pytestbot commented Jul 9, 2018

nicoddemus commented Jul 10, 2018

mvahowe commented Jul 10, 2018

mvahowe commented Jul 10, 2018

nicoddemus commented Jul 10, 2018

TvoroG commented Jul 19, 2018

m3nu commented Nov 4, 2018

nicoddemus commented May 1, 2019

daveliu-riviera commented Aug 8, 2019

mvahowe commented Aug 9, 2019

AvnerCohen commented Oct 28, 2019 • edited Loading

mvahowe commented Oct 28, 2019

AvnerCohen commented Oct 28, 2019

blueyed commented Oct 28, 2019

RonnyPfannschmidt commented Oct 28, 2019

AvnerCohen commented Oct 28, 2019

blueyed commented Oct 28, 2019

AvnerCohen commented Oct 28, 2019

blueyed commented Oct 28, 2019

AvnerCohen commented Oct 31, 2019

blueyed commented Oct 31, 2019

AvnerCohen commented Oct 31, 2019

mvahowe commented Jul 9, 2018 •

edited

Loading

AvnerCohen commented Oct 28, 2019 •

edited

Loading