[core dumped] core dumped when running pytest tests

[Jackwin]

Hi folks,

Development environment:

1. Ubuntu 18.04
2. Python 3.7 in the virtual environment

Following the instruction, clone the master branch and make the installation, and everything goes well during the process. But an error about core dumped occurs when the pytest tests command executes the test_generator2.py. And the logs are as follows. Could you give any hints?

Thanks ahead

============================================= test session starts ============================================= [13/220]
platform linux -- Python 3.6.9, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /home/chunjie/workspace/aha/magma
collected 759 items

tests/test_compile_errors.py sss [ 0%]
tests/test_conversion.py .FF.FF [ 1%]
tests/test_ext.py ...F..F..F [ 2%]
tests/test_family.py . [ 2%]
tests/test_flatten.py . [ 2%]
tests/test_generator.py F. [ 3%]
tests/test_generator2.py ..Fatal Python error: Aborted

Current thread 0x00007f08b5c31740 (most recent call first):
File "/home/chunjie/env/python3/lib/python3.6/site-packages/lib/python3.6/site-packages/coreir-2.0.128-py3.6-linux-x86_64.egg/coreir/module.py", line 82 in connect
File "/home/chunjie/workspace/aha/magma/magma/backend/coreir_transformer.py", line 327 in connect
File "/home/chunjie/workspace/aha/magma/magma/backend/coreir_transformer.py", line 269 in connect_non_outputs
File "/home/chunjie/workspace/aha/magma/magma/backend/coreir_transformer.py", line 257 in get_coreir_defn
File "/home/chunjie/workspace/aha/magma/magma/backend/coreir_transformer.py", line 235 in run_self
File "/home/chunjie/workspace/aha/magma/magma/backend/coreir_transformer.py", line 99 in run
File "/home/chunjie/workspace/aha/magma/magma/backend/coreir_transformer.py", line 98 in run
File "/home/chunjie/workspace/aha/magma/magma/backend/coreir_.py", line 78 in compile
File "/home/chunjie/workspace/aha/magma/magma/backend/coreir_compiler.py", line 71 in compile
File "/home/chunjie/workspace/aha/magma/magma/compile.py", line 62 in compile
File "/home/chunjie/workspace/aha/magma/tests/test_generator2.py", line 59 in test_compilation
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/python.py", line 184 in pytest_pyfunc_call
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/manager.py", line 87 in
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in call
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/python.py", line 1627 in runtest
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/runner.py", line 163 in pytest_runtest_call
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/manager.py", line 87 in
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in call
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/runner.py", line 256 in
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/runner.py", line 310 in from_call
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/runner.py", line 256 in call_runtest_hook
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/runner.py", line 216 in call_and_report
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/runner.py", line 127 in runtestprotocol
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/runner.py", line 110 in pytest_runtest_protocol
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/manager.py", line 87 in
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in call
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/main.py", line 338 in pytest_runtestloop
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/manager.py", line 87 in
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in call
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/main.py", line 313 in _main
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/main.py", line 257 in wrap_session
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/main.py", line 306 in pytest_cmdline_main
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/manager.py", line 87 in
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
File "/home/chunjie/env/python3/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in call
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/config/init.py", line 165 in main
File "/home/chunjie/env/python3/lib/python3.6/site-packages/_pytest/config/init.py", line 187 in console_main
File "/home/chunjie/env/python3/bin/pytest", line 8 in
[1] 19344 abort (core dumped) pytest tests

[leonardt]

In your output I see: /home/chunjie/env/python3/lib/python3.6/, it looks like you're running in python 3.6, not 3.7. I think the version of the coreir python package distributed with 3.6 is out of date (we stopped maintaining it). Can you try upgrading to 3.7 and see if it works? Since you noted you're using 3.7 in your virtualenv, it could be some pathing issue with your environment. Did you install pytest in your virtualenv? If not, it may be using the pytest from your 3.6 installation rather than your 3.7 environment

[Jackwin]

Yes, you are right. The virtualenv is configured to python 3.6.9 uncarefully. After updating to 3.7, the pytest flow is Okay. Surprisingly, it pops out 3 failed logs and these could be accepted?

=========================================== short test summary info ===========================================
FAILED tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py::test_simple_def[coreir-json]
FAILED tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py::test_for_loop_def[coreir-json]
FAILED tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py::test_interleaved_instance_wiring[coreir-json]
============================ 3 failed, 750 passed, 6 skipped in 152.60s (0:02:32) =============================

One failed case log:

________________________________ test_interleaved_instance_wiring[coreir-json] ________________________________

target = 'coreir', suffix = 'json'

@pytest.mark.parametrize("target,suffix",
                         [("verilog", "v"), ("coreir", "json")])
def test_interleaved_instance_wiring(target, suffix):
    m.config.set_debug_mode(True)
    m.set_codegen_debug_info(True)
    with pytest.warns(DeprecationWarning):
        And2 = m.DeclareCircuit('And2', "I0", m.In(m.Bit), "I1", m.In(m.Bit),
                                "O", m.Out(m.Bit))

        main = m.DefineCircuit("main", "I", m.In(m.Bits[2]), "O", m.Out(m.Bit))

        and2_0 = And2()
        and2_1 = And2()

        m.wire(main.I[0], and2_0.I0)
        m.wire(main.I[1], and2_0.I1)
        m.wire(and2_0.O, and2_1.I0)
        m.wire(main.I[1], and2_1.I1)
        and2_2 = And2()
        m.wire(and2_1.O, and2_2.I0)
        m.wire(main.I[0], and2_2.I1)

        m.wire(and2_2.O, main.O)

        m.EndCircuit()

    m.compile("build/test_interleaved_instance_wiring", main, output=target)
    m.set_codegen_debug_info(False)
    m.config.set_debug_mode(False)

  assert check_files_equal(__file__, f"build/test_interleaved_instance_wiring.{suffix}",

                             f"gold/test_interleaved_instance_wiring.{suffix}")

E AssertionError: assert False
E + where False = check_files_equal('/home/chunjie/workspace/aha/magma/tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py', 'build/test_interleaved_instance_wiring.json', 'gold/test_interleaved_$nstance_wiring.json')

tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py:112: AssertionError

[leonardt]

Hmm, in the test output, when it fails, it should report what the difference between the two files are (causing the failure), can you copy that here so we can take a look? Also, can you do pip show coreir so we can double check what version is being run?

[Jackwin]

Hi, the version and error log is listed as follows. The error log is long, and maybe be not reading-friendly. If any puzzles, please let me know.

BTW: A small piece of advice about the Development setup in the README is that the log about testing results could be updated, as the python 3.6.1 may mislead users.

============================= test session starts ==============================
platform darwin -- Python 3.6.1, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: ..../repos/magmacore, inifile:
collected 70 items

Thanks for your time.

The version info:

Name: coreir
Version: 2.0.128
Summary: Python bindings for CoreIR
Home-page: https://github.com/leonardt/pycoreir
Author: Leonard Truong
Author-email: lenny@cs.stanford.edu
License: BSD License
Location: /home/chunjie/env/python3.7/lib/python3.7/site-packages
Requires: hwtypes
Required-by: fault, CoSA, magma-lang, peak

The error logs are as follows:

================================================================================================================== test session starts ==================================================================================================================
platform linux -- Python 3.7.9, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /home/chunjie/workspace/aha/lassen
collected 1058 items

tests/test_complex.py .................................................................................... [ 7%]
tests/test_config.py ................................................ [ 12%]
tests/test_lut.py ...... [ 13%]
tests/test_mapper.py . [ 13%]
tests/test_mem.py . [ 13%]
tests/test_micro.py ..........................................................s.......................................................................................... [ 27%]
tests/test_pe.py ....................................................................................................................ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 48%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 71%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 94%]
sssssssssssssssssssssssssssssssssss................ [ 98%]
tests/test_reset.py F [ 98%]
tests/test_rtl.py ...... [ 99%]
tests/test_smt.py ..... [100%]

======================================================================================================================= FAILURES ========================================================================================================================
______________________________________________________________________________________________________________________ test_reset _______________________________________________________________________________________________________________________

def test_reset():
    tester = fault.Tester(pe_circuit, clock=pe_circuit.CLK)

    inst = add(ra_mode=Mode_t.DELAY, rb_mode=Mode_t.DELAY)
    tester.circuit.inst = assembler(inst)
    data = [0, 0]
    for i in range(2):
        while data[i] == 0:
            data[i] = hwtypes.BitVector.random(16)
    tester.circuit.data0 = data[0]
    tester.circuit.data1 = data[1]
    tester.circuit.CLK = 0
    tester.circuit.clk_en = 1
    tester.circuit.ASYNCRESET = 0
    tester.step(1)
    tester.circuit.O0.expect(data[0] + data[1])
    tester.circuit.ASYNCRESET = 1
    tester.eval()
    tester.circuit.O0.expect(0)
    tester.step(2)
    tester.circuit.O0.expect(0)
    tester.circuit.ASYNCRESET = 0
    tester.step(2)
    tester.circuit.O0.expect(data[0] + data[1])
    tester.compile_and_run("verilator", flags=["-Wno-UNUSED", "-Wno-fatal"],
                           directory="tests/build",
                          magma_opts={"coreir_libs": {"float_DW"}})

tests/test_reset.py:37:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ $
../../../env/python3.7/lib/python3.7/site-packages/fault/tester/staged_tester.py:290: in compile_and_run
self._compile_and_run(target=target, **kwargs)
../../../env/python3.7/lib/python3.7/site-packages/fault/tester/staged_tester.py:274: in _compile_and_run
self._compile(target, **kwargs)
../../../env/python3.7/lib/python3.7/site-packages/fault/tester/staged_tester.py:227: in _compile
self.targets[target] = self.make_target(target, **kwargs)
../../../env/python3.7/lib/python3.7/site-packages/fault/tester/staged_tester.py:102: in make_target
return VerilatorTarget(self._circuit, **kwargs)
../../../env/python3.7/lib/python3.7/site-packages/fault/verilator_target.py:137: in init
coverage=coverage)
../../../env/python3.7/lib/python3.7/site-packages/fault/verilog_target.py:60: in init
**self.magma_opts)
../magma/magma/compile.py:62: in compile
result = compiler.compile()
../magma/magma/backend/coreir_compiler.py:72: in compile
backend.context.run_passes(self.passes, self.namespaces)
../../../env/python3.7/lib/python3.7/site-packages/coreir/context.py:231: in run_passes
self.get_lib(ns)

---

self = <coreir.context.Context object at 0x7f8dddb4d0d0>, name = 'global'

def method(self, name: str):
   c_addr = ct.addressof(self.context)

E TypeError: invalid type

../../../env/python3.7/lib/python3.7/site-packages/coreir/context.py:51: TypeError
================================================================================================================ short test summary info ================================================================================================================
FAILED tests/test_reset.py::test_reset - TypeError: invalid type
================================================================================================ 1 failed, 431 passed, 626 skipped in 151.74s (0:02:31) ========================

[leonardt]

That error looks like it's coming from the lassen test suite, but should be fixed by this PR StanfordAHA/lassen#181 I'll see about getting that merged.

Can you post the output of the error from this (the magma) test suite?

[Jackwin]

When running the magma test suite, three assertion errors appear, which are so long that only capture some. Hope they could give you some hints.

========================================================================================================== FAILURES ==========================================================================================================
________________________________________________________________________________________________ test_simple_def[coreir-json] ________________________________________________________________________________________________

target = 'coreir', suffix = 'json'

@pytest.mark.parametrize("target,suffix",
                         [("verilog", "v"), ("coreir", "json")])
def test_simple_def(target, suffix):
    m.config.set_debug_mode(True)
    m.set_codegen_debug_info(True)
    with pytest.warns(DeprecationWarning):
        And2 = m.DeclareCircuit('And2', "I0", m.In(m.Bit), "I1", m.In(m.Bit),
                                "O", m.Out(m.Bit))

        main = m.DefineCircuit("main", "I", m.In(m.Bits[2]), "O", m.Out(m.Bit))

        and2 = And2()

        m.wire(main.I[0], and2.I0)
        m.wire(main.I[1], and2.I1)
        m.wire(and2.O, main.O)

        m.EndCircuit()

    m.compile("build/test_simple_def", main, output=target)

  assert check_files_equal(__file__, f"build/test_simple_def.{suffix}",

                             f"gold/test_simple_def.{suffix}")

E AssertionError: assert False
E + where False = check_files_equal('/home/chunjie/workspace/aha/magma/tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py', 'build/test_simple_def.json', 'gold/test_simple_def.json')

tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py:28: AssertionError
---------------------------------------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------------------------------------
/home/chunjie/workspace/aha/magma/tests/test_deprecated/test_old_io_syntax/build/test_simple_def.json
/home/chunjie/workspace/aha/magma/tests/test_deprecated/test_old_io_syntax/build/test_simple_def.json /home/chunjie/workspace/aha/magma/tests/test_deprecated/test_old_io_syntax/gold/test_simple_def.json
---------------------------------------------------------------------------------------------------- Captured stderr call ----------------------------------------------------------------------------------------------------
WARNING:magma:tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py:15: 'IO = [...]' syntax is deprecated, use 'io = IO(...)' syntax instead

                            "O", m.Out(m.Bit))

WARNING:magma:tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py:17: 'IO = [...]' syntax is deprecated, use 'io = IO(...)' syntax instead

    main = m.DefineCircuit("main", "I", m.In(m.Bits[2]), "O", m.Out(m.Bit))

--- gold/test_simple_def.json
+++ build/test_simple_def.json
@@ -8,7 +8,7 @@
["I1","BitIn"],
["O","Bit"]
]],

•    "metadata":{"filename":"tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py","lineno":"14"}
  

[1]❰ 1 ..ha/aha/garnet ❰ 2 ..orkspace/mlir ❰ 3 ../docker_share ❰ 4 ..ace

       main = m.DefineCircuit("main", "I", m.In(m.Bits[2]), "O", m.Out(m.Bit))

        and2_0 = And2()
        and2_1 = And2()

        m.wire(main.I[0], and2_0.I0)
        m.wire(main.I[1], and2_0.I1)
        m.wire(and2_0.O, and2_1.I0)
        m.wire(main.I[1], and2_1.I1)
        and2_2 = And2()
        m.wire(and2_1.O, and2_2.I0)
        m.wire(main.I[0], and2_2.I1)

        m.wire(and2_2.O, main.O)

        m.EndCircuit()

    m.compile("build/test_interleaved_instance_wiring", main, output=target)
    m.set_codegen_debug_info(False)
    m.config.set_debug_mode(False)

  assert check_files_equal(__file__, f"build/test_interleaved_instance_wiring.{suffix}",

                             f"gold/test_interleaved_instance_wiring.{suffix}")

E AssertionError: assert False
E + where False = check_files_equal('/home/chunjie/workspace/aha/magma/tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py', 'build/test_interleaved_instance_wiring.json', 'gold/test_interleaved_i
nstance_wiring.json')

tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py:112: AssertionError
---------------------------------------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------------------------------------
/home/chunjie/workspace/aha/magma/tests/test_deprecated/test_old_io_syntax/build/test_interleaved_instance_wiring.json
/home/chunjie/workspace/aha/magma/tests/test_deprecated/test_old_io_syntax/build/test_interleaved_instance_wiring.json /home/chunjie/workspace/aha/magma/tests/test_deprecated/test_old_io_syntax/gold/test_interleaved_instan
ce_wiring.json
---------------------------------------------------------------------------------------------------- Captured stderr call ----------------------------------------------------------------------------------------------------
WARNING:magma:tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py:90: 'IO = [...]' syntax is deprecated, use 'io = IO(...)' syntax instead

                            "O", m.Out(m.Bit))

WARNING:magma:tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py:92: 'IO = [...]' syntax is deprecated, use 'io = IO(...)' syntax instead

    main = m.DefineCircuit("main", "I", m.In(m.Bits[2]), "O", m.Out(m.Bit))

--- gold/test_interleaved_instance_wiring.json
+++ build/test_interleaved_instance_wiring.json
@@ -8,7 +8,7 @@
["I1","BitIn"],
["O","Bit"]
]],

•    "metadata":{"filename":"tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py","lineno":"89"}
  

•    "metadata":{"filename":"tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py","lineno":"90"}
   },
   "main":{
     "type":["Record",[
  

----------------------------------------------------------------------------------------------------- Captured log call ------------------------------------------------------------------------------------------------------
WARNING magma:logging.py:91 tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py:90: 'IO = [...]' syntax is deprecated, use 'io = IO(...)' syntax instead

                            "O", m.Out(m.Bit))

WARNING magma:logging.py:91 tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py:92: 'IO = [...]' syntax is deprecated, use 'io = IO(...)' syntax instead

    main = m.DefineCircuit("main", "I", m.In(m.Bits[2]), "O", m.Out(m.Bit))

================================================================================================== short test summary info ===================================================================================================
FAILED tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py::test_simple_def[coreir-json] - AssertionError: assert False
FAILED tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py::test_for_loop_def[coreir-json] - AssertionError: assert False
FAILED tests/test_deprecated/test_old_io_syntax/test_old_io_syntax_define.py::test_interleaved_instance_wiring[coreir-json] - AssertionError: assert False

[leonardt]

Ah, those look like they are related to #790 and #704

Basically, Python 3.7 and 3.8 generated different metadata for line number information causing the test to fail for Python 3.7. It's ok to ignore those test failures. If you want them to pass, you can try upgrading to Python 3.8

[leonardt]

Should be fixed by #888 (it skips the tests if you're running an older version of Python)

[Jackwin]

In the Python 3.7, the preceding issues are fixed after updating to #888. Thanks a lot.