0.22.1 test_stacking ends with Bus error on ARMv7 arch (Python 3.5) #16443
Comments
|
I tried to test joblib package alone (as in stack trace there are calls to it): standard test finishes with single failure (non-fatal) for test_weak_array_key_map test joblib/joblib#1010. Parallel tests pass successfully. |
|
The problem still there in 0.22.2.post1 |
|
I tried to search for exact place where bus errors occur: and it fails in functools.partial(<function check_regressors_train at 0xae988a98>, 'StackingRegressor', readonly_memmap=True) check (the same test with readonly_memmap=False successfully passes). It additionally calls create_memmap_backed_data routine. Memmapped outputs can be successfully printed. |
|
Maybe some additional information is needed? |
|
We're also getting Here's a minimal use case that reliably triggers the bus error for me (extracted from on the from sklearn.tree import DecisionTreeClassifier
from sklearn.utils.estimator_checks import check_estimator
estimator = DecisionTreeClassifier(random_state=0)
check_estimator(estimator)Backtrace: Furthermore, we're also getting |
|
Thank for the confirmation. For of if that doesn't work try, from sklearn.utils.estimator_checks import parametrize_with_checks
@parametrize_with_checks([DecisionTreeClassifier(random_state=0)])
def test_common_checks(estimator, check):
check(estimator)to see which common test fails specifically (check_estimator runs all of them)? |
|
Here you go:
produces (edit: I accidentally dropped the 'Bus error' from output, sorry) |
|
I took another look at the core dump: I've attached The backtrace indicates the cause to emanate from scikit-learn/sklearn/tree/_splitter.pyx Lines 446 to 453 in fd23727 To my untrained eye, that doesn't look all too suspicious, though. At least, not suspicious enough to fail on only this particular architecture. |
cmarmo
added
arch:arm
|
Sorry to jump in. This bug was referenced on Debian PowerPC mailing list. The issue is present on PowerPC, too.
I have some experience flushing these unaligned accesses/SIGBUS faults... You almost certainly have unaligned data causing this (based on my experience with chasing these). Here is how I would approach it. First, switch back to x86_64. It is an easier platform, and the tools work best. Second, build a debug build with
Fourth, run your test program. The problems areas of the code produce a finding like below. The keywords are runtime error:. Finally, fix the findings. Retest on ARM32 and PowerPC. If you get unlucky, then the problem will only surface on ARM32 and PowerPC. In this case, try the same experiment on ARM or PPC instead of x86_64. |
|
@noloader, the last build on PowerPC doesn't seem related to SIGBUS though? There's just one test failure not related to the above. SIGBUS seems to be an issue only on 32-bit ARM. In any case, I agree that it's probably an alignment issue, and @glaubitz already discovered a build process issue that needs to be fixed first. If that doesn't resolve the issue, then it's time to follow the steps you outlined above (thanks!) |
|
Closing due to no activity for long time and no interest in fixing issue |
Describe the bug
Running standard test suite for Scikit-learn 0.22.1 on ARMv7 architecture for Ubuntu 16.04 (Python 3.5) results in fatal bus error.
Versions
System:
python: 3.5.2 (default, Oct 8 2019, 13:06:37) [GCC 5.4.0 20160609]
machine: Linux-3.8.13.30-armv7l-with-Ubuntu-16.04-xenial
executable: /usr/bin/python3
Python dependencies:
matplotlib: None
scipy: 1.4.1 (test suite run without failures)
numpy: 1.18.1 (test suite run without failures)
pip: 20.0.2
sklearn: 0.22.1
setuptools: 45.2.0
pandas: None
Cython: None (was installed latest from pip and deleted after building scikit-learn)
joblib: 0.14.1
Built with OpenMP: True
The text was updated successfully, but these errors were encountered: