Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Segfault in classification notebook #135

Closed
benureau opened this issue Sep 25, 2023 · 5 comments · Fixed by #139
Closed

Segfault in classification notebook #135

benureau opened this issue Sep 25, 2023 · 5 comments · Fixed by #139

Comments

@benureau
Copy link

If I run the code below, copied from the classification notebook in tutorial/:

from reservoirpy.datasets import japanese_vowels
from reservoirpy import set_seed

set_seed(42)

X_train, Y_train, X_test, Y_test = japanese_vowels(repeat_targets=True)

from reservoirpy.nodes import Reservoir, Ridge, Input

source = Input()
reservoir = Reservoir(500, sr=0.9, lr=0.1)
readout = Ridge(ridge=1e-6)

model = [source >> reservoir, source] >> readout

Y_pred = model.fit(X_train, Y_train, stateful=False, warmup=2).run(X_test, stateful=False)

I get this output:

§ python tests/classification.py
Running Model-1: 20it [00:00, 5418.65it/s]
Running Model-1: 26it [00:00, 2019.74it/s]
Running Model-1: 22it [00:00, 3156.31it/s]
Running Model-1: 20it [00:00, 3284.63it/s]
Running Model-1: 21it [00:00, 3504.71it/s]
Running Model-1: 23it [00:00, 3763.47it/s]
Running Model-1: 22it [00:00, 5930.25it/s]
Running Model-1: 18it [00:00, 5817.79it/s]
Running Model-1: 24it [00:00, 5459.56it/s]
Running Model-1: 15it [00:00, 2721.69it/s]
Running Model-1: 23it [00:00, 5941.67it/s]
Running Model-1: 15it [00:00, 5997.57it/s]
Running Model-1: 17it [00:00, 6030.38it/s]
Running Model-1: 14it [00:00, 2722.69it/s]
Running Model-1: 14it [00:00, 5824.27it/s]
Running Model-1: 15it [00:00, 2590.25it/s]
Running Model-1: 15it [00:00, 2175.17it/s]
Running Model-1: 21it [00:00, 6020.53it/s]
Running Model-1: 16it [00:00, 2420.17it/s]
Running Model-1: 15it [00:00, 5649.66it/s]
Running Model-1: 15it [00:00, 5745.10it/s]
Running Model-1: 19it [00:00, 6107.59it/s]
Running Model-1: 22it [00:00, 6226.78it/s]
Running Model-1: 20it [00:00, 3068.93it/s]
Running Model-1: 17it [00:00, 2264.24it/s]
Running Model-1: 17it [00:00, 6055.99it/s]
Running Model-1: 13it [00:00, 5842.90it/s]
Running Model-1: 16it [00:00, 2800.17it/s]
Running Model-1: 13it [00:00, 2753.00it/s]
Running Model-1: 14it [00:00, 5985.14it/s]
Running Model-1: 18it [00:00, 6085.07it/s]
Running Model-1: 17it [00:00, 6150.54it/s]
Running Model-1: 16it [00:00, 6213.21it/s]
Running Model-1: 17it [00:00, 6204.59it/s]
Running Model-1: 15it [00:00, 6198.48it/s]
Running Model-1: 15it [00:00, 6162.66it/s]
Running Model-1: 15it [00:00, 6172.94it/s]
Running Model-1: 18it [00:00, 6209.18it/s]
Running Model-1: 18it [00:00, 6018.61it/s]
Running Model-1: 21it [00:00, 4130.77it/s]
Running Model-1: 17it [00:00, 6070.94it/s]
Running Model-1: 13it [00:00, 6167.40it/s]
Running Model-1: 13it [00:00, 6152.78it/s]
Running Model-1: 17it [00:00, 3196.02it/s]
Running Model-1: 15it [00:00, 6107.62it/s]
Running Model-1: 16it [00:00, 3457.26it/s]
Running Model-1: 15it [00:00, 5973.09it/s]
Running Model-1: 18it [00:00, 6152.01it/s]
Running Model-1: 15it [00:00, 6085.17it/s]
Running Model-1: 11it [00:00, 6023.94it/s]
Running Model-1: 17it [00:00, 6229.53it/s]
Running Model-1: 15it [00:00, 6144.00it/s]
Running Model-1: 13it [00:00, 6126.51it/s]
Running Model-1: 12it [00:00, 5698.14it/s]
Running Model-1: 14it [00:00, 5996.15it/s]
Running Model-1: 16it [00:00, 2603.74it/s]
Running Model-1: 12it [00:00, 2308.26it/s]
Running Model-1: 16it [00:00, 6123.07it/s]
Running Model-1: 15it [00:00, 3697.38it/s]
Running Model-1: 15it [00:00, 2682.35it/s]
Running Model-1: 21it [00:00, 2747.87it/s]
Running Model-1: 19it [00:00, 4115.25it/s]
Running Model-1: 14it [00:00, 5985.75it/s]
Running Model-1: 12it [00:00, 3327.93it/s]
Running Model-1: 14it [00:00, 2595.49it/s]
Running Model-1: 15it [00:00, 1662.08it/s]
Running Model-1: 20it [00:00, 6069.90it/s]
Running Model-1: 14it [00:00, 6205.25it/s]
Running Model-1: 7it [00:00, 2614.90it/s]
Running Model-1: 17it [00:00, 2683.09it/s]
Running Model-1: 11it [00:00, 2591.84it/s]
Running Model-1: 15it [00:00, 2624.06it/s]
Running Model-1: 15it [00:00, 4094.13it/s]
Running Model-1: 9it [00:00, 5990.91it/s]
Running Model-1: 12it [00:00, 6163.56it/s]
Running Model-1: 18it [00:00, 3341.48it/s]
Running Model-1: 13it [00:00, 2922.08it/s]
Running Model-1: 12it [00:00, 6119.35it/s]
Running Model-1: 12it [00:00, 6126.05it/s]
Running Model-1: 16it [00:00, 6188.00it/s]
Running Model-1: 13it [00:00, 2467.24it/s]
Running Model-1: 12it [00:00, 2334.60it/s]
Running Model-1: 17it [00:00, 5608.24it/s]
Running Model-1: 15it [00:00, 6195.43it/s]
Running Model-1: 13it [00:00, 3274.64it/s]
Running Model-1: 19it [00:00, 4368.83it/s]
Running Model-1: 11it [00:00, 6138.55it/s]
Running Model-1: 13it [00:00, 6220.16it/s]
Running Model-1: 10it [00:00, 2472.77it/s]
Running Model-1: 15it [00:00, 5796.44it/s]
Running Model-1: 21it [00:00, 2416.60it/s]
Running Model-1: 18it [00:00, 5553.73it/s]
Running Model-1: 23it [00:00, 6100.61it/s]
Running Model-1: 23it [00:00, 6263.81it/s]
Running Model-1: 21it [00:00, 6237.10it/s]
Running Model-1: 21it [00:00, 5564.49it/s]
Running Model-1: 23it [00:00, 3130.48it/s]
Running Model-1: 21it [00:00, 2733.63it/s]
Running Model-1: 24it [00:00, 1711.00it/s]
Running Model-1: 19it [00:00, 1424.08it/s]
Running Model-1: 23it [00:00, 4196.31it/s]
Running Model-1: 23it [00:00, 2382.72it/s]
Running Model-1: 17it [00:00, 4003.77it/s]
Running Model-1: 23it [00:00, 6235.88it/s]
Running Model-1: 19it [00:00, 2760.84it/s]
Running Model-1: 17it [00:00, 3090.73it/s]
Running Model-1: 18it [00:00, 6169.10it/s]
Running Model-1: 19it [00:00, 6089.38it/s]
Running Model-1: 15it [00:00, 6157.83it/s]
Running Model-1: 22it [00:00, 6300.33it/s]
Running Model-1: 18it [00:00, 6285.17it/s]
Running Model-1: 15it [00:00, 6250.20it/s]
Running Model-1: 17it [00:00, 5810.23it/s]
Running Model-1: 25it [00:00, 6323.20it/s]
Running Model-1: 21it [00:00, 6342.19it/s]
Running Model-1: 19it [00:00, 6355.01it/s]
Running Model-1: 23it [00:00, 6365.49it/s]
Running Model-1: 16it [00:00, 6264.25it/s]
Running Model-1: 20it [00:00, 6315.30it/s]
Running Model-1: 22it [00:00, 6349.32it/s]
Running Model-1: 13it [00:00, 6157.65it/s]
Running Model-1: 11it [00:00, 6154.93it/s]
Running Model-1: 12it [00:00, 6211.48it/s]
Running Model-1: 10it [00:00, 6189.03it/s]
Running Model-1: 11it [00:00, 6131.21it/s]
Running Model-1: 10it [00:00, 6120.39it/s]
Running Model-1: 16it [00:00, 3802.42it/s]
Running Model-1: 13it [00:00, 5855.45it/s]
Running Model-1: 11it [00:00, 4444.40it/s]
Running Model-1: 15it [00:00, 2782.85it/s]
Running Model-1: 11it [00:00, 5397.44it/s]
Running Model-1: 15it [00:00, 6208.27it/s]
Running Model-1: 16it [00:00, 6220.70it/s]
Running Model-1: 13it [00:00, 4231.74it/s]
Running Model-1: 17it [00:00, 6231.70it/s]
Running Model-1: 14it [00:00, 6213.78it/s]
Running Model-1: 16it [00:00, 6267.17it/s]
Running Model-1: 15it [00:00, 5696.72it/s]
Running Model-1: 15it [00:00, 6102.29it/s]
Running Model-1: 10it [00:00, 6045.41it/s]
Running Model-1: 15it [00:00, 6239.67it/s]
Running Model-1: 14it [00:00, 6238.21it/s]
Running Model-1: 18it [00:00, 6269.51it/s]
Running Model-1: 15it [00:00, 6284.54it/s]
Running Model-1: 15it [00:00, 6255.18it/s]
Running Model-1: 12it [00:00, 6198.48it/s]
Running Model-1: 12it [00:00, 6200.77it/s]
Running Model-1: 10it [00:00, 6094.60it/s]
Running Model-1: 12it [00:00, 6226.08it/s]
Running Model-1: 10it [00:00, 2174.79it/s]
Running Model-1: 17it [00:00, 2905.00it/s]
Running Model-1: 16it [00:00, 5939.36it/s]
Running Model-1: 16it [00:00, 6026.30it/s]
Running Model-1: 21it [00:00, 6281.58it/s]
Running Model-1: 18it [00:00, 2661.55it/s]
Running Model-1: 13it [00:00, 3049.38it/s]
Running Model-1: 16it [00:00, 6245.01it/s]
Running Model-1: 17it [00:00, 5763.27it/s]
Running Model-1: 21it [00:00, 3391.36it/s]
Running Model-1: 23it [00:00, 6105.63it/s]
Running Model-1: 19it [00:00, 2361.10it/s]
Running Model-1: 16it [00:00, 5604.08it/s]
Running Model-1: 18it [00:00, 5538.25it/s]
Running Model-1: 18it [00:00, 2898.51it/s]
Running Model-1: 15it [00:00, 6053.55it/s]
Running Model-1: 16it [00:00, 6263.08it/s]
Running Model-1: 18it [00:00, 4801.42it/s]
Running Model-1: 19it [00:00, 4498.29it/s]
Running Model-1: 16it [00:00, 5201.83it/s]
Running Model-1: 16it [00:00, 3174.05it/s]
Running Model-1: 18it [00:00, 4196.87it/s]
Running Model-1: 14it [00:00, 5482.24it/s]
Running Model-1: 16it [00:00, 5572.90it/s]
Running Model-1: 19it [00:00, 5563.90it/s]
Running Model-1: 23it [00:00, 5589.17it/s]
Running Model-1: 17it [00:00, 2838.16it/s]
Running Model-1: 18it [00:00, 5255.65it/s]
Running Model-1: 16it [00:00, 2754.43it/s]
Running Model-1: 18it [00:00, 2835.69it/s]
Running Model-1: 15it [00:00, 5409.21it/s]
Running Model-1: 16it [00:00, 2592.38it/s]
Running Model-1: 15it [00:00, 4398.70it/s]
Running Model-1: 15it [00:00, 5476.55it/s]
Running Model-1: 17it [00:00, 3069.71it/s]
Running Model-1: 15it [00:00, 3108.27it/s]
Running Model-1: 12it [00:00, 5343.06it/s]
Running Model-1: 16it [00:00, 5118.52it/s]
Running Model-1: 18it [00:00, 4731.60it/s]
Running Model-1: 18it [00:00, 5471.23it/s]
Running Model-1: 14it [00:00, 5428.52it/s]
Running Model-1: 20it [00:00, 2847.36it/s]
Running Model-1: 19it [00:00, 2472.60it/s]
Running Model-1: 17it [00:00, 4650.91it/s]
Running Model-1: 14it [00:00, 3098.04it/s]
Running Model-1: 14it [00:00, 1385.60it/s]
Running Model-1: 18it [00:00, 2903.64it/s]
Running Model-1: 17it [00:00, 3394.74it/s]
Running Model-1: 20it [00:00, 4078.28it/s]
Running Model-1: 17it [00:00, 3601.72it/s]
Running Model-1: 14it [00:00, 3342.07it/s]
Running Model-1: 13it [00:00, 2108.75it/s]
Running Model-1: 16it [00:00, 1445.75it/s]
Running Model-1: 16it [00:00, 2601.62it/s]
Running Model-1: 21it [00:00, 2314.31it/s]
Running Model-1: 16it [00:00, 2513.72it/s]
Running Model-1: 15it [00:00, 2122.77it/s]
Running Model-1: 21it [00:00, 2230.90it/s]
Running Model-1: 16it [00:00, 1845.88it/s]
Running Model-1: 21it [00:00, 2705.17it/s]
Running Model-1: 25it [00:00, 2188.00it/s]
Running Model-1: 10it [00:00, 3289.39it/s]
Running Model-1: 12it [00:00, 3159.55it/s]
Running Model-1: 11it [00:00, 2731.64it/s]
Running Model-1: 11it [00:00, 3127.74it/s]
Running Model-1: 10it [00:00, 1868.37it/s]
Running Model-1: 11it [00:00, 2692.74it/s]
Running Model-1: 11it [00:00, 1056.19it/s]
Running Model-1: 14it [00:00, 3013.46it/s]
Running Model-1: 13it [00:00, 3212.13it/s]
Running Model-1: 12it [00:00, 2677.50it/s]
Running Model-1: 12it [00:00, 3968.43it/s]
Running Model-1: 13it [00:00, 4086.48it/s]
Running Model-1: 17it [00:00, 2444.65it/s]
Running Model-1: 13it [00:00, 3751.87it/s]
Running Model-1: 11it [00:00, 3868.96it/s]
Running Model-1: 12it [00:00, 1538.25it/s]
Running Model-1: 11it [00:00, 3102.92it/s]
Running Model-1: 12it [00:00, 1925.91it/s]
Running Model-1: 11it [00:00, 3305.44it/s]
Running Model-1: 12it [00:00, 1467.14it/s]
Running Model-1: 14it [00:00, 1658.16it/s]
Running Model-1: 15it [00:00, 1418.27it/s]
Running Model-1: 14it [00:00, 2333.69it/s]
Running Model-1: 15it [00:00, 2502.07it/s]
Running Model-1: 13it [00:00, 2312.28it/s]
Running Model-1: 10it [00:00, 2472.18it/s]
Running Model-1: 13it [00:00, 2666.70it/s]
Running Model-1: 15it [00:00, 1989.08it/s]
Running Model-1: 13it [00:00, 3031.75it/s]
Running Model-1: 16it [00:00, 3354.94it/s]
Running Model-1: 17it [00:00, 2031.31it/s]
Running Model-1: 12it [00:00, 3441.01it/s]
Running Model-1: 18it [00:00, 1775.16it/s]
Running Model-1: 16it [00:00, 2435.72it/s]
Running Model-1: 17it [00:00, 1252.85it/s]
Running Model-1: 19it [00:00, 2405.06it/s]
Running Model-1: 20it [00:00, 1120.56it/s]
Running Model-1: 19it [00:00, 2875.82it/s]
Running Model-1: 18it [00:00, 2007.80it/s]
Running Model-1: 11it [00:00, 3684.80it/s]
Running Model-1: 11it [00:00, 949.58it/s]
Running Model-1: 17it [00:00, 2451.29it/s]
Running Model-1: 15it [00:00, 2792.85it/s]
Running Model-1: 12it [00:00, 2263.72it/s]
Running Model-1: 15it [00:00, 2469.08it/s]
Running Model-1: 14it [00:00, 2252.93it/s]
Running Model-1: 14it [00:00, 2461.24it/s]
Running Model-1: 11it [00:00, 2234.80it/s]
Running Model-1: 13it [00:00, 1930.74it/s]
Running Model-1: 14it [00:00, 2684.60it/s]
Running Model-1: 12it [00:00, 3034.95it/s]
Running Model-1: 17it [00:00, 1462.24it/s]
Running Model-1: 11it [00:00, 2706.00it/s]
Running Model-1: 10it [00:00, 2146.41it/s]
Running Model-1: 14it [00:00, 864.14it/s]
Running Model-1: 15it [00:00, 3280.90it/s]
Running Model-1: 17it [00:00, 2116.01it/s]
Running Model-1: 12it [00:00, 3195.66it/s]
Running Model-1: 14it [00:00, 2393.62it/s]
Running Model-1: 9it [00:00, 1270.45it/s]
Running Model-1: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 270/270 [00:02<00:00, 104.37it/s]
Fitting node Ridge-0...
Bus error: 10
(echoenv) λ:reservoirpy § /Users/fcyb/.pyenv/versions/3.10.11/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d

Sometimes "Bus error: 10" is replaced by a "Segmentation fault: 11".

I am running the code on macOS 13.4 (22F66), Apple Silicon (Arm) with python 3.10.11.
@benureau
Copy link
Author

Code is freshly cloned from bb00cb0

@benureau
Copy link
Author

Ok, this is solved for me by upgrading scipy and numpy to the latest versions (1.11.2 and 1.26.0, resp.), even though reservoirpy wants scipy<=1.7.3. Consider updating the requirements or adding a warning in the installation section.

@benureau
Copy link
Author

The segfault was triggered by this line btw:

reservoirpy/reservoirpy/nodes/readouts/ridge.py

Line 17 in bb00cb0

return linalg.solve(XXT + ridge, YXT.T, assume_a="sym")

Any ideas of issues I might encounter while using a newer scipy?

@neuronalX
Copy link
Collaborator

Hi @benureau, thanks for your interest in reservoirpy and for finding this issue!

Usually this kind of error, related to the regression when training the readout, could come from a bad conditioning of the matrix to be pseudo-inversed or if the size of the training data (nr of timesteps / number of reservoir dimensions) is too big.
Changing the ridge parameter could solve the first problem and reducing the data solve the second kind of problem.
But in your case this seems very related to the version of scipy in which the internal computations of linalg.solve seems to have changed.
No clear idea yet.

@PAUL-BERNARD PAUL-BERNARD mentioned this issue Nov 7, 2023
Merged
@PAUL-BERNARD PAUL-BERNARD linked a pull request Nov 7, 2023 that will close this issue
V0.3.10 #139
Merged
@PAUL-BERNARD
Copy link
Contributor

Hello @benureau , thank you for your extensive report.

Even if I wasn't able to reproduce on my machine, which makes it difficult to debug, it seems that this is an issue with older scipy versions (<1.9.0), specifically on the ARM architecture.

As you mentionned, updating your scipy version should fix this. Feel free to reopen if your problem persists on recent scipy versions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

V0.3.10 reservoirpy/reservoirpy
3 participants
@benureau @neuronalX @PAUL-BERNARD