Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segmentation fault in tests on 32-bit architectures (i686 and arm) #223

Closed
ViralBShah opened this issue Dec 7, 2019 · 16 comments
Closed

Comments

@ViralBShah
Copy link
Contributor

ViralBShah commented Dec 7, 2019

I just tried running the Sundials tests on Raspberry Pi and they failed with 1.3. Just reporting here for the record.

signal (11): Segmentation fault
in expression starting at /home/pi/.julia/packages/Sundials/MllUG/test/cvode_Roberts_dns.jl:66
_ZL22isExtendedBUILD_VECTORPN4llvm6SDNodeERNS_12SelectionDAGEb at /home/pi/julia-1.3.0/bin/../lib/julia/libLLVM-6.0.so (unknown line)
Allocations: 9997384 (Pool: 9996517; Big: 867); GC: 21
ERROR: Package Sundials errored during testing
julia> versioninfo()
Julia Version 1.3.0
Commit 46ce4d7933 (2019-11-26 06:09 UTC)
Platform Info:
  OS: Linux (arm-linux-gnueabihf)
  CPU: ARMv7 Processor rev 4 (v7l)
  WORD_SIZE: 32
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, cortex-a53)
@ChrisRackauckas
Copy link
Member

For reference, the Sundials build is: https://github.com/JuliaDiffEq/SundialsBuilder/releases/tag/sundials-3.1.1.6 . I assume that segfault means we need to fix that binary builder repo build script.

@ViralBShah
Copy link
Contributor Author

Is there a minimal example I can try? It could even be Julia itself crashing.

@ChrisRackauckas
Copy link
Member

using Sundials
function lorenz(du,u,p,t)
 du[1] = 10.0(u[2]-u[1])
 du[2] = u[1]*(28.0-u[3]) - u[2]
 du[3] = u[1]*u[2] - (8/3)*u[3]
end
u0 = [1.0;0.0;0.0]
tspan = (0.0,100.0)
prob = ODEProblem(lorenz,u0,tspan)
sol = solve(prob,CVODE_Adams())
using Plots; plot(sol,vars=(1,2,3))

is what I'd try.

@ViralBShah
Copy link
Contributor Author

ViralBShah commented Dec 8, 2019

That example works fine. It's crashing here (which I can verify manually and is what the failing test above showed as well):

https://github.com/JuliaDiffEq/Sundials.jl/blob/90ec5466df22a61129c3d84665bb113cf6aafd81/test/cvode_Roberts_dns.jl#L66

@ViralBShah
Copy link
Contributor Author

For reference, the Sundials build is: https://github.com/JuliaDiffEq/SundialsBuilder/releases/tag/sundials-3.1.1.6 . I assume that segfault means we need to fix that binary builder repo build script.

Isn't it the Yggdrasil one? https://github.com/JuliaPackaging/Yggdrasil/tree/master/S/Sundials

@ViralBShah
Copy link
Contributor Author

Off topic, but we should just build KLU as part of the regular SuiteSparse build, and simplify the Sundials build.

@ChrisRackauckas
Copy link
Member

This Sundials build was the first BinaryBuilder setup, and it hasn't really been modified since. So anything like Yggdrasil isn't apart of this unless that happened outside of these repos.

No BLAS here, that's an open issue. And getting KLU out of this build would indeed be nice.

@ViralBShah
Copy link
Contributor Author

ViralBShah commented Dec 8, 2019

I feel like we should get the build upgraded first. KLU would be next. But not sure if any of this would help fix the crash unless it is happening due to some bad combination of things being done in the old way and it is only triggered on arm.

@ChrisRackauckas
Copy link
Member

We might as well update to Sundials 5.0 at the same time too. It's an easy enough API change. It mostly just needs someone who wants to make the binaries build

@ViralBShah ViralBShah changed the title Tests fail on arm Segmentation fault on arm tests Dec 13, 2019
@ViralBShah
Copy link
Contributor Author

@ChrisRackauckas Should I try this again now? Will it pull Sundials 5 and the new changes? I don't have any use for this myself, but just figured I can close htis out if if it works.

@ChrisRackauckas
Copy link
Member

Yes, it should just test it on Sundials 5.

@ViralBShah
Copy link
Contributor Author

ViralBShah commented Apr 14, 2020

Looks like an error in the same place. Using Julia 1.4 and Sundials 5 through artifacts.

signal (11): Segmentation fault
in expression starting at /home/viral/.julia/packages/Sundials/0yJ3G/test/cvode_Roberts_simplified.jl:14
SUNMatGetID at /home/viral/.julia/artifacts/ee6fe6e3b73ab790eab0ab0d686f5df2933d0498/lib/libsundials_sunmatrixdense.so.3 (unknown line)
Allocations: 19394760 (Pool: 19392171; Big: 2589); GC: 42
ERROR: Package Sundials errored during testing

Stacktrace:
 [1] pkgerror(::String, ::Vararg{String,N} where N) at /buildworker/worker/package_linuxarmv7l/build/usr/share/julia/stdlib/v1.4/Pkg/src/Types.jl:53
 [2] test(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing) at /buildworker/worker/package_linuxarmv7l/build/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:1503
 [3] test(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Cmd, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /buildworker/worker/package_linuxarmv7l/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:316
 [4] test(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linuxarmv7l/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:303
 [5] #test#68 at /buildworker/worker/package_linuxarmv7l/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:297 [inlined]
 [6] test at /buildworker/worker/package_linuxarmv7l/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:297 [inlined]
 [7] #test#67 at /buildworker/worker/package_linuxarmv7l/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:296 [inlined]
 [8] test at /buildworker/worker/package_linuxarmv7l/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:296 [inlined]
 [9] test(::String; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /buildworker/worker/package_linuxarmv7l/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:295
 [10] test(::String) at /buildworker/worker/package_linuxarmv7l/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:295
 [11] top-level scope at REPL[2]:1

@ViralBShah
Copy link
Contributor Author

ViralBShah commented Apr 14, 2020

using Sundials
function lorenz(du,u,p,t)
 du[1] = 10.0(u[2]-u[1])
 du[2] = u[1]*(28.0-u[3]) - u[2]
 du[3] = u[1]*u[2] - (8/3)*u[3]
end
u0 = [1.0;0.0;0.0]
tspan = (0.0,100.0)
prob = ODEProblem(lorenz,u0,tspan)
sol = solve(prob,CVODE_Adams())
using Plots; plot(sol,vars=(1,2,3))

is what I'd try.

This fails now.


julia> sol = solve(prob,CVODE_Adams())

[CVODES WARNING]  CVode
  Internal t = 0 and h = 0 are such that t + h = t on the next step. The solver will continue anyway.

Thread 1 "julia" received signal SIGSEGV, Segmentation fault.
0x59360bac in N_VLinearSum_Serial ()
   from /home/viral/.julia/artifacts/ee6fe6e3b73ab790eab0ab0d686f5df2933d0498/lib/libsundials_nvecserial.so
(gdb) bt
#0  0x59360bac in N_VLinearSum_Serial ()
   from /home/viral/.julia/artifacts/ee6fe6e3b73ab790eab0ab0d686f5df2933d0498/lib/libsundials_nvecserial.so

@ViralBShah
Copy link
Contributor Author

ViralBShah commented Apr 14, 2020

I get a failure using 32-bit i686 Julia as well, which suggests that this failure is not arm specific:


[CVSLS ERROR]  CVodeSetLinearSolver
  LS must be non-NULL

┌ Error: Sundials.CVDlsSetLinearSolver failed with error code = 
│   flag = -3
└ @ Sundials ~/.julia/packages/Sundials/0yJ3G/src/simple.jl:18

[CVODE ERROR]  cvNlsInit
  Setting linear solver solve function failed


[CVODES ERROR]  cvInitialSetup
  The nonlinear solver's init routine failed.

┌ Error: CVode failed with error code = 
│   flag = -13
└ @ Sundials ~/.julia/packages/Sundials/0yJ3G/src/simple.jl:18

[CVODE ERROR]  cvNlsInit
  Setting linear solver solve function failed


[CVODES ERROR]  cvInitialSetup
  The nonlinear solver's init routine failed.

┌ Error: CVode failed with error code = 
│   flag = -13
└ @ Sundials ~/.julia/packages/Sundials/0yJ3G/src/simple.jl:18

[CVODE ERROR]  cvNlsInit
  Setting linear solver solve function failed


[CVODES ERROR]  cvInitialSetup
  The nonlinear solver's init routine failed.

┌ Error: CVode failed with error code = 
│   flag = -13
└ @ Sundials ~/.julia/packages/Sundials/0yJ3G/src/simple.jl:18

[CVODE ERROR]  cvNlsInit
  Setting linear solver solve function failed


[CVODES ERROR]  cvInitialSetup
  The nonlinear solver's init routine failed.

┌ Error: CVode failed with error code = 
│   flag = -13
└ @ Sundials ~/.julia/packages/Sundials/0yJ3G/src/simple.jl:18

[CVODE ERROR]  cvNlsInit
  Setting linear solver solve function failed


[CVODES ERROR]  cvInitialSetup
  The nonlinear solver's init routine failed.

┌ Error: CVode failed with error code = 
│   flag = -13
└ @ Sundials ~/.julia/packages/Sundials/0yJ3G/src/simple.jl:18

[CVODE ERROR]  cvNlsInit
  Setting linear solver solve function failed


[CVODES ERROR]  cvInitialSetup
  The nonlinear solver's init routine failed.

┌ Error: CVode failed with error code = 
│   flag = -13
└ @ Sundials ~/.julia/packages/Sundials/0yJ3G/src/simple.jl:18

[CVODE ERROR]  cvNlsInit
  Setting linear solver solve function failed


[CVODES ERROR]  cvInitialSetup
  The nonlinear solver's init routine failed.

┌ Error: CVode failed with error code = 
│   flag = -13
└ @ Sundials ~/.julia/packages/Sundials/0yJ3G/src/simple.jl:18

[CVODE ERROR]  cvNlsInit
  Setting linear solver solve function failed


[CVODES ERROR]  cvInitialSetup
  The nonlinear solver's init routine failed.

┌ Error: CVode failed with error code = 
│   flag = -13
└ @ Sundials ~/.julia/packages/Sundials/0yJ3G/src/simple.jl:18

[CVODE ERROR]  cvNlsInit
  Setting linear solver solve function failed


[CVODES ERROR]  cvInitialSetup
  The nonlinear solver's init routine failed.

┌ Error: CVode failed with error code = 
│   flag = -13
└ @ Sundials ~/.julia/packages/Sundials/0yJ3G/src/simple.jl:18

signal (11): Segmentation fault
in expression starting at /home/viralbshah/.julia/packages/Sundials/0yJ3G/test/cvode_Roberts_dns.jl:67
N_VMin_Serial at /home/viralbshah/.julia/artifacts/7a2115d417d3b49a2b4e62ef70b96cfe312c6135/lib/libsundials_nvecserial.so (unknown line)
CVodeSVtolerances at /home/viralbshah/.julia/artifacts/7a2115d417d3b49a2b4e62ef70b96cfe312c6135/lib/libsundials_cvodes.so (unknown line)
CVodeSVtolerances at /home/viralbshah/.julia/packages/Sundials/0yJ3G/src/API/cvodes.jl:36 [inlined]
CVodeSVtolerances at /home/viralbshah/.julia/packages/Sundials/0yJ3G/src/API/cvodes.jl:41
unknown function (ip: 0xdc617754)
_jl_invoke at /buildworker/worker/package_linux32/build/src/gf.c:2158 [inlined]
jl_apply_generic at /buildworker/worker/package_linux32/build/src/gf.c:2322
jl_apply at /buildworker/worker/package_linux32/build/src/julia.h:1692 [inlined]
do_call at /buildworker/worker/package_linux32/build/src/interpreter.c:369
eval_value at /buildworker/worker/package_linux32/build/src/interpreter.c:458
eval_stmt_value at /buildworker/worker/package_linux32/build/src/interpreter.c:409 [inlined]
eval_body at /buildworker/worker/package_linux32/build/src/interpreter.c:817
jl_interpret_toplevel_thunk at /buildworker/worker/package_linux32/build/src/interpreter.c:911
jl_toplevel_eval_flex at /buildworker/worker/package_linux32/build/src/toplevel.c:814
jl_parse_eval_all at /buildworker/worker/package_linux32/build/src/ast.c:872
jl_load at /buildworker/worker/package_linux32/build/src/toplevel.c:872
jl_load_ at /buildworker/worker/package_linux32/build/src/toplevel.c:879
include at ./client.jl:439
_jl_invoke at /buildworker/worker/package_linux32/build/src/gf.c:2144 [inlined]
jl_apply_generic at /buildworker/worker/package_linux32/build/src/gf.c:2322
jl_apply at /buildworker/worker/package_linux32/build/src/julia.h:1692 [inlined]
do_call at /buildworker/worker/package_linux32/build/src/interpreter.c:369
eval_value at /buildworker/worker/package_linux32/build/src/interpreter.c:458
eval_stmt_value at /buildworker/worker/package_linux32/build/src/interpreter.c:409 [inlined]
eval_body at /buildworker/worker/package_linux32/build/src/interpreter.c:817
eval_body at /buildworker/worker/package_linux32/build/src/interpreter.c:744
eval_body at /buildworker/worker/package_linux32/build/src/interpreter.c:744
eval_body at /buildworker/worker/package_linux32/build/src/interpreter.c:744
eval_body at /buildworker/worker/package_linux32/build/src/interpreter.c:744
jl_interpret_toplevel_thunk at /buildworker/worker/package_linux32/build/src/interpreter.c:911
jl_toplevel_eval_flex at /buildworker/worker/package_linux32/build/src/toplevel.c:814
jl_parse_eval_all at /buildworker/worker/package_linux32/build/src/ast.c:872
jl_load at /buildworker/worker/package_linux32/build/src/toplevel.c:872
jl_load_ at /buildworker/worker/package_linux32/build/src/toplevel.c:879
include at ./client.jl:439
_jl_invoke at /buildworker/worker/package_linux32/build/src/gf.c:2144 [inlined]
jl_apply_generic at /buildworker/worker/package_linux32/build/src/gf.c:2322
jl_apply at /buildworker/worker/package_linux32/build/src/julia.h:1692 [inlined]
do_call at /buildworker/worker/package_linux32/build/src/interpreter.c:369
eval_value at /buildworker/worker/package_linux32/build/src/interpreter.c:458
eval_stmt_value at /buildworker/worker/package_linux32/build/src/interpreter.c:409 [inlined]
eval_body at /buildworker/worker/package_linux32/build/src/interpreter.c:817
jl_interpret_toplevel_thunk at /buildworker/worker/package_linux32/build/src/interpreter.c:911
jl_toplevel_eval_flex at /buildworker/worker/package_linux32/build/src/toplevel.c:814
jl_toplevel_eval_flex at /buildworker/worker/package_linux32/build/src/toplevel.c:764
jl_toplevel_eval at /buildworker/worker/package_linux32/build/src/toplevel.c:823
jl_toplevel_eval_in at /buildworker/worker/package_linux32/build/src/toplevel.c:843
eval at ./boot.jl:331
_jl_invoke at /buildworker/worker/package_linux32/build/src/gf.c:2144 [inlined]
jl_apply_generic at /buildworker/worker/package_linux32/build/src/gf.c:2322
exec_options at ./client.jl:264
_start at ./client.jl:484
jfptr__start_2064.clone_1 at /tmp/julia-1.4.0/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux32/build/src/gf.c:2144 [inlined]
jl_apply_generic at /buildworker/worker/package_linux32/build/src/gf.c:2322
jl_apply at /buildworker/worker/package_linux32/build/ui/../src/julia.h:1692 [inlined]
true_main at /buildworker/worker/package_linux32/build/ui/repl.c:96
main at /buildworker/worker/package_linux32/build/ui/repl.c:217
__libc_start_main at /lib32/libc.so.6 (unknown line)
_start at /tmp/julia-1.4.0/bin/julia (unknown line)
Allocations: 17250912 (Pool: 17248697; Big: 2215); GC: 35
ERROR: Package Sundials errored during testing

@ViralBShah
Copy link
Contributor Author

@jd-lara Any thoughts?

@ViralBShah ViralBShah changed the title Segmentation fault on arm tests Segmentation fault in tests on 32-bit architectures (i686 and arm) Apr 14, 2020
@jd-lara
Copy link
Contributor

jd-lara commented Apr 14, 2020

It seems that the SUNLinearSolver wrap has some problem. We would have to review this when fixing #250.

I would test the basic tests first to see if the problem is in the DiffEqCommon interface set-up of the problem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants