Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Segmentation fault in cutest_cigr_r_ #63

Closed
amontoison opened this issue Aug 26, 2024 · 27 comments
Closed

Segmentation fault in cutest_cigr_r_ #63

amontoison opened this issue Aug 26, 2024 · 27 comments
Labels
bug Something isn't working

Comments

@amontoison
Copy link
Member

amontoison commented Aug 26, 2024 •
edited
Loading

I wanted to use cifn and cigr in the Julia interface CUTEst.jl but I have segmentation faults when I call cigr.
All my CI builds on Linux and Windows failed.
I compiled CUTEst in debug mode to reproduce the error and I have the following stack-trace:

[25278] signal (11.1): Erreur de segmentation
in expression starting at /home/alexis/Bureau/git/CUTEst.jl/test/multiple_precision.jl:19
group_s at /home/alexis/Bureau/git/CUTEst.jl/deps/files/GROUP_s.f:19 # <- group_s_ at /home/alexis/Bureau/git/CUTEst.jl/src/../deps/files/libALLINITA_single.so (unknown line)
cutest_cigr_threadsafe_s at /home/alexis/Applications/CUTEst/builddir/../src/tools/cigr.F90:426
cutest_cigr_s at /home/alexis/Applications/CUTEst/builddir/../src/tools/cigr.F90:37
cutest_cigr_s_ at /home/alexis/Bureau/git/CUTEst.jl/src/libcutest.jl:925
cigr at /home/alexis/Bureau/git/CUTEst.jl/src/core_interface.jl:3055

The issue seems to be in cigr.F90 at line 426.

Ref JuliaSmoothOptimizers/CUTEst.jl#375
@ralna ralna deleted a comment Aug 26, 2024
@ralna ralna deleted a comment Aug 26, 2024
@amontoison amontoison added the bug Something isn't working label Aug 26, 2024
@nimgould
Copy link
Contributor

nimgould commented Aug 26, 2024 •
edited
Loading

No issues here, @amontoison

runcutest -p test -sp -D ALLINITA

Problem name: ALLINITA

Single precision version will be formed

The objective function uses 2 linear groups
The objective function uses 10 nonlinear groups

There is 1 linear equality constraint
There is 1 linear inequality constraint
There is 1 nonlinear equality constraint
There is 1 nonlinear inequality constraint

There is 1 free variable
There is 1 variable bounded only from below
There is 1 variable bounded from below and above
There is 1 fixed variable

File successfully decoded
CUTEST: tools (single precision version) compiled successfully
CUTEst: test (single precision version) compiled successfully

This problem is constrained

CALL CUTEST_cdimen

  • n = 4, m = 4
    .....

CALL CUTEST_cigr for the objective function

  •   i       G

  •   1 -8.0000E+00

  •   2  0.0000E+00

  •   3  1.0000E+00

  •   4 -1.0000E+00

....

CALL CUTEST_cigr for a constraint

  •   i   J(1)

  •   1  1.0000E+00

  •   2  0.0000E+00

  •   3  1.0000E+00

  •   4  0.0000E+00

...

Call CUTEST_cterminate

Sound like it might be related to your libALLINITA_single.so ? In particular, it seems to be the call to group_s that you are providing usied in this library. Can you get line numbers here?

Of course these are fortran tests. Perhaps one of you should produce equivalent tests for the C interfaces? That would be a good sanity check, and is definitely missing at the moment. It needs to match the functionality of utest.F90 and ctest.f90 in src/test/

@amontoison
Copy link
Member Author

amontoison commented Aug 26, 2024 •
edited
Loading

The line number in group_s is 19.
I will try to isolate the culprit with other problems.
What I don't understand is that I don't have any error on Mac.

Good idea for the C interface.
A script to generate them based on the Fortran tests should not be too hard.

@nimgould
Copy link
Contributor

LIne 19 is IGRTYP = ITYPEG(IGROUP), so either ITYPEG is not being set or IGROUP is outside the
bounds. You will need to put print statements inside the group_s you are generating to find which. As I said, it is fine here in Linuxland. One thing I do notice, though, is that the subroutine has a logical argument. Is this being passed properly? But I supose that this is all on the fortran side, so this shouldn't be an issue for C.

@amontoison
Copy link
Member Author

amontoison commented Aug 26, 2024 •
edited
Loading

I have a violation of memory on Windows so maybe IGROUP is outside the bounds?
I can reproduce the issue on my laptop so let's add some print statements.

@nimgould
Copy link
Contributor

nimgould commented Aug 26, 2024 •
edited
Loading

OK, I would
write(6, * ) ' ltypeg = ', ltypeg
write(6, * ) 'itypeg = ', itypeg(:)
before the loop and then inside it
write(6, * ) 'igroup = ', igroup
after igroup is set

@amontoison
Copy link
Member Author

The content of GROUP_s.f is:

      SUBROUTINE GROUP_s( GVALUE, LGVALU, FVALUE, GPVALU, NCALCG, 
     *                   ITYPEG, ISTGPA, ICALCG, LTYPEG, LSTGPA, 
     *                   LCALCG, LFVALU, LGPVLU, DERIVS, IGSTAT )
      INTEGER LGVALU, NCALCG, LTYPEG, LSTGPA
      INTEGER LCALCG, LFVALU, LGPVLU, IGSTAT
      LOGICAL DERIVS
      INTEGER ITYPEG(LTYPEG), ISTGPA(LSTGPA), ICALCG(LCALCG)
      REAL             GVALUE(LGVALU,3), FVALUE(LFVALU), GPVALU(LGPVLU)
C
C  Problem name : ALLINITA  
C
C  -- produced by SIFdecode 2.6.0
C
      INTEGER IGRTYP, IGROUP, IPSTRT, JCALCG
      REAL             GVAR  
      IGSTAT = 0
      DO     2 JCALCG = 1, NCALCG
       IGROUP = ICALCG(JCALCG)
       IGRTYP = ITYPEG(IGROUP)
       IF ( IGRTYP == 0 ) GO TO     2
       IPSTRT = ISTGPA(IGROUP) - 1
C
C  Group type : L2      
C
       GVAR  = FVALUE(IGROUP)
       IF ( .NOT. DERIVS ) THEN
        GVALUE(IGROUP,1)= GVAR * GVAR                              
       ELSE
        GVALUE(IGROUP,2)= GVAR + GVAR                              
        GVALUE(IGROUP,3)= 2.0                                      
       END IF
    2 CONTINUE
      RETURN
      END

@nimgould
Copy link
Contributor

nimgould commented Aug 26, 2024 •
edited
Loading

Yes, so add the lines I just suggested to this code before you compile it

Remember to indent to column 7 though, it is fortran 77 (old form) so columns 1-6 are used for labels, etc

@amontoison
Copy link
Member Author

amontoison commented Aug 26, 2024 •
edited
Loading

@nimgould

  ltypeg =           16
 itypeg =            0           0           0           0           0           0           1           1           1           1           1           1           0           0           0           0
 igroup =            1
 igroup =            2
 igroup =            3
 igroup =            4
 igroup =            5
 igroup =            6
 igroup =            7
 igroup =            8
 igroup =            9
 igroup =           10
 igroup =           11
 igroup =           12
  ltypeg =           16
 itypeg =            0           0           0           0           0           0           1           1           1           1           1           1           0           0           0           0
 igroup =            1
 igroup =            2
 igroup =            3
 igroup =            4
 igroup =            5
 igroup =            6
 igroup =            7
 igroup =            8
 igroup =            9
 igroup =           10
 igroup =           11
 igroup =           12
  ltypeg =           16
 itypeg =            0           0           0           0           0           0           1           1           1           1           1           1           0           0           0           0
 igroup =            1
 igroup =            2
 igroup =            3
 igroup =            4
 igroup =            5
 igroup =            6
 igroup =            7
 igroup =            8
 igroup =            9
 igroup =           10
 igroup =           11
 igroup =           12
  ltypeg =           16
 itypeg =            0           0           0           0           0           0           1           1           1           1           1           1           0           0           0           0
 igroup =            1
 igroup =            2
 igroup =            3
 igroup =            4
 igroup =            5
 igroup =            6
 igroup =            7
 igroup =            8
 igroup =            9
 igroup =           10
 igroup =           11
 igroup =           12
 igroup =     50033768

[7514] signal (11.1): Erreur de segmentation
in expression starting at /home/alexis/Bureau/git/CUTEst.jl/test/multiple_precision.jl:19
group_s at /home/alexis/Bureau/git/CUTEst.jl/deps/files/GROUP_s.f:22
cutest_cigr_threadsafe_s at /home/alexis/Applications/CUTEst/builddir/../src/tools/cigr.F90:426
cutest_cigr_s at /home/alexis/Applications/CUTEst/builddir/../src/tools/cigr.F90:37
cutest_cigr_s_ at /home/alexis/Bureau/git/CUTEst.jl/src/libcutest.jl:925
cigr at /home/alexis/Bureau/git/CUTEst.jl/src/core_interface.jl:3055
  ltypeg =           16
 itypeg =            0           0           0           0           0           0           1           1           1           1           1           1           0           0           0           0
 igroup =            1
 igroup =            2
 igroup =            3
 igroup =            4
 igroup =            5
 igroup =            6
 igroup =            7
 igroup =            8
 igroup =            9
 igroup =           10
 igroup =           11
 igroup =           12
  ltypeg =           16
 itypeg =            0           0           0           0           0           0           1           1           1           1           1           1           0           0           0           0
 igroup =            1
 igroup =            2
 igroup =            3
 igroup =            4
 igroup =            5
 igroup =            6
 igroup =            7
 igroup =            8
 igroup =            9
 igroup =           10
 igroup =           11
 igroup =           12
  ltypeg =           16
 itypeg =            0           0           0           0           0           0           1           1           1           1           1           1           0           0           0           0
 igroup =            1
 igroup =            2
 igroup =            3
 igroup =            4
 igroup =            5
 igroup =            6
 igroup =            7
 igroup =            8
 igroup =            9
 igroup =           10
 igroup =           11
 igroup =           12
  ltypeg =           16
 itypeg =            0           0           0           0           0           0           1           1           1           1           1           1           0           0           0           0
 igroup =            1
 igroup =            2
 igroup =            3
 igroup =            4
 igroup =            5
 igroup =            6
 igroup =            7
 igroup =            8
 igroup =            9
 igroup =           10
 igroup =           11
 igroup =           12
 igroup =    233881008

@amontoison amontoison mentioned this issue Aug 26, 2024
Open
@nimgould
Copy link
Contributor

right, more writes needed. Before loop
write(6, * ) ' lcalcg, ncalcg = ', lcalcg, ncalcg
write(6, * ) ' icalcg ', icalcg(:)

@nimgould
Copy link
Contributor

(also, what cutest routines are you calling and in what order. This is going wrong on the 4th call to group as far as I can tell)

@amontoison
Copy link
Member Author 

  lcalcg, ncalcg =           16          12
  icalcg            1           2           3           4           5           6           7           8           9          10          11          12  1660348536       32759    30038672           0
  ltypeg =           16
 itypeg =            0           0           0           0           0           0           1           1           1           1           1           1           0           0           0           0
 igroup =            1
 igroup =            2
 igroup =            3
 igroup =            4
 igroup =            5
 igroup =            6
 igroup =            7
 igroup =            8
 igroup =            9
 igroup =           10
 igroup =           11
 igroup =           12
  lcalcg, ncalcg =           16          12
  icalcg            1           2           3           4           5           6           7           8           9          10          11          12  1660348536       32759    30038672           0
  ltypeg =           16
 itypeg =            0           0           0           0           0           0           1           1           1           1           1           1           0           0           0           0
 igroup =            1
 igroup =            2
 igroup =            3
 igroup =            4
 igroup =            5
 igroup =            6
 igroup =            7
 igroup =            8
 igroup =            9
 igroup =           10
 igroup =           11
 igroup =           12
  lcalcg, ncalcg =           16          12
  icalcg            1           2           3           4           5           6           7           8           9          10          11          12  1660348536       32759    30038672           0
  ltypeg =           16
 itypeg =            0           0           0           0           0           0           1           1           1           1           1           1           0           0           0           0
 igroup =            1
 igroup =            2
 igroup =            3
 igroup =            4
 igroup =            5
 igroup =            6
 igroup =            7
 igroup =            8
 igroup =            9
 igroup =           10
 igroup =           11
 igroup =           12
  lcalcg, ncalcg =           16          16
  icalcg            1           2           3           4           5           6           7           8           9          10          11          12  1660348536       32759    30038672           0
  ltypeg =           16
 itypeg =            0           0           0           0           0           0           1           1           1           1           1           1           0           0           0           0
 igroup =            1
 igroup =            2
 igroup =            3
 igroup =            4
 igroup =            5
 igroup =            6
 igroup =            7
 igroup =            8
 igroup =            9
 igroup =           10
 igroup =           11
 igroup =           12
 igroup =   1660348536

@amontoison
Copy link
Member Author

amontoison commented Aug 26, 2024 •
edited
Loading

The routines that I'm calling are:

  • cifn (1st call)
  • cifn (2nd call)
  • cigr (3rd call)

@nimgould
Copy link
Contributor

Thank you. One further question, what is iprob here?

@amontoison
Copy link
Member Author

iprob is 0, I just want the gradient of the objective.

@amontoison
Copy link
Member Author

amontoison commented Aug 26, 2024 •
edited
Loading

Nick, I get the same error if I don't call cifn. The issue occurs after the first call to cigr, which internally calls group twice.

@nimgould
Copy link
Contributor

OK, this doesn't make sense. The array kndofc that should be imported and adjusted in csetup - you are calling this aren't you, not usetup - says which of the groups are objectives and which aren't. For the ALLINITA example, I get
KNDOFC
0 0 0 0 0 0 0 0 0 0
0 0 1 2 3 4
those with the value 0 correspond to the objective function., and there are 12 of them. cigr then computes ncalcg by counting the number of entries of KNDOFC that are 0, i.e., ncalcg = 12. So why does your run report ncalcg=16? It is 12 here in linuxland

@amontoison
Copy link
Member Author

amontoison commented Aug 26, 2024 •
edited
Loading

I confirm that I'm calling csetup if the problem is constrained:
https://github.com/JuliaSmoothOptimizers/CUTEst.jl/blob/main/src/model.jl#L153

I use Ubuntu 22.04 with gfortran v11.4 on my laptop so it's also linuxland here.

@nimgould
Copy link
Contributor

OK, can you edit cigr.f90, and print out the values of the array data%KNDOFC( : ) just after the else on line 365 (before the loop that calculates ncalcg)

@nimgould
Copy link
Contributor

Oh no, I now see the bug. The second call to GROUP_r is using data%ng = 16 not ncalcg = 12. I thought I fixed this before! Presumably on my linux, the later array weren't filled with junk, so the bug didn't show. I fixed this in cisgr a while ago. Fix on its way ...

@nimgould
Copy link
Contributor

Committed. Well found

@amontoison
Copy link
Member Author

amontoison commented Aug 26, 2024 •
edited
Loading

Nick, you pushed a few files but not cigr.f90:
b964eb9

@nimgould
Copy link
Contributor

Yup, that's the trouble with working on a public holiday, my brain is on the beach!

@amontoison
Copy link
Member Author

amontoison commented Aug 26, 2024 •
edited
Loading

A last question before that I close the issue.
If I just want to evaluate all contraints, what is the most efficient approach:

  • Evaluate objective + constraints
  • evaluate all constraints one by one with cifn.

I didn't find a routine to just evaluate all constraints (without the objective) that takes as input the vector c.

@amontoison
Copy link
Member Author

Yup, that's the trouble with working on a public holiday, my brain is on the beach!

Sorry for that! I need to finish my boxes for Chicago after so I will not annoy you anymore for today ;)

@nimgould
Copy link
Contributor

You could also try ccfsg with grad = false and lj = 0.

@nimgould
Copy link
Contributor

Maybe there could be a ccf function, but as I said before, every function we add takes a day+ to write all the tests, documentation and interfaces. My usual fee is a very reasonable $2000

@amontoison
Copy link
Member Author

amontoison commented Aug 26, 2024 •
edited
Loading

No worries!
I’ll be sending you the bill soon for my work on GALAHAD, SIFDecode, libHSL, and CUTEst. 😜

The fix is in release 2.2.4. I can close the issue.
I will just add a note to keep in mind the ccf function for a potential future feature.

@amontoison amontoison mentioned this issue Aug 26, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nimgould @amontoison