Use -std=gnu99 #485

jhunkeler · 2020-05-14T05:08:28Z

This is compatible with Apple Clang
(non-GNU) strdup() was producing garbage

Let's see if the tests fair better...

* This is compatible with Apple Clang * (non-GNU) strdup() was producing garbage

mdlpstsci · 2020-05-14T12:20:23Z

@pllim Can you confirm this works on your computer?

pllim · 2020-05-14T13:42:50Z

I'll try it out when I get the chance. Thanks!

jhunkeler · 2020-05-14T13:48:52Z

PS - Here's how I came across the problem in the first place...

$ ./waf configure --debug --prefix=/tmp/hstcal && ./waf install
Setting top to                           : /home/jhunk/Downloads/hstcal 
Setting out to                           : /home/jhunk/Downloads/hstcal/build.Linux-5.4.34-1-MANJARO-x86_64-with-glibc2.2.5 
Building app                             : HSTCAL 
Version                                  : 2.3.2-6-g810ad66-dirty 
git HEAD commit                          : 810ad66cd4000866bcde7cae9ee2e6f0875fa4ae 
git branch                               : master 
Checking for 'gcc' (C compiler)          : /usr/bin/gcc 
Checking for 'gfortran' (Fortran compiler) : /usr/bin/gfortran 
Compiling a simple fortran app             : yes 
Checking for 'gcc' (C compiler)            : /usr/bin/gcc 
Checking for program 'pkg-config'          : /usr/bin/pkg-config 
Checking for 'cfitsio'                     : yes 
Checking for OpenMP: -fopenmp              : yes 
Checking for sizeof(int)                   : yes 
Checking for compiler flags -std=c99       : yes 
Checking for compiler flags -g             : yes 
Checking for compiler flags -O0            : yes 
Checking for compiler flags -Wall          : yes 
C compiler flags (CFLAGS)                  : -D APPNAME="HSTCAL" -D VERSION="2.3.2-6-g810ad66-dirty" -D BRANCH="master" -D COMMIT="810ad66cd4000866bcde7cae9ee2e6f0875fa4ae" -std=c99 -g -O0 -Wall 
Fortran compiler flags (FCFLAGS)           : not found 
Linker flags (LDFLAGS)                     : not found 
'configure' finished successfully (0.361s)
[...]

$ export PATH=/tmp/hstcal/bin:$PATH
$ gdb -ex run --args cs0.e -v -s o5cl02040_raw.fits
[...]
Reading symbols from cs0.e...
Starting program: /tmp/hstcal/bin/cs0.e -v -s o5cl02040_raw.fits
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".

*** CALSTIS-0 -- Version 3.4.2 (19-Jan-2018) ***
Begin    14-May-2020 09:25:22 EDT

Input    o5cl02040_raw.fits
Wavecal  o5cl02040_wav.fits

*** CALSTIS-1 -- Version 3.4.2 (19-Jan-2018) ***
Begin    14-May-2020 09:25:22 EDT
Input    o5cl02040_raw.fits
Output   o5cl02040_flt.fits
OBSMODE  ACCUM
APERTURE 6X0.2
OPT_ELEM E140H
DETECTOR FUV-MAMA

Imset 1  Begin 09:25:22 EDT
         Heliocentric radial velocity = 11.103 (km/s)

DQICORR  PERFORM
DQITAB   oref$hcm14405o_bpx.fits
DQITAB   PEDIGREE=GROUND
DQITAB   DESCRIP =Prel. Ground Calib + edge flag
DQICORR  COMPLETE
         Uncertainty array initialized.

LORSCORR PERFORM
LORSCORR COMPLETE

GLINCORR PERFORM
LFLGCORR PERFORM
MLINTAB  oref$j9r16559o_lin.fits
MLINTAB  PEDIGREE=GROUND
MLINTAB  DESCRIP =T. Danks gathered Info
MLINTAB  DESCRIP =T. Danks Gathered Info
GLINCORR COMPLETE
LFLGCORR COMPLETE

DARKCORR PERFORM
DARKFILE oref$ma71317ro_drk.fits
DARKFILE PEDIGREE=INFLIGHT 10/01/2000 15/05/2001
DARKFILE DESCRIP =Average of 137 1380s darks from 7604
DARKCORR COMPLETE

FLATCORR PERFORM
PFLTFILE oref$mbj1658bo_pfl.fits
PFLTFILE PEDIGREE=INFLIGHT 17/03/1998 17/05/2002
PFLTFILE DESCRIP =On-orbit FUV P-flat created from proposals 7645,8428,8862,8922
LFLTFILE oref$m1b2139lo_lfl.fits
LFLTFILE PEDIGREE=DUMMY
LFLTFILE DESCRIP =Dummy file created by M. Mcgrath. Modified by C Proffitt
FLATCORR COMPLETE

PHOTCORR OMIT

DOPPCORR applied to DQICORR, DARKCORR, FLATCORR

STATFLAG PERFORM
STATFLAG COMPLETE
Imset 1  End 09:25:23 EDT

End      14-May-2020 09:25:23 EDT

*** CALSTIS-1 complete ***

*** CALSTIS-1 -- Version 3.4.2 (19-Jan-2018) ***
Begin    14-May-2020 09:25:23 EDT
Input    o5cl02040_wav.fits
Output   o5cl02040_fwv_tmp.fits
OBSMODE  ACCUM
APERTURE 6X0.2
OPT_ELEM E140H
DETECTOR FUV-MAMA

Imset 1  Begin 09:25:23 EDT

DQICORR  PERFORM
DQITAB   oref$hcm14405o_bpx.fits
DQITAB   PEDIGREE=GROUND
DQITAB   DESCRIP =Prel. Ground Calib + edge flag
DQICORR  COMPLETE
         Uncertainty array initialized.

LORSCORR OMIT

GLINCORR PERFORM
LFLGCORR PERFORM
MLINTAB  oref$j9r16559o_lin.fits
MLINTAB  PEDIGREE=GROUND
MLINTAB  DESCRIP =T. Danks gathered Info
MLINTAB  DESCRIP =T. Danks Gathered Info
GLINCORR COMPLETE
LFLGCORR COMPLETE

DARKCORR PERFORM
DARKFILE oref$ma71317ro_drk.fits
DARKFILE PEDIGREE=INFLIGHT 10/01/2000 15/05/2001
DARKFILE DESCRIP =Average of 137 1380s darks from 7604
DARKCORR COMPLETE

FLATCORR PERFORM
PFLTFILE oref$mbj1658bo_pfl.fits
PFLTFILE PEDIGREE=INFLIGHT 17/03/1998 17/05/2002
PFLTFILE DESCRIP =On-orbit FUV P-flat created from proposals 7645,8428,8862,8922
LFLTFILE oref$m1b2139lo_lfl.fits
LFLTFILE PEDIGREE=DUMMY
LFLTFILE DESCRIP =Dummy file created by M. Mcgrath. Modified by C Proffitt
FLATCORR COMPLETE

PHOTCORR OMIT

DOPPCORR OMIT

STATFLAG PERFORM
STATFLAG COMPLETE
Imset 1  End 09:25:23 EDT

End      14-May-2020 09:25:23 EDT

*** CALSTIS-1 complete ***

*** CALSTIS-4 -- Version 3.4.2 (19-Jan-2018) ***
Begin    14-May-2020 09:25:23 EDT
Input    o5cl02040_fwv_tmp.fits
OBSMODE  ACCUM
APERTURE 6X0.2
OPT_ELEM E140H
DETECTOR FUV-MAMA
         Wavecal parameters are: \
         WL_TRIM1 = 10 \
         WL_TRIM2 = 10

Imset 1  Begin 09:25:23 EDT
         Section to use for finding shift: \
             11:1014, 11:1014 \
         Image pixels are reference pixel size.

WAVECORR PERFORM
WCPTAB   oref$lag1815lo_wcp.fits
WCPTAB   PEDIGREE=GROUND
WCPTAB   DESCRIP =Parameters used by calstis4 (wavecal processing)
WCPTAB   DESCRIP =TRIM values modified, WL_RANGE and SP_RANGE increased
LAMPTAB  oref$l421050oo_lmp.fits
LAMPTAB  PEDIGREE=GROUND
LAMPTAB  DESCRIP =Ground+Inflight Cal PT lamp, 3.8 & 10 mA
LAMPTAB  DESCRIP =Ground Cal for non-PRISM, PT lamp 10 mA
APDESTAB oref$n7p10329o_apd.fits
APDESTAB PEDIGREE=INFLIGHT 01/03/1997 01/03/1997
APDESTAB DESCRIP =Microscope Meas./Hartig Postlaunch Offsets
APDESTAB DESCRIP =Microscope Meas./Hartig Post-launch Offsets
DISPTAB  oref$m7p16110o_dsp.fits
DISPTAB  PEDIGREE=GROUND
DISPTAB  DESCRIP =Dispersion coefficients reference table
DISPTAB  DESCRIP =Prelaunch Calibration/Lindler
INANGTAB oref$h1v1541eo_iac.fits
INANGTAB PEDIGREE=GROUND
INANGTAB DESCRIP =Prelaunch Calibration/Lindler and Models
INANGTAB DESCRIP =Model
SPTRCTAB oref$mam15290o_1dt.fits
SPTRCTAB PEDIGREE=INFLIGHT 27/02/97 05/05/98
SPTRCTAB DESCRIP =Initial post-launch calibration
SPTRCTAB DESCRIP =Lindler, Initial Postlaunch Calibration
         Shift in dispersion direction is 0.120 pixels.
         Shift in spatial direction is -2.463 pixels.
WAVECORR COMPLETE
Imset 1  End 09:25:24 EDT

End      14-May-2020 09:25:24 EDT

*** CALSTIS-4 complete ***

*** CALSTIS-12 -- Version 3.4.2 (19-Jan-2018) ***
Begin    14-May-2020 09:25:24 EDT
Wavecal  o5cl02040_fwv_tmp.fits
Science  o5cl02040_flt.fits
OBSMODE  ACCUM
APERTURE 6X0.2
OPT_ELEM E140H
DETECTOR FUV-MAMA
Imset 1  Begin 09:25:24 EDT
         SHIFTA1 set to 0.119566
         SHIFTA2 set to -2.46287
Imset 1  End 09:25:24 EDT

End      14-May-2020 09:25:24 EDT

*** CALSTIS-12 complete ***

*** CALSTIS-6 -- Version 3.4.2 (19-Jan-2018) ***
Begin    14-May-2020 09:25:24 EDT
2020135092524-I--------------- CALSTIS-6 started -------------------------------

2-D scattering correction algorithm activated.

Warning  Grating-aperture throughput correction table (GACTAB) was not found,
         and no gac corrections will be applied
Read scattering functions and build kernels.
Reading ECHSCTAB: oref$k8m09585o_ech.fits
Reading EXSTAB: oref$k8m09586o_exs.fits
Reading CDSTAB: oref$k8m09584o_cds.fits
Reading RIPTAB: oref$k8m09588o_rip.fits
Reading SRWTAB: oref$k8m09589o_srw.fits
Reading HALOTAB: oref$k8m09587o_hal.fits
Reading TELTAB: oref$k8m0958ao_tel.fits
Computing Fourier Transforms of kernel at wavelength  1450
Computing Fourier Transforms of kernel at wavelength  1600


Program received signal SIGSEGV, Segmentation fault.
-----------------------------------------------------------------------------------------------------------------------[regs]
  RAX: 0x00000000556163F0  RBX: 0x0000000000000001  RBP: 0x00007FFFFFFEDE90  RSP: 0x00007FFFFFFED628  o d I t s z a p c 
  RDI: 0x00000000556163E0  RSI: 0x000000000000002F  RDX: 0x0000000000000013  RCX: 0x0000000000000010  RIP: 0x00007FFFF7A6C077
  R8 : 0x00005555556163F0  R9 : 0x0000000000000007  R10: 0x00007FFFF7FEF710  R11: 0x00007FFFF7A6C000  R12: 0x0000000000000001
  R13: 0x00007FFFFFFFDDE0  R14: 0x0000000000000000  R15: 0x0000000000000000
  CS: 0033  DS: 0000  ES: 0000  FS: 0000  GS: 0000  SS: 002B				
[0x002B:0x00007FFFFFFED628]-------------------------------------------------------------------------------------------[stack]
0x00007FFFFFFED678 : 40 DD FE FF FF 7F 00 00 - C0 C3 AC F7 FF 7F 00 00 @...............
0x00007FFFFFFED668 : 00 DC FE FF FF 7F 00 00 - 68 10 60 55 55 55 00 00 ........h.`UUU..
0x00007FFFFFFED658 : F0 63 61 55 00 00 00 00 - F0 DB FE FF FF 7F 00 00 .caU............
0x00007FFFFFFED648 : 9A B3 FF FF FF 7F 00 00 - 02 00 00 00 00 00 00 00 ................
0x00007FFFFFFED638 : 50 7C FF FF FF 7F 00 00 - 40 78 FF FF FF 7F 00 00 P|......@x......
0x00007FFFFFFED628 : 88 44 59 55 55 55 00 00 - 60 80 FF FF FF 7F 00 00 .DYUUU..`.......
[0x0000:0x00007FFFFFFED628]--------------------------------------------------------------------------------------------[data]
0x00007FFFFFFED628 : 88 44 59 55 55 55 00 00 - 60 80 FF FF FF 7F 00 00 .DYUUU..`.......
0x00007FFFFFFED638 : 50 7C FF FF FF 7F 00 00 - 40 78 FF FF FF 7F 00 00 P|......@x......
0x00007FFFFFFED648 : 9A B3 FF FF FF 7F 00 00 - 02 00 00 00 00 00 00 00 ................
0x00007FFFFFFED658 : F0 63 61 55 00 00 00 00 - F0 DB FE FF FF 7F 00 00 .caU............
0x00007FFFFFFED668 : 00 DC FE FF FF 7F 00 00 - 68 10 60 55 55 55 00 00 ........h.`UUU..
0x00007FFFFFFED678 : 40 DD FE FF FF 7F 00 00 - C0 C3 AC F7 FF 7F 00 00 @...............
0x00007FFFFFFED688 : 98 5A 97 F7 FF 7F 00 00 - 30 67 F9 56 55 55 00 00 .Z......0g.VUU..
0x00007FFFFFFED698 : 17 00 00 00 00 00 00 00 - 01 00 00 00 00 00 00 00 ................
-----------------------------------------------------------------------------------------------------------------------[code]
=> 0x7ffff7a6c077 <__strrchr_avx2+119>:	vmovdqa ymm1,YMMWORD PTR [rdi]
   0x7ffff7a6c07b <__strrchr_avx2+123>:	vpcmpeqb ymm2,ymm0,ymm1
   0x7ffff7a6c07f <__strrchr_avx2+127>:	vpcmpeqb ymm3,ymm4,ymm1
   0x7ffff7a6c083 <__strrchr_avx2+131>:	vpmovmskb edx,ymm2
   0x7ffff7a6c087 <__strrchr_avx2+135>:	vpmovmskb eax,ymm3
   0x7ffff7a6c08b <__strrchr_avx2+139>:	shr    edx,cl
   0x7ffff7a6c08d <__strrchr_avx2+141>:	shr    eax,cl
   0x7ffff7a6c08f <__strrchr_avx2+143>:	add    rdi,0x20
-----------------------------------------------------------------------------------------------------------------------------
0x00007ffff7a6c077 in __strrchr_avx2 () from /usr/lib/libc.so.6

# Ok, so strrchr blew up... where?
gdb$ stack
#0  0x00007ffff7a6c077 in __strrchr_avx2 () from /usr/lib/libc.so.6
#1  0x0000555555594488 in BuildTempNames (basename=0x7fffffffb39a "o5cl02040_x1d.fits", out1=0x7fffffff7840 "\034\205\377\377\377\177", out2=0x7fffffff7c50 "", out3=0x7fffffff8060 "\016") at ../pkg/stis/calstis/cs6/idtalg/calstis6idt.c:2106 /* <- HERE */
#2  0x000055555558c5c7 in CalStis6IDT (input=0x7fffffffaf98 "o5cl02040_flt.fits", output=0x7fffffffb39a "o5cl02040_x1d.fits", phdr=0x7fffffff8e70, backcorr=0x1, dispcorr=0x1, fluxcorr=0x1, helcorr=0x1, sgeocorr=0x0, ctecorr=0x0, cl_a2center=-1, maxsearch=0xffffffff, extrsize=-1, bk1size=-1, bk2size=-1, bk1offset=-1, bk2offset=-1, bktilt=-100, bkord=0xffffffff, sporder=0xffffffff, xtracalg=0x7fffffff9092 "", printtime=0x0, verbose=0x1, extrloc=0x1, ccglobal=0x0, ccthresh=0, do_profile=0x0, pstep=0x1, wstep=1, minsn=1, rejranges=0x7fffffff9093 "", profilefile=0x7fffffff9094 "", fluxfile=0x7fffffff9095 "", outw=0x7fffffff9096 "", backval=0, backerr=0, variance=0x0, fflux=0x1, psclip=0, sclip=0, lfilter=0x11, imset=0x0, pipeline=0x1, idtfile=0x7fffffff9097 "", blazeshift=1.7976931348623157e+308, bks_mode=0x0, bks_order=0x3, xoffset=0) at ../pkg/stis/calstis/cs6/idtalg/calstis6idt.c:362
#3  0x00005555555736ad in CalStis6 (input=0x7fffffffaf98 "o5cl02040_flt.fits", output=0x7fffffffb39a "o5cl02040_x1d.fits", backcorr=0x1, dispcorr=0x1, fluxcorr=0x1, helcorr=0x1, sgeocorr=0x0, ctecorr=0x0, sc2dcorr=0x1, cl_a2center=-1, maxsearch=0xffffffff, extrsize=-1, bk1size=-1, bk2size=-1, bk1offset=-1, bk2offset=-1, bktilt=-100, bkord=0xffffffff, sporder=0xffffffff, xtracalg=0x7fffffff9092 "", printtime=0x0, verbose=0x1, extrloc=0x1, ccglobal=0x0, ccthresh=0, do_profile=0x0, pstep=0x1, wstep=1, minsn=1, rejranges=0x7fffffff9093 "", profilefile=0x7fffffff9094 "", fluxfile=0x7fffffff9095 "", outw=0x7fffffff9096 "", backval=0, backerr=0, variance=0x0, fflux=0x1, psclip=0, sclip=0, lfilter=0x11, idtfile=0x7fffffff9097 "", subscale=10, blazeshift=1.7976931348623157e+308, bks_mode=0x1, bks_order=0x3, xoffset=0, pipeline=0x1) at ../pkg/stis/calstis/cs6/calstis6.c:151
#4  0x000055555555a863 in CalStis6_0 (input=0x7fffffffaf98 "o5cl02040_flt.fits", output=0x7fffffffb39a "o5cl02040_x1d.fits", sci_sw=0x7fffffff9110, printtime=0x0, verbose=0x1) at ../pkg/stis/calstis/cs0/calstis0.c:822
#5  0x0000555555559b28 in CalStis0 (rawfile=0x555555616450 "o5cl02040_raw.fits", wavfile=0x555555616860 "", outroot=0x555555616c70 "", printtime=0x0, save_tmp=0x1, verbose=0x1) at ../pkg/stis/calstis/cs0/calstis0.c:434
#6  0x0000555555558349 in main (argc=0x4, argv=0x7fffffffdde8) at ../pkg/stis/calstis/cs0/cs0.c:259

# Walk up the stack to the error
gdb$ up
#1  0x0000555555594488 in BuildTempNames (basename=0x7fffffffb39a "o5cl02040_x1d.fits", out1=0x7fffffff7840 "\034\205\377\377\377\177", out2=0x7fffffff7c50 "", out3=0x7fffffff8060 "\016") at ../pkg/stis/calstis/cs6/idtalg/calstis6idt.c:2106
2106		slash = strrchr (aname, '/');  /* <- the call that generated the segfault */

# Examine the surrounding code
gdb$ list
2101		char separator[2];
2102		char *slash;
2103		char *aname;
2104	
2105		aname = strdup (basename);  /* <-- basename=0x7fffffffb39a "o5cl02040_x1d.fits" is obviously not NULL, and this should have worked */
2106		slash = strrchr (aname, '/');  /* <-- The immediate use of aname throws a segfault */
2107	
2108		if (slash != NULL) {
2109		    *slash = '\0';
2110		    strcpy (file_path, aname);

PS - There's also a minor memory leak in general, but isn't the cause of the segfault:

char *aname is never free()ed

pllim · 2020-05-14T17:36:54Z

I can build but I cannot install. Not sure if it is related or not.

I did ./waf configure prefix=.../miniconda/envs/py37 but ./waf install tries to stick the exe files in /usr/local/bin/.

pllim

I can reproduce the install path problem with master, so this LGTM. Thanks!

UPDATE: Install issue was user error. 🤦

mdlpstsci · 2020-05-14T17:40:11Z

@jhunkeler @pllim Thank you both!

pllim · 2020-05-14T17:43:04Z

Note for future self: This is the more correct fix for #472 and supersedes #473

Use -std=gnu99

5a7c74b

* This is compatible with Apple Clang * (non-GNU) strdup() was producing garbage

pllim self-requested a review May 14, 2020 13:42

pllim added bug build labels May 14, 2020

pllim approved these changes May 14, 2020

View reviewed changes

pllim merged commit d392d8c into spacetelescope:master May 14, 2020

mdlpstsci mentioned this pull request May 14, 2020

Modified "dodqi" to accommodate the A-to-D saturation threshold #475

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use -std=gnu99 #485

Use -std=gnu99 #485

jhunkeler commented May 14, 2020 •

edited

mdlpstsci commented May 14, 2020

pllim commented May 14, 2020

jhunkeler commented May 14, 2020

pllim commented May 14, 2020

pllim left a comment •

edited

mdlpstsci commented May 14, 2020

pllim commented May 14, 2020

Use -std=gnu99 #485

Use -std=gnu99 #485

Conversation

jhunkeler commented May 14, 2020 • edited

mdlpstsci commented May 14, 2020

pllim commented May 14, 2020

jhunkeler commented May 14, 2020

pllim commented May 14, 2020

pllim left a comment • edited

Choose a reason for hiding this comment

mdlpstsci commented May 14, 2020

pllim commented May 14, 2020

jhunkeler commented May 14, 2020 •

edited

pllim left a comment •

edited