Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] Fatal glibc error: malloc assertion failure in _int_malloc: (unsigned long) (size) >= (unsigned long) (nb) #2631

Open
sebastic opened this issue Nov 6, 2022 · 4 comments
Open

[Bug] Fatal glibc error: malloc assertion failure in _int_malloc: (unsigned long) (size) >= (unsigned long) (nb) #2631

sebastic opened this issue Nov 6, 2022 · 4 comments
Labels
bug Something isn't working

Comments

@sebastic
Copy link
Contributor

sebastic commented Nov 6, 2022 •
edited
Loading

Describe the bug
Running ogrinfo on the spearfish dataset with gdal-grass plugins installed on 32bit architectures fails:

$ ogrinfo -so -al /tmp/spearfish*/PERMANENT/vector/roads/head ; echo $?
Warning 1: GRASS warning: GISBASE environment variable was not set, using:
/usr/lib/grass82
Fatal glibc error: malloc assertion failure in _int_malloc: (unsigned long) (size) >= (unsigned long) (nb)
Aborted
134

Backtrace:

# gdb -args ogrinfo -so -al /tmp/spearfish*/PERMANENT/vector/roads/head
GNU gdb (Debian 12.1-4) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "i686-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ogrinfo...
Reading symbols from /usr/lib/debug/.build-id/67/97d000a32909a3b958be61463aacae3b1a9452.debug...
(gdb) run
Starting program: /usr/bin/ogrinfo -so -al /tmp/spearfish60_grass7/PERMANENT/vector/roads/head
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Warning 1: GRASS warning: GISBASE environment variable was not set, using:
/usr/lib/grass82
Fatal glibc error: malloc assertion failure in _int_malloc: (unsigned long) (size) >= (unsigned long) (nb)

Program received signal SIGABRT, Aborted.
0xf7fc7559 in __kernel_vsyscall ()
(gdb) bt
#0  0xf7fc7559 in __kernel_vsyscall ()
#1  0xf65081d7 in __pthread_kill_implementation (threadid=threadid@entry=4002894912, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:43
#2  0xf650824b in __pthread_kill_internal (signo=6, threadid=4002894912) at ./nptl/pthread_kill.c:78
#3  0xf64b70d1 in __GI_raise (sig=6) at ../sysdeps/posix/raise.c:26
#4  0xf64a026a in __GI_abort () at ./stdlib/abort.c:79
#5  0xf64fab76 in __libc_message (action=<optimized out>, fmt=<optimized out>) at ../sysdeps/posix/libc_fatal.c:155
#6  0xf6516d69 in __malloc_assert (function=0xf663a290 <__PRETTY_FUNCTION__.7> "_int_malloc", line=4298, file=<synthetic pointer>, assertion=0xf6639e30 "(unsigned long) (size) >= (unsigned long) (nb)") at ./malloc/malloc.c:299
#7  _int_malloc (av=av@entry=0xf669b7c0 <main_arena>, bytes=bytes@entry=48) at ./malloc/malloc.c:4298
#8  0xf651781f in __GI___libc_malloc (bytes=48) at ./malloc/malloc.c:3315
#9  0xee84230b in RTreeAllocBoundary (t=0x56603a80) at ./lib/vector/rtree/rect.c:85
#10 0xee83e87b in RTreeCreateTree (fd=-1, rootpos=0, ndims=2) at ./lib/vector/rtree/index.c:180
#11 0xee8749f4 in dig_spidx_init (Plus=0x56603f00) at ./lib/vector/diglib/spindex.c:80
#12 0xee86e2e1 in dig_init_plus (Plus=0x56603f00) at ./lib/vector/diglib/plus.c:40
#13 0xee8c4a63 in Vect__open_old (Map=0x56603ef4, name=0x56609ed0 "roads", mapset=0x56609f90 "PERMANENT", layer=0x0, update=0, head_only=0, is_tmp=0) at ./lib/vector/Vlib/open.c:196
#14 0xee8c58b8 in Vect_open_old (Map=0x56603ef4, name=0x56609ed0 "roads", mapset=0x56609f90 "PERMANENT") at ./lib/vector/Vlib/open.c:575
#15 0xee91a323 in OGRGRASSDataSource::Open (this=0x56603e20, pszNewName=0x566073c0 "/tmp/spearfish60_grass7/PERMANENT/vector/roads/head", bTestOpen=1) at ./ogrgrassdatasource.cpp:189
#16 0xee919daf in OGRGRASSDriver::Open (this=0x5659fb60, pszFilename=0x566073c0 "/tmp/spearfish60_grass7/PERMANENT/vector/roads/head", bUpdate=0) at ./ogrgrassdriver.cpp:62
#17 0xf785a170 in OGRSFDriverRegistrar::OpenWithDriverArg (poDriver=0x5659fb60, poOpenInfo=0xffffd304) at ./ogr/ogrsf_frmts/generic/ogrsfdriverregistrar.cpp:196
#18 0xf7891319 in GDALOpenEx (pszFilename=<optimized out>, nOpenFlags=<optimized out>, papszAllowedDrivers=<optimized out>, papszOpenOptions=<optimized out>, papszSiblingFiles=<optimized out>) at ./gcore/gdaldataset.cpp:3525
#19 0x56557d58 in main (nArgc=<optimized out>, papszArgv=<optimized out>) at ./apps/ogrinfo.cpp:999
(gdb) bt full
#0  0xf7fc7559 in __kernel_vsyscall ()
No symbol table info available.
#1  0xf65081d7 in __pthread_kill_implementation (threadid=threadid@entry=4002894912, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:43
        resultvar = <optimized out>
        tid = 783538
        ret = <optimized out>
        pd = 0xee975440
        old_mask = {__val = {4134115904, 4294967295}}
        ret = <optimized out>
#2  0xf650824b in __pthread_kill_internal (signo=6, threadid=4002894912) at ./nptl/pthread_kill.c:78
No locals.
#3  0xf64b70d1 in __GI_raise (sig=6) at ../sysdeps/posix/raise.c:26
        ret = <optimized out>
#4  0xf64a026a in __GI_abort () at ./stdlib/abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0x20, sa_sigaction = 0x20}, sa_mask = {__val = {0, 0, 1966434816, 0, 720822983, 0, 4134123456, 4134121460, 4134123456, 4134121460, 321, 4134121460, 4134123512, 4294966996, 12, 16, 1, 
              1, 16, 0, 3, 4096, 4133042146, 4294944152, 4294943792, 4134121460, 4133069431, 0, 4294967295, 0, 1966434816, 4096}}, sa_flags = -160845836, sa_restorer = 0xffffa470}
#5  0xf64fab76 in __libc_message (action=<optimized out>, fmt=<optimized out>) at ../sysdeps/posix/libc_fatal.c:155
        ap = <optimized out>
        fd = <optimized out>
        list = <optimized out>
        nlist = <optimized out>
        cp = <optimized out>
#6  0xf6516d69 in __malloc_assert (function=0xf663a290 <__PRETTY_FUNCTION__.7> "_int_malloc", line=4298, file=<synthetic pointer>, assertion=0xf6639e30 "(unsigned long) (size) >= (unsigned long) (nb)") at ./malloc/malloc.c:299
No locals.
#7  _int_malloc (av=av@entry=0xf669b7c0 <main_arena>, bytes=bytes@entry=48) at ./malloc/malloc.c:4298
        p = <optimized out>
        iters = <optimized out>
        nb = <optimized out>
        idx = 6
        bin = <optimized out>
        victim = <optimized out>
        size = 0
        victim_index = <optimized out>
        remainder = <optimized out>
        remainder_size = <optimized out>
        block = 2
        bit = <optimized out>
        map = <optimized out>
        fwd = <optimized out>
        bck = <optimized out>
        tcache_unsorted_count = <optimized out>
        tcache_nb = 64
        tc_idx = 3
        return_cached = <optimized out>
        __PRETTY_FUNCTION__ = "_int_malloc"
#8  0xf651781f in __GI___libc_malloc (bytes=48) at ./malloc/malloc.c:3315
        ar_ptr = <optimized out>
        victim = <optimized out>
        tbytes = <optimized out>
        tc_idx = 3
        __PRETTY_FUNCTION__ = "__libc_malloc"
#9  0xee84230b in RTreeAllocBoundary (t=0x56603a80) at ./lib/vector/rtree/rect.c:85
        boundary = <optimized out>
#10 0xee83e87b in RTreeCreateTree (fd=-1, rootpos=0, ndims=2) at ./lib/vector/rtree/index.c:180
        new_rtree = 0x56603a80
        n = 0x56601e30
        i = <optimized out>
        j = <optimized out>
        k = <optimized out>
#11 0xee8749f4 in dig_spidx_init (Plus=0x56603f00) at ./lib/vector/diglib/spindex.c:80
        ndims = 2
#12 0xee86e2e1 in dig_init_plus (Plus=0x56603f00) at ./lib/vector/diglib/plus.c:40
No locals.
#13 0xee8c4a63 in Vect__open_old (Map=0x56603ef4, name=0x56609ed0 "roads", mapset=0x56609f90 "PERMANENT", layer=0x0, update=0, head_only=0, is_tmp=0) at ./lib/vector/Vlib/open.c:196
        xname = "\000\000\000\000\000\000\000\000\314\247\377\377(\250\377\377\314\307\377\377", '\000' <repeats 76 times>, "\377\377\377\377", '\000' <repeats 40 times>, "\200\223i\366\000\275i\366", '\000' <repeats 12 times>, "Warning 1: GRASS warning: GISBASE enviro"...
        xmapset = '\000' <repeats 255 times>
        path = '\000' <repeats 2672 times>...
        fp = <optimized out>
        level = <optimized out>
        level_request = 2
        format = <optimized out>
        ret = <optimized out>
        ogr_mapset = <optimized out>
        fmapset = <optimized out>
#14 0xee8c58b8 in Vect_open_old (Map=0x56603ef4, name=0x56609ed0 "roads", mapset=0x56609f90 "PERMANENT") at ./lib/vector/Vlib/open.c:575
No locals.
#15 0xee91a323 in OGRGRASSDataSource::Open (this=0x56603e20, pszNewName=0x566073c0 "/tmp/spearfish60_grass7/PERMANENT/vector/roads/head", bTestOpen=1) at ./ogrgrassdatasource.cpp:189
        stat = {st_dev = 2051, __pad1 = 0, st_ino = 7753983, st_mode = 33188, st_nlink = 1, st_uid = 1338, st_gid = 500, st_rdev = 0, __pad2 = 0, st_size = 258, st_blksize = 4096, st_blocks = 8, st_atim = {tv_sec = 1667751698, 
            tv_nsec = 49335245}, st_mtim = {tv_sec = 1142443082, tv_nsec = 0}, st_ctim = {tv_sec = 1667751658, tv_nsec = 718502396}, __glibc_reserved4 = 0, __glibc_reserved5 = 0}
        level = <optimized out>
        ncidx = <optimized out>
#16 0xee919daf in OGRGRASSDriver::Open (this=0x5659fb60, pszFilename=0x566073c0 "/tmp/spearfish60_grass7/PERMANENT/vector/roads/head", bUpdate=0) at ./ogrgrassdriver.cpp:62
        poDS = 0x56603e20
#17 0xf785a170 in OGRSFDriverRegistrar::OpenWithDriverArg (poDriver=0x5659fb60, poOpenInfo=0xffffd304) at ./ogr/ogrsf_frmts/generic/ogrsfdriverregistrar.cpp:196
        poDS = <optimized out>
#18 0xf7891319 in GDALOpenEx (pszFilename=<optimized out>, nOpenFlags=<optimized out>, papszAllowedDrivers=<optimized out>, papszOpenOptions=<optimized out>, papszSiblingFiles=<optimized out>) at ./gcore/gdaldataset.cpp:3525
        papszTmpOpenOptions = <optimized out>
        bIdentifyRes = <optimized out>
        poDS = 0x0
        poDriver = 0x5659fb60
        papszTmpOpenOptionsToValidate = <optimized out>
        papszOptionsToValidate = <optimized out>
        iDriver = 0
        poDM = 0x565859a0
        oOpenInfo = {bHasGotSiblingFiles = false, papszSiblingFiles = 0x0, nHeaderBytesTried = 1024, pszFilename = 0x566073c0 "/tmp/spearfish60_grass7/PERMANENT/vector/roads/head", papszOpenOptions = 0x0, eAccess = GA_ReadOnly, 
          nOpenFlags = 4, bStatOK = 1, bIsDirectory = 0, fpL = 0x56609f50, nHeaderBytes = 258, 
          pabyHeader = 0x56605030 "ORGANIZATION: US Army Const. Eng. Rsch. Lab\nDIGIT DATE:   5/26/90\nDIGIT NAME:   youngs\nMAP NAME:     Output from Vpatch\nMAP DATE:     1961\nMAP SCALE:    24000\nOTHER INFO:   new roads at 1:24000 for Sp"..., papszAllowedDrivers = 0x0}
        sAntiRecursion = <optimized out>
        osAllowedDrivers = ""
        dsCtxt = {osFilename = "/tmp/spearfish60_grass7/PERMANENT/vector/roads/head", nOpenFlags = 4, osAllowedDrivers = ""}
        papszOpenOptionsCleaned = <optimized out>
        nDriverCount = <optimized out>
#19 0x56557d58 in main (nArgc=<optimized out>, papszArgv=<optimized out>) at ./apps/ogrinfo.cpp:999
        pszWHERE = <optimized out>
        pszDataSource = <optimized out>
        papszLayers = <optimized out>
        poSpatialFilter = <optimized out>
        nRepeatCount = <optimized out>
        bAllLayers = <optimized out>
        pszSQLStatement = <optimized out>
        pszDialect = <optimized out>
        nRet = 0
        pszGeomField = <optimized out>
        papszOpenOptions = <optimized out>
        papszExtraMDDomains = <optimized out>
        bListMDD = <optimized out>
        bShowMetadata = <optimized out>
        bFeatureCount = <optimized out>
        bExtent = <optimized out>
        bGeomType = <optimized out>
        bDatasetGetNextFeature = <optimized out>
        bReadOnly = <optimized out>
        bUpdate = <optimized out>
        pszWKTFormat = <optimized out>
        osFieldDomain = ""
        poDS = <optimized out>
        poDriver = <optimized out>
        nLayerCount = <optimized out>

This was revealed by the autopkgtest in the Debian package on armel, armhf, i386

To Reproduce
Steps to reproduce the behavior:

  1. Login to i386 VM or chroot (e.g. sudo cowbuilder --login --basepath /var/cache/pbuilder/base-sid-i386.cow)
  2. apt install ca-certificates gdal-bin libgdal-grass wget
  3. wget https://grass.osgeo.org/sampledata/spearfish_grass70data-0.3.tar.gz -P /tmp
  4. tar xavf /tmp/spearfish_grass70data-0.3.tar.gz -C /tmp
  5. ogrinfo -so -al /tmp/spearfish*/PERMANENT/vector/roads/head

Expected behavior
No error as on 64bit architectures:

INFO: Open of `/tmp/spearfish60_grass7/PERMANENT/vector/roads/head'
      using driver `OGR_GRASS' successful.

Layer name: roads
Geometry: Line String
Feature Count: 825
Extent: (589434.856469, 4914006.337837) - (609527.210215, 4928063.398015)
Layer SRS WKT:
PROJCRS["unknown",
    BASEGEOGCRS["clark66",
        DATUM["North American Datum 1927",
            ELLIPSOID["Clarke_1866",6378206.4,294.9786982,
                LENGTHUNIT["metre",1]],
            ID["EPSG",6267]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8901]]],
    CONVERSION["UTM zone 13N",
        METHOD["Transverse Mercator",
            ID["EPSG",9807]],
        PARAMETER["Latitude of natural origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",-105,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["Scale factor at natural origin",0.9996,
            SCALEUNIT["unity",1],
            ID["EPSG",8805]],
        PARAMETER["False easting",500000,
            LENGTHUNIT["metre",1],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8807]],
        ID["EPSG",16013]],
    CS[Cartesian,2],
        AXIS["easting",east,
            ORDER[1],
            LENGTHUNIT["metre",1,
                ID["EPSG",9001]]],
        AXIS["northing",north,
            ORDER[2],
            LENGTHUNIT["metre",1,
                ID["EPSG",9001]]]]
Data axis to CRS axis mapping: 1,2
cat: Integer (0.0)
label: String (0.0)

System description

  • Operating System: Debian unstable
  • GRASS GIS version: 8.2.0
  • gdal-grass version: 1.0.2

Additional context
The issue only affects vector data, gdalinfo works as expected:

$ GISBASE=$(ls -d /usr/lib/grass??) PROJ_NETWORK=ON gdalinfo /tmp/spearfish*/PERMANENT/cellhd/geology
Driver: GRASS/GRASS Rasters (7+)
Files: /tmp/spearfish60_grass7/PERMANENT/cellhd/geology
Size is 190, 140
Coordinate System is:
PROJCRS["unknown",
    BASEGEOGCRS["clark66",
        DATUM["North American Datum 1927",
            ELLIPSOID["Clarke_1866",6378206.4,294.9786982,
                LENGTHUNIT["metre",1]],
            ID["EPSG",6267]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8901]]],
    CONVERSION["UTM zone 13N",
        METHOD["Transverse Mercator",
            ID["EPSG",9807]],
        PARAMETER["Latitude of natural origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",-105,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["Scale factor at natural origin",0.9996,
            SCALEUNIT["unity",1],
            ID["EPSG",8805]],
        PARAMETER["False easting",500000,
            LENGTHUNIT["metre",1],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8807]],
        ID["EPSG",16013]],
    CS[Cartesian,2],
        AXIS["easting",east,
            ORDER[1],
            LENGTHUNIT["metre",1,
                ID["EPSG",9001]]],
        AXIS["northing",north,
            ORDER[2],
            LENGTHUNIT["metre",1,
                ID["EPSG",9001]]]]
Data axis to CRS axis mapping: 1,2
Origin = (590000.000000000000000,4928000.000000000000000)
Pixel Size = (100.000000000000000,-100.000000000000000)
Corner Coordinates:
Upper Left  (  590000.000, 4928000.000) (103d52' 4.42"W, 44d30' 5.97"N)
Lower Left  (  590000.000, 4914000.000) (103d52'13.17"W, 44d22'32.31"N)
Upper Right (  609000.000, 4928000.000) (103d37'44.25"W, 44d29'56.54"N)
Lower Right (  609000.000, 4914000.000) (103d37'54.84"W, 44d22'22.93"N)
Center      (  599500.000, 4921000.000) (103d44'59.16"W, 44d26'14.67"N)
Band 1 Block=190x1 Type=Byte, ColorInterp=Palette
  Min=1.000 Max=9.000 
  NoData Value=0
  Metadata:
    COLOR_TABLE_RULES_COUNT=0
  Color Table (RGB with 10 entries)
    0: 197,129,125,255
    1: 107,250,75,255
    2: 226,83,250,255
    3: 246,222,188,255
    4: 123,225,27,255
    5: 134,190,1,255
    6: 48,86,221,255
    7: 113,70,15,255
    8: 102,134,101,255
    9: 89,135,169,255
@sebastic sebastic added the bug Something isn't working label Nov 6, 2022
@metzm
Copy link
Contributor

metzm commented Nov 12, 2022

This line https://github.com/OSGeo/grass/blob/main/lib/vector/rtree/rect.c#L85 should be fool-proof:

    RectReal *boundary = (RectReal *) malloc(t->rectsize);

It seems that GDAL without the GDAL-GRASS plugin and GRASS itself are working properly, it is only the GDAL-GRASS plugin that is triggering this bug?

@marisn
Copy link
Contributor

marisn commented Nov 12, 2022

Please run the command under valgrind as memory corruption could happen way earlier.

@sebastic
Copy link
Contributor Author

valgrind.txt

@sebastic
Copy link
Contributor Author

sebastic commented Jul 4, 2023

Still present in GRASS 8.3.0 with GDAL 3.7.0 & gdal-grass 1.0.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@sebastic @marisn @metzm