Infinite Loop(?) with msSHPLayerNextShape

[akrherz]

I am testing out current master mapserver on RHEL7 64 bit. I am having a problem with what appears to be an infinite loop with mapshape.c#msSHPLayerNextShape. I suspect that the underlying shapefile is being updated while mapserver attempts to read from it.

The current mapshape.c has this:

int msSHPLayerNextShape(layerObj *layer, shapeObj *shape)
{
 //snipped

  msSHPReadShape(shpfile->hSHP, i, shape);
  if(shape->type == MS_SHAPE_NULL) {
    msFreeShape(shape);
    msSHPLayerNextShape(layer, shape); /* skip NULL shapes */
  }

whereas 6.4.1 has this (which I don't see this issue with)

  do {
  //snipped

    msSHPReadShape(shpfile->hSHP, i, shape);
    if(shape->type == MS_SHAPE_NULL) {
      msFreeShape(shape);
      continue; /* skip NULL shapes */
    }
}

So I am wondering if msSHPLayerNextShape is in a recursive loop. The mapserv CGI process spins 100% CPU and eats more and more memory when this happens. Thanks

[sdlime]

The change in the code (removing the do-while loop) is because FILTERing was moved out to the layer-level msLayerNextShape() function as it applies to all driver types. Functionally it should be the same. This one is going to be impossible to debug with out a test case - any chance you have something that is repeatable?

Steve

[rouault]

@sdlime Probably that prefixing "msSHPLayerNextShape(layer, shape); /* skip NULL shapes */" with "return" would be better (otherwise you're going to set shape->values multiple times). But this this assumes that the compiler handles tail recursion correctly. The loop approach was safer in this respect

[akrherz]

@sdlime I can reproduce this at-will with my production system testing, but not on my dev laptop in isolation :). I am currently testing with this patch as suggested by @rouault

diff --git a/mapshape.c b/mapshape.c
index e139b5e..174407c 100644
--- a/mapshape.c
+++ b/mapshape.c
@@ -2688,7 +2688,7 @@ int msSHPLayerNextShape(layerObj *layer, shapeObj *shape)
   msSHPReadShape(shpfile->hSHP, i, shape);
   if(shape->type == MS_SHAPE_NULL) {
     msFreeShape(shape);
-    msSHPLayerNextShape(layer, shape); /* skip NULL shapes */
+    return msSHPLayerNextShape(layer, shape); /* skip NULL shapes */
   }
   shape->numvalues = layer->numitems;
   shape->values = msDBFGetValueList(shpfile->hDBF, i, layer->iteminfo, layer->numitems);

and am not seeing the issue, but will watch it closely.

[sdlime]

Ignore my first comment. Actually returning NULL isn't possible given the function sig. Could return MS_SUCCESS if shape->type=MS_NULL_SHAPE then add the:

if(shape->type == MS_SHAPE_NULL) {
msFreeShape(shape);
msLayerNextShape(layer, shape); /* skip NULL shapes */
}

Check to msLayerNextShape()...However, null shapes see to be a shapefile phenomenon only aren't they?

Steve

[rouault]

OGR datasources could return NULL shapes (haven't checked the OGR mapserver provider, but OGR can certainly return a feature without geometry)

[akrherz]

I have been running the patch mentioned above and am not seeing the CPU/memory usage issue. I am unsure if there are other side effects or not.

[sdlime]

What do you think @rouault, should we go with the simple patch? Probably safest... --Steve

[rouault]

The patch with the added return, you mean ? Yes, that shouldn't hurt (and hopefully in optimized mode the compiler will turn the call into a jump)

[sdlime]

Calling this one closed.@akherz, thanks for the detailed bug report and @rouault for the easy fix. --Steve