Labeling - Compressed Label Overwrite

[DonaldKerr]

For some time, and with various Mapserver versions (currently, I'm using MS4W 6.2-beta4 ), I am getting a strange compressed label overwrite as demonstrated in the following video file (5Mb): http://www.dkerr.co.uk/LabelIssue.wmv

Apologies for the file size but a video seems like the best way to demonstrate the issue.

It appears to be completely random in that the label will be written correctly most times but will then, for no apparent reason, will appear to compress and overwrite.

Here is the layer definition:

LAYER
    NAME "Test"
    TYPE POINT

    STATUS OFF

    METADATA
        "wms_title" "Test"
        "wfs_title" "Test"
        "wfs_extent" "0 0 670000 1230000"
        "wms_extent" "0 0 670000 1230000"
        "gml_exclude_items" "all" # No spaces
        "gml_featureid" "MyID"
    END # METADATA

    HEADER "header.html"
    TEMPLATE "blank.html"
    FOOTER "footer.html"

    PROJECTION # Required for WFS reprojection
        "init=epsg:27700"
    END # PROJECTION

    TOLERANCE 0
    TOLERANCEUNITS pixels # "pixels" broken in Mapserver 6.0.2

    CONNECTIONTYPE OGR
    CONNECTION "ODBC:Risks,Layer_Test" # Requires GEOMETRY_COLUMNS table (SPATIAL_REF_SYS optional) - Fails if DATA included in this map file

    CLASS
        Name "FullOILabelShadow"
        STYLE
            SYMBOL "WarningShadow"
            SIZE 35
            MAXSIZE 35
            MINSIZE 35
            OFFSET 9 0
        END #STYLE
        STYLEbbb
            SYMBOL "WarningRed"
            SIZE 35
            MAXSIZE 35
            MINSIZE 35
        END #STYLE
        TEXT ('[NameStr]')
        LABEL
            PRIORITY 10
            FONT arialbold
            SIZE 11
            TYPE truetype
            ANTIALIAS TRUE
            COLOR  0 0 0
            OUTLINECOLOR 255 255 192
            OUTLINEWIDTH 3
            #SHADOWCOLOR 192 192 192
            #SHADOWSIZE 12 12
            POSITION lr
            FORCE TRUE
            PARTIALS TRUE
            OFFSET -48 3
            WRAP "|"
        END #  LABEL
    END # CLASS

END # LAYER

Is there a problem with the LABEL part of the above layer definition or is this a bug?

Some sample images (screenshots):

Correct display of the label:

Incorrect display of the label:

The above labels are multiline with WRAP using the "|" character. This is also a problem with single line (non-wrapped) labels as shown in the video link above and the examples below.

Correct display of the label:

Incorrect display of the label:

Many thanks.

Regards,

Donald

[dmorissette]

You say that it is random, but when it happens, could you capture the GetMap URL (with firebug or other) and confirm that a given GetMap URL always reproduces the issue? That would get us closer to producing a testcase to reproduce the issue which is really what we need in order to troubleshoot this.

[DonaldKerr]

Daniel,

I have captured the following url using Firebug:

http://maps1/cgi-bin/mapserv.exe?MAP=D:\mapserver\my.map&LAYERS=MyLayer&TRANSPARENT=TRUE&FORMAT=png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&SRS=EPSG%3A27700&BBOX=238650.81271696,655984.48710239,273872.2164931,679167.32513473&WIDTH=1536&HEIGHT=1011

This above url was causing the problem labels; not all labels but a significant number. When I viewed the url in a browser on it's own, it repeatedly showed the problem labels.

I restarted my laptop and tried out the url and all was working well i.e. no problem labels.

I am thinking that the problem labels appear only after a great number of requests; maybe > 50 or 60 or more. That's anecdotal as I cannot replicate this absolutely.

Regards,

Donald

[dmorissette]

Based on the info we have so far, that could be a side-effect of some kind of memory/buffer corruption/management problem. The best way to verify this would be to run this same request under Valgrind (valgrind.org). Even if the problem is intermittent, Valgrind would tell us if there is a memory management issue.

Unfortunately Valgrind is only available on Linux. Do you have access to a Linux box to install your data, mapfile, symbolset/fontset and run this same request under Valgrind? (I can provide advice to use Valgrind if needed)

[DonaldKerr]

Thanks, Daniel. That was all sounding great until you said Linux only :( I will see what I can do about a Valgrind alternative or maybe a virtual Linux install on my Windows (spit) laptop. Will get back to you. Thanks.

Regards,

Donald

[dmorissette]

I was going to suggest you send me a testcase, but since the data source is ODBC that makes it harder to port it.

Actually, are you able to test using a shapefile as data source instead of ODBC? Just to rule out the (likely) possibility of the problem coming from the ODBC driver or its support libs causing some kind of memory corruption.

[DonaldKerr]

I'll see what I can do. Will look at it shortly as I need to pop out.

I've found this which may help as an alternative to Valgrind: http://code.google.com/p/drmemory/

Will also try to look at porting to shapefile to see if the problem can be replicated with that as data source.

[dmorissette]

I had never heard of Dr.Memory but would love to hear your feedback once you try it

[DonaldKerr]

No joy for me with Dr Memory - Ran the following with IE, Firefox and Chrome but never got any mention of Mapserver related dlls:

drmemory.exe -- "C:\Program Files\Internet Explorer\iexplore.exe" "http://maps1/cgi-bin/mapserv.
exe?MAP=D:\mapserver\my.map&LAYERS=FullOIsLabelsShadowClustered&TRANSPARENT=TRUE&FORMAT=p
ng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&SRS=EPSG%3A27700&BBOX=257272.18968193,664608.280
59996,258410.11195777,665357.26459794&WIDTH=1536&HEIGHT=1011"

Not sure where to go from here. I'll see what I can do about converting the MS Access data to shapfile though I'm not sure that's going to be easy.

[dmorissette]

The command above runs and tests the internet explorer process. We need to run it on mapserv.exe. Please try the following command:

drmemory.exe -- C:\path\to\mapserv.exe "QUERY_STRING=MAP=D:\mapserver\my.map&LAYERS=FullOIsLabelsShadowClustered&TRANSPARENT=TRUE&FORMAT=p
ng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&SRS=EPSG%3A27700&BBOX=257272.18968193,664608.280
59996,258410.11195777,665357.26459794&WIDTH=1536&HEIGHT=1011"

FYI the QUERY_STRING=... arg is a special hook for debugging MapServer at the command-line.

[DonaldKerr]

Thanks, Daniel.

Here's the results of the above: http://www.dkerr.co.uk/DrMemory-mapserv.exe.5720.zip

Regards,

Donald

[dmorissette]

Ouch! Lots of uninitialized reads, that's scary! Many seem to come from initialization of underlying libs on which MapServer has no control so MapServer itself cannot be the direct cause of all of them for sure. This makes me wonder if Dr.Memory is giving false positives or just too picky. I do however see several errors related to reading ODBC data so we cannot completely rule out the possibility of your problem being related to the OGR ODBC driver (or its support libs).

I'm not sure what to suggest at this point... it would be interesting if one of the MapServer devs on Windows could run regular MapServer requests under Dr. Memory and see if those issues are real or not... and fix them if they are. This may not be a small task though.

BTW, is your server setup with FastCGI? Maybe a way to work around your issue would be to disable FastCGI if it was enabled? (But that's not a real fix I agree)

[DonaldKerr]

There's a bit about Dr. memory uninitialized reads here: http://www.drmemory.org/docs/page_uninit.html - This is way out my comfort zone!

OGR ODBC may well be the cause of the labelling problem and I will work on a test (shapefile) that will hopefully eliminate that as soon as I can. Having said that, and having studied the rendered labels, it does look like some are being drawn with some overwrite rather than characters always being missing; to me, this would imply that the problem may be related to when the string is rendered rather than an incorrect string being returned to Mapserver for rendering. The above image examples show missing characters only which could mean that an incorrect string is being returned. The video in the above link shows an example of a label with overwrite rather than characters missing,

I do have FastCGI running with IIS 5.1 and this is pretty much essential for an end to end system that's at all useable. I could not turn that off. I'm not sure how that would affect the labelling problem.

[DonaldKerr]

We can discount ODBC as being the issue as I can confirm that I am also getting this in a PostgreSQL layer. See images below:

Correct:

Incorrectly rendered:

Incorrectly rendered:

Relevant part of map file:

    CLASS
        EXPRESSION ([featurecode] = 10213)
        LABEL
            TYPE truetype
            FONT [font]
            SIZE [height]
            MAXSIZE 21
            MINSIZE 4
            POSITION [anchorpositiontxt]
            COLOR [oscolor]
            ANTIALIAS TRUE
            ANGLE [orientation]
            FORCE TRUE
        END # LABEL
    END # CLASS

One other point to note is that the above images are from a Windows Server 2003 installation whilst all other sample images further above are from a Windows XP SP3 installation. Both MS4W 6.2-beta4).

[dmorissette]

I'd be tempted to say that it's good news that it is not specific to ODBC and that it happens on another system. The next step would be to package a small test case for a dev to reproduce it.

[DonaldKerr]

I have put together a test setup and I've made it available here: SAMPLE REMOVED AND REPLACED SEE NEXT MESSAGE. This has data contained in a shapefile rather than a PostGIS table; the former is an export from the latter. There's some info in ReadMe.txt about setting up the demo. Basically, the file MyMap.htm will reload a map image every second until it breaks. Unfortunately, or maybe fortunately, it has only broken and produced a compressed label, as described in this thread, once today despite me running this for a few hours. There's a stop button on the htm page that will stop the image refresh should a faulty label appear.

[DonaldKerr]

I have removed the above test setup and replaced it with this one: http://www.dkerr.co.uk/MyMapV2.zip (538K). This falls over for me very quickly i.e. within a minute and shows the label overwrite consistently.

Look for a label coming in from the top. It should say "Standing Stone". There's another label that comes in from the right. This should say "Rocks". If you leave it to cycle round for a few times, the labels will materialise in various forms (including correctly displayed).

Should say "Standing Stone" with no overwrite:

Should say "Rocks" with no overwrite:

[unicolet]

Donald,
could you capture the http request that produces the compressed labels?
By replacing the extent on the map file with the bbox from the request
perhaps we can find a way to consistently reproduce the error.

/Umberto

[DonaldKerr]

Umberto,

I capture the following request but when I post it in another browser, it renders as it should do i.e. no compressed labels; even with multiple F5 refreshes. This is the exact same WMS request that showed faulty labels.

http://localhost/cgi-bin/mapserv.exe?MAP=D:\MyMap\my.map&LAYERS=MyLayer&FORMAT=png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&SRS=EPSG%3A27700&BBOX=97970,886522,98262,886711&WIDTH=1200&HEIGHT=400&0.4836536909974589

Regards,

Donald

[DonaldKerr]

I have narrowed down the problem a good bit. I have the following OUTPUTFORMAT statement in my map file. This uses CAIRO to render output. If I change CAIRO to AGG then I DO NOT get the label overwrite problem. Is this a CAIRO bug? I've tried commenting out some of the options like FORMATOPTION AND TRANSPARENT to no avail.

OUTPUTFORMAT
    NAME "png"
    DRIVER "CAIRO/PNG"
    MIMETYPE "image/png"
    IMAGEMODE "RGBA"
    EXTENSION "png"
    FORMATOPTION "GAMMA=0.75"
    TRANSPARENT ON
END

[DonaldKerr]

Just to say that I have FONTSET "osfonts.txt" in the map file. The file osfonts.txt contains: "arial arial.ttf" (no quotes). I have changed this physical file (arial.ttf) to another TTF and the problem persists thus eliminating the font file.

[jmckenna]

@DonaldKerr I will grab your test case and try in a new Cairo build. Is the test case here? http://www.dkerr.co.uk/MyMapV2.zip

[DonaldKerr]

Yes, Jeff. Try with the old Cairo first to ensure that you can see the issue. For me, if falls over almost within the first few image creations using MyMap.htm (in the zip file) which requests a new image every second. If I change "CAIRO/PNG" to "AGG/PNG" in the map file live whilst MyMap.htm is requesting images, it goes from creating bad labels to creating good labels and vice versa.

Many thanks.

Regards,

Donald

[jmckenna]

@DonaldKerr I've tried with the old Cairo in MS4W. Here are my commands:

    mapserv -nh "QUERY_STRING=map=my.map&mode=map" > test1.png
    mapserv -nh "QUERY_STRING=map=my.map&mode=map" > test2.png
    mapserv -nh "QUERY_STRING=map=my.map&mode=map" > test3.png
    mapserv -nh "QUERY_STRING=map=my.map&mode=map" > test4.png
    ...

I don't see any problems. Unfortunately I must run now so I cannot test your full application.

[DonaldKerr]

Thanks for looking at that Jeff. All looks as it should. I suggest that you try to set up MyMap.htm which requests images once every second. Within a few images it falls over. There is definitely a repeatable problem.

Many thanks.

Regards,

Donald

[jmckenna]

@DonaldKerr I just ran this command (it will create 100 pngs). Give it a try. I don't see any changes in the results.

for /L %A in (1, 1, 100) do (mapserv -nh "QUERY_STRING=map=my.map&mode=map" > test%A.png)

When I get time I will configure you application. I wish I was cloned right now :) Until I'm cloned, try reproducing your problem with that script, through mapserv or shp2img calls.

[DonaldKerr]

I'll give that a go right now and will post back. The only difference is that my htm page changes the BBOX values very slightly for each call below:

http://localhost/cgi-bin/mapserv.exe?MAP=D:\MyMap\my.map&LAYERS=MyLayer&FORMAT=png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&SRS=EPSG%3A27700&BBOX="+BLE+","+BLN+","+TRE+","+TRN+"&WIDTH=1200&HEIGHT=400

var BLEOrig = 97883;
var BLNOrig = 886464;

BLE = BLEOrig;
BLN = BLNOrig;
TRE = BLE+292;
TRN = BLN+189;

Each time the image is refreshed, this is done to change the BBOX slightly and to move the image on the htm page so that the viewer can actually see something happening:

BLE+=3;
BLN+=2;
TRE+=3;
TRN+=2; 

[DonaldKerr]

Jeff,

The following is a Windows batch file (save as test.bat):

setlocal ENABLEDELAYEDEXPANSION
set BLE=97883
set BLN=886464
set /a TRE=%BLE%+292
set /a TRN=%BLN%+189
for /L %%A in (1, 1, 50) do (
set /a BLE=!BLE!+3
set /a BLN=!BLN!+2
set /a TRE=!TRE!+3
set /a TRN=!TRN!+2
mapserv.exe -nh "QUERY_STRING=MAP=my.map&mode=map&LAYERS=MyLayer&map_size=1200+400&MAPEXT=!BLE!+!BLN!+!TRE!+!TRN!" > test%%A.png
)

This basically replicates what I'm doing with the html test file (MyMap.htm) by changing the BBOX (or MAPEXT as this is mapserv CGI).

However, this DOES NOT replicate the issue and all images are created correctly.

The WMS request, as per the MyMap.htm example, DOES replicate the issue. So, I still think this is maybe a Cairo related problem but when called via a WMS: When I change CAIRO to AGG in the map file it starts working. Maybe it's a WMS related issue when coupled with Cairo.

[DonaldKerr]

Here is a test example that shows the error both with CGI and WMS calls using Mapserver's built-in OpenLayers viewer.

Download and extract the following file: http://www.dkerr.co.uk/MyMapV3.zip (269K). You will need to edit my.map and replace "D:\MyMap", "D:\MyMap" and "d:/MyMap" with the loction to which you've extracted the example. There are some changes to my.map that makes it different from MyMapV2.zip (above) so don't mix up the example files.

CGI Url:
http://localhost/cgi-bin/mapserv.exe?map=d:\MyMap\my.map&mode=browse&template=openlayers&layer=MyLayer

WMS Url:
http://localhost/cgi-bin/mapserv.exe?map=d:\MyMap\my.map&LAYERS=MyLayer&VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&FORMAT=application/openlayers&WIDTH=512&HEIGHT=512&SRS=EPSG:27700&BBOX=96883,885464,99175,887653

To replicate the issue, enter one or other of the above urls into a browser. Zoom in four times then drag the map about. You should see the compressed labels with both the CGI and WMS calls. Change DRIVER "CAIRO/PNG" to DRIVER "AGG/PNG" in the OUTPUTFORMAT section of the map file and the issue goes away.

N.B. I had to use double-backslashes for my map file location in the urls above when pasted into a browser address bar..

[dmorissette]

Guys, I believe this will be reproducible only with FastCGI... so it is normal that a loop of calls to mapserv.exe from a script or command window doesn't reproduce it.

[DonaldKerr]

Further information: I have both IIS and Apache (on port 81) set up; both use FastCGI. I tested the example in the previous post in Apache by adding ":81" to the url e.g. "http://localhost:81/cgi-bin/..." and the issue DOES NOT appear i.e. labels are drawn correctly. I turned off FastCGI in IIS and again, the issue DOES NOT appear. The version of IIS on this machine is 5.1 (Win XP SP3) but the same thing is happening with IIS 7 (Server 2003) on another machine that I can access.

So, this issue appears to involve FastCGI in IIS (@dmorissette - you had previously suggested disabling FastCGI) though turning off FastCGI is not an option.

The combination of Cairo and IIS FastCGI appears to consistently cause this problem.

[dmorissette]

Since the issue would apparently involve IIS FastCGI specifically (i.e. not reproducible with Apache+FastCGI), then I'll step out of this ticket for now and let other Windows devs look into it. My short term fix would be "use Apache" ;-) but I realize that this is not an option for everyone.

@DonaldKerr, it may be fair at this point to contract with one of the Windows devs to work on this if this issue is really important to you.

[DonaldKerr]

Thanks for your input, Daniel.

Much as I would love to, I cannot use Apache in these circumstances.

This still appears to be a CAIRO, IIS and FastCGI issue. In the meantime, I've changed the map file thus:

#DRIVER "CAIRO/PNG"
DRIVER "AGG/PNG"

i.e. I'm using AGG instead of CAIRO until this is resolved.

Many thanks.

Regards,

Donald

[jratike80]

Entertaining ticket to read. @DonaldKerr , have you tried if current versions of Mapserver, IIS, and Cairo show the same issue?

[DonaldKerr]

Thank you. I haven't carried out any recent development as I have changed jobs and no longer support or develop the system in question. I might revisit this in the future as I may look at developing a replacement system on a commercial basis.

[tbonfort]

closing now as label handling has been rewritten for mapserver 7.0