Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SLD - Missing escaped characters when converting OGC propertyIsLike to Regex #5658

Closed
pmauduit opened this issue Sep 5, 2018 · 3 comments
Closed

Comments

@pmauduit
Copy link
Contributor

@pmauduit pmauduit commented Sep 5, 2018

Using the following getmap request:

/usr/lib/cgi-bin/mapserv QUERY_STRING='MAP=/root/resources/version-shapefile/mapfile.map&
SERVICE=WMS&
VERSION=1.3.0&
REQUEST=GetMap&
FORMAT=image%2Fpng&
TRANSPARENT=true&
LAYERS=surval_30140_all_point_19_12_2017&
SLD=http://host-where-my-sld-is-located/sld-shapefile.xml&
CRS=EPSG%3A3857&
STYLES=&
WIDTH=2036&
HEIGHT=686&
BBOX=-79555169.84223041%2C-33479467.308605302%2C79555169.84223041%2C33479467.308605302'

The SLD used as parameter above, containing a filter which has the following form (truncated XML content):

          <ogc:Filter xmlns:xs="http://www.w3.org/2001/XMLSchema">
            <ogc:PropertyIsLike escapeChar="\" matchCase="false" singleChar="?" wildCard="*">
              <ogc:PropertyName>SUPPORTS_N</ogc:PropertyName>
              <ogc:Literal>*Support : Eau - Niveau : Surface (0-1m)*</ogc:Literal>
            </ogc:PropertyIsLike>
          </ogc:Filter>

Mapserver will parse the SLD and generate a temporary mapfile to produce the actual output, this mapfile will translate the previous PopertyIsLike property into the following expression:

    CLASS
      NAME "Unknown"
      EXPRESSION (("[SUPPORTS_N]" ~* ".*Support : Eau - Niveau : Surface (0-1m).*"))
      STYLE
        COLOR 0 92 230
        SIZE 6
        SYMBOL "sld_mark_symbol_circle_filled"
      END # STYLE
    END # CLASS

The wildcard character (*) is correctly escaped (-> .* ) as a REGEXP, but the parenthesis are not ((0-1m) instead of \(0-1m\)). The expected expression should have every REGEXP control characters escaped when generating the temporary mapfile.

Note: The issue was encountered on a oracle spatial datasource but adapted to a shapefile source to simplify its reproduction. In case of an Oracle datasource, another part of the temporary mapfile might be affected by a similar issue, about the FILTER directive as below:

 LAYER
    CONNECTION "aaa/bbb@oracle"
    CONNECTIONTYPE ORACLESPATIAL
    DATA "..."
    DEBUG 5
    ENCODING "UTF-8"
    EXTENT -180 -90 180 90
    FILTER (((("[SUPPORTS_NIVEAUX_PRELEVEMENT]" ~* ".*Support : Eau - Niveau : Surface (0-1m).*"))))
 [...]
rouault added a commit to rouault/mapserver that referenced this issue Sep 7, 2018
@rouault rouault closed this in b023c9c Sep 7, 2018
rouault added a commit that referenced this issue Sep 7, 2018
SLD / Filter encoding: fix escaping of REGEX special characters (fixes #5658)
@pmauduit
Copy link
Contributor Author

@pmauduit pmauduit commented Sep 7, 2018

thanks @rouault !

@pmauduit
Copy link
Contributor Author

@pmauduit pmauduit commented Jan 10, 2019

Actually, the pr #5660 introduced a regression in regards to how the regexp is built / character are escaped, I reproduced using a shapefile to simplify ; using the following layer entry:

	LAYER
            NAME "surval_30140_all_point_19_12_2017"
        TYPE POINT
	DEBUG 5
        DUMP TRUE
        STATUS OFF
        EXTENT -180 -90 180 90
        UNITS DD
        DATA surval/surval 
	    PROJECTION
                "init=epsg:4326"
            END
	    ENCODING UTF-8 
            METADATA
                wms_title "surval_30140_all_point_19_12_2017"
                wms_name "surval_30140_all_point_19_12_2017"
                wms_srs "EPSG:4326"
                wms_connectiontimeout "120"
                wms_server_version "1.1.1"
                wms_include_items "all"
                gml_include_items "all"		
	        wms_attribution_title ""
	        wms_attribution_onlineresource ""
                wms_metadataurl_format "text/xml"
                wms_metadataurl_type "TC211"
            END
        CLASS
            NAME ""
            STYLE
             	SYMBOL "circle"
		MINSIZE 5
		MAXSIZE 20
		COLOR 0 92 230
            END
	    LABEL
                COLOR 0 92 230
                POSITION auto
                SIZE small
	    END
        END
    END	

Here is the shape mentioned: surval.zip

Calling mapserver using:

/usr/lib/cgi-bin/mapserv QUERY_STRING='MAP=/root/resources/version-shapefile/mapfile.map
&SERVICE=WMS
&VERSION=1.3.0
&REQUEST=GetMap
&FORMAT=image%2Fpng
&TRANSPARENT=true
&LAYERS=surval_30140_all_point_19_12_2017
&STYLES=default
&SLD=http://172.17.0.1/mantis-44221.xml
&CRS=EPSG%3A3857&WIDTH=2036&HEIGHT=750&BBOX=-10014840.59554642%2C765984.6328891446%2C9897433.320100393%2C8098264.663286176'

The SLD passed as parameter looking like:

          <ogc:Filter xmlns:xs="http://www.w3.org/2001/XMLSchema">
            <ogc:PropertyIsLike escapeChar="\" matchCase="false" singleChar="?" wildCard="*">
              <ogc:PropertyName>SUPPORTS_N</ogc:PropertyName>
                   <ogc:Literal>*Support : Eau - Niveau : Surface-Fond ?profondeur &lt;3 m?*</ogc:Literal>
            </ogc:PropertyIsLike>
          </ogc:Filter>

Breaking in gdb in the ms_regcomp() function, before the PR being applied, analysing the memory gives the following hexdump (xxd is a specific macro I defined in my .gdbinit):

(gdb) xxd 0x55555578fab0 512
0000000: 2e2a 5375 7070 6f72 7420 3a20 4561 7520 .*Support : Eau 
0000010: 2d20 4e69 7665 6175 203a 2053 7572 6661 - Niveau : Surfa
0000020: 6365 2d46 6f6e 6420 2e70 726f 666f 6e64 ce-Fond .profond
0000030: 6575 7220 3c33 206d 2e2e 2a00 4d50 2054 eur <3 m..*.MP T

After the PR:

(gdb) xxd 0x64f740 512
0000000: 2e2a 5375 7070 6f72 7420 3a20 4561 7520 .*Support : Eau 
0000010: 5c2d 204e 6976 6561 7520 3a20 5375 7266 \- Niveau : Surf
0000020: 6163 655c 2d46 6f6e 6420 2e70 726f 666f ace\-Fond .profo
0000030: 6e64 6575 7220 5c3c 3320 6d2e 2e2a 0000 ndeur \<3 m..*..

the characters "-" and "<" are escaped. Normally in our SLD filter, we make use of parenthesis instead of the "?" char, which does not work in both version, so for now I was using '?' instead. From my understanding, I can sum up the 2 problems encountered here:

  • Is it necessary to escape both previous characters ('-' and '<') ?
  • Why is the filter not working when containing parenthesis in both cases (before and after the PR) ?

I have not been able to reopen the current issue, I can open a new one if necessary, and/or provide more information to ease reproduction of the problem.

@pmauduit
Copy link
Contributor Author

@pmauduit pmauduit commented Jan 14, 2019

ping @rouault

@rouault rouault reopened this Jan 30, 2019
rouault added a commit to rouault/mapserver that referenced this issue Jan 30, 2019
…is taken into account to avoid extent inconsistencies (found during refs MapServer#5658 investigation)
rouault added a commit to rouault/mapserver that referenced this issue Jan 30, 2019
…fixes MapServer#5658)7
rouault added a commit to rouault/mapserver that referenced this issue Jan 30, 2019
…is taken into account to avoid extent inconsistencies (found during refs MapServer#5658 investigation)
rouault added a commit to rouault/mapserver that referenced this issue Jan 30, 2019
…fixes MapServer#5658)7
rouault added a commit to rouault/mapserver that referenced this issue Jan 30, 2019
…fixes MapServer#5658)
rouault added a commit to rouault/mapserver that referenced this issue Jan 30, 2019
…is taken into account to avoid extent inconsistencies (found during refs MapServer#5658 investigation)
@rouault rouault closed this in 9e654b3 Jan 30, 2019
rouault added a commit that referenced this issue Jan 30, 2019
 SLD / Filter encoding: fix again escaping of REGEX special characters (fixes #5658)
sdlime added a commit that referenced this issue Mar 22, 2019
WMS: apply FILTER (RFC 118) vendor parameter after the CRS parameter is taken into account to avoid extent inconsistencies (found during refs #5658 investigation)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants