Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WFS request with chinese keywords does not have any result #5215

Merged
merged 1 commit into from Jan 6, 2016
Merged

WFS request with chinese keywords does not have any result #5215

merged 1 commit into from Jan 6, 2016

Conversation

tbonfort
Copy link
Member

@tbonfort tbonfort commented Jan 5, 2016

My WFS request is the following code:

          <?xml version='1.0' encoding='UTF-8'?>
            <wfs:GetFeature  service='WFS' version='1.0.0' 
            xmlns:wfs='http://www.opengis.net/wfs' 
            xmlns:gml='http://www.opengis.net/gml' 
            xmlns:ogc='http://www.opengis.net/ogc' 
           xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' 
            xsi:schemaLocation='http://www.opengis.net/wfs http://schemas.opengis.net/wfs/1.0.0/wfs.xsd'>           
 <wfs:Query typeName='district' srsName='EPSG:3830'>
            <ogc:Filter>
            <ogc:PropertyIsEqualTo>
            <ogc:PropertyName>TField</ogc:PropertyName>
            <ogc:Literal>高新区</ogc:Literal>
            </ogc:PropertyIsEqualTo>
           </ogc:Filter>
           </wfs:Query>
           </wfs:GetFeature>

I want to search the feature that the value of field named TField is 高新区. I set the the layer's encoding to GBK, ant the above request returns the message that No matching record(s) found. however, if I set the keyword to english words or number, then I can get the result. The code above works well on Geoserver WFS, but can't got result from mapserver' WFS. Additional, "高新区" is Chinese words.

@tbonfort
Copy link
Member

tbonfort commented Jan 4, 2016

Please give more info:

  • What is your mapserver version
  • What is the encoding/type of your underlaying datasource
  • How are you setting the layer encoding in the mapfile

@bruce6135
Copy link
Author

bruce6135 commented Jan 4, 2016

Thanks for your atention. My mapserver version is 7.0.0, which is build in ms4w 3.1.0. My datasource is ESRI Shapefile format, and I modified the data source's encode to GBK. In my mapfile, I set the Map and Layer's encoding to GBK.

@bruce6135
Copy link
Author

bruce6135 commented Jan 5, 2016

In order to help you understand my question, I copied the log message from mapserver's log file.
Here is a wfs request that got the result features:

[Tue Jan 05 10:52:43 2016].848000 CGI Request 1 on process 3464
[Tue Jan 05 10:52:43 2016].848000 msWFSParseRequest(): WFS post request: <wfs:GetFeature service='WFS' version='1.0.0' xmlns:wfs='http://www.opengis.net/wfs' xmlns:gml='http://www.opengis.net/gml' xmlns:ogc='http://www.opengis.net/ogc' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:schemaLocation='http://www.opengis.net/wfs http://schemas.opengis.net/wfs/1.0.0/wfs.xsd'><wfs:Query typeName='district' srsName='EPSG:3830'>ogc:Filterogc:PropertyIsEqualToogc:PropertyNameTField/ogc:PropertyNameogc:Literal440402/ogc:Literal/ogc:PropertyIsEqualTo/ogc:Filter/wfs:Query/wfs:GetFeature
[Tue Jan 05 10:52:43 2016].851000 FLTLayerApplyPlainFilterToLayer(): ([TField] =440402)
[Tue Jan 05 10:52:43 2016].865000 mapserv request processing time (msLoadMap not incl.): 0.017s
[Tue Jan 05 10:52:43 2016].865000 msFreeMap(): freeing map at 05C1A5B0.

And here is a wfs request that can not got the result features:

[Tue Jan 05 10:19:25 2016].525000 CGI Request 1 on process 3220
[Tue Jan 05 10:19:25 2016].525000 msWFSParseRequest(): WFS post request: <wfs:GetFeature service='WFS' version='1.0.0' xmlns:wfs='http://www.opengis.net/wfs' xmlns:gml='http://www.opengis.net/gml' xmlns:ogc='http://www.opengis.net/ogc' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:schemaLocation='http://www.opengis.net/wfs http://schemas.opengis.net/wfs/1.0.0/wfs.xsd'><wfs:Query typeName='district' srsName='EPSG:3830'>ogc:Filterogc:PropertyIsEqualToogc:PropertyNameTField/ogc:PropertyNameogc:Literal高新区/ogc:Literal/ogc:PropertyIsEqualTo/ogc:Filter/wfs:Query/wfs:GetFeature
[Tue Jan 05 10:19:25 2016].526000 FLTLayerApplyPlainFilterToLayer(): ("[TField]" ="高新区")
[Tue Jan 05 10:19:25 2016].527000 msQueryByFilter(): Search returned no results. No matching record(s) found.
[Tue Jan 05 10:19:25 2016].527000 mapserv request processing time (msLoadMap not incl.): 0.002s
[Tue Jan 05 10:19:25 2016].527000 msFreeMap(): freeing map at 05B1A5B0.

I don't know how to resolve this problem, can anybody tell me how to do?

@tbonfort
Copy link
Member

tbonfort commented Jan 5, 2016

Can you try the attached patch? It would be nice if you could provide a sample shapefile so we can include this check in our test suite

@bruce6135
Copy link
Author

bruce6135 commented Jan 5, 2016

I can not attach a patch in the comment, so I upload the file into oneDrive.
The link is :
https://onedrive.live.com/redir?resid=660C89D3266533C6!109&authkey=!AIAeACYiK62s0rY&ithint=file%2czip
https://onedrive.live.com/redir?resid=660C89D3266533C6!110&authkey=!AOoFQdF6ItRjpv4&ithint=file%2cmap
My data's EPSG is 4326.
When I enter the URL like this:

http://localhost:88/cgi-bin/mapserv.exe?map=D:/ms4w/Apache/htdocs/map/query.map&SERVICE=WFS&VERSION=1.0.0&REQUEST=getFeature&TYPENAME=city&Filter=<Filter><PropertyIsEqualTo><PropertyName>NAME</PropertyName><Literal>塔河县</Literal></PropertyIsEqualTo></Filter>

then show the following:

<gml:boundedBy>
<gml:null>missing</gml:null>
</gml:boundedBy>

The information in the log file is :
[Tue Jan 05 23:07:33 2016].399000 CGI Request 1 on process 4908
[Tue Jan 05 23:07:33 2016].400000 FLTLayerApplyPlainFilterToLayer(): ("[NAME]" ="塔河县")
[Tue Jan 05 23:07:33 2016].438000 msQueryByFilter(): Search returned no results. No matching record(s) found.
[Tue Jan 05 23:07:33 2016].438000 mapserv request processing time (msLoadMap not incl.): 0.039s
[Tue Jan 05 23:07:33 2016].438000 msFreeMap(): freeing map at 05BAA990.

tbonfort added a commit to MapServer/msautotest_DEPRECATED that referenced this pull request Jan 6, 2016
@tbonfort tbonfort merged commit 100e6af into MapServer:branch-7-0 Jan 6, 2016
@tbonfort
Copy link
Member

tbonfort commented Jan 6, 2016

Pushed the fix to branch-7-0, along with an autotest in MapServer/msautotest_DEPRECATED@b52c73c . Without the current fix, the autotest result is

<gml:boundedBy>
<gml:null>missing</gml:null>
</gml:boundedBy>

instead of the correct feature.

Thanks for the testcase @bruce6135 , you can delete the data now, I was able to create the autotest with an exisiting dataset

@bruce6135
Copy link
Author

bruce6135 commented Jan 6, 2016

Congratulations for you fixed this problem, and thanks for you help. I will try to compile in my computer to fix this problem.

@bruce6135
Copy link
Author

bruce6135 commented Jan 8, 2016

Sombody helped me to find another method. In QGIS, saving the layer as another layer and set the new layer's code to UTF-8.The new layer's property sheet is now in UTF-8 code and shows correctly. Then I can query chinese words with WFS service. The disadvantage of this method is that UTF-8 is not contain all the chinese words, by save as the layer, some chinese words will be garbled and must be reenter with the correct content. However, those garbled words usually is not often used and account for a small proportion of chinese words.

@tbonfort
Copy link
Member

tbonfort commented Jan 8, 2016

I doubt this has something to do with UTF-8 not supporting those glyphs. You are probably giving qgis an incorrect version of gbk (with respect to the version actually used by your dataset) when asking it to convert it to utf8.
ps: always use utf-8 and you'll never have to deal with encoding issues again :)

@tbonfort tbonfort deleted the issues/5215-encoding-before-filter branch Jun 3, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants