Make the output of GeoFeatureInfo exportable

[GoogleCodeExporter]

GetFeatureInfo responses are now displayed in a widget, the 
GetFeatureInfoPanel. In some cases users would like the raw data of the GFI 
response, basically an export. This issue should make it possible for users to 
export/download the GFI response data.

Original issue reported on code.google.com by jus...@gmail.com on 2 Feb 2012 at 7:56

[GoogleCodeExporter]

Original comment by milovand...@gmail.com on 14 Aug 2012 at 10:59

• Added labels: Milestone-Release0.7

[GoogleCodeExporter]

Feedback needed: working on a client-only solution using browser "Data URI 
scheme"
http://en.wikipedia.org/wiki/Data_URI_scheme as specified in 
http://tools.ietf.org/html/rfc2397. This allows any data generated within the 
browser to be linked as if it was an external file using the URL format 
"data:[<MIME-type>][;charset=<encoding>][;base64],<data>". When encoding a GFI 
response as CSV (or even Excel) within the browser, we can add an "export" link 
to the GFI Panel and it will open e.g. Excel (or LibreOffice in my case) with 
the GFI data. Unfortunately this does not work in IE. There are hacks using 
Flash SWF Objects with IE but this is not clean either.

For now the GFI data will be opened in IE as text data. You can try a demo on 
any of the Heron examples. e.g.
http://lib.heron-mc.org/heron/latest/examples/default/index-nl.html
- select Themalaag e.g. "RD Stations", 
- enable GFI with I button
- Click on map 
- click the "Exporteer" link in the bottom toolbar of the GFI Panel. 

In FF, Chrome it may open your configured app for .csv in IE a new window 
should popup.

Let me know if this direction is ok. The cleanest cross-browser solution would 
be with a server-side script that returns a real file. In the future HTML local 
storage may be used but this is not yet common.

Original comment by jus...@gmail.com on 20 Aug 2012 at 3:55

[GoogleCodeExporter]

Original comment by jus...@gmail.com on 20 Aug 2012 at 3:55

• Changed state: Started

[GoogleCodeExporter]

It works but as mentioned IE behaviour is clumsy. For the user it is a bit 
difficult to copy/paste with keeping the format.

Maybe we can optimise this solution a bit by adding an option (in the config) 
to either post the (same) data to a server-side script or send it, with the 
known limitations, directly as shown in the example.

It would also be nice if the location of the button is configurable. There 
isn't that much space on the screen so an additional button-bar should be 
avoided if possible.

Original comment by rvob...@gmail.com on 21 Aug 2012 at 7:54

[GoogleCodeExporter]

There is now  a server-side cgi python script that will echo the client posted 
content with an indicated mime-type. By sending "Content-Disposition: 
attachment; filename=layer_name.csv|xlsx" a download is triggered on the 
client. Later we may do more advanced stuff like generating an ESRI Shapefile 
as in issue #79. 

All examples should now have two links in the bottom: [Export CSV] and [Export 
XLS]. For the latter I saw some problems with Excel (Starter), but this may be 
due to a wrong install where the Excel XML format was disabled.
Can you test the Excel download ? 

Next is best placement/configuration of the buttons etc but wanted to test the 
server option first.

Original comment by jus...@gmail.com on 27 Aug 2012 at 8:56

[GoogleCodeExporter]

Strange: when the extension of the exported file is .xlsx Excel refuses to load 
the file (LIbre Office loads ok) but when using ".xls the file loads ok in 
Excel (Starter Win7) though with some warnings.

Original comment by jus...@gmail.com on 27 Aug 2012 at 10:18

[GoogleCodeExporter]

Improved: 
- Excel export as .xls (now Excel accepts the Excel 2003 XML Workbook file)
- true file download via simple server-side CGI script (also for later services 
like ESRI Shapefile export)
- single menu in toolbar for display options (Grid, XML, Tree) and export 
options (CSV, Excel)
- both display options and export options are configurable in the GFIPanel 
config

e.g.
{{
                {
                    xtype: 'hr_featureinfopanel',
                    id: 'hr-feature-info',
                    region: "south",
                    border: true,
                    collapsible: true,
                    collapsed: true,
                    height: 205,
                    split: true,
                    // Option values are 'Grid', 'Tree' and 'XML', default is 'Grid' (results in no tabs)
                    displayPanels: ['Grid', 'XML'],
                    // Export to download file. Option values are 'CSV', 'Excel', default is no export.
                    exportFormats: ['CSV', 'Excel'],
                    maxFeatures: 10
                }
}}

Please test. Any of the examples will do, e.g.
http://lib.heron-mc.org/heron/latest/examples/default
- enable "cities" layer
- press I button
- click features
etc

Original comment by jus...@gmail.com on 28 Aug 2012 at 8:32

[GoogleCodeExporter]

Testing the above url gives an empty FI pane when displayed in Grid.  Setting 
display to xml wil show the right data. Export is not working and gives message 
"No features available or none-grid display chosen" even when in grid mode.

Original comment by rvob...@gmail.com on 28 Aug 2012 at 11:17

[GoogleCodeExporter]

Strange, I have tested using IE8 on Windows 7 with MS Excel Starter and found 
no problems.
Can you give me more details of your environment (system en browser versions) ? 
A lot of resources have been adapted so emptying the browser cache may help. 
Have you also first enabled the "cities" layer? Export will detect if there are 
no features to export or not within the grid panel.

Original comment by jus...@gmail.com on 28 Aug 2012 at 2:52

[GoogleCodeExporter]

Hmm, I now see similar errors in some cases (FF locally and IE on Kadaster, but 
never in Chrome), but always for the URL 
http://lib.heron-mc.org/heron/latest/examples/default, the other example URLs 
work ok, like 
http://lib.heron-mc.org/heron/latest/examples/default/index-nl.html

It looks like it is more an issue with the "cities" layer GFI response not 
handled well by OpenLayers in some cases. During debugging I see that the GFI 
response text comes back but the "features" array that we should get from OL is 
empty. So OL was unable to parse. I have seen earlier issues with the WMS 
server of the cities layer (hosted by FAO in Rome). There seem to be subtle 
changes.

With the format/export menus in the toolbar we have more screen real-estate. We 
could even put in the tabs as group-buttons in that toolbar as well...Let me 
know.

Original comment by jus...@gmail.com on 28 Aug 2012 at 4:23

[GoogleCodeExporter]

Within OL there is a parser error when parsing the GFI response for cities 
layer:
<parsererror>
XML Parsing Error: not well-formed Location:  Line Number 1, Column 851:

<sourcetext><?xml version="1.0" encoding="UTF-8"?><wfs:FeatureCollection 
xmlns="http://www.opengis.net/wfs" xmlns:wfs="http://www.opengis.net/wfs" 
xmlns:GEONETWORK="http://www.fao.org/geonetwork" 
xmlns:gml="http://www.opengis.net/gml" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://www.fao.org/geonetwork 
http://data.fao.org:80/geoserver/wfs?service=WFS&version=1.0.0&request=DescribeF
eatureType&typeName=GEONETWORK%3A12764_esri_cities http://www.opengis.net/wfs 
http://data.fao.org:80/geoserver/schemas/wfs/1.0.0/WFS-basic.xsd"><gml:boundedBy
><gml:Box 
srsName="http://www.opengis.net/gml/srs/epsg.xml#4326"><gml:coordinates 
xmlns:gml="http://www.opengis.net/gml" decimal="." cs="," ts=" 
">-68.28333636,-38.94490909 
-68.28333636,-38.94490909</gml:coordinates></gml:Box></gml:boundedBy><gml:featur
eMember><GEONETWORK:12764_esri_cities 
fid="12764_esri_cities.34"><gml:boundedBy><gml:Box 
srsName="http://www.opengis.net/gml/srs/epsg.xml#4326"><gml:coordinates 
xmlns:gml="http://www.opengis.net/gml" decimal="." cs="," ts=" 
">-68.28333636,-38.94490909 
-68.28333636,-38.94490909</gml:coordinates></gml:Box></gml:boundedBy><GEONETWORK
:the_geom><gml:Point 
srsName="http://www.opengis.net/gml/srs/epsg.xml#4326"><gml:coordinates 
xmlns:gml="http://www.opengis.net/gml" decimal="." cs="," ts=" 
">-68.28333636,-38.94490909</gml:coordinates></gml:Point></GEONETWORK:the_geom><
GEONETWORK:ObjectID>262148</GEONETWORK:ObjectID><GEONETWORK:CITY_NAME>Neuquen</G
EONETWORK:CITY_NAME><GEONETWORK:GMI_ADMIN>ARG-NEU</GEONETWORK:GMI_ADMIN><GEONETW
ORK:ADMIN_NAME>Neuquen</GEONETWORK:ADMIN_NAME><GEONETWORK:FIPS_CNTRY>AR</GEONETW
ORK:FIPS_CNTRY><GEONETWORK:CNTRY_NAME>Argentina</GEONETWORK:CNTRY_NAME><GEONETWO
RK:STATUS>Provincial 
capital</GEONETWORK:STATUS><GEONETWORK:POP_RANK>6</GEONETWORK:POP_RANK><GEONETWO
RK:POP_CLASS>50,000 to 
100,000</GEONETWORK:POP_CLASS><GEONETWORK:PORT_ID>0</GEONETWORK:PORT_ID><GEONETW
ORK:LABEL_FLAG>1</GEONETWORK:LABEL_FLAG></GEONETWORK:12764_esri_cities></gml:fea
tureMember></wfs:FeatureCollection> 

Original comment by jus...@gmail.com on 28 Aug 2012 at 6:39

[GoogleCodeExporter]

The problem (column 851) is in the element <GEONETWORK:12764_esri_cities>. XML 
Elements are not allowed to start with a number, see e.g. 
http://stackoverflow.com/questions/2087108/encoding-xml-element-name-beginning-w
ith-a-number

Some parsers are more forgiving....

Original comment by jus...@gmail.com on 29 Aug 2012 at 8:13

[GoogleCodeExporter]

Indeed the index-nl example works fine. The default example fails in FF and IE 
on multiple systems.

Also I did some additional testing on my private server. The grid display and 
export on all layers gives the expected result. 

Original comment by rvob...@gmail.com on 29 Aug 2012 at 8:52

[GoogleCodeExporter]

We have taken Heron as it is today (30 aug) in (semi)production. The export to 
Excel is on IE7 extremely slow. IE7' processor usage goes sky-high. 
On IE8 it works as expected. Since IE7 is obsolete now I don't think we need to 
spend more time on this. Once we have migrated to IE8 we will re-eneble the 
export to Excel.

Original comment by rvob...@gmail.com on 30 Aug 2012 at 9:00

[GoogleCodeExporter]

@rvobict : is the functionality for this issue ok with you ? Then we can close 
this issue. The issue with the 'cities' layer is a problem with the ESRI layer 
naming of the FAO Geoserver. There is nothing we can do about this, except  to 
use another public WMS server.

Original comment by jus...@gmail.com on 3 Sep 2012 at 2:03

[GoogleCodeExporter]

Yes, you can close this issue. Once we dig deeper into the excel export we 
might create a new issue at a later date. For now the added functionality on 
the FI panel is highly appreciated by our users. 

Original comment by rvob...@gmail.com on 3 Sep 2012 at 6:15

[GoogleCodeExporter]

Fixed, so can be closed

Original comment by jus...@gmail.com on 4 Sep 2012 at 8:31

• Changed state: Fixed