Enable libxml2 XInclude functionality for style sheets

[artemp]

Leverage the XInclude functionality provided by libxml2 to enable style sheets to exist as multiple files, enhancing maintainability.

This changeset adds a new xml tag, Include, which serves as a document wrapper of included files. All tags within the Include tag are merged into the Map container in the enclosing style sheet.

Style sheets wishing to use XInclude must add

xmlns:xi ="http://www.w3.org/2001/XInclude

to the Map tag

Files can then be included using

<xi:include href="style_include.xml">

Simple examples are attached for use as test cases

[artemp]

[springmeyer] This patch looks great, thanks for the contribution. I've tested, and touched up the tests to be able to go into the main tests directory.

One question, would you be able to write a bit of documentation on the xinclude support here:

http://trac.mapnik.org/wiki/ManagingLargeXmlFiles

You could create a new page or find a place within that one. Let me know what you think.

[artemp]

[springmeyer] Also I'm thinking its minor, but I like the simple naming of "Include" more than "MapFrag". Would you be open to using "Include" instead?

[artemp]

[springmeyer] one last thing... any idea why this script does not work with xincludes?

http://trac.mapnik.org/browser/trunk/utils/upgrade_map_xml/upgrade_map_xml.py

I did add:

{{{
#!diff

Index: utils/upgrade_map_xml/upgrade_map_xml.py

--- utils/upgrade_map_xml/upgrade_map_xml.py (revision 2031)
+++ utils/upgrade_map_xml/upgrade_map_xml.py (working copy)
@@ -45,6 +45,7 @@

 xml = sys.argv[1]
 tree = objectify.parse(xml)

• tree.xinclude()
  root = tree.getroot()
  for style in root.Style:
  if len(style.Rule):
  }}}

But still get AttributeError: no such child: Style because the 'Include' is pulled in rather that just the 'Style' within it.

e.g.:
{{{dane-3:trunk dane$ upgrade_map_xml.py tests/data/good_maps/xinclude/map.xml t.xml

/usr/local/bin/upgrade_map_xml.py(49)()
-> tree.xinclude()
(Pdb) etree.tostring(tree,pretty_print=True,standalone=True)
'\n\n\n \n <xi:include href="styles.xml"/>\n <xi:include href="layers.xml"/>\n\n'
(Pdb) tree.xinclude()
(Pdb) etree.tostring(tree,pretty_print=True,standalone=True)
'\n\n\n \n\n \n <Style name="point_style">\n \n \n \n \n </Style>\n <Style name="world_borders_style">\n \n \n \n </Style>\n \n\n\n \n \n world_borders_style\n \n ../../shp/world_merc.shp\n shape\n \n \n \n point_style\n \n -100.276283619,-23.5207203374,136.782396088,45.7946830856\n ../../sqlite/qgis_spatiallite.sqlite\n geometry\n pkuid\n point\n sqlite\n 1\n spatialite\n \n \n \n\n\n'
(Pdb) print etree.tostring(tree,pretty_print=True,standalone=True)

[artemp]

[springmeyer] trying again:

{{{
dane-3:trunk dane$ upgrade_map_xml.py tests/data/good_maps/xinclude/map.xml t.xml

/usr/local/bin/upgrade_map_xml.py(49)()
-> tree.xinclude()
(Pdb) etree.tostring(tree,pretty_print=True,standalone=True)
'\n\n\n \n <xi:include href="styles.xml"/>\n <xi:include href="layers.xml"/>\n\n'
(Pdb) tree.xinclude()
(Pdb) etree.tostring(tree,pretty_print=True,standalone=True)
'\n\n\n \n\n \n <Style name="point_style">\n \n \n \n \n </Style>\n <Style name="world_borders_style">\n \n \n \n </Style>\n \n\n\n \n \n world_borders_style\n \n ../../shp/world_merc.shp\n shape\n \n \n \n point_style\n \n -100.276283619,-23.5207203374,136.782396088,45.7946830856\n ../../sqlite/qgis_spatiallite.sqlite\n geometry\n pkuid\n point\n sqlite\n 1\n spatialite\n \n \n \n\n\n'
(Pdb) print etree.tostring(tree,pretty_print=True,standalone=True)

[artemp]

[oldtopos] Replying to [comment:2 springmeyer]:

This patch looks great, thanks for the contribution. I've tested, and touched up the tests to be able to go into the main tests directory.

One question, would you be able to write a bit of documentation on the xinclude support here:

http://trac.mapnik.org/wiki/ManagingLargeXmlFiles

You could create a new page or find a place within that one. Let me know what you think.

http://trac.mapnik.org/wiki/ManagingLargeXmlFiles and
http://trac.mapnik.org/wiki/XMLConfigReference

modified to reflect addition of Include functionality. Using Include as tag name rather than MapFrag is perfectly acceptable.

[artemp]

[springmeyer] okay, great to see the docs - excellent, and that you are okay with the "Include" name.

I've turned a few more of the docs examples you wrote into tests, and have hit a problem with using both includes and entities together.

I'll attach a patch with the tests next, but basically libxml2 appears to be having trouble getting access to the entities when used in an include. Maybe I am using wrong, but at least we should give thought to having a less mysterious error.

{{{
#!sh

this one works:

$ nik2img.py tests/data/good_maps/xinclude/map.xml test.png

this one does not:

$ nik2img.py tests/data/good_maps/xinclude/map_with_entities_and_includes.xml test.png
Traceback (most recent call last):
File "/usr/local/bin/nik2img.py", line 233, in
main()
File "/usr/local/bin/nik2img.py", line 223, in main
nik_map = ComposeDebug(mapfile,**options.dict)
File "/Library/Python/2.6/site-packages/mapnik_utils/composer.py", line 248, in init
Compose.init(self,mapfile,**kwargs)
File "/Library/Python/2.6/site-packages/mapnik_utils/composer.py", line 67, in init
self.build()
File "/Library/Python/2.6/site-packages/mapnik_utils/composer.py", line 265, in build
builder = super(ComposeDebug,self).build()
File "/Library/Python/2.6/site-packages/mapnik_utils/composer.py", line 111, in build
self.map = builder.build_map(self.width,self.height)
File "/Library/Python/2.6/site-packages/mapnik_utils/loader.py", line 105, in build_map
self.load_mapfile(m)
File "/Library/Python/2.6/site-packages/mapnik_utils/loader.py", line 97, in load_mapfile
load(m)
File "/Library/Python/2.6/site-packages/mapnik_utils/loader.py", line 67, in load_xml
return mapnik.load_map(m,self.mapfile)
RuntimeError: conversion of type "Pc" to data failed
}}}

[artemp]

[springmeyer] please apply and test with that last attachment.

I note that if I take away

{{{
minzoom="&zoom02min;" maxzoom="&zoom00max;"
}}}

the xml can be read

I'm running on osx with the stock version of libxml2:

{{{
$ otool -L /usr/local/lib/libmapnik2.dylib
/usr/local/lib/libmapnik2.dylib:
libmapnik2.dylib (compatibility version 0.8.0, current version 0.8.0)
/Library/Frameworks/FreeType.framework/Versions/2.3/FreeType (compatibility version 11.0.0, current version 11.0.0)
/usr/lib/libltdl.7.dylib (compatibility version 9.0.0, current version 9.2.0)
/usr/X11/lib/libpng12.0.dylib (compatibility version 42.0.0, current version 42.0.0)
/usr/local/Cellar/libtiff/3.9.2/lib/libtiff.3.dylib (compatibility version 13.0.0, current version 13.2.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.3)
/usr/local/Cellar/jpeg/8a/lib/libjpeg.8.dylib (compatibility version 9.0.0, current version 9.1.0)
/usr/local/Cellar/proj/4.7.0/lib/libproj.0.dylib (compatibility version 7.0.0, current version 7.6.0)
/usr/local/Cellar/icu4c/4.3.1/lib/libicuuc.43.1.dylib (compatibility version 43.0.0, current version 43.1.0)
libboost_filesystem.dylib (compatibility version 0.0.0, current version 0.0.0)
libboost_regex.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.3.0)
libboost_thread.dylib (compatibility version 0.0.0, current version 0.0.0)
libboost_system.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.0)
}}}

[artemp]

[springmeyer] int iXIncludeReturn = xmlXIncludeProcessFlags( doc, options_ ); fixed the above problem, applied in r2112.

Thanks oldtopos!

[artemp]

[springmeyer] and tests added in r2113