New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SVG output file is just a wrapper for a PNG image!? #3749

Closed
alexandervlpl opened this Issue Sep 1, 2017 · 17 comments

Comments

Projects
None yet
6 participants
@alexandervlpl
Copy link

alexandervlpl commented Sep 1, 2017

I want to generate SVG maps in Python using Mapnik 3.0.13 (built from source). I'm using version 3.0.13 of the Python bindings as well, built with PYCAIRO as described by @artemp in #3438. My code:

f = open('/tmp/test.svg', 'w')
surface = cairo.SVGSurface(f.name, map.width, map.height)
mapnik.render(map, surface)
surface.finish()

The output SVG file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="4961pt" height="3508pt" viewBox="0 0 4961 3508" version="1.1">
<defs>
<image id="image3198" width="20671" height="14617" xlink:href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAUL8A ...

Followed by ~200 MB of base64-encoded PNG garbage. 😨

Guys, I've been using your amazing library for years. Is this a joke? There's a secret undocumented way to create an actual SVG, isn't there? What is it? How do I use Cairo or whatever svg_renderer is to get something usable here?

@alexandervlpl

This comment has been minimized.

Copy link
Author

alexandervlpl commented Sep 1, 2017

Update: after recompiling my library and bindings to use the experimental svg_renderer, I'm getting the exact same result.

@talaj

This comment has been minimized.

Copy link
Member

talaj commented Sep 1, 2017

The experimental svg_renderer is something different than SVG from Cairo backend. The first one is not usable at all and no development is going there.

The Cairo SVG rendering should work fine. Rasters may cause problems. Isn't that 200 MB PNG garbage just a raster in the background? Vector only maps should look fine with reasonably sized output.

@alexandervlpl

This comment has been minimized.

Copy link
Author

alexandervlpl commented Sep 1, 2017

Thanks for the tip, switching back to Cairo.

No, this isn't a raster in the background. I'm experimenting with the default OpenStreetMap style. I'm getting a single huge PNG with all the streets, buildings, etc. The PNG is base64 encoded and embedded in the SVG. Nothing is actually rendered as vectors.

@alexandervlpl

This comment has been minimized.

Copy link
Author

alexandervlpl commented Sep 1, 2017

Here's an example output file:
https://drive.google.com/open?id=0BxCAc6zVaSK6Y1JuZUJsM0VGQjg

It's ~75MB, best not to open it in a browser.

@alexandervlpl

This comment has been minimized.

Copy link
Author

alexandervlpl commented Sep 1, 2017

The image ends up as the only item in a group (defined at the bottom of the file):

<g id="surface1">
<use xlink:href="#image8" transform="matrix(0.24,0,0,0.24,0,0)"/>
</g>
@talaj

This comment has been minimized.

Copy link
Member

talaj commented Sep 1, 2017

I just tried to render this style with

mapnik.register_fonts('...')

m = mapnik.Map(512, 512)
mapnik.load_map(m, 'style.xml')
bbox = mapnik.Envelope(-1, -1, 1, 1)
m.zoom_to_box(bbox)

surface = cairo.SVGSurface('output.svg', m.width, m.height)
mapnik.render(m, surface)
surface.finish()

And got this:
output.txt

@alexandervlpl

This comment has been minimized.

Copy link
Author

alexandervlpl commented Sep 1, 2017

Interesting, that works for me too. What doesn't work is this full OpenStreetMap stylesheet with Shapefile and PostGIS layers:
https://gist.github.com/alexandervlpl/d2cafd995f8b2c32b19a6d5579924ee3

Could it be that these Datasource types are not supported?

@alexandervlpl

This comment has been minimized.

Copy link
Author

alexandervlpl commented Sep 1, 2017

To be clear, that stylesheet was generated from the OpenStreetMap Carto project, nothing special. It works perfectly and the embedded PNG looks great.

I'll try to isolate this with a simpler sheet, perhaps with just one Shapefile or PostGIS layer.

@talaj

This comment has been minimized.

Copy link
Member

talaj commented Sep 1, 2017

The problem are compositing operations like comp-op="multiply". These lead to rasterization of the map.

@talaj

This comment has been minimized.

Copy link
Member

talaj commented Sep 1, 2017

Most interestingly PDF output does not suffer from this problem and can also handle fonts better. The output is really small (6.2 kB) compared to SVG (232 kB) for this style.

surface = cairo.PDFSurface('output.pdf', m.width, m.height)
mapnik.render(m, surface)
surface.finish()
@alexandervlpl

This comment has been minimized.

Copy link
Author

alexandervlpl commented Sep 1, 2017

Thanks for the quick investigation, @talaj !

Yes, composting operations were the problem. Unfortunately they're pretty important in some of my styles.

The PDF output is excellent though, and I can probably use it as a workaround for now. Will do some careful testing to see how comp-ops and raster layers are actually handled.

@lightmare

This comment has been minimized.

Copy link
Contributor

lightmare commented Jul 17, 2018

If I understand correctly, the issue has been explained, closing.

@tomhughes

This comment has been minimized.

Copy link
Contributor

tomhughes commented Sep 18, 2018

Has it really been explained? It doesn't seem to me that we understand why those operations cause the SVG output to be rasterised but not the the PDF output?

@hholzgra

This comment has been minimized.

Copy link

hholzgra commented Sep 18, 2018

I have seen this on my maposmatic instance, and tracked it down to styles using comp-op, too, but in all the cases I checked it always affected SVG and PDF at the same time.

For now I pre-process all styles to remove comp-op on installation, for the future I plan to keep the original version along with the pre-processed one, using the original for bitmap, and the pre-processed one for vector formats ...

@dieterdreist

This comment has been minimized.

Copy link

dieterdreist commented Sep 18, 2018

if your pdf contains a raster as well, it could depend on the pdf version.
The SVG specs seems to support in principle this kind of operation (with filters):
https://www.w3.org/TR/filter-effects/#feMergeElement
https://www.w3.org/TR/compositing-1/

@talaj

This comment has been minimized.

Copy link
Member

talaj commented Sep 18, 2018

@tomhughes Rendering code for SVG and PDF in Mapnik is exactly the same. Only difference is supplied Cairo surface, but type of surface is opaque for Mapnik.

SVG and PDF formats support different sets of compositing operations. It also depends on version of Cairo. The example comp-op="multiply" I gave should work without rasterization for both PDF and SVG with current Cairo.

@talaj

This comment has been minimized.

Copy link
Member

talaj commented Feb 8, 2019

Please see #4029.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment