Skip to content
This repository

optimize expression parsing #1167

Open
springmeyer opened this Issue · 8 comments

3 participants

Dane Springmeyer sehe Artem Pavlenko
Dane Springmeyer
Owner

OSM.org Mapnik stylesheets classically have verbose rules matching specific types of features, e.g. <Filter>([type] = 'golf_course')</Filter> repeated numerous times with slight variations.

Profiling a large osm stylesheet shows that a significant amount of map loading time is taken in mapnik::parse_expression, so any speedups in expression parsing will benefit loading times greatly:

screenshot 2014-04-29 09 02 43

Relevant grammars:

Dane Springmeyer
Owner

fyi @artemp the above ^^ docs have full weight of the spirit guys: http://boost-spirit.com/home/2011/07/23/how-to-optimize-qi/

sehe
sehe commented

Do you still need help optimizing this? What has already been optimized? (Where can I start :)?)

sehe
sehe commented

@artemp Okay. I've looked at the benchmark in test_expression_parse. It looks like the runtime is completely dominated by dynamic memory allocations. In fact, it doesn't even matter whether the grammar is reused or re-instantiated ("soft-compiled" from the expression template).

I'm not sure it's worth looking at the parser code as the first line to optimizing. Of course, some changes in the parser might reduce the number of dyn-allocations. I'll have a look at the classes you linked

(I was just happy to get the benchmarks to link, so I haven't given the actual grammar a second's look)


OT

I've manually compiled the benchmark by

  • setting the optimization mode to -O2 or higher (otherwise there is breakage because methods aren't inlined and also not exported from the library)
  • likewise, hard-adding ../src/params.cpp as a TU
  • somehow the params weren't being parsed so I hardcoded them :) (call me cheap)

In fact I've stooped to using a Makefile, here it is:

all: test_expression_parse

%: %.cpp ../src/params.cpp
        g++ -O3 -march=native -flto -std=c++11 -I ../include/ $^ -o $@ -L ../src/ -lmapnik -lboost_system

Shouldn't scons -j4 or scons benchmark just have built those executables for me?

Dane Springmeyer
Owner

@sehe thank you for helping!! To get the test_expression_parse to be built via scons in benchmark/out/test_expression_parse do: scons BENCHMARK=True

sehe
sehe commented

Is there a realistic payload? This issue opened with the observation

Profiling a large osm stylesheet shows that a significant amount of map loading time is taken in mapnik::parse_expression, so any speedups in expression parsing will benefit loading times greatly

It seems to me to be most fruitful to profile that specific task, rather than focusing on micro-optimizing the grammar. I've thought a bit about things and I have an intuition that the overhead might actually be lurking elsewhere. Still haven't found the time to read the grammar, though. I'll do so when I return to this "todo list" item :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.