Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor markers_placement_finder #3338

Merged
merged 1 commit into from Mar 2, 2016

Conversation

lightmare
Copy link
Contributor

Replace member variant of placement-type implementations with plain
union. The active implementation is chosen at construction time like
before.

Make placement-type implementation classes virtual to allow invoking
the active union member through a base class pointer.

clang 3.6

this
  43.61s  mem:  774MB [0 page faults]     src/renderer_common/render_markers_symbolizer.cpp
master
  64.61s  mem: 1325MB [0 page faults]     src/renderer_common/render_markers_symbolizer.cpp

gcc 4.8

this
  74.46s  mem: 1740MB [0 page faults]     src/renderer_common/render_markers_symbolizer.cpp
master
  85.71s  mem: 3459MB [6 page faults]     src/renderer_common/render_markers_symbolizer.cpp

- refs mapnik#3327

Replace member variant of placement-type implementations with plain
union. The active implementation is chosen at construction time like
before.

Make placement-type implementation classes virtual to allow invoking
the active union member through a base class pointer.
@tomhughes
Copy link
Contributor

I've kicked off a scratch build on Fedora of 3.0.10 with this patch applied: http://koji.fedoraproject.org/koji/taskinfo?taskID=13194331

@tomhughes
Copy link
Contributor

Looks good - all three platforms built successfully on Fedora.

@artemp
Copy link
Member

artemp commented Mar 2, 2016

Hmm.. but travis is still failing https://travis-ci.org/mapnik/mapnik/jobs/113005661.

@tomhughes - have you tried latest master on Fedora? I have a feeling it might pass as well, curious to find out.
/cc @lightmare @springmeyer

@artemp
Copy link
Member

artemp commented Mar 2, 2016

Just gathering some stats so we can make an intelligent decisions

If travis is to believe (big ?) current master got much further with the compilation

/cc @lightmare @tomhughes @springmeyer

refactor markers_placement_finder PR

Command execution time: test/unit/datasource/shapeindex.o: 45.599106 seconds
clang++ -o test/unit/geometry/geometry_envelope_test.o -c -std=c++11 -stdlib=libc++ -fvisibility=hidden -fvisibility-inlines-hidden -Wall -ftemplate-depth-300 -Wsign-compare -Wshadow -Wno-unsequenced -O3 -DMAPNIK_MEMORY_MAPPED_FILE -DMAPNIK_HAS_DLCFN -DBIGINT -DBOOST_REGEX_HAS_ICU -DHAVE_JPEG -DMAPNIK_USE_PROJ4 -DHAVE_PNG -DHAVE_WEBP -DHAVE_TIFF -DDARWIN -DMAPNIK_THREADSAFE -DBOOST_SPIRIT_NO_PREDEFINED_TERMINALS=1 -DBOOST_PHOENIX_NO_PREDEFINED_TERMINALS=1 -DBOOST_SPIRIT_USE_PHOENIX_V3=1 -DNDEBUG -DMAPNIK_MEMORY_MAPPED_FILE -DMAPNIK_HAS_DLCFN -DBIGINT -DBOOST_REGEX_HAS_ICU -DHAVE_JPEG -DMAPNIK_USE_PROJ4 -DHAVE_PNG -DHAVE_WEBP -DHAVE_TIFF -DDARWIN -DMAPNIK_THREADSAFE -DBOOST_SPIRIT_NO_PREDEFINED_TERMINALS=1 -DBOOST_PHOENIX_NO_PREDEFINED_TERMINALS=1 -DBOOST_SPIRIT_USE_PHOENIX_V3=1 -DNDEBUG -DHAVE_CAIRO -DGRID_RENDERER -DSVG_RENDERER -DHAVE_CAIRO -Itest -Imason_packages/.link/include/cairo -Imason_packages/.link/include/pixman-1 -Ideps -Ideps/agg/include -Iinclude -Imason_packages/.link/include -Imason_packages/.link/include/freetype2 -Imason_packages/.link/include/libpng16 test/unit/geometry/geometry_envelope_test.cpp
The job exceeded the maxmimum time limit for jobs, and has been terminated.

current master

* Running visual tests...
..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
The job exceeded the maxmimum time limit for jobs, and has been terminated.

@tomhughes
Copy link
Contributor

Travis is only failing because the compile took too long right? I don't know what the timeout is in travis but i386 and x86_64 each took about 1h20 on the Fedora build farm, and arm took just under 4 hours.

When you say "try master" do you mean with this patch? or without it?

@artemp
Copy link
Member

artemp commented Mar 2, 2016

@tomhughes - just realised that refactor markers_placement_finder is already rebased on latest master so no need to try, sry for the noise:).

If memory requirements are dropped this a very positive sign, thanks @lightmare. But I believe we can(must) also squash compile times to fit travis imposed 50 min limit.

@artemp artemp merged commit 66e7ef5 into mapnik:master Mar 2, 2016
@artemp
Copy link
Member

artemp commented Mar 2, 2016

Giving a travis another chance ;)

@tomhughes
Copy link
Contributor

One thing about the Fedora builds is that I deliberately don't pass -j when building so the build is not parallel at all - that was done to fix previous problems running out of memory

I think I'm right that mapnik now avoids building problem files in parallel though, so I can likely try adding that back in?

@artemp
Copy link
Member

artemp commented Mar 2, 2016

libmapnik.dylib size + compile speeds on OS X with clang++

master before #3338

ls -l /opt/mapnik/lib/libmapnik.dylib 
-rwxr-xr-x  1 root  wheel  15778820  2 Mar 13:32 /opt/mapnik/lib/libmapnik.dylib
Total build time: 519.722278 seconds
Total SConscript file execution time: 0.779022 seconds
Total SCons execution time: 5.986600 seconds
Total command execution time: 512.956656 seconds

real    8m41.786s
user    59m55.261s
sys 1m33.937s

master after #3338

ls -l /opt/mapnik/lib/libmapnik.dylib 
-rwxr-xr-x  1 root  wheel  17258300  2 Mar 12:44 /opt/mapnik/lib/libmapnik.dylib
Total build time: 500.249299 seconds
Total SConscript file execution time: 0.707408 seconds
Total SCons execution time: 4.925966 seconds
Total command execution time: 494.615925 seconds

real    8m20.571s
user    58m16.777s
sys 1m31.321s

/cc @springmeyer @lightmare @tomhughes

@lightmare
Copy link
Contributor Author

+1.5MB in lib size? That's another surprise 😕 The difference the is even bigger on Linux: 17.2 => 18.9

@lightmare
Copy link
Contributor Author

@artemp - regarding travis build times, I think they vary wildly, perhaps depending on overall load. My favourite build is 5006. There were a few green builds before it, but none after. And it coincides with the time we started fixing noexcept specifiers in variant... 💡

As for this commit increasing the library size by 10%, I don't get it. I removed the whole variant move-construction and apply_visitor machinery, and replaced it with 6 virtual function tables, that's all. Where does it come from?

@artemp
Copy link
Member

artemp commented Mar 3, 2016

@lightmare - it's hard to be 100% sure, but v-tables means compiler can't do certain optimisations/inlining. The binary size increase is probably a trickle-down side-effect of this change.
This one of the reasons we don't usually use virtual inheritance in mapnik :)
Lets keep this change in master as it helps with linux packaging which is important improvement but I'm keeping my eyes open as old approach results in tighter binaries - important too!

lightmare added a commit to lightmare/mapnik that referenced this pull request Mar 5, 2016
- move Locator/Detector-dependent stuff back to markers_point_placement
- slightly reduces library size, by about 20% of what mapnik#3338 added
artemp pushed a commit that referenced this pull request Mar 8, 2016
- move Locator/Detector-dependent stuff back to markers_point_placement
- slightly reduces library size, by about 20% of what #3338 added
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants