Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

-flto breaks grid rendering on OS X #2232

Closed
springmeyer opened this issue May 1, 2014 · 5 comments
Closed

-flto breaks grid rendering on OS X #2232

springmeyer opened this issue May 1, 2014 · 5 comments

Comments

@springmeyer
Copy link
Member

Adding -flto to LDFLAGS and CXXFLAGS in master works with the exception that grid rendering breaks.

$ clang++ -v
Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin13.1.0
Thread model: posix

~/projects/mapnik/master[fast-static-grammars]$ make test
./run_tests
*** Running visual tests...
........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Visual rendering: 22 failed / 1058 passed
1) 99999999 different pixels:
    /tmp/mapnik-visual-images/map-background-image-compositing-512-512-1.0-grid.json (actual)
    tests/visual_tests/grids/map-background-image-compositing-512-512-1.0-grid-reference.json (expected)
2) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-on-hex-grid-600-400-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-on-hex-grid-600-400-1.0-grid-reference.json (expected)
3) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-on-hex-grid-400-600-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-on-hex-grid-400-600-1.0-grid-reference.json (expected)
4) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-with-background-image-600-400-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-with-background-image-600-400-1.0-grid-reference.json (expected)
5) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-with-background-image-400-600-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-with-background-image-400-600-1.0-grid-reference.json (expected)
6) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-with-background-image-257-256-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-with-background-image-257-256-1.0-grid-reference.json (expected)
7) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-svg-opacity-500-100-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-svg-opacity-500-100-1.0-grid-reference.json (expected)
8) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-on-line-and-line-placement-600-400-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-on-line-and-line-placement-600-400-1.0-grid-reference.json (expected)
9) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-path-expression-500-100-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-path-expression-500-100-1.0-grid-reference.json (expected)
10) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-multi-policy-600-400-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-multi-policy-600-400-1.0-grid-reference.json (expected)
11) 99999999 different pixels:
    /tmp/mapnik-visual-images/whole-centroid-600-400-1.0-grid.json (actual)
    tests/visual_tests/grids/whole-centroid-600-400-1.0-grid-reference.json (expected)
12) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-svg-opacity2-500-100-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-svg-opacity2-500-100-1.0-grid-reference.json (expected)
13) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-on-line-spacing-eq-width-600-400-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-on-line-spacing-eq-width-600-400-1.0-grid-reference.json (expected)
14) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-on-line-600-400-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-on-line-600-400-1.0-grid-reference.json (expected)
15) 99999999 different pixels:
    /tmp/mapnik-visual-images/text-bug1533-600-600-1.0-grid.json (actual)
    tests/visual_tests/grids/text-bug1533-600-600-1.0-grid-reference.json (expected)
16) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-with-background-image-and-hsla-transform-600-400-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-with-background-image-and-hsla-transform-600-400-1.0-grid-reference.json (expected)
17) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-with-background-image-and-hsla-transform-400-600-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-with-background-image-and-hsla-transform-400-600-1.0-grid-reference.json (expected)
18) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-with-background-image-and-hsla-transform-257-256-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-with-background-image-and-hsla-transform-257-256-1.0-grid-reference.json (expected)
19) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-on-line-spacing-eq-width-overlap-600-400-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-on-line-spacing-eq-width-overlap-600-400-1.0-grid-reference.json (expected)
20) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-vs-point-512-512-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-vs-point-512-512-1.0-grid-reference.json (expected)
21) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker_line_placement_on_points-500-100-1.0-grid.json (actual)
    tests/visual_tests/grids/marker_line_placement_on_points-500-100-1.0-grid-reference.json (expected)
22) 99999999 different pixels:
    /tmp/mapnik-visual-images/marker-svg-empty-g-element-500-100-1.0-grid.json (actual)
    tests/visual_tests/grids/marker-svg-empty-g-element-500-100-1.0-grid-reference.json (expected)
*** Running C++ tests...
..............

*** Running python tests...
Notice: skipping postgis tests (connection)
................................................................................................................................F.........................................................................................................................................................................................................................................FF.................................................................
======================================================================
FAIL: python_tests.load_map_test.test_broken_files
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/python-libs/nose-1.3.0-py2.7.egg/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/dane/projects/mapnik/master/tests/python_tests/load_map_test.py", line 37, in test_broken_files
    eq_(len(failures),0,'\n'+'\n'.join(failures))
AssertionError: 
Loading broken map (../data/broken_maps/css_value_broken.xml) did not raise RuntimeError!

======================================================================
FAIL: test render_grid method
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/python-libs/nose-1.3.0-py2.7.egg/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/dane/projects/mapnik/master/tests/python_tests/render_grid_test.py", line 140, in test_render_grid
    eq_(utf1,grid_correct_new3,show_grids('new-markers',utf1,grid_correct_new3))
AssertionError: JSON does not match  ->
 X grid does not match
 X data does not match
 X keys do not

    /tmp/mapnik-new-markers-actual.json
    /tmp/mapnik-new-markers-expected.json

======================================================================
FAIL: test using feature id
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/python-libs/nose-1.3.0-py2.7.egg/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/dane/projects/mapnik/master/tests/python_tests/render_grid_test.py", line 182, in test_render_grid3
    eq_(utf1,grid_feat_id3,show_grids('id-markers',utf1,grid_feat_id3))
AssertionError: JSON does not match  ->
 X grid does not match
 X data does not match
 X keys do not

    /tmp/mapnik-id-markers-actual.json
    /tmp/mapnik-id-markers-expected.json

----------------------------------------------------------------------
Ran 429 tests in 11.786s

FAILED (failures=3)
make: *** [test] Error 2
@springmeyer
Copy link
Member Author

looks like a clang++ over-eager optimization bug. Putting a std::clog << "foo\n" statement within add_path (

mapnik/include/mapnik/grid/grid_marker_helpers.hpp

Lines 85 to 152 in 15d3cef

void add_path(T & path)
{
marker_placement_enum placement_method = get<marker_placement_enum>(sym_, keys::markers_placement_type, MARKER_POINT_PLACEMENT);
bool ignore_placement = get<bool>(sym_, keys::ignore_placement, false);
bool allow_overlap = get<bool>(sym_, keys::allow_overlap, false);
double spacing = get<double>(sym_, keys::spacing, 100.0);
double max_error = get<double>(sym_, keys::max_error, 0.2);
box2d<double> bbox_(0,0, src_.width(),src_.height());
if (placement_method != MARKER_LINE_PLACEMENT ||
path.type() == geometry_type::types::Point)
{
double x = 0;
double y = 0;
if (path.type() == geometry_type::types::LineString)
{
if (!label::middle_point(path, x, y))
return;
}
else if (placement_method == MARKER_INTERIOR_PLACEMENT)
{
if (!label::interior_position(path, x, y))
return;
}
else
{
if (!label::centroid(path, x, y))
return;
}
agg::trans_affine matrix = marker_trans_;
matrix.translate(x,y);
box2d<double> transformed_bbox = bbox_ * matrix;
if (allow_overlap ||
detector_.has_placement(transformed_bbox))
{
render_raster_marker(matrix);
if (!ignore_placement)
{
detector_.insert(transformed_bbox);
}
if (!placed_)
{
pixmap_.add_feature(feature_);
placed_ = true;
}
}
}
else
{
markers_placement<T, label_collision_detector4> placement(path, bbox_, marker_trans_, detector_,
spacing * scale_factor_,
max_error,
allow_overlap);
double x, y, angle;
while (placement.get_point(x, y, angle, ignore_placement))
{
agg::trans_affine matrix = marker_trans_;
matrix.rotate(angle);
matrix.translate(x, y);
render_raster_marker(matrix);
if (!placed_)
{
pixmap_.add_feature(feature_);
placed_ = true;
}
}
}
}
) fixes all the failing tests, which indicates to me that clang++ is optimizing away valid code which similar to a previous clang++ bug: #1696.

@springmeyer springmeyer mentioned this issue May 5, 2014
Closed
@springmeyer
Copy link
Member Author

http://clang-developers.42468.n3.nabble.com/need-help-to-log-optimization-bug-td3821572.html notes bugpoint (http://llvm.org/docs/Bugpoint.html) to help debug things like this. More details at http://quetzalcoatal.blogspot.com/2012/03/how-to-use-bugpoint.html

@springmeyer springmeyer mentioned this issue May 5, 2014
Closed
@springmeyer
Copy link
Member Author

now hitting similar errors on travis with clang (https://travis-ci.org/mapnik/mapnik/jobs/24497210#L1440). The travis guys must have upgraded the clang version. So perhaps is not just -flto but clang 5.1 is general that is broken and #2235 is a dupe.

springmeyer pushed a commit that referenced this issue May 7, 2014
travis: try -Os with -flto to see if #2235 and #2232 are still broken…
e137985 
… on travis
springmeyer pushed a commit that referenced this issue May 8, 2014
revert usage of -flto on linux due to #2238 - refs #2232
88701b8
@springmeyer
Copy link
Member Author

http://llvm.org/bugs/show_bug.cgi?id=19824#c0

@springmeyer
Copy link
Member Author

turns out to be a mapnik bug / incorrect c++11 lambda usage - nothing to do with clang++ or -flto. Now solved by #2264

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@springmeyer