Rendring raster border bug

[artemp]

When tile border is on a raster border the tile border isn't properly rendered.[[BR]]
Here is an example :[[BR]]
Wrong : http://i455.photobucket.com/albums/qq279/ghurdyl/work/mapnik-1.png (with mapnik & gdal plugin) [[BR]]
Right : http://i455.photobucket.com/albums/qq279/ghurdyl/work/gqis.png (with QGis)[[BR]]
[[BR]]
Raster are 4096x4096px png generated with mapnik from shape files data. (and an additional .pgw world file)[[BR]]
They are generated in such a way that there are exactly 256 small tiles (16x16) of 256px inside one raster[[BR]]
[[BR]]
On the wrong images, tiles have been generated from the rasters with mapnik (and gdal plugin ) and the border of wrong tiles are exactly on the border of rasters.

[artemp]

[springmeyer] how are you generating the .pgw files?

Also, it would be helpful to have a test script attached to replicate.

[artemp]

[mar_rud] I think I have similar problem, but in my case it wasn't as visible to easily notice effects. Not sure if it same problem, but prepared patch potentially should fix mentioned issue.

What I found is the fact, that returned in gdal and raster plugin returned new raster(intersect,image) is using intersect variable computed from query, but real image extend can be a little different because of rounding to pixel grid. Also in agg_renderer (cario_renderer too), there is error when rounding negative values: int(ext.minx()+0.5), ie: int(-3.0+0.5) == -2 but round(-3.0) == -3. There is still left potential subpixel misalignment due to rounding position and size of target raster after resizing, but it should have less impact.

In my map I rarely see effects of this issues, because in hillshading 1px offset is hardly visible and also my geotiffs are optimized for tiled resolutions. In attachment there is gif showing original and patched version of same region resized 4x. In fact this is a little different then described in ticket, because I have one big raster and misalignment appears very rarely on tile edges. Unfortunately this ticket doesn't have any test examples and details, so I can't say patch will definitely fix reported problem.

I prepared patches for both 0.7.1-dev and trunk, but only first one I could compile (ubuntu doesn't have new enough libboost). I tested only gdal input plugin. Changes to trunk and to raster plugin are done as mirror copy, but not tested.

BTW. codes in trunk are using mixed indention (tab and spaces).

[artemp]

[springmeyer] Just had a chance to test quickly the patch against 0.7.1. It perfectly fixes shifts I have been seeing in comparison to QGIS when loading raster data via Quantumnik. There is still apparent a slight different in pixel aspect ratio after nearest neighbor resampling, but I think this is likely a bug in the sampling in QGIS because I can push a raster through mapnik and get the exact same visual output for a tile rendered at the same size as the input raster using this patch.

I am going to test more later in the afternoon, but as far as I can tell this is an excellent improvement and should be applied to 0.7.1 before release, so I am going to switch the milestone.

Thanks again for all of your excellent work Marcin!!!

[artemp]

[mar_rud] There are still some problems with scaling, because when I render given bbox with different resolutions (ie. same area in size 128x128 and 256x256) and when I resize and compare in gimp, raster background is often shifted a little. This bothers me for a long time. In OpenLayers while zooming, already available images are initially resized and shown by browser to avoid showing empty space before current zoom is downloaded. Now I have effect of moving mountains while zooming.

I thing one of causes could be scale_bilinear that assumes point 0,0 before and after scaling is in the same position, but it isn't true (imagine where should be placed middle of (0,0) pixel before and after scaling). After some thinking and drawing, additional offset of source (0,0) in target image should be: (target_size-source_size)/(2*source_size).

In attached patch I tried to implement it with additional support for passing any subpixel offset (i.e. in agg_renderer, difference between rounded start_x and real ext.minx() ). I used integer arithmetic so offset is evaluated with 1/source_size accuracy. This fixes gimp resizing test.

Changes to agg_renderer and cairo_renderer are also needed to use new feature, but even if ext boundaries of target raster are integer (no rounding) I have some shifts, in some rare cases in other tests.

Some very strange test cases for catching potential errors, that I use to torture mapnik:

• mentioned test with rendering in different resolutions and resizing with gimp
• rendering vector map and then using rendered image as gdal/raster input in same vector map, using different resolutions. This way I have image with known features that should match with vector source regardless of scale. Roads on bigger zoom should be in the middle of ones in resized raster.
• render map second time with few pixel offset and test if raster background is still in same relative position compared to vector data (I use layers in gimp in "difference" mode)

Probably most of mentioned issues are only potential or without any significant negative effect in real mapping situation.

[artemp]

[mar_rud] Latest patch (trunk or 0.7.1-dev one) combines changes from my previous patches in this ticket and latests experiments.

It contains fixed size, position and extend calculation in raster and gdal plugins, more accurate target image positioning in *_renderer and subpixel offset support in bilinear method + fixed initial offset and some improvements for special cases in image_util.hpp. scale_image using NN is not changed.

With these changes most of my extreme tests are ok.

[artemp]

[springmeyer] Amazing, We should apply yes? We should also get you svn ci access!

[artemp]

[artem] +1 for svn access!

[artemp]

[springmeyer] after this patch getting:

{{{
plugins/input/gdal/gdal_featureset.cpp: In member function 'mapnik::feature_ptr gdal_featureset::get_feature(const mapnik::query&)':
plugins/input/gdal/gdal_featureset.cpp:117: warning: comparison between signed and unsigned integer expressions
plugins/input/gdal/gdal_featureset.cpp:119: warning: comparison between signed and unsigned integer expressions
}}}

ideas on avoiding?

[artemp]

[mar_rud] Replying to [comment:10 springmeyer]:

ideas on avoiding?

in 'if' statement, cast unsigned to signed typ. In practice end_x/end_y will always be >=0, so other way is to declare them unsigned.

For testing svn :) , I commited to 0.7.1-dev first approach.

[artemp]

[springmeyer] whoops, forgot to close out. great job all.

[artemp]

[mar_rud] I just compared my map using 0.6.1 and 0.7.1-dev @ 1700 and in 0.7.1-dev there is little difference between 0.6.1 vs 0.7.1 when upscaling. First is sharp and second blurry in such case.

So i.e. 512x512 should be resized to 256x256 tile, but to have some margin around, source raster is bigger at most 1px in each direction (floor/ceil on floating 24.999992,32.0000001) resulting 514x514 image scaled by gdal to 257x257 but it is shifted by 0.5px (1px before scaling) so when this shift is corrected later in scale_image_bilinear, raster is blurred and in effect looks like 2x lower resolution. If upscaling is done in scale_image_bilinear, this doesn't happen, because there is enough resolution to do 0.5px shift without blur.

Rounding is to pixels of source raster not considering output
resolution and in effect it introduces unnecessary subpixel shift when downscaling.
This could be avoided, if x_off,y_off,end_x,end_y in
gdal_featureset.cpp are rounded using output pixel grid (r1702, r1703).

Still, if query is not in sync with gdal source, blurry effect can happen in case of resolution same as raster/gdal input, when scale_image_bilinear tries to fix any misalignment.

Other side effect of subpixel positioning is case of adjacent rasters. If they are not in sync with resolution of output image, some strange things could potentially happened on joints (didn't test it).

Eventually we could disable passing additional x_off,y_off to scale_image_bilinear, because in practice it only tries to fix < 0.5px shift but introduces more visible side effects.

[artemp]

[springmeyer] Just tested an large raster with gdal overviews in Quantumnik, and r1702 fixes another issue I just encountered when setting up to test 1702, whereby a few pixels at the edge of the image are distorted in color when using bilinear scaling. So, great stuff! I'll just document what was fixed below for reference:

r1701 with bilinear scaling:
[[BR]]
[[Image(r_1701_bilinear.png)]]

Image in QGIS:
[[BR]]
[[Image(qgis_nearest_neightbor.png)]]