cv::remap artifacts with bilinear interpolation and cv::UMat

[sergiud]

System information (version)

• OpenCV => 3.4.6 (release) and 3.4.7 (dev) 1e9e2aa
• Operating System / Platform => Arch Linux
• Compiler => GCC 9.1

Detailed description

I use cv::remap in combination with CV_32F maps to rectify images using a non-standard lens model. With cv::INTER_LINEAR, I observe regular artifacts at the left and top borders of the transformed image. The problem does not occur at the right and bottom borders or with cv::INTER_CUBIC. Due to the image size, what follows is only the top-left corner of the image transformed using bilinear and bicubic interpolation for comparison purposes. The border color is deliberately set to magenta to illustrate the problem.

Unfortunately, it is not easy to extract a minimum working example due to complexity of the original code. However, I'm providing the values for the x and y coordinate maps for the above image transform:

x:

[-1.2421091, -0.28069085, 0.6807881, 1.6423278, 2.6039281, 3.565589, 4.5273104, 5.4890928, 6.4509354, 7.4128385, 8.3748016, 9.3368254, 10.29891, 11.261054, 12.223259, 13.185524, 14.147849, 15.110234, 16.07268, 17.035185, 17.997751, 18.960375, 19.923061, 20.885805, 21.84861;
 -1.2633712, -0.30193734, 0.65955734, 1.6211128, 2.5827289, 3.5444057, 4.5061431, 5.4679408, 6.4297991, 7.3917179, 8.3536968, 9.3157368, 10.277837, 11.239997, 12.202218, 13.164498, 14.12684, 15.08924, 16.051701, 17.014223, 17.976805, 18.939445, 19.902145, 20.864906, 21.827728;
 -1.284614, -0.32316425, 0.6383462, 1.5999173, 2.5615492, 3.5232418, 4.4849949, 5.4468083, 6.4086823, 7.3706169, 8.332612, 9.2946672, 10.256783, 11.21896, 12.181195, 13.143492, 14.105849, 15.068266, 16.030743, 16.99328, 17.955877, 18.918533, 19.88125, 20.844027, 21.806864;
 -1.305837, -0.34437162, 0.61715454, 1.5787414, 2.5403891, 3.5020971, 4.4638662, 5.4256954, 6.3875852, 7.3495355, 8.3115463, 9.2736177, 10.235749, 11.197941, 12.160193, 13.122505, 14.084878, 15.047311, 16.009804, 16.972357, 17.934969, 18.897642, 19.860374, 20.823166, 21.786018;
 -1.3270406, -0.3655594, 0.59598249, 1.5575851, 2.5192485, 3.4809723, 4.4427567, 5.4046021, 6.3665075, 7.3284736, 8.2905006, 9.2525873, 10.214734, 11.176942, 12.13921, 13.101538, 14.063926, 15.026375, 15.988883, 16.951452, 17.91408, 18.876768, 19.839518, 20.802326, 21.765194;
 -1.3482244, -0.3867276, 0.57483, 1.5364482, 2.4981272, 3.459867, 4.4216671, 5.3835278, 6.3454494, 7.3074312, 8.2694731, 9.231576, 10.193739, 11.155962, 12.118246, 13.080589, 14.042994, 15.005458, 15.967982, 16.930567, 17.893211, 18.855915, 19.81868, 20.781504, 21.744387;
 -1.3693887, -0.40787622, 0.55369705, 1.515331, 2.4770257, 3.438781, 4.4005971, 5.3624735, 6.3244104, 7.2864079, 8.2484665, 9.2105846, 10.172763, 11.135002, 12.097301, 13.059661, 14.02208, 14.984561, 15.947101, 16.9097, 17.87236, 18.835081, 19.797861, 20.7607, 21.7236;
 -1.3905334, -0.4290053, 0.53258365, 1.4942333, 2.4559436, 3.4177146, 4.3795462, 5.3414383, 6.3033915, 7.2654047, 8.227478, 9.1896124, 10.151807, 11.114061, 12.076376, 13.038752, 14.001187, 14.963683, 15.926238, 16.888855, 17.85153, 18.814266, 19.777061, 20.739918, 21.702833;
 -1.4116585, -0.45011476, 0.51148981, 1.4731551, 2.4348812, 3.396668, 4.3585153, 5.3204231, 6.2823915, 7.2444205, 8.2065096, 9.1686592, 10.13087, 11.09314, 12.05547, 13.017861, 13.980313, 14.942824, 15.905396, 16.868027, 17.830719, 18.79347, 19.756281, 20.719152, 21.682083;
 -1.4327641, -0.47120464, 0.49041557, 1.4520966, 2.4138381, 3.3756406, 4.3375034, 5.299427, 6.2614112, 7.2234559, 8.1855602, 9.1477261, 10.109952, 11.072238, 12.034584, 12.996991, 13.959458, 14.921985, 15.884572, 16.847219, 17.809927, 18.772694, 19.735521, 20.698408, 21.661354;
 -1.45385, -0.49227494, 0.46936092, 1.4310575, 2.3928149, 3.3546329, 4.3165112, 5.2784505, 6.2404504, 7.2025104, 8.1646309, 9.126812, 10.089054, 11.051355, 12.013718, 12.97614, 13.938622, 14.901165, 15.863768, 16.826431, 17.789154, 18.751936, 19.714779, 20.677683, 21.640644;
 -1.4749162, -0.51332563, 0.44832584, 1.410038, 2.3718109, 3.3336446, 4.2955389, 5.2574935, 6.2195091, 7.1815848, 8.1437206, 9.1059179, 10.068175, 11.030493, 11.99287, 12.955308, 13.917807, 14.880364, 15.842983, 16.805662, 17.7684, 18.731199, 19.694057, 20.656977, 21.619953;
 -1.495963, -0.53435671, 0.42731035, 1.3890381, 2.3508267, 3.312676, 4.2745857, 5.2365561, 6.1985869, 7.1606784, 8.1228304, 9.085043, 10.047316, 11.009648, 11.972042, 12.934496, 13.89701, 14.859584, 15.822218, 16.784912, 17.747665, 18.71048, 19.673355, 20.636288, 21.599283;
 -1.5169901, -0.55536819, 0.40631443, 1.3680578, 2.3298619, 3.2917268, 4.2536521, 5.2156382, 6.1776848, 7.139792, 8.1019592, 9.064187, 10.026476, 10.988825, 11.951233, 12.913703, 13.876232, 14.838821, 15.801472, 16.764181, 17.726952, 18.689781, 19.652672, 20.615622, 21.57863;
 -1.5379975, -0.57636011, 0.38533813, 1.347097, 2.3089168, 3.270797, 4.232738, 5.1947398, 6.1568017, 7.1189246, 8.0811081, 9.0433512, 10.005655, 10.968019, 11.930444, 12.892929, 13.855474, 14.818079, 15.780745, 16.743471, 17.706255, 18.669102, 19.632008, 20.594973, 21.557997;
 -1.5589852, -0.59733236, 0.3643814, 1.3261559, 2.287991, 3.249887, 4.2118435, 5.173861, 6.1359386, 7.0980768, 8.0602751, 9.0225344, 9.9848547, 10.947234, 11.909675, 12.872175, 13.834736, 14.797357, 15.760037, 16.722778, 17.685579, 18.648441, 19.611362, 20.574343, 21.537384;
 -1.5799534, -0.618285, 0.34344429, 1.3052343, 2.2670851, 3.2289965, 4.1909685, 5.1530013, 6.1150947, 7.0772486, 8.039463, 9.0017376, 9.9640722, 10.926468, 11.888924, 12.85144, 13.814016, 14.776652, 15.739349, 16.702106, 17.664923, 18.6278, 19.590736, 20.553734, 21.51679;
 -1.6009021, -0.63921803, 0.32252675, 1.2843324, 2.2461987, 3.2081256, 4.1701136, 5.1321616, 6.0942702, 7.0564399, 8.0186701, 8.9809599, 9.9433107, 10.905722, 11.868193, 12.830725, 13.793317, 14.755968, 15.71868, 16.681454, 17.644285, 18.607178, 19.570129, 20.533142, 21.496214;
 -1.6218309, -0.66013145, 0.30162886, 1.2634499, 2.2253318, 3.1872742, 4.1492777, 5.1113415, 6.0734658, 7.0356507, 7.9978957, 8.9602013, 9.9225683, 10.884995, 11.847482, 12.810029, 13.772636, 14.735304, 15.698031, 16.66082, 17.623667, 18.586575, 19.549543, 20.512571, 21.475658;
 -1.6427402, -0.68102527, 0.28075057, 1.2425871, 2.2044845, 3.1664424, 4.1284614, 5.0905404, 6.0526805, 7.0148807, 7.9771419, 8.9394627, 9.901845, 10.864287, 11.826789, 12.789352, 13.751975, 14.714658, 15.677402, 16.640205, 17.603067, 18.565992, 19.528975, 20.492018, 21.455122;
 -1.6636299, -0.70189941, 0.25989187, 1.2217439, 2.1836567, 3.1456304, 4.1076646, 5.0697594, 6.0319147, 6.9941306, 7.9564071, 8.9187441, 9.8811407, 10.843598, 11.806117, 12.768695, 13.731334, 14.694032, 15.656791, 16.61961, 17.582489, 18.545427, 19.508427, 20.471485, 21.434605;
 -1.6844999, -0.72275394, 0.23905282, 1.2009203, 2.1628487, 3.1248376, 4.0868874, 5.0489974, 6.0111685, 6.9733996, 7.9356918, 8.8980436, 9.8604565, 10.82293, 11.785463, 12.748057, 13.710711, 14.673426, 15.6362, 16.599033, 17.56193, 18.524883, 19.487898, 20.450972, 21.414106;
 -1.7053502, -0.74358881, 0.21823338, 1.1801164, 2.14206, 3.1040645, 4.0661297, 5.0282555, 5.9904418, 6.9526887, 7.9149961, 8.8773642, 9.8397923, 10.80228, 11.76483, 12.727439, 13.690108, 14.652838, 15.615628, 16.578478, 17.541388, 18.504358, 19.467388, 20.430479, 21.393629;
 -1.7261809, -0.76440406, 0.19743356, 1.159332, 2.1212912, 3.0833111, 4.0453916, 5.0075331, 5.9697347, 6.9319968, 7.89432, 8.8567028, 9.8191471, 10.781651, 11.744215, 12.706841, 13.669525, 14.63227, 15.595076, 16.557941, 17.520866, 18.483852, 19.446898, 20.410004, 21.373169;
 -1.7469919, -0.78519964, 0.17665339, 1.1385672, 2.1005418, 3.0625772, 4.024673, 4.9868298, 5.9490471, 6.911325, 7.8736629, 8.8360615, 9.798521, 10.761041, 11.72362, 12.686261, 13.648961, 14.611722, 15.574543, 16.537424, 17.500364, 18.463366, 19.426426, 20.389547, 21.352728]

y:

[-0.80142921, -0.82269335, -0.84394169, -0.86517435, -0.88639116, -0.90759224, -0.92877752, -0.94994706, -0.97110081, -0.99223876, -1.0133609, -1.0344672, -1.0555578, -1.0766326, -1.0976915, -1.1187347, -1.139762, -1.1607735, -1.1817693, -1.2027491, -1.2237132, -1.2446613, -1.2655938, -1.2865102, -1.307411;
 0.16792351, 0.14667894, 0.12545012, 0.10423703, 0.083039708, 0.061858147, 0.040692344, 0.019542305, -0.0015919654, -0.022710463, -0.043813184, -0.064900123, -0.085971281, -0.10702665, -0.12806623, -0.14909001, -0.17009799, -0.19109017, -0.21206655, -0.23302712, -0.25397187, -0.27490079, -0.29581392, -0.31671122, -0.33759266;
 1.1373272, 1.1161022, 1.0948929, 1.0736994, 1.0525216, 1.0313596, 1.0102133, 0.98908263, 0.96796787, 0.94686884, 0.92578554, 0.90471804, 0.88366628, 0.86263031, 0.84161007, 0.8206057, 0.79961705, 0.7786442, 0.75768715, 0.73674589, 0.71582043, 0.69491076, 0.67401695, 0.65313888, 0.63227665;
 2.1067817, 2.0855763, 2.0643866, 2.0432127, 2.0220544, 2.0009117, 1.979785, 1.958674, 1.9375786, 1.916499, 1.8954352, 1.8743871, 1.8533548, 1.8323382, 1.8113374, 1.7903523, 1.7693831, 1.7484295, 1.7274919, 1.7065699, 1.6856637, 1.6647733, 1.6438987, 1.62304, 1.6021971;
 3.0762873, 3.0551014, 3.0339313, 3.0127769, 2.9916379, 2.970515, 2.9494078, 2.9283161, 2.9072404, 2.8861802, 2.8651357, 2.8441072, 2.8230941, 2.8020971, 2.7811155, 2.76015, 2.7392001, 2.7182658, 2.6973474, 2.6764448, 2.6555579, 2.6346869, 2.6138315, 2.5929921, 2.5721684;
 4.0458436, 4.0246773, 4.0035267, 3.9823916, 3.9612725, 3.9401691, 3.9190812, 3.8980091, 3.8769529, 3.8559122, 3.8348873, 3.8138781, 3.7928846, 3.7719066, 3.7509446, 3.7299984, 3.7090678, 3.688153, 3.667254, 3.6463706, 3.6255031, 3.6046512, 3.5838153, 3.562995, 3.5421906;
 5.0154505, 4.9943037, 4.9731727, 4.9520574, 4.9309578, 4.909874, 4.8888054, 4.867753, 4.8467159, 4.825695, 4.8046894, 4.7836995, 4.7627258, 4.7417674, 4.7208247, 4.6998978, 4.6789865, 4.6580911, 4.6372113, 4.6163473, 4.595499, 4.5746665, 4.5538497, 4.5330486, 4.5122638;
 5.9851084, 5.9639812, 5.9428697, 5.9217739, 5.9006939, 5.8796296, 5.8585806, 5.8375478, 5.8165302, 5.7955284, 5.7745423, 5.753572, 5.7326174, 5.7116785, 5.6907554, 5.669848, 5.6489558, 5.6280799, 5.6072197, 5.5863748, 5.565546, 5.5447326, 5.5239353, 5.5031533, 5.4823875;
 6.9548168, 6.9337091, 6.9126172, 6.891541, 6.8704805, 6.8494358, 6.8284063, 6.8073931, 6.7863951, 6.7654128, 6.7444463, 6.7234955, 6.7025599, 6.6816406, 6.6607366, 6.6398487, 6.6189761, 6.5981193, 6.5772786, 6.5564532, 6.5356436, 6.5148497, 6.4940715, 6.473309, 6.4525623;
 7.9245758, 7.9034882, 7.8824158, 7.8613591, 7.8403182, 7.8192925, 7.7982831, 7.7772889, 7.7563105, 7.7353477, 7.7144008, 7.693469, 7.6725535, 7.6516533, 7.6307688, 7.6099, 7.589047, 7.5682096, 7.5473881, 7.5265822, 7.5057917, 7.4850173, 7.4642582, 7.4435148, 7.4227877;
 8.8943853, 8.8733168, 8.8522644, 8.8312273, 8.8102064, 8.7891998, 8.7682104, 8.7472353, 8.7262764, 8.7053337, 8.6844053, 8.6634941, 8.6425972, 8.6217165, 8.600852, 8.5800028, 8.5591688, 8.5383511, 8.5175486, 8.4967613, 8.4759903, 8.4552355, 8.4344959, 8.4137716, 8.3930635;
 9.8642454, 9.8431969, 9.8221636, 9.8011465, 9.7801447, 9.7591581, 9.7381878, 9.7172327, 9.6962929, 9.6753693, 9.6544609, 9.6335688, 9.6126919, 9.5918303, 9.5709848, 9.5501547, 9.5293407, 9.5085421, 9.4877596, 9.4669924, 9.4462404, 9.4255047, 9.4047842, 9.384079, 9.36339;
 10.834157, 10.813128, 10.792114, 10.771116, 10.750134, 10.729167, 10.708216, 10.687281, 10.666361, 10.645456, 10.624568, 10.603695, 10.582837, 10.561996, 10.541169, 10.520358, 10.499563, 10.478785, 10.458021, 10.437273, 10.41654, 10.395823, 10.375123, 10.354438, 10.333768;
 11.804118, 11.783109, 11.762115, 11.741137, 11.720173, 11.699226, 11.678294, 11.657378, 11.636478, 11.615593, 11.594725, 11.573871, 11.553033, 11.53221, 11.511403, 11.490612, 11.469837, 11.449077, 11.428332, 11.407604, 11.386891, 11.366194, 11.345512, 11.324846, 11.304195;
 12.77413, 12.753139, 12.732165, 12.711206, 12.690264, 12.669335, 12.648424, 12.627527, 12.606647, 12.585781, 12.564931, 12.544097, 12.523279, 12.502476, 12.481688, 12.460917, 12.440161, 12.41942, 12.398695, 12.377986, 12.357292, 12.336615, 12.315952, 12.295305, 12.274674;
 13.744191, 13.723221, 13.702267, 13.681328, 13.660404, 13.639496, 13.618604, 13.597726, 13.576865, 13.556019, 13.535189, 13.514375, 13.493575, 13.472792, 13.452024, 13.431272, 13.410535, 13.389813, 13.369108, 13.348418, 13.327744, 13.307085, 13.286442, 13.265815, 13.245203;
 14.714304, 14.693353, 14.672418, 14.651499, 14.630594, 14.609706, 14.588833, 14.567976, 14.547133, 14.526308, 14.505497, 14.484702, 14.463923, 14.443158, 14.42241, 14.401677, 14.38096, 14.360258, 14.339572, 14.318901, 14.298246, 14.277607, 14.256983, 14.236375, 14.215782;
 15.684466, 15.663535, 15.64262, 15.62172, 15.600836, 15.579967, 15.559114, 15.538276, 15.517453, 15.496646, 15.475855, 15.455079, 15.434319, 15.413575, 15.392846, 15.372132, 15.351435, 15.330752, 15.310085, 15.289434, 15.268799, 15.248178, 15.227574, 15.206985, 15.186412;
 16.654678, 16.633768, 16.612871, 16.591991, 16.571127, 16.550278, 16.529444, 16.508625, 16.487823, 16.467035, 16.446264, 16.425507, 16.404766, 16.384041, 16.363333, 16.342638, 16.32196, 16.301296, 16.280649, 16.260017, 16.239401, 16.2188, 16.198215, 16.177647, 16.157091;
 17.624943, 17.60405, 17.583174, 17.562313, 17.541468, 17.520639, 17.499825, 17.479025, 17.458242, 17.437475, 17.416723, 17.395987, 17.375265, 17.354559, 17.333868, 17.313194, 17.292536, 17.271891, 17.251263, 17.230652, 17.210054, 17.189472, 17.168907, 17.148357, 17.127823;
 18.595255, 18.574383, 18.553526, 18.532686, 18.51186, 18.491051, 18.470255, 18.449476, 18.428713, 18.407965, 18.387232, 18.366514, 18.345812, 18.325127, 18.304455, 18.2838, 18.263161, 18.242537, 18.221928, 18.201334, 18.180758, 18.160194, 18.139648, 18.119118, 18.098602;
 19.565619, 19.544767, 19.52393, 19.503107, 19.482302, 19.461512, 19.440737, 19.419977, 19.399233, 19.378504, 19.35779, 19.337093, 19.31641, 19.295744, 19.275093, 19.254457, 19.233837, 19.213232, 19.192642, 19.17207, 19.15151, 19.130968, 19.110441, 19.08993, 19.069433;
 20.536032, 20.5152, 20.494383, 20.473579, 20.452793, 20.432022, 20.411266, 20.390528, 20.369802, 20.349092, 20.3284, 20.307722, 20.287058, 20.266411, 20.245779, 20.225163, 20.204563, 20.183977, 20.163408, 20.142853, 20.122314, 20.101791, 20.081284, 20.060791, 20.040314;
 21.506496, 21.485683, 21.464886, 21.444103, 21.423336, 21.402584, 21.381847, 21.361128, 21.340424, 21.319733, 21.299059, 21.2784, 21.257757, 21.237129, 21.216516, 21.195919, 21.175339, 21.154772, 21.134222, 21.113688, 21.093168, 21.072664, 21.052176, 21.031702, 21.011246;
 22.477009, 22.456215, 22.435438, 22.414675, 22.393927, 22.373196, 22.35248, 22.331778, 22.311092, 22.290422, 22.269768, 22.249128, 22.228504, 22.207897, 22.187304, 22.166727, 22.146164, 22.125618, 22.105087, 22.084572, 22.064072, 22.043587, 22.023117, 22.002665, 21.982225]

I am able to backtrace the problem to cv::UMat usage as input to cv::remap. The image does not exhibit any artifacts with cv::Mat.