Adapted to use on IOS and work (android/android / ios/ios) but don't authenticate in cross platform (ios/android - android/ios)

[fnoceda]

Hi @MCarlomagno thanks for share your job, is amazing.

I have a problem, I save my photo and the array resulting of predictedData on my back end database with a API REST, when I Sign In and Sign Up in a ios-ios or android-android no problem, but when i try authenticate cross platform the distance result different i see the array save on my back end and are different in this case. (can you see this more low)

I think the problem is in the function _convertCameraImage, because android and ios use different formats (android -> yuv and ios -> bgra), but i understand this function _convertCameraImage convert to the yuv -> bgra, so i don't understanding because the result on the predictedData is very different

this is my code adapted to the android and ios support

imglib.Image _convertCameraImage(CameraImage image, CameraLensDirection _dir) {
    print('_convertCameraImage');
    if( GetPlatform.isAndroid == true ){
      print(' Android');
      int width = image.width; print('11'); print(width);
      int height = image.height; print('12'); print(height);
      var img = imglib.Image(width, height); print('13');
      const int hexFF = 0xFF000000; print('14');
      print( 'image.planes.length=>' + image.planes.length.toString());
      print(image.planes[0].bytesPerRow);
      final int uvyButtonStride = image.planes[1].bytesPerRow; print('15');
      final int uvPixelStride = image.planes[1].bytesPerPixel; print('16');
      for (int x = 0; x < width; x++) {
        for (int y = 0; y < height; y++) {
          final int uvIndex = uvPixelStride * (x / 2).floor() + uvyButtonStride * (y / 2).floor();
          final int index = y * width + x;
          final yp = image.planes[0].bytes[index];
          final up = image.planes[1].bytes[uvIndex];
          final vp = image.planes[2].bytes[uvIndex];
          int r = (yp + vp * 1436 / 1024 - 179).round().clamp(0, 255);
          int g = (yp - up * 46549 / 131072 + 44 - vp * 93604 / 131072 + 91).round().clamp(0, 255);
          int b = (yp + up * 1814 / 1024 - 227).round().clamp(0, 255);
          img.data[index] = hexFF | (b << 16) | (g << 8) | r;
        }
      }
      print('17');
      var img1 = imglib.copyRotate(img, -90);
      print(img1.toString());
      return img1;
    }else{
      print(' IOS');
      imglib.Image img = imglib.Image.fromBytes(
        image.width,
        image.height,
        image.planes[0].bytes,
        format: imglib.Format.bgra,
      );
      var img1 = (_dir == CameraLensDirection.front)
          ? imglib.copyRotate(img, -90)
          : imglib.copyRotate(img, 90);
      print('retorno');
      print(img1.toString());
      return img1;
    }
    
  }

This is the result of my test

IOS SignUp-IOS Sign In [works fine]
ANDROID SignUp-ANDROID Sign In [works fine]
IOS SignUp-ANDROID Sign In [don't authenticate]
ANDROID SignUp-IOS Sign In [don't authenticate]

I log my results
IOS SignUp-IOS SignIn
// flutter: calculandoDistancia
// flutter: currDist=>0.7708154422560634
// flutter: threshold=>1.0
// flutter: minDist=>0.7708154422560634
// flutter: end setPredictedData
// flutter: authenticate OK

ANDROID SignUp-ANDROID SignIn
// flutter ( 8968): calculandoDistancia
// flutter ( 8968): currDist=>0.6422662304939313
// flutter ( 8968): threshold=>1.0
// flutter ( 8968): minDist=>0.6422662304939313
// flutter ( 8968): end setPredictedData
// flutter ( 8968): authenticate OK

IOS SignUp-ANDROID SignIn
// flutter (19459): calculandoDistancia
// flutter (19459): currDist=>1.4399978617184572
// flutter (19459): threshold=>1.0
// flutter (19459): minDist=>999.0
// flutter (19459): end setPredictedData
// flutter (19459): don't authenticate

ANDROID SignUp-IOS SignIn
// flutter: calculandoDistancia
// flutter: currDist=>1.4992889685122506
// flutter: threshold=>1.0
// flutter: minDist=>999.0
// flutter: end setPredictedData
// flutter: don't authenticate

The predictedData saved (for the same face) is here, and is very different in android and ios

With IOS
[-0.005366702564060688, 0.03501761332154274, 0.016875134781003, 0.0008109764894470572, -0.07175101339817047, 0.09405621886253357, -0.05950654298067093, 0.05654291436076164, -0.0009066364145837724, -0.030182218179106712, -0.029889754951000214, 0.0075845494866371155, -0.00888113770633936, 0.02600974030792713, 0.0004703785525634885, 0.00661942083388567, -0.025093356147408485, -0.015354325994849205, -0.00023305673676077276, 0.009817020036280155, -0.19247987866401672, 0.09031268209218979, -0.03433519974350929, 0.015412818640470505, -0.0217202790081501, 0.014701157808303833, -0.05307235196232796, 0.11292985081672668, 0.11714132130146027, -0.040593914687633514, -0.006721782963722944, 0.2812327742576599, 0.00556167820468545, 0.002247093478217721, -0.025229839608073235, 0.11261788755655289, 0.1094202920794487, -0.022480683401226997, 0.0022312516812235117, 0.03486163169145584, 0.013287585228681564, 0.00656580226495862, 0.01938057132065296, -0.017791520804166794, 0.009392947889864445, -0.06839743256568909, -0.11292985081672668, -0.035972993820905685, -0.01327783614397049, 0.07424669712781906, -0.04000898823142052, -0.008154853247106075, -0.18608468770980835, -0.0032780268229544163, -0.0868811160326004, 0.008096360601484776, 0.1081724464893341, 0.004223658237606287, -0.12166475504636765, 0.03511510044336319, 0.05743980407714844, -0.08188974112272263, -0.04991374537348747, -0.0866471454501152, -0.019146600738167763, -0.08508733659982681, -0.007964751683175564, 0.008457065559923649, 0.010167975910007954, 0.0048524546436965466, -0.038644157350063324, 0.09202846884727478, -0.05233144387602806, 0.010655415244400501, -0.09382224828004837, 0.01086988765746355, 0.003938506357371807, -0.002282432746142149, 0.2416137307882309, -0.019507305696606636, -0.012897633947432041, -0.027121102437376976, -0.014798645861446857, 0.1291518211364746, -0.11238391697406769, -0.0034656908828765154, -0.0029051359742879868, 0.011152602732181549, 0.05307235196232796, -0.1511450558900833, 0.0439474955201149, -0.0017815892351791263, 0.013560551218688488, -0.03493962064385414, -0.10770450532436371, -0.09124856442213058, -0.038215212523937225, 0.0656287744641304, 0.0009505058987997472, -0.0028832012321799994, 0.003543680999428034, -0.011874012649059296, -0.007121482864022255, -0.001653636572882533, -0.006395198870450258, 0.006283087655901909, -0.20339851081371307, 0.0008712970884516835, -0.005746904760599136, 0.015822267159819603, -0.02010198123753071, 0.010557927191257477, 0.004179788753390312, 0.27436962723731995, 0.014720655977725983, 0.14420393109321594, -0.00717022642493248, -0.03051367774605751, 0.00400918535888195, 0.10754852741956711, 0.26610267162323, -0.004287025425583124, -0.14404794573783875, -0.0010973468888550997, -0.0003805070009548217, -0.005079113412648439, 0.0077697765082120895, 0.002900261664763093, 0.005322833079844713, -0.03796174377202988, 0.01333632878959179, 0.028037486597895622, -0.003499811515212059, -0.02388450689613819, 0.06870938837528229, -0.014730404131114483, -0.1974712610244751, 0.011347577907145023, 0.0060296193696558475, 0.013950502499938011, -0.008208471350371838, 0.0019278209656476974, -0.0006574332364834845, -0.02831045351922512, -0.07572851330041885, 0.060676395893096924, -0.017918255180120468, -0.0022641539108008146, 0.01422346755862236, -0.014408694580197334, -0.009558677673339844, -0.029070857912302017, -0.05318933725357056, -0.031956497579813004, -0.009612295776605606, -0.0028417690191417933, 0.0025029988028109074, 0.013541053049266338, -0.07841917127370834, -0.0004280323046259582, -0.08391748368740082, 0.00179255660623312, -0.011825268156826496, 0.0017815892351791263, 0.005439818371087313, 0.02858341857790947, 0.016894632950425148, -0.13804270327091217, -0.0028637037612497807, 0.0006805866141803563, 0.23412667214870453, 0.11207196116447449, -0.0027832763735204935, -0.013433816842734814, 0.015968499705195427, 0.006487811915576458, -0.012429692782461643, -0.11277387291193008, -0.01536407507956028, 0.010450690984725952, -0.0436355322599411, 0.013316831551492214, -0.006326957140117884, -0.002563928719609976, 0.131959468126297, 0.10123131424188614, -0.13734079897403717, 0.0655507892370224, 0.10723656415939331, -0.08625718951225281, -0.07693736255168915, -0.008364452049136162]

With ANDROID
[0.017560526728630066, -0.0025206489954143763, -0.001565065118484199, -0.004763653036206961, -0.006009773351252079, 0.04073874652385712, 0.016540179029107094, 0.20969343185424805, 0.013223133981227875, 0.20780882239341736, 0.005943980999290943, -0.0026168033946305513, -0.006616274360567331, -0.01967291533946991, -0.004560244735330343, 0.0022333874367177486, -0.0006560852052643895, 0.006724789272993803, 0.007219155319035053, -0.006807046011090279, 0.059210970997810364, 0.008006574586033821, -0.0298028364777565, 0.0026111663319170475, -0.06040123477578163, -0.003707980504259467, 0.0029578227549791336, -0.03018888272345066, 0.1781667321920395, 0.011050385423004627, 0.005854886025190353, 0.017274057492613792, -0.09191303700208664, -0.006212593521922827, -0.2262837141752243, -0.15064898133277893, 0.1591326892375946, -0.006933901458978653, -0.001802279381081462, -0.10352537781000137, -0.0010443583596497774, 0.0018612570129334927, 0.006398417986929417, 0.0012554702116176486, 0.0074211121536791325, 0.010717593133449554, 0.049051254987716675, 0.24959337711334229, 0.006417354568839073, -0.057666826993227005, -0.0468457005918026, -0.002862254623323679, -0.00486677186563611, -0.002375381998717785, -0.01682371459901333, 0.009011342190206051, 0.12074223905801773, 0.001989581622183323, -0.009173679165542126, -0.013144731521606445, -0.015070175752043724, -0.04363885894417763, -0.0003053119871765375, -0.006338771432638168, 0.004932073410600424, -0.15332861244678497, -0.004816148895770311, -0.0023492116015404463, 0.016545282676815987, -0.001709619304165244, -0.013169904239475727, -0.11412511020898819, -0.202517569065094, 0.009383699856698513, 0.1297224462032318, 0.0056001124903559685, -0.01269744522869587, 0.0021330846939235926, -0.08546653389930725, -0.1334666609764099, -0.0027801108080893755, -0.07388244569301605, 0.00505793234333396, 0.07371390610933304, 0.03501145541667938, -0.0006730420864187181, -0.002135098446160555, 0.004249595105648041, -0.036232441663742065, 0.263942152261734, 0.0027228561230003834, -0.0020696944557130337, 0.0017612290102988482, -0.002822405658662319, 0.1190698966383934, 0.04764315485954285, 0.02588491700589657, -0.09784920513629913, -0.0023657327983528376, -0.02211504615843296, 0.004360306076705456, -0.0026493342593312263, -0.001022880314849317, 0.00408570934087038, -0.0034248982556164265, 0.003120097564533353, -0.07066097110509872, 0.003962109796702862, -0.020672880113124847, 0.002750005107372999, -0.10889820009469986, -0.0069341775961220264, 0.010038419626653194, 0.1742521971464157, 0.0003855641698464751, -0.0016570232110098004, 0.012623189948499203, 0.007401735056191683, 0.030959144234657288, -0.04500763118267059, -0.018015660345554352, -0.004637634847313166, 0.1341673582792282, -0.002675049938261509, 0.0010797850554808974, -0.004457265138626099, 0.0026313276030123234, 0.0004421667836140841, 0.02306954190135002, 0.21203938126564026, -0.0042820824310183525, 0.0033936691470444202, -0.004675364587455988, -0.006049075163900852, -0.08900583535432816, -0.002515271073207259, -0.06748009473085403, 0.02633572369813919, -0.007937341928482056, -0.0005898139788769186, -0.000987825682386756, -0.0029705087654292583, -0.0008315134327858686, -0.26069924235343933, 0.0367228239774704, 0.08291738480329514, -0.012987925671041012, -0.00524356821551919, 0.010866782627999783, -0.0017442327225580812, 0.0024614043068140745, -0.24548490345478058, -0.0642000138759613, 0.009642115794122219, 0.0014905520947650075, 0.004344462417066097, -0.001906131743453443, 0.008299516513943672, 0.05748245120048523, 0.0031748428009450436, 0.0020936853252351284, 0.0012538537848740816, 0.004880981519818306, -0.0021094498224556446, 0.0012627202086150646, -0.0025581330992281437, -0.0043145096860826015, 0.17136064171791077, -0.0027595553547143936, 0.0014381170039996505, 0.1392369121313095, 0.13462324440479279, -0.0060705929063260555, 0.023846164345741272, -0.041592974215745926, 0.0018410480115562677, 0.12148106843233109, -0.03168937936425209, -0.00026864526444114745, -0.005666567012667656, 0.0920085608959198, 0.026908613741397858, 0.0023466504644602537, 0.0051388428546488285, -0.11932720988988876, -0.19283035397529602, 0.024459129199385643, -0.06673143804073334, 0.08777308464050293, 0.01421852596104145, -0.005940374452620745, 0.007013058289885521]

I hope can you help me, thanks a lot

[MCarlomagno]

Hi! Thank you so much for the contribution, I didnt try this with a server app, buth the array result should be the same since it doesnt depend on the platform, it depends of the ML model.
Im currently working on upgrade the project to Flutter 2.x and doing some bugfixing, but then I want to check the integration with server apps, I'm probably missing something.
Again, thank you very much and we keep in touch 😃

[matejsvajger]

+1 on this, I have the same issue when registration is done on one platform and sign-in on another, the data is significantly different.

Also on iOS all the faces seem to be in the 0.6-0.8 range. Could it be something wrong with _convertYUV420 function? 🤔