Sending a more explicit header when doing a scan/query from Smoothie

[teolemon]

What

• Sending a more explicit header when doing a scan/query from Smoothie

At the moment the log looks like this:

xxxx - - [10/Jun/2022:15:02:48 +0200] "POST /cgi/display.pl?api/v0/product/8059070744132.json HTTP/1.0" 200 20617 "-" "Dart/2.17 (dart:io)" 0/54837

• When a scan it could say Open Food Facts vXXX - Scan
• When a search it could say Open Food Facts vXXX - Search

[stephanegigandet]

Please also add the platform ios / android

[VaiTon]

Can we use User-Agent? @teolemon
Or the uuid like we wanted to do in the Android native app?
openfoodfacts/openfoodfacts-androidapp#4438

[teolemon]

https://github.com/openfoodfacts/openfoodfacts-dart/pull/213/files

[monsieurtanuki]

You're already supposed to see something like that, isn't that the case?

  UserAgent(
    name: 'Smoothie - ${packageInfo.appName}',
    version: '${packageInfo.version}+${packageInfo.buildNumber}',
    system: Platform.operatingSystemVersion,
    url: 'https://world.openfoodfacts.org/',
  );

Regarding using the user agent for a distinction between scan and search, it would mean changing the user agent for each query (ok, we have the technology for that) while it's supposed to be kind of static.

[monsieurtanuki]

Correction: the "user agent" parameters are supposed to appear as get/post parameters (cf. HttpHelper)

      map['app_name'] = OpenFoodAPIConfiguration.userAgent!.name!;
      map['app_version'] = OpenFoodAPIConfiguration.userAgent!.version!;
      map['app_uuid'] = OpenFoodAPIConfiguration.uuid!;

[monsieurtanuki]

@teolemon Actually looking back at the OP we don't call /cgi/display.pl from off-dart; maybe it's a redirection done on the server where we unfortunately dismiss the POST parameters.

[teolemon]

@monsieurtanuki is that ok for @cli1005 to work on this ? As we get Dart info in the logs, I believe it's a client side issue (eg Smoothie)

[monsieurtanuki]

@teolemon @cli1005 Sure, go ahead!

[cli1005]

The issue might be in Open Food Facts API... and the API accepts only app_name, app_version, app_uuid actually, if you want send more informations, like device system(ios/android) or comment (instead of what I did in #2262), we might have to modify the Open Food Facts API as well.

Another question, @teolemon could you please tell me how to find the log as you mentioned in the issue description? thanks

[monsieurtanuki]

@cli1005 Indeed. We should probably add something like app_system (e.g. 'android 12').
Beyond that, a more generic optional comment would make sense (e.g. scan). Or specific to barcode search, a source parameter (e.g. scan/search).
Don't know which would be the best solution as I don't see the full picture, but a generic dev_comment would be versatile.

[cli1005]

app_platform and comment have been added into UserAgentParameters (openfoodfacts/openfoodfacts-dart#482).
As monsieurtanuki mentioned, we do not call display.pl in OFF API, among all perl called, we do not sent UserAgentParameters for only 4 following PL:

• test.pl
• product_image_upload.pl (is it normal? why?)
• user.pl
• reset_password.pl

I could add Scan/ Search in parameter comment will sending in Smoothie, but it might be better to do it in OFF API...

[stephanegigandet]

Regarding using the user agent for a distinction between scan and search, it would mean changing the user agent for each query (ok, we have the technology for that) while it's supposed to be kind of static.

Hi @monsieurtanuki, having the app name + platform + source of query (scan, or search, or list) in the UserAgent is very useful server side as we get the UserAgent in the nginx + Apache logs directly, before we interpret GET or POST parameters.

This is used currently to get scan statistics from our app for products: we just parse nginx logs.

[stephanegigandet]

we do not call display.pl in OFF API,

It's an internal redirect server side, any call to /api/ will be handled by display.pl

[stephanegigandet]

we don't call /cgi/display.pl from off-dart; maybe it's a redirection done on the server where we unfortunately dismiss the POST parameters.

There is redirection server side done by a nginx reverse proxy. This proxy does not analyze the POST parameters, it just passes them along. But it is very useful for us to be able to identify our app, and scans, in this nginx reverse proxy.

[monsieurtanuki]

@stephanegigandet Ok then. We go a little beyond the normal use of User Agent, fair enough. A little problem is that the User Agent is static: that made sense to use a static field for characteristics that don't change often (e.g. device or app version), that's a bit lousy for something that is supposed to be changed at every request (e.g. 2 requests in parallel and the second request overwrites the first one's user agent). I guess we can live with that, it's not like one user scans two products at the very same time. With @cli1005's #484. now we have a "comment" field in user agent, so we are ok I guess:

  const UserAgent({
    this.name,
    this.version,
    this.system,
    this.url,
    this.comment,
  });