Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Risk scores are 0 on Android #79

Closed
havart-netcompany opened this issue Jun 10, 2020 · 10 comments · Fixed by xamarin/XamarinComponents#901
Closed

Risk scores are 0 on Android #79

havart-netcompany opened this issue Jun 10, 2020 · 10 comments · Fixed by xamarin/XamarinComponents#901

Comments

@havart-netcompany
Copy link

I think there is a problem with risk scores on Android. When ExposureNotificationHandler.ExposureDetectedAsync runs, I get a summary with matchedKeyCount 10, but all risk scores are 0. If I run getExposureInfo, I get a bunch of ExposureInfo objects with risk score 0. Doing the same on iOS, gives ExposureDetectionSummary and ExposureInfo objects with risk scores > 0.

Breakpoint here:
image

This is the summary object in Locals:
image

The screenshots are for 0.7.0, but it's the same problem on 0.8.0

@mattleibow
Copy link
Contributor

What were the scores when they were submitted to the OS (in the files from the server)? It should be using that. I am assuming you are using the same files for iOS and Android?

@havart-netcompany
Copy link
Author

havart-netcompany commented Jun 10, 2020

I'm not sure what scores you are referring to here. But in the .zip files from the server that we provide with submitBatches, there are keys I have been exposed to with various TransmissionRiskLevels, e.g. Medium and MediumLow (we know this because that's what I get when printing the ExposureInfo objects after running getExposureInfo).

iOS uses the same files, so the fact that it's only Android that gets 0 for the various RiskScores probably means it's lost somewhere between receiving it from Play Services and presenting it to our app in ExposureDetectedAsync

@havart-netcompany
Copy link
Author

havart-netcompany commented Jun 10, 2020

Just to show that the problem is still there on 0.8.0, there's summary in Locals at the same breakpoint:
image

@mattleibow
Copy link
Contributor

Well, thanks for reporting this. We will have to investigate this.

I am looking at the code now and we are just passing the value from Google to the shared code. It could very well be that the beta version of the API has a bug. It is almost 30 days old, and Apple is still changing code. Maybe we are due for an update from Google soon, or they are going stable... I'll try find things out and see where the issue lies.

@havart-netcompany
Copy link
Author

I think the 0 scores were because of the Configuration we were using. I changed to a default configuration, just with maxed out duration scores and min risk score 1, and was able to get:
image
image
I'll try to see if we can get rid of the anomalies we are seeing by adjusting the Configuration

@havart-netcompany
Copy link
Author

havart-netcompany commented Jun 11, 2020

Edit: This is not actually a bug because "1 matched key" does not mean "1 exposure incident". There were several exposure incidents to the same key on the same day

It is still unkown how we were able to record this anomaly, though:
image
1 matched key, MaximumRiskScore 64, SummationRiskScore 320. Do you suspect that this is a bug in Play Services @mattleibow ?

@havart-netcompany
Copy link
Author

I did 4 experiments with with breakpoints here:

  • The first line of ExposureNotificationHandler.ExposureDetectedAsync()
  • The first line after getting out the IEnumerable<ExposureInfo> with the getExposureInfo() parameter of ExposureNotificationHandler.ExposureDetectedAsync()

My current conclusion: On Android, the only thing Configuration.MinimumRiskScore does is to tell the EN API "if an exposure incident has TotalRiskScore less than MinimumRiskScore, then set TotalRiskScore to 0, but otherwise treat it exactly the same as other exposure incidents". On iOS, Configuration.MinimumRiskScore doesn't do anything.

The differences I observed between iOS and Android could just be because the EN API is different on each platform, or maybe the MinimumRiskScore is not provided correctly in the iOS part of Xamarin.ExposureNotification.

Experiment 1: Running on Android. Using a Configuration with the default values, except that we set Configuration.MinimumRiskScore to 4096
At the first breakpoint, Locals shows us this ExposureDetectionSummary:
image
At the second breakpoint, Locals shows us this IEnumerable<ExposureInfo>:
image

Experiment 2: Running on Android. Using a Configuration with the default values, except that we set Configuration.MinimumRiskScore to 1
ExposureDetectionSummary:
image
IEnumerable<ExposureInfo>:
image

Experiment 3: Running on iOS. Using a Configuration with the default values, except that we set Configuration.MinimumRiskScore to 4096
image
IEnumerable<ExposureInfo>:
image

Experiment 4: Running on iOS. Using a Configuration with the default values, except that we set Configuration.MinimumRiskScore to 1
ExposureDetectionSummary:
image
IEnumerable<ExposureInfo>:
image

@mattleibow
Copy link
Contributor

mattleibow commented Jun 11, 2020

There is a new xcode beta out... you could see what ios did there...

Maybe they had a bug and fixed it...

Also, we haven't heard anything from Android in a long time...

@havart-netcompany
Copy link
Author

havart-netcompany commented Jun 11, 2020

Oh no... iOS minimum risk score is also capped at 255. I saw this thing in the documentation for minimum risk score:
image
So I think we need a nice fix like this one: #81

@havart-netcompany
Copy link
Author

My new current conclusion: iOS exposure incidents with risk score below the threshold are actually ignored except that they count towards "matched key count", while on Android, it's just the risk score in an exposure incident below the threshold that gets ignored. This is probably not a problem in Xamarin.ExposureNotification

Experiment 5: Running on iOS. Using a Configuration with AttenuationScores, AttenuationScores, DurationScores, TransmissionRiskScores all being integer arrays containing {2,2,2,2,2,2,2,2}, and with minimum risk score 255.
ExposureDetectionSummary:
image
IEnumerable<ExposureInfo>:
image

Experiment 6: Same as experiment 5, but minimum risk score = 1.
ExposureDetectionSummary:
image
IEnumerable<ExposureInfo>:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants