Configuration of the apps changed overnight

[pdehaye]

As detailed here, the configuration of the apps changed overnight.

In short: there no longer seems to be any attempt at leveraging Bluetooth as a proxy for distance, only duration counts, and pretty much all the beacons heard.

Has there been any public communication or explanation on the topic?

[immuniopensource]

Hi @pdehaye, thank you for your questions and your interest in the project. We strive to configure Immuni’s risk model to adhere as closely as possible to the requirements set forth by Italy’s Ministry of Health, in particular to the definition of “contatto stretto” (close contact): “a person that had a direct contact (face to face) with a COVID-19-positive person at a distance less than 2 meters and for at least 15 minutes.” Several experiments were conducted to calibrate the parameters of the algorithm. As more experiments are performed and we gather more information about the problem, it cannot be ruled out that the parameters will change again, just as they did on June 5.
By conducting additional physical experiments, using both iOS and Android devices, to observe the distribution of attenuation at varying distances and physical conditions (i.e., leaving the phone on a non-metallic table, or holding the device in a hand or a pocket), we concluded that when the devices were within two meters from each other, the majority of the attenuation measurements was distributed in the range of 50-70 dBm, and when the devices were at a distance greater than two meters, the the majority of the attenuation measurements was distributed in the range of 70-100 dBm. Based on our observations and on the instructions of the Ministry of Health, the threshold of 73 dBm was chosen to minimize false positives and false negatives.
As a final note, the exchange of Random Proximity Identifiers via Bluetooth Low Energy happens independently of those parameters and is controlled solely by the A/G Exposure Notification framework. The Technology Description document offers a thorough explanation of how the A/G Exposure Notification framework works.

[pdehaye]

By conducting additional physical experiments, using both iOS and Android devices, to observe the distribution of attenuation at varying distances and physical conditions (i.e., leaving the phone on a non-metallic table, or holding the device in a hand or a pocket), we concluded that when the devices were within two meters from each other, the majority of the attenuation measurements was distributed in the range of 50-70 dBm, and when the devices were at a distance greater than two meters, the the majority of the attenuation measurements was distributed in the range of 70-100 dBm. Based on our observations and on the instructions of the Ministry of Health, the threshold of 73 dBm was chosen to minimize false positives and false negatives.

There is a lot to unpack in the statement above, especially in conjunction with the data shared by the DP-3T team (and others).

Note that the Swiss team configures the app very differently: see the discussion here), or directly download their data from here.

While they do have a bad attenuation threshold at 73dB, their choice of higher threshold to be passed to the API is 55dB, not 73dB. This is a very substantial difference.

Statistics can definitely be misleading, so there is a distinct possibility that by considering the same data you would be led to vastly different configuration than the Swiss team, if you are optimizing for different functions (yours seems to be focused on just being better than pure luck). Alternatively you could somehow have obtained very different data, which naturally would lead you to different conclusions.

Hence my question to you: does your data look anything like the Swiss' ?

[pdehaye]

This apparently made some little news in Italy.

https://www.key4biz.it/immuni-modificati-i-parametri-dellalert-senza-una-comunicazione-istituzionale-ora-lapp-non-tiene-piu-conto-della-distanza/309449/

[Telefonorosso]

The "distance measurement" is still present in the official documentation.

https://github.com/immuni-app/immuni-documentation#product

@immunistaff @immuniteam correct that and issue an official statement of said modification.

[Seio]

By conducting additional physical experiments, using both iOS and Android devices, to observe the distribution of attenuation at varying distances and physical conditions (i.e., leaving the phone on a non-metallic table, or holding the device in a hand or a pocket), we concluded that when the devices were within two meters from each other, the majority of the attenuation measurements was distributed in the range of 50-70 dBm, and when the devices were at a distance greater than two meters, the the majority of the attenuation measurements was distributed in the range of 70-100 dBm. Based on our observations and on the instructions of the Ministry of Health, the threshold of 73 dBm was chosen to minimize false positives and false negatives.

How can you mathematically minimize false positve and false negative at the same time by acting on the threshold?

[alessandro-gentilini]

@Seio I assume that the attenuation measurements are distributed according to two Gaussian (or normal) probability density functions; I further assume that the given intervals cover three standard deviations (so called 3σ).
With these assumptions you can numerically compute the sum of the false positives and the false negatives and see that there is a minimum.
I am not able to give you a mathematical proof but just a Python simulation; I am not a mathematician but I think that assuming bell-shaped probability density functions it can be proved that the function false positives plus false negatives has a global unique minimum.

import math
import numpy as np
import scipy.special
import scipy.stats
import matplotlib.pyplot as plt

near_attenuation = [50,70]
far_attenuation = [70,100]

# mu is the average
mu_near = (near_attenuation[0]+near_attenuation[1])/2
# three sigma is the length of the interval
sigma_near = (near_attenuation[1]-near_attenuation[0])/3

mu_far = (far_attenuation[0]+far_attenuation[1])/2
sigma_far = (far_attenuation[1]-far_attenuation[0])/3

t = np.arange(10,200)

# https://mathworld.wolfram.com/NormalDistribution.html
# See formulae 8,9,10 in
# Weisstein, Eric W. "Normal Distribution." 
# From MathWorld--A Wolfram Web Resource. 
# https://mathworld.wolfram.com/NormalDistribution.html
def area_under_gaussian_at_left(t,mu,sigma):
    a = t-mu
    b = math.sqrt(2)*sigma
    return .5*(1+scipy.special.erf(a/b))

def area_under_gaussian_at_right(t,mu,sigma):
    return 1-area_under_gaussian_at_left(t,mu,sigma)

false_positive = area_under_gaussian_at_left(t,mu_far,sigma_far)
false_negative = area_under_gaussian_at_right(t,mu_near,sigma_near)

threshold = t[np.argmin(false_negative+false_positive)]

print('Threshold minimizing the sum of false positives and false negatives: {0}'.format(threshold))

fig, ax = plt.subplots(1)
ax.plot(t,scipy.stats.norm.pdf(t,mu_near,sigma_near),
t,scipy.stats.norm.pdf(t,mu_far,sigma_far)
)
ax.axvline(threshold,color='black')
ax.set_title('The black vertical line is the threshold that minimizes\nthe sum of false positives and negatives: {0}dBm'.format(threshold))
ax.set_xlabel('Attenuation/dBm')
ax.set_ylabel('Probability density')
ax.legend(['near','far'],loc='upper right')
fig.savefig('far_vs_near.png',bbox_inches='tight')

fig, ax = plt.subplots(1)
ax.plot(
t,false_positive,
t,false_negative,
t,false_negative+false_positive
)
ax.set_xlabel('Threshold value for Attenuation/dBm')
ax.set_ylabel('Proportion of falses (1 means 100%)')
ax.legend(['False positives','False negatives','False negatives plus false positives'],loc='upper right')
fig.savefig('falses.png',bbox_inches='tight')

In this simulation the threshold that minimizes the false positives and negatives is 71dBm.

The above is theory, in practice you can have a set of "far" attenuation measurements and a set of "near" attenuation measurements and you can apply a classifier like for example a Naive Bayes one (see https://scikit-learn.org/stable/modules/naive_bayes.html) in order to get a threshold.

[pdehaye]

@alessandro-gentilini that minimizes the sum of false positives and false negatives, not each simultaneously.

[vincenzoiovino]

Dear all,
is this correct that currently Immuni for Android flags as risk signal with attenuation below 63 dB (not 73) for >15 minutes?
This seems changed from July 09.
Why? Any rationale for this choice?

[vincenzoiovino]

Dear all,
is this correct that currently Immuni for Android flags as risk signal with attenuation below 63 dB (not 73) for >15 minutes?
This seems changed from July 09.
Why? Any rationale for this choice?

@immuniopensource , could you kindly clarify?