How to handle null-AIDs

[sebastian]

This is branching off from: #21

In his comment, @yoid2000 asked some questions. I'll add a comment to his question #2 here.

Regarding 2, can we avoid this question by always knowing what the actual AID is?

The AID would be null if the value is missing in the dataset, rather than as an artefact of the query itself. So it will in fact not always be possible to know what an AID is. In Aircloak Insights we explicitly filtered out the data where there was no AID value present.

In practice, this will lead to severe data loss for certain datasets (I saw this a couple of times with Aircloak insights), but in all the instances that I encountered the problem would not have been solved by including this data. Had it all been represented by a null-AID it would have been suppressed as an extreme outlier. Had it been assigning random AID values in place of the null-value, then we would have lost all ability to make any claims about the resulting anonymity. This all leads me to think we must drop null-AIDed data in Open Diffix too.

[edongashi]

A user could create a view to fallback nulls to something which makes sense for them.

This all leads me to think we must drop null-AIDed data in Open Diffix too.

Agreed. We shouldn't assume anything and ignore null AIDs.

[cristianberneanu]

After checking the code, it seems to me that NULL AIDs are already ignored, so this can be closed.