Add validation of GSM / UMTS cell ids ranges

[hannosch]

We currently have some GSM cells with cell ids larger than 64k and some UMTS cells with cell ids lower than 64k.

In both of these cases the radio type is likely wrong. Many older Android chipsets don't properly distinguish between GSM/UMTS and so we end up getting the wrong type.

We should normalize the radio type based on the cell id value. This applies both to the submission data as well as the search interface and needs a migration of existing cells.

[Gitschubser]

I have do a simple check with the file MLS-full-cell-export-2014-10-23T000000.csv.

Radio Type GSM:
24892 cells have the radio type GSM and a cellid >65535

-> 7721 cells have only 1 sample

Radio Type UMTS:
19906 have the radio type UMTS and a cellid <65536

-> 6638 cells have only 1 sample

Maybe there are 44798 cellids wrong or doesn't exist?

A good task could validate the data (only a short example from me, maybe there are another ideas)

GSM

1. Check if radio type is GSM and the cellid has a range between 0 and 65535 (<65536).
2. If yes, radio type is correct, if no data is false.
3. If false, check if the cellid (mcc_mnc_lac_cellid) exists with another radio type (duplicate)
4. If cellid exists with another radio type, delete the wrong data (how we find the wrong data?)
5. If cellid exists not with another radio type change to the right radio type.

UMTS

1. Check if radio type is UMTS and the cellid is >65535.
2. If yes, radio type is correct, if no radio type is false.
3. If false, check if the cellid (mcc_mnc_lac_cellid) exists with another radio type (duplicate)
4. If cellid exists with another radio type, delete the wrong data (how we find the wrong data?)
5. If cellid exists not with another radio type change to the right radio type.

Maybe i have not regard all possibilities.

[E3V3A]

From Open Cell ID site:

CID     must be in the range of 1 to 65535 for GSM
CID     must be in the range of 1 to 4294967295 for UMTS
CID     must be in the range 1 to 268435455 for LTE
CID/BID     must be in the range 1 to 65535 for CDMA 

[jas19]

How can we check the missing cell_id counts while migrating the data from one source to another ?
I am getting 10000 events on the target one but the actual records should be 30000.
How to check this ?