Several oonimeasurements bugs #1011
Description
This issue will list some minor oonimeasurements bugs we found browsing AWS logs
1. Bad datetime parsing
There seems to be an error when parsing since and until datetimes from the request in list_measurements:
Example request to reproduce:
curl https://backend-fsn.ooni.org:443/api/v1/measurements?probe_cc=SY&since=2025-09-29T00:00:00&until=2025-09-29T23:59:59&limit=2000&since_index=20250910T105502Z_tor_SY_29256_n1_C8NFgxmJpyaP5Bsd
Server logs:
File "/usr/local/lib/python3.11/site-packages/oonimeasurements/routers/v1/measurements.py", line 687, in list_measurements
until_dt = datetime.strptime(until, "%Y-%m-%d")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/_strptime.py", line 567, in _strptime_datetime
tt, fraction, gmtoff_fraction = _strptime(data_string, format)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/_strptime.py", line 352, in _strptime
raise ValueError("unconverted data remains: %s" %
ValueError: unconverted data remains: T23:59:59
INFO: 10.0.1.210:30530 - "GET /api/v1/measurements?probe_cc=SY&since=2025-09-29T00:00:00&until=2025-09-29T23:59:59&limit=2000&since_index=20250910T105502Z_tor_SY_29256_n1_C8NFgxmJpyaP5Bsd HTTP/1.1" 500 Internal Server Error
2. Typing error in measurement_meta
There's a typing assertion failing when requesting a measurement meta:
29 September 2025, 09:21
INFO: 10.0.0.40:52346 - "GET /api/v1/measurement_meta?report_id=20241209T185959Z_openvpn_MZ_37223_n1_x51ewx3QwQcqXNgW&full=true&input=openvpn:%2F%2Foonivpn.corp%3Faddress%3D37.218.243.98%253A1194 HTTP/1.1" 200 OK
ERROR:oonimeasurements.routers.v1.measurements:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/oonimeasurements/routers/v1/measurements.py", line 490, in get_measurement_meta
assert isinstance(msmt_meta.report_id, str) and isinstance(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This happens bc sometimes the measurement could not be found, but somehow still passes the not-found check here
The final result is actually correct but for the wrong reasons
Example request:
curl https://backend-fsn.ooni.org:443/api/v1/measurement_meta?report_id=20241023T063837Z_openvpn_US_10796_n1_q2mw3znrlqNNgfJ0&full=true&input=openvpn:%2F%2Foonivpn.corp%2F%3Faddress%3D37.218.243.98:1194
3. Another typing issue
There's another typing bug in the same function:
File "/usr/local/lib/python3.11/site-packages/oonimeasurements/routers/v1/measurements.py", line 496, in get_measurement_meta
assert isinstance(body, bytes)
AssertionError
This is because the measurement_meta function expects the result of _fetch_measurement_body to be bytes, but is already converted to string:
This is a silent error, any full=true request to /api/v1/measurement_meta will trigger it and the results will be valid by pure chance, but an error will be logged
4. Inconsistent behavior against the monolith
While we were investigating the previous issues we noticed that the probe_asn encoding is different between the monolith and the microservice. This is an error. For example, running this in the monolith:
curl -s 'https://api.ooni.org/api/v1/measurement_meta?measurement_uid=20250930131336.512528_NZ_browserweb_72527abda374191b&full=true' > new-api.txt
curl -s 'localhost:8000/api/v1/measurement_meta?measurement_uid=20250930131336.512528_NZ_browserweb_72527abda374191b&full=true' > monolith.txt
diff <(jq --sort-keys . monolith.txt) <(jq --sort-keys . new-api.txt)
we gett
7c7
< "measurement_start_time": "2025-09-30T13:13:35Z",
---
> "measurement_start_time": "2025-09-30T13:13:35.000000Z",
9c9
< "probe_asn": 136557,
---
> "probe_asn": "136557",
15c15
< "test_start_time": "2025-09-30T13:12:20Z"
---
> "test_start_time": "2025-09-30T13:12:20.000000Z"
The time stamp issue is not so bad butt the probe_asn being an int in the monolith is an issue that should be fixed