-
Notifications
You must be signed in to change notification settings - Fork 142
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
failure: "json_processing_error" #911
Comments
Another similar failure: Logs:
|
Another report from AS197207 sent by the same user: I think there are two different json errors on the server side and on the client side. |
On the client side, the app mistakenly detects HTML (Probably the content sent by the censorship system) as object and wants to parse it with json. ( I guess)
|
@xhdix yes, I believe your analysis is right. |
To reproduce this issue locally, I did the following:
import socket
import sys
import os
import time
HOST, PORT = '127.0.0.1', 7001
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(10)
conn, addr = s.accept()
conn.recv(65536)
conn.sendall("HTTP/1.0 503 Service Unavailable\r\n")
conn.sendall("Connection: close\r\n\r\n")
conn.sendall("""
<HTML><HEAD>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>503 Service Unavailable</TITLE>
</HEAD><BODY><H1>503 Service Unavailable</H1></BODY></HTML>""")
s.close()
measurement_kit -vn http_header_field_manipulation -b http://127.0.0.1:7001
{
"annotations": {
"engine_name": "libmeasurement_kit",
"engine_version": "0.10.3",
"engine_version_full": "v0.10.3",
"failure_asn_lookup": "true",
"failure_cc_lookup": "true",
"failure_network_name_lookup": "true",
"platform": "macos"
},
"data_format_version": "0.2.0",
"id": "27b7b4b7-2584-432c-a742-40bb19eff1c4",
"input": null,
"input_hashes": [],
"measurement_start_time": "2019-05-23 08:39:10",
"options": [],
"probe_asn": "AS0",
"probe_cc": "ZZ",
"probe_city": null,
"probe_ip": "127.0.0.1",
"report_id": "",
"software_name": "measurement_kit",
"software_version": "0.10.3",
"test_helpers": {
"backend": "http://127.0.0.1:7001"
},
"test_keys": {
"agent": "agent",
"client_resolver": "91.80.37.84",
"failure": null,
"requests": [
{
"failure": null,
"request": {
"body": "",
"headers": {
"UsER-agEnT": "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2) Gecko/20100115 Firefox/3.6",
"aCCEPT": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"aCCePt-enCODIng": "gzip,deflate,sdch",
"accEPT-CHarSet": "ISO-8859-1,utf-8;q=0.7,*;q=0.3",
"accepT-LAnguaGE": "en-US,en;q=0.8",
"hoST": "5iTz7F7FiPo99SC.com"
},
"method": "GET",
"tor": {
"exit_ip": null,
"exit_name": null,
"is_tor": false
},
"url": "http://127.0.0.1:7001/"
},
"response": {
"body": "\n <HTML><HEAD>\n <meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n <TITLE>503 Service Unavailable</TITLE>\n </HEAD><BODY><H1>503 Service Unavailable</H1></BODY></HTML>",
"code": 503,
"headers": {
"Connection": "close"
},
"response_line": "HTTP/1.0 503 Service Unavailable"
}
}
],
"socksproxy": null,
"tampering": {
"header_field_name": null,
"header_name_diff": null,
"request_line_capitalization": true,
"total": true
}
},
"test_name": "http_header_field_manipulation",
"test_runtime": 0.006891012191772461,
"test_start_time": "2019-05-23 08:39:09",
"test_version": "0.0.1"
} The relevant part of the JSON seems to me in line with the report that @xhdix shared while the log messages seem in line with what @xhdix reported. I believe the behaviour of MK is correct in this case, modulo the need to check whether the spec says what should be the value of the tampering entry in case the response is not JSON. It may also be that we could do a better job at clearly flagging a result like this as tampering (even though On the contrary, the original issue reported by @anadahz looks more problematic since it seems to happen really early, and specifically so early that we don't even know the test name (!?). |
Follow-up checks (to become maybe integration tests):
|
Now, regarding the first measurement, here's what I did:
curl "https://api.ooni.io/api/v1/measurements?report_id=20190428T192329Z_AS197207_whIbylNGE3kRug3AJEnC9qQTiZJeF8wKjnUqZY3VD5LpaAZWLW"
to obtain:
```json
{
"metadata": {
"count": 1,
"current_page": 1,
"limit": 100,
"next_url": null,
"offset": 0,
"pages": 1,
"query_time": 0.00869607925415039
},
"results": [
{
"anomaly": false,
"confirmed": false,
"failure": true,
"input": null,
"measurement_id": "temp-id-296207424",
"measurement_start_time": "2019-04-28T19:23:27Z",
"measurement_url": "https://api.ooni.io/api/v1/measurement/temp-id-296207424",
"probe_asn": "AS197207",
"probe_cc": "IR",
"report_id": "20190428T192329Z_AS197207_whIbylNGE3kRug3AJEnC9qQTiZJeF8wKjnUqZY3VD5LpaAZWLW",
"test_name": "dash"
}
]
}
to obtain: {
"annotations": {
"engine_name": "libmeasurement_kit",
"engine_version": "0.9.3",
"engine_version_full": "v0.9.3",
"flavor": "fdroid",
"network_type": "mobile",
"platform": "android"
},
"backend_version": null,
"bucket_date": "2019-04-28",
"data_format_version": "0.2.0",
"id": "35730c9e-9554-4aba-96fb-4f960e2ce31f",
"input": null,
"input_hashes": [],
"measurement_start_time": "2019-04-28 19:23:27",
"options": [],
"probe_asn": "AS197207",
"probe_cc": "IR",
"probe_city": null,
"probe_ip": "127.0.0.1",
"report_filename": "2019-04-28/20190428T192323Z-IR-AS197207-dash-20190428T192329Z_AS197207_whIbylNGE3kRug3AJEnC9qQTiZJeF8wKjnUqZY3VD5LpaAZWLW-0.2.0-probe.json",
"report_id": "20190428T192329Z_AS197207_whIbylNGE3kRug3AJEnC9qQTiZJeF8wKjnUqZY3VD5LpaAZWLW",
"software_name": "ooniprobe-android",
"software_version": "2.0.3",
"test_helpers": {},
"test_keys": {
"client_resolver": "2.188.21.131",
"failure": "json_processing_error"
},
"test_name": "dash",
"test_runtime": 0.515253067,
"test_start_time": "2019-04-28 19:23:23",
"test_version": "0.7.0"
}
Where the script is now: import socket
import sys
import os
import time
HOST, PORT = '127.0.0.1', 7007
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(10)
conn, addr = s.accept()
conn.recv(65536)
conn.sendall("HTTP/1.0 200 OK\r\n")
conn.sendall("Connection: close\r\n\r\n")
conn.sendall("""
<HTML><HEAD>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>503 Service Unavailable</TITLE>
</HEAD><BODY><H1>503 Service Unavailable</H1></BODY></HTML>""")
s.close() and the test keys matches the one in the original report. |
To conclude, I believe there may be app related actions to better visualise these failure, but it seems to me that the behaviour we see is consistent with the spirit of these two tests. @lorenzoPrimi do we have a way to see what the app shows given a specific JSON? @sarathms same question but for Explorer (maybe the next version not the legacy one) |
Meanwhile, I'll close the related MK bug as I don't think there's an MK bug here. |
Moving to the icebox because no-one is assigned. |
Added wontfix label because this is an MK bug that we are not going to fix. We'll do better with the new Go engine, and we'll keep MK unchanged, so it will have this bug. |
I guess this issue can be closed. |
A strange measurement reported by @xhdix in #ooni:
https://explorer.ooni.io/measurement/20190428T192329Z_AS197207_whIbylNGE3kRug3AJEnC9qQTiZJeF8wKjnUqZY3VD5LpaAZWLW
Not sure if this is the right place to report this issue.
Specifications
The text was updated successfully, but these errors were encountered: