Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

webconnectivity: x_status is 16384 with android_dns_cache_no_data #2499

Closed
bassosimone opened this issue Jul 4, 2023 · 1 comment
Closed

webconnectivity: x_status is 16384 with android_dns_cache_no_data #2499

bassosimone opened this issue Jul 4, 2023 · 1 comment
Assignees
@bassosimone
Labels
bug Something isn't working data quality methodology issues related to the testing methodology needs investigation This issue needs extra data and investigation ooni/probe-engine platform/android priority/high releaseBlocker This issue blocks releasing

Comments

@bassosimone
Copy link
Member

bassosimone commented Jul 4, 2023 •
edited
Loading

See for example https://explorer.ooni.org/chart/mat?probe_cc=PK&since=2023-03-05&time_grain=day&axis_x=measurement_start_day&test_name=web_connectivity&domain=www.dw.com. The issue here is that:

  1. the Android resolver is notoriously broken and does not report the real error why it failed, but it just says "no data";
  2. this fact causes a check for DNS failures to be skipped because the Android resolver is treated specially;
  3. so we end up wondering why we don't have requests and set the 16384 value for x_status.

I suspect switching users to Web Connectivity LTE would fix the issue. Fixing the issue with Web Connectivity v0.4 is possible but also dubious because we cannot be sure about what the system resolver is hiding from us. We should investigate whether using Web Connectivity LTE fixes the issue.

Another approach would be to call connectivityManager.getLinkProperties method and then call the linkProperties.getDnsServers method to get the DNS resolvers IP addresses.
@bassosimone bassosimone added bug Something isn't working priority/high platform/android methodology issues related to the testing methodology data quality needs investigation This issue needs extra data and investigation ooni/probe-engine labels Jul 4, 2023
@bassosimone bassosimone self-assigned this Jul 4, 2023
@FedericoCeratto FedericoCeratto mentioned this issue Jul 5, 2023
Closed
@bassosimone bassosimone mentioned this issue Aug 24, 2023
Merged
bassosimone added a commit to ooni/probe-cli that referenced this issue Aug 24, 2023
@bassosimone
feat(webconnectivityqa): add test for android_dns_cache_no_data (#1210)
eb4f82b 
Part of ooni/probe#2029. The general idea is
to modify v0.4 in a subsequent PR to make it WAI for this test case.
Currently v0.4 does not correctly support this case, which is why we're
doing this work. (See also ooni/probe#2499).
bassosimone added a commit to ooni/probe-cli that referenced this issue Aug 24, 2023
@bassosimone
fix(webconnectivity): handle android_dns_cache_no_data as dns anomaly
23b4371 
This diff changes the handling of android_dns_cache_no_data to mark it and
any other DNS inconsistenct as DNS anomaly.

The previous implementation was not handling any DNS inconsistency as an
anomaly, therefore I needed to adjust a bunch of summary_test.go tests.

Part of ooni/probe#2499 and
ooni/probe#2029.

I bumped the version number because this is a significant change
in the analysis algorithm implemented by the probe.
@bassosimone bassosimone mentioned this issue Aug 24, 2023
Merged
4 tasks
@bassosimone bassosimone added the releaseBlocker This issue blocks releasing label Oct 4, 2023
bassosimone added a commit to ooni/probe-cli that referenced this issue Oct 10, 2023
@bassosimone
fix: anomaly with android_dns_cache_no_data and inconsistent dns (#1211)
29e8c88 
## Checklist

- [x] I have read the [contribution
guidelines](https://github.com/ooni/probe-cli/blob/master/CONTRIBUTING.md)
- [x] reference issue for this pull request:
ooni/probe#2499
- [x] if you changed anything related to how experiments work and you
need to reflect these changes in the ooni/spec repository, please link
to the related ooni/spec pull request: N/A
- [x] if you changed code inside an experiment, make sure you bump its
version number

## Summary

This diff changes the `summary.go` algorithm of Web Connectivity v0.4 to
handle `android_dns_cache_no_data` as an anomaly when the DNS is
inconsistent. We continue handling `dns_nxdomain_error` as an anomaly
when the DNS is inconsistent, as demonstrated by the fact that the
corresponding netem test is still passing.

This diff also bumps the version number to v0.4.3. Version v0.4.2 did
not handle this case, which caused measurements to be marked as failed
as documented by ooni/probe#2499.

This diff is also related to ooni/probe#2029,
in the sense that it is slightly improving our analysis results when the
is an NXDOMAIN error (even if it's masked by Android's DNS cache
behavior).

While there, add empty lines to improve the code readability.
@bassosimone
Copy link
Member Author

Done in ooni/probe-cli#1211

Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
feat(webconnectivityqa): add test for android_dns_cache_no_data (ooni…
b735b3d 
…#1210)

Part of ooni/probe#2029. The general idea is
to modify v0.4 in a subsequent PR to make it WAI for this test case.
Currently v0.4 does not correctly support this case, which is why we're
doing this work. (See also ooni/probe#2499).
Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
fix: anomaly with android_dns_cache_no_data and inconsistent dns (oon…
6e9a9ac 
…i#1211)

## Checklist

- [x] I have read the [contribution
guidelines](https://github.com/ooni/probe-cli/blob/master/CONTRIBUTING.md)
- [x] reference issue for this pull request:
ooni/probe#2499
- [x] if you changed anything related to how experiments work and you
need to reflect these changes in the ooni/spec repository, please link
to the related ooni/spec pull request: N/A
- [x] if you changed code inside an experiment, make sure you bump its
version number

## Summary

This diff changes the `summary.go` algorithm of Web Connectivity v0.4 to
handle `android_dns_cache_no_data` as an anomaly when the DNS is
inconsistent. We continue handling `dns_nxdomain_error` as an anomaly
when the DNS is inconsistent, as demonstrated by the fact that the
corresponding netem test is still passing.

This diff also bumps the version number to v0.4.3. Version v0.4.2 did
not handle this case, which caused measurements to be marked as failed
as documented by ooni/probe#2499.

This diff is also related to ooni/probe#2029,
in the sense that it is slightly improving our analysis results when the
is an NXDOMAIN error (even if it's masked by Android's DNS cache
behavior).

While there, add empty lines to improve the code readability.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bassosimone bassosimone

Labels
bug Something isn't working data quality methodology issues related to the testing methodology needs investigation This issue needs extra data and investigation ooni/probe-engine platform/android priority/high releaseBlocker This issue blocks releasing
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@bassosimone