Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

th: missing cookie jar leads to twitter.com failure #2488

Closed
bassosimone opened this issue Jun 2, 2023 · 3 comments
Closed

th: missing cookie jar leads to twitter.com failure #2488

bassosimone opened this issue Jun 2, 2023 · 3 comments
Assignees
@bassosimone
Labels
bug Something isn't working data quality ooni/probe-engine priority/high

Comments

@bassosimone
Copy link
Contributor

Describe the bug

The lack of a cookie jar leads to twitter.com failures in the TH. Other URL may also fail. We just noticed this behavior for twitter.com. Here is a measurement that failed. We noticed this for Russia but I was able to reproduce the same problem by running the TH locally.

What is puzzling is that we do not see this failure for measurements run in, say, Italy. Why is that? The answer is that the probe ignores the TH result for Web Connectivity v0.4 when it could successfully use URL scheme is HTTPS. Take a look, for example, at this measurement. You would see that the x_status is equal to 1, which means StatusSuccessSecure. If you check the analysis code, you would see that we set the result as accessible when the URL is HTTPS and we succeded.

To Reproduce

It suffices to run ./miniooni -i https://twitter.com urlgetter and inspect the measurement. See, for example, this measurement that I have collected. You would see that the first request redirects us to / with a cookie set.

Expected behavior

The TH should use a cookie jar.

Screenshots

N/A

System information (if applicable):

Not relevant.

Additional context

None.
@bassosimone bassosimone self-assigned this Jun 2, 2023
bassosimone added a commit to ooni/probe-cli that referenced this issue Jun 2, 2023
@bassosimone
fix(oohelperd): use cookiejar for HTTP/HTTPS/HTTP2/HTTP3 measurements
11597c6 
See ooni/probe#2488
This was referenced Jun 2, 2023
Merged
Closed
bassosimone added a commit to ooni/probe-cli that referenced this issue Jun 5, 2023
@bassosimone
fix(oohelperd): use cookiejar for HTTP measurements (#1149)
6aceca1 
See ooni/probe#2488.

This patch needs to be backported to the release/3.17 branch.
bassosimone added a commit to ooni/probe-cli that referenced this issue Jun 5, 2023
@bassosimone
[backport] fix(oohelperd): use cookiejar for HTTP measurements (#1149)
a3af554 
See ooni/probe#2488.

This patch backports 6aceca1 to
the release/3.17 branch.
@bassosimone
Copy link
Contributor Author

I have run some A/B testing and did not see any big regression with respect to the previous TH version.

@bassosimone
Copy link
Contributor Author

We cannot close the issue yet, because we still need to roll out the change more widely.

@bassosimone
Copy link
Contributor Author

We can close this issue now!

cyBerta pushed a commit to cyBerta/probe-cli that referenced this issue Aug 4, 2023
@bassosimone @cyBerta
fix(oohelperd): use cookiejar for HTTP measurements (ooni#1149)
f6ded68 
See ooni/probe#2488.

This patch needs to be backported to the release/3.17 branch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bassosimone bassosimone

Labels
bug Something isn't working data quality ooni/probe-engine priority/high
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@bassosimone