Skip to content

Releases: ooni/probe-cli

OONI Probe CLI v3.22.0

22 May 09:10
@github-actions github-actions
v3.22.0
This tag was signed with the committer’s verified signature. The key has expired.
bassosimone Simone Basso
GPG key ID: 80B691277733D95B
Expired
Learn about vigilant mode.
97f1df3
Compare
Choose a tag to compare
OONI Probe CLI v3.22.0 Latest
Latest

This is a stable release. As usual, this release includes both an OONI Probe CLI release and the corresponding release of the measurement engine, which is hosted by the same repository.

Here's a thematic summary of what changed since v3.21.0:

  • ✨ Preparing Web Connectivity LTE for use in production
  • 🐛 Prioritize DNS results when communicating with backends
  • 🚧 Refactor and document communication with the OONI backend
  • 📖 Documenting the release process
  • 🚧 Minor changes & bug fixes

More in detail, here's what has changed:

✨ Preparing Web Connectivity LTE for use in production

We worked towards closing all the open Web Connectivity LTE issues, such that we can finally be well placed to run an A/B testing between the v0.4 and LTE using this version of the OONI Probe Engine.

🐛 Prioritize DNS results when communicating with backends

We documented the strategy to communicate with backends, including the OONI backend, and we discussed the design as part of the #1552. During such a review, it was requested to modify the policy with which we communicate with the backends, so that the DNS results have higher priority. The main concern of this change request was to avoid long recovery periods if the currently deployed bridge becomes unavailable as well as to tilt the overall design such that uncensored OONI Probe users communicate with the backends in the simplest possible way.

🚧 Refactor and document communication with the OONI backend

We wanted to document the interaction with the OONI backend and specifically with the "probe services" (i.e., the collection of APIs used by OONI Probe), per ooni/probe#2700. To this end, we seized the opportunity to simplify the code interactions, reduce the number of used packages, and refactor the code to improve its reliability and future usability.

📖 Documenting the release process

We were asked to document the release process in ooni/probe#2698, and the resulting work is this:

🚧 Releasing & maintenance

This set of changes deal with preparing a release. In particular, in this case, we took on the chore of reducing the number of warnings emitted by gosec, such that its output can actually be informative and actionable (there was lots of noise). In terms of behavioral changes in the ooniprobe binary, these are the two most "impactful" changes:

  1. I explicitly ignore errors when resetting with -f the OONI_HOME directory, to keep the previous idempotent behavior, because I think that otherwise the desktop app may emit errors if asked to delete the OONI_HOME multiple times: https://github.com/ooni/probe-cli/pull/1603/files#diff-2e841e5388e3f75b0ab808e74d5b34a9ebb72466a508daecda35625a7eab5c7fR29

  2. I have improved how we manage survey errors: https://github.com/ooni/probe-cli/pull/1603/files#diff-3e09d6a095d686f531f06efe21b6e430b311340af5ce74989f110dc05424cbcdR24 and #1608

  3. I am now reporting an error if we fail to update the database to set the upload status: https://github.com/ooni/probe-cli/pull/1603/files#diff-2a4cd822f78eb174dc7c6b88e3ab71be4810624b358b98109c10bcb5371e4844R249

  4. I have now configured timeouts for receiving headers in oohelperd: https://github.com/ooni/probe-cli/pull/1603/files#diff-663f720a0015bc642e1957e965f9bc9115cea500fc0268bf5a6e22b5ad7cb44dR111

Here's the full list of pull requests:

Read more

Contributors

bassosimone, aanorbel, and 3 other contributors
Assets 45

OONI Probe CLI v3.22.0-alpha

15 May 07:56
@github-actions github-actions
v3.22.0-alpha
This tag was signed with the committer’s verified signature. The key has expired.
bassosimone Simone Basso
GPG key ID: 80B691277733D95B
Expired
Learn about vigilant mode.
ad70998
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
OONI Probe CLI v3.22.0-alpha Pre-release
Pre-release

This is an alpha release for creating a branching point for the release/3.22 branch. As usual, this release includes both an OONI Probe CLI release and the corresponding release of the measurement engine, which is hosted by the same repository.

Here's a thematic summary of what changed since v3.21.0-alpha:

  • ✨ Preparing Web Connectivity LTE for use in production
  • 🐛 Prioritize DNS results when communicating with backends
  • 🚧 Refactor and document communication with the OONI backend
  • 📖 Documenting the release process
  • 🚧 Minor changes & bug fixes

In terms of testing requirements, this release mostly changed the interaction with backend services, so we need

  1. QA testing of ooniprobe and miniooni operating in normal and restricted operating conditions, to see how the new algorithms are faring, collect feedback, and plan for future changes (or revert patches if needed).

  2. We should also probably test whether we're able to trigger the Web Connectivity LTE A/B testing using the check-in API, so we're actually able to perform the A/B testing 😅.

  3. We should also test the new oohelperd in production, for now the code has a generous 8 seconds timeout for receiving HTTP headers rather than leaving the timeout unbounded (we should be fine, but we should nonetheless test).

  4. We should ensure in the desktop app that we can reset the OONI_HOME multiple times without errors.

  5. We should test in the CLI that the "panics" added for handling survey errors have no practical implication unless something very weird like not being able to write the standard output happens.

More in detail, here's what has changed:

✨ Preparing Web Connectivity LTE for use in production

We worked towards closing all the open Web Connectivity LTE issues, such that we can finally be well placed to run an A/B testing between the v0.4 and LTE using this version of the OONI Probe Engine.

🐛 Prioritize DNS results when communicating with backends

We documented the strategy to communicate with backends, including the OONI backend, and we discussed the design as part of the #1552. During such a review, it was requested to modify the policy with which we communicate with the backends, so that the DNS results have higher priority. The main concern of this change request was to avoid long recovery periods if the currently deployed bridge becomes unavailable as well as to tilt the overall design such that uncensored OONI Probe users communicate with the backends in the simplest possible way.

🚧 Refactor and document communication with the OONI backend

We wanted to document the interaction with the OONI backend and specifically with the "probe services" (i.e., the collection of APIs used by OONI Probe), per ooni/probe#2700. To this end, we seized the opportunity to simplify the code interactions, reduce the number of used packages, and refactor the code to improve its reliability and future usability.

📖 Documenting the release process

We were asked to document the release process in ooni/probe#2698, and the resulting work is this:

🚧 Releasing & maintenance

This set of changes deal with preparing a release. In particular, in this case, we took on the chore of reducing the number of warnings emitted by gosec, such that its output can actually be informative and actionable (there was lots of noise). In terms of behavioral changes in the ooniprobe binary, these are the two most "impactful" changes:

  1. I explicitly ignore errors when resetting with -f the OONI_HOME directory, to keep the previous idempotent behavior, because I think that otherwise the desktop app may emit errors if asked to delete the OONI_HOME multiple times: https://github.com/ooni/probe-cli/pull/1603/files#diff-2e841e5388e3f75b0ab808e74d5b34a9ebb72466a508daecda35625a7eab5c7fR29

  2. I am now panicking if survey fails (which should not happen unless we misconfigure survey in the code or we're not able to write to the standard output): https://github.com/ooni/probe-cli/pull/1603/files#diff-3e09d6a095d686f531f06efe21b6e430b311340af5ce74989f110dc05424cbcdR24

  3. I am now reporting an error if we fail to update the database to set the upload status: https://github.com/ooni/probe-cli/pull/1603/files#diff-2a4cd822f78eb174dc7c6b88e3ab71be4810624b358b98109c10...

Read more

Contributors

bassosimone, aanorbel, and 3 other contributors
Assets 45
Loading

OONI Probe CLI v3.21.1

05 Apr 05:59
@github-actions github-actions
v3.21.1
This tag was signed with the committer’s verified signature. The key has expired.
bassosimone Simone Basso
GPG key ID: 80B691277733D95B
Expired
Learn about vigilant mode.
028d7ac
Compare
Choose a tag to compare
OONI Probe CLI v3.21.1

This is a stable release. On a high-level, this is what changed since 3.19:

  • ✨ 🐛 🚧 Web Connectivity LTE improvements
  • ✨ 🐛 🚧 Web Connectivity Test Helper improvements
  • 🐛 TorSf experiment: fixed data quality issue
  • ✨ Add the ./script/go.bash script to help development
  • 🚧 OONI Run v2: more flexible Engine<->App interface
  • ✨ Adding Wikimedia DNS-over-HTTPS resolver
  • 🚧 Removing the run experiment
  • 🚧 Improve SummaryKeys management
  • 🚧 Updating dependencies
  • 🚧 Minor Miscellaneous Maintenance
  • 🐛 Minor Miscellaneous Bug Fixes

For more details, please see OONI Probe CLI v3.21.0's ChangeLog.

Since v3.21.0 we additionally:

  • ✨ backport: feat(miniooni): add --software-name and --software-version 294ce23

This release does not include a linux/armv6 binary because of ooni/probe#2694.

Assets 43
Loading

OONI Probe CLI v3.21.0

27 Mar 12:29
@github-actions github-actions
v3.21.0
This tag was signed with the committer’s verified signature. The key has expired.
bassosimone Simone Basso
GPG key ID: 80B691277733D95B
Expired
Learn about vigilant mode.
df7f2fd
Compare
Choose a tag to compare
OONI Probe CLI v3.21.0

This is a stable release. On a high-level, this is what changed since 3.19:

  • ✨ 🐛 🚧 Web Connectivity LTE improvements
  • ✨ 🐛 🚧 Web Connectivity Test Helper improvements
  • 🐛 TorSf experiment: fixed data quality issue
  • ✨ Add the ./script/go.bash script to help development
  • 🚧 OONI Run v2: more flexible Engine<->App interface
  • ✨ Adding Wikimedia DNS-over-HTTPS resolver
  • 🚧 Removing the run experiment
  • 🚧 Improve SummaryKeys management
  • 🚧 Updating dependencies
  • 🚧 Minor Miscellaneous Maintenance
  • 🐛 Minor Miscellaneous Bug Fixes

More in detail:

✨ 🐛 🚧 Web Connectivity LTE improvements

We made sure that Web Connectivity LTE handles internationalized domain names correctly:

We improved Web Connectivity LTE to correctly handle malformed URLs:

We ensured that we correctly handle measurements containing loopback addresses:

  • fix(webconnectivitylte): handle measurements with loopback addrs by @bassosimone in #1462

We fixed a bug where Web Connectivity LTE was not correctly counting bytes sent and received:

We ensured that Web Connectivity LTE correctly handle cases of websites with missing A/AAAA addresses:

  • fix(webconnectivitylte): handle domains w/o A/AAAA records by @bassosimone in #1473

We fixed a bug where Web Connectivity LTE was not including the network events:

We updated Web Connectivity LTE's "classic" analysis engine to generate the extended blocking flags, such that it is now possible to always use the "classic" engine. In practical term, this implied improving the minipipeline package and making sure the "classic" engine, which already generates v0.4-compatible tests keys, could also generate the test keys that were previously generated by the "orig" engine. After this change, we removed the now-unneded "orig" analysis engine.

We added more test Web Connectivity LTE and minipipeline test cases:

We improved Web Connectivity LTE to correctly handle more blocking=null accessible=null cases:

We improved (and modified) the implementation of StreamAllContext and moved it inside the netxlite package:

We implemented support for SummaryKeys in Web Connectivity LTE:

We factored Web Connectivity LTE algorithms into the webconnectivityalgo package and added support for cycling through several DNS-over-UDP resolvers, thus implementing a feature request demanded by users in the SEA region:

We did work to reduce to ~zero the churn caused by regenerating minipipeline test cases, after we realized the non-determinism in generating the test cases was going to cause too many large and ~hard-to-review properly diffs:

We ensured that Web Connectivity LTE correctly includes the client_resolver field:

We ensured that Web Connectivity LTE correctly uses the tcptls_experiment tag when needed:

  • fix(webconnectivitylte): add classic and tcptls_experiment tags by @bassosimone in #1502

We implemented additional, misc changes to improve the codebase

  • refactor(webconnectivitylte): use NewHTTPTransportWithOptions by @bassosimone in #1494
  • doc(webconnectivitylte): clarify http_transaction_{start,end} semantics by @bassosimone in #1495

✨ 🐛 🚧 Web Connectivity Test Helper improvements

We implemented protection against oohelperd overload, which can be caused by synchronized clients:

  • ✨ feat(oohelperd): protect against overload and add metrics by @bassosimone in #1442

We implemented logs and tests to make sure oohelperd does not try connecting to bogons:

  • 🐛 fix(oohelperd): make sure endpoints don't connect to 127.0.0.1 by @bassosimone in #1463

We removed technical debt in the oohelperd, which was making GVisor-based integration tests tricky. Specifically, we ensured we have a single constructor for oohelperd and that such a constructor depends on a *netxlite.Netx. In turn, this allowed us to use the exact same constructor for running oohelperd in production and in integration testing, thus allowing for less fragile integration testing.

  • 🐛 fix(oohelperd,netemx): construct equivalent HTTPTransports by @bassosimone in #1464
  • 🚧 cleanup(netxlite): remove first-order implicit-Netx wrappers by @bassosimone in https://...
Read more

Contributors

bassosimone, hellais, and 3 other contributors
Assets 45
Loading

OONI Probe CLI v3.21.0-alpha

26 Mar 14:59
@github-actions github-actions
v3.21.0-alpha
This tag was signed with the committer’s verified signature. The key has expired.
bassosimone Simone Basso
GPG key ID: 80B691277733D95B
Expired
Learn about vigilant mode.
5f8a767
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
OONI Probe CLI v3.21.0-alpha Pre-release
Pre-release

This is an alpha release for creating a branching point for the release/3.21 branch.

Please see release 3.21.0 for an explanatory changelog.

This build lacks linux/armv6 binaries because of ooni/probe#2694.

What's Changed

Read more

Contributors

bassosimone, hellais, and 3 other contributors
Assets 43
Loading

OONI Probe CLI v3.20.1

21 Dec 19:01
@github-actions github-actions
v3.20.1
This tag was signed with the committer’s verified signature. The key has expired.
bassosimone Simone Basso
GPG key ID: 80B691277733D95B
Expired
Learn about vigilant mode.
b51640c
Compare
Choose a tag to compare
OONI Probe CLI v3.20.1

This is a stable release. On a high-level, this is what changed since 3.19:

  • ✨ Web Connectivity v0.5 is nearly production ready
  • ✨ Initial support for Go 1.21
  • 🚧 Scripting to make research faster
  • 🚧 Refactoring and improvements for the experimental dslx package
  • 🚧 Release engineering (aka keeping the lights on)
  • 🐛 Miscellaneous bug fixes

See https://github.com/ooni/probe-cli/releases/tag/v3.20.0 for a detailed changelog with respect to 3.19.

On top of 3.20.0, this release adds the following commits:

  • chore(deps): bump golang.org/x/crypto from 0.16.0 to 0.17.0 2d3173a
  • feat(oohelperd): protect against overload and add metrics b51640c
Assets 45
Loading

OONI Probe CLI v3.20.0

13 Dec 19:50
@github-actions github-actions
v3.20.0
This tag was signed with the committer’s verified signature. The key has expired.
bassosimone Simone Basso
GPG key ID: 80B691277733D95B
Expired
Learn about vigilant mode.
c34bdf6
Compare
Choose a tag to compare
OONI Probe CLI v3.20.0

This is a stable release. On a high-level, this is what changed since 3.19:

  • ✨ Web Connectivity v0.5 is nearly production ready
  • ✨ Initial support for Go 1.21
  • 🚧 Scripting to make research faster
  • 🚧 Refactoring and improvements for the experimental dslx package
  • 🚧 Release engineering (aka keeping the lights on)
  • 🐛 Miscellaneous bug fixes

More in detail:

✨ Web Connectivity v0.5 is nearly production ready

We implemented the minipipeline package, borrowing the https://github.com/ooni/data design, to make sure that Web Connectivity v0.5 produces the same top-level keys produced by Web Connectivity v0.4 for our GVisor-based QA test suite. Because of that, we're now ready to start enabling Web Connectivity v0.5 for all users for limited periods of time using the check-in API's feature flags. By doing this, we would be able to more thoroughly A/B test v0.4 and v0.5. In turn, this would allow us to fix additional bugs and hopefully v0.5 would be the default implementation of Web Connectivity in the 3.21 or 3.22 release cycles.

Related commits

✨ Initial support for Go 1.21

Community members asked us to add Go 1.21 support. It is now possible to compile OONI Probe using Go 1.21. Because Psiphon does not support Go 1.21 yet, when you're building using Go 1.21, it would not be possible to use Psiphon as a circumvention tunnel or to run the Psiphon experiment. (Psiphon should finish migrating to Go 1.21 soon, so OONI Probe 3.21 should have full Go 1.21 support.)

Related commits

🚧 Scripting to make research faster

As part of our richer input efforts (https://github.com/ooni/2023-05-richer-input), we developed DSL code designed to executed from a scripting language environment. We have been also using this code for research purposes, to address cases where we need to iterate faster and deploying new binaries frequently slows down the process significantly. In this release, we are starting to merge part of this code into this repository and to expose it (as an experimental preview) to the miniooni tool. Namely, we have merged the underlying DSL engine and the corresponding bindings for the https://github.com/dop251/goja JavaScript engine. We will add additional documentation about this functionality in the next release cycles, as we continue experimenting and using it for research. In the meanwhile, see https://github.com/bassosimone/2023-12-09-ooni-javascript/ for additional information and details about this experimental feature.

Related commits

🚧 Refactoring and improvements for the experimental dslx package

We continued working on our experimental internal DSL prototype, by trying to simplify and make it suitable for production usage. We also used some of the refactored code in the scripting functionality implementation described above.

Related commits

🚧 Release engineering (aka keeping the lights on)

🐛 Miscellaneous bug fixes

Contributors

bassosimone
Assets 45
Loading

OONI Probe CLI v3.20.0-alpha

13 Dec 18:37
@github-actions github-actions
v3.20.0-alpha
This tag was signed with the committer’s verified signature. The key has expired.
bassosimone Simone Basso
GPG key ID: 80B691277733D95B
Expired
Learn about vigilant mode.
e5c1f6a
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
OONI Probe CLI v3.20.0-alpha Pre-release
Pre-release

This is an alpha release for creating a branching point for the release/3.20 branch.

Please see release 3.20 for an explanatory changelog.

What's Changed

Full Changelog: v3.19.0-alpha...v3.20.0-alpha

Contributors

bassosimone, hellais, and kelmenhorst
Assets 45
Loading

OONI Probe CLI v3.19.2

01 Dec 11:41
@github-actions github-actions
v3.19.2
This tag was signed with the committer’s verified signature. The key has expired.
bassosimone Simone Basso
GPG key ID: 80B691277733D95B
Expired
Learn about vigilant mode.
85c6d1b
Compare
Choose a tag to compare
OONI Probe CLI v3.19.2

This is a stable release. On a high-level, this is what changed in 3.19:

  • ✨ Better Support for Measuring Throttling
  • ✨ Introducing OONI Probe Bridges
  • ✨ Improving Measurements Scrubbing
  • ✨ Netemx: A Better Unit/Integration Testing Framework
  • ✨ OONI Run v2 Support
  • ✨ Unit Testing Improvements (go test -short ./... does not use the host network anymore)
  • 🐛 OONI Probe Bootstrap fixes
  • 🐛 Web Connectivity v0.4 Fixes
  • 🐛 Snowflake Fixes

We recommend reading v3.19.0-alpha's changelog because it contains an exhaustive description of the most significant changes in 3.19 compared to 3.18. The following is the complete list of all the releases that led to this release:

On top of the v3.19.1 release, this follow-up release adds the following changes backported from the master branch:

  • 🐛 fix(signal): trim down tested endpoints 85c6d1b
Assets 45
Loading

OONI Probe CLI v3.19.1

21 Nov 14:41
@github-actions github-actions
v3.19.1
This tag was signed with the committer’s verified signature. The key has expired.
bassosimone Simone Basso
GPG key ID: 80B691277733D95B
Expired
Learn about vigilant mode.
96e96be
Compare
Choose a tag to compare
OONI Probe CLI v3.19.1

This is a stable release. On a high-level, this is what changed in 3.19:

  • ✨ Better Support for Measuring Throttling
  • ✨ Introducing OONI Probe Bridges
  • ✨ Improving Measurements Scrubbing
  • ✨ Netemx: A Better Unit/Integration Testing Framework
  • ✨ OONI Run v2 Support
  • ✨ Unit Testing Improvements (go test -short ./... does not use the host network anymore)
  • 🐛 OONI Probe Bootstrap fixes
  • 🐛 Web Connectivity v0.4 Fixes
  • 🐛 Snowflake Fixes

We recommend reading v3.19.0-alpha's changelog because it contains an exhaustive description of the most significant changes in 3.19 compared to 3.18. The following is the complete list of all the releases that led to this release:

On top of the v3.19.0 release, this follow-up release adds the following changes backported from the master branch:

  • 🐛 Update signal test endpoints 6b32fbf
Assets 45
Loading