cli: release 3.19 #2524

bassosimone · 2023-08-23T09:32:36Z

probe-cli: pin to the latest staging-client commit using go get -u -v: chore: upgrade to Psiphon-Labs/psiphon-tunnel-core@3f91b1b probe-cli#1338
probe-cli: make sure go list -json ./cmd/ooniprobe does not include any pinned package in psiphon's go.mod
oocrypto: merge with upstream if needed chore: update to go1.20.8 oocrypto#21
oohttp: merge with upstream if needed chore: update to go1.20.8 oohttp#47 chore: update examples to use go1.20.8 oohttp#48
probe-cli: take a look at go report card
probe-cli: sync stunreachability targets with snowflake (see ./internal/stuninput) chore: update the user agent we use probe-cli#1344
probe-cli: update GOVERSION if needed chore: use go1.20.8 probe-cli#1340
probe-cli: update the Go version mentioned in the Readme.md file chore: use go1.20.8 probe-cli#1340
probe-cli: update user-agent at internal/model/http.go chore: update the user agent we use probe-cli#1344
probe-cli: update github.com/ooni/probe-assets dependency chore: update go dependencies historical-geoip#7 Bump certifi from 2022.12.7 to 2023.7.22 historical-geoip#8 Bump urllib3 from 1.26.12 to 1.26.17 historical-geoip#9 ooni/probe-assets@8bc10a7 chore: use 2023-10 geoip database probe-cli#1341
probe-cli: update bundled certs (using go generate ./...) chore: run go generate ./... probe-cli#1343
probe-cli: ensure no dependency bumped its major version number using https://github.com/icholy/gomajor chore: use gomajor to update direct deps probe-cli#1342
probe-cli: update cdeps: chore: upgrade C dependencies probe-cli#1339
probe-cli: update NDKVERSION, and MOBILE/android/ensure if needed: chore: upgrade the NDK version probe-cli#1345
probe-cli: update dependencies with go get -u -v -d ./... chore: upgrade most deps except snowflake probe-cli#1346 chore: update obfs4, goptlib, snowflake probe-cli#1347 chore: replace obfs4 with lyrebird probe-cli#1348
probe-cli: tag a new version
probe-cli: update internal/version/version.go to be alpha
probe: create issue for next routine release: cli: release 3.20 #2556
probe-engine: run ./script/autoexport.bash ooni/probe-engine@b8025d7 https://github.com/ooni/probe-engine/releases/tag/v0.26.0
probe-cli: update internal/version/version.go chore: we're now hacking on v3.20.0-alpha probe-cli#1356
probe-cli: make sure all workflows are green ooni/probe-cli@29e8c88
probe-cli: go test -race -count 1 ./... must pass locally
probe-cli: try to address all the issues marked as "releaseBlocker" fix: update Snowflake domain front probe-cli#1337 fix(logx): log when an operation is starting probe-cli#1350 fix: make bootstrap more robust (and slower 😢) probe-cli#1351 refactor: go test -short does not use the host network probe-cli#1352 fix(engineresolver): support HTTP/HTTPS proxies probe-cli#1353 chore(oonimkall): re-enable TestTaskRunnerRun probe-cli#1354 fix: anomaly with android_dns_cache_no_data and inconsistent dns probe-cli#1211 feat: mvp of conditionally enabling experiments probe-cli#1355 fix(torsf): bump the version number probe-cli#1357 feat: add echcheck to the experimental suite probe-cli#1217 fix(ooniprobe): make sure we actually run echcheck probe-cli#1358 fix(registry): mark torsf as disabled by default probe-cli#1359 cleanup(riseupvpn): remove summary keys probe-cli#1360 refactor(riseupvpn): rename structs, change progress, bump version probe-cli#1361 chore: use go1.20.10 probe-cli#1362 refactor(riseupvpn): handle failing API and simplify test keys probe-cli#1363 feat(riseupvpn): include bootstrap in progress probe-cli#1364 chore(ooniprobe): add riseupvpn to experimental test suite probe-cli#1365
ios: replace ooni/go-libtor with scripts compiling tor #2564
probe-cli: address any outstanding TODO in the diff since last release (or create an issue for it)
probe-cli: check warnings emitted by gosec runs

The text was updated successfully, but these errors were encountered:

Part of ooni/probe#2524. Prompted by observing a failing Linux build test for zlib. Upgrading to OpenSSL v3 is tracked by ooni/probe#2498 and implemented by #1191. While there, fix ./pkg/oonimkall integration tests as well.

jefferyto · 2023-09-22T03:34:45Z

It would be great to have 3.19 released with support for Go 1.21.

bassosimone · 2023-10-04T09:29:17Z

@jefferyto ACK, I will see of this is possible!

See ooni/probe#2524

Part of ooni/probe#2524

See ooni/probe#2524

Part of ooni/probe#2524

See ooni/probe#2524

Part of ooni/probe#2524

See ooni/probe#2524

Part of ooni/probe#2524

This diff uses the https://github.com/icholy/gomajor tool to upgrade *direct* dependencies. More work is needed to take care of indirect dependencies, snowflake, obfs4, etc. Part of ooni/probe#2524

This diff backports #1365 to the release/3.19 branch. Concludes riseupvpn work for release 3.19. Reference issue: ooni/probe#2524.

Part of ooni/probe#2524

bassosimone · 2023-10-17T07:53:15Z

I forked off follow-up issues. We can now close this issue.

Part of ooni/probe#2524 ## Proposed Changes - Enable `echcheck` ooni/probe#2567. - Enable. `riseupvpn`. - Add support for appending errors --------- Co-authored-by: Simone Basso <bassosimone@gmail.com>

Part of ooni/probe#2524 ## Proposed Changes - Enable `echcheck` ooni/probe#2567. - Add support for appending errors

Part of ooni/probe#2524. Prompted by observing a failing Linux build test for zlib. Upgrading to OpenSSL v3 is tracked by ooni/probe#2498 and implemented by ooni#1191. While there, fix ./pkg/oonimkall integration tests as well.

See ooni/probe#2524

Part of ooni/probe#2524

This diff uses the https://github.com/icholy/gomajor tool to update *direct* dependencies. More work is needed to take care of indirect dependencies, snowflake, obfs4, etc. Part of ooni/probe#2524

The https://github.com/iancoleman/strcase library changed its default behavior between v0.2.0 and v0.3.0 such that: - before `DNS_NXDOMAIN_error` became `DNSNXDOMAINError`; - now it becomes `DnsNxdomainError`. In the grand scheme of things, perhaps, this change makes sense for the library. However, as a user, what I see is a breaking change between two minor versions (admittedly we're at v0.x). Because I don't know what would happen next and I don't want any more surprises, I am forking strcase@v2.0.0. This work is part of ooni/probe#2524

While there, make sure there are no changes in STUN servers. Part of ooni/probe#2524

This diff updates C dependencies to the latest version. Part of ooni/probe#2524.

Part of ooni/probe#2524

This diff upgrades most deps except for snowflake and related dependencies such as goptlib. The following diff contains the packages I *did not* update: ```diff diff --git a/go.mod b/go.mod index 09b51d97..54eecd98 100644 --- a/go.mod +++ b/go.mod @@ -37,7 +37,7 @@ require ( github.com/rubenv/sql-migrate v1.5.2 github.com/schollz/progressbar/v3 v3.13.1 github.com/upper/db/v4 v4.6.0 - gitlab.com/yawning/obfs4.git v0.0.0-20230519154740-645026c2ada4 + gitlab.com/yawning/obfs4.git v0.0.0-20231005123604-19f5a37fe427 gitlab.com/yawning/utls.git v0.0.12-1 golang.org/x/crypto v0.13.0 golang.org/x/net v0.15.0 @@ -74,13 +74,14 @@ require ( github.com/spf13/pflag v1.0.5 // indirect github.com/stretchr/testify v1.8.4 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect + gitlab.com/yawning/edwards25519-extra v0.0.0-20231005122941-2149dcafc266 // indirect gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/goptlib v1.5.0 // indirect go.uber.org/mock v0.3.0 // indirect golang.org/x/exp v0.0.0-20230905200255-921286631fa9 // indirect golang.org/x/exp/typeparams v0.0.0-20230522175609-2e198f4a06a1 // indirect golang.org/x/time v0.3.0 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect - gvisor.dev/gvisor v0.0.0-20230603040744-5c9219dedd33 // indirect + gvisor.dev/gvisor v0.0.0-20231005075148-09be6cec5e98 // indirect ) require ( @@ -89,7 +90,7 @@ require ( github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect github.com/armon/go-proxyproto v0.0.0-20210323213023-7e956b284f0a // indirect github.com/beorn7/perks v1.0.1 // indirect - github.com/bifurcation/mint v0.0.0-20180306135233-198357931e61 // indirect + github.com/bifurcation/mint v0.0.0-20210616192047-fd18df995463 // indirect github.com/cespare/xxhash/v2 v2.2.0 // indirect github.com/cheekybits/genny v1.0.0 // indirect github.com/cognusion/go-cache-lru v0.0.0-20170419142635-f73e2280ecea // indirect @@ -132,7 +133,7 @@ require ( github.com/prometheus/common v0.44.0 // indirect github.com/prometheus/procfs v0.12.0 // indirect github.com/refraction-networking/gotapdance v1.7.4 // indirect - github.com/refraction-networking/utls v1.3.3 // indirect + github.com/refraction-networking/utls v1.5.3 // indirect github.com/sergeyfrolov/bsbuffer v0.0.0-20180903213811-94e85abb8507 // indirect github.com/sirupsen/logrus v1.9.3 // indirect github.com/spf13/cobra v1.7.0 @@ -142,11 +143,11 @@ require ( github.com/templexxx/xorsimd v0.4.2 // indirect github.com/tjfoc/gmsm v1.4.1 // indirect github.com/wader/filtertransport v0.0.0-20200316221534-bdd9e61eee78 // indirect - github.com/xtaci/kcp-go/v5 v5.6.2 // indirect + github.com/xtaci/kcp-go/v5 v5.6.3 // indirect github.com/xtaci/smux v1.5.24 // indirect gitlab.com/yawning/bsaes.git v0.0.0-20190805113838-0a714cd429ec // indirect gitlab.com/yawning/edwards25519-extra.git v0.0.0-20230925102617-375edaeb7d45 // indirect - golang.org/x/mod v0.12.0 // indirect + golang.org/x/mod v0.13.0 // indirect golang.org/x/term v0.12.0 // indirect golang.org/x/text v0.13.0 // indirect golang.org/x/tools v0.13.0 // indirect ``` The following updates would break the build: - gvisor because the default branch is not go.mod ready; - kcp-go because v5.6.3 requires go1.21; - obfs4 because it requires changing goptlib, which I'll do next; - mint because Psiphon requires that version of mint. This work is part of ooni/probe#2524

This diff upgrades obfs4, goptlib, and snowflake. I tried to import lyrebird instead of obfs4, but that did not work as intended. It seems tags aren't compatible with the format expected by Go (vX.Y.Z) and I did not want to pin to the latest commit. I will report this issue to lyrebird developers later today. While there, also upgrade x/net and x/exp. This work is part of ooni/probe#2524. Here's what we could not upgrade (based on the content of `go.mod` after we committed this diff and ran `go get -u -v -d ./...`): ```diff diff --git a/go.mod b/go.mod index 67e0af02..18175d73 100644 --- a/go.mod +++ b/go.mod @@ -80,7 +80,7 @@ require ( golang.org/x/exp/typeparams v0.0.0-20230522175609-2e198f4a06a1 // indirect golang.org/x/time v0.3.0 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect - gvisor.dev/gvisor v0.0.0-20230603040744-5c9219dedd33 // indirect + gvisor.dev/gvisor v0.0.0-20231006032704-15cc3fcbbd77 // indirect ) require ( @@ -89,7 +89,7 @@ require ( github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect github.com/armon/go-proxyproto v0.0.0-20210323213023-7e956b284f0a // indirect github.com/beorn7/perks v1.0.1 // indirect - github.com/bifurcation/mint v0.0.0-20180306135233-198357931e61 // indirect + github.com/bifurcation/mint v0.0.0-20210616192047-fd18df995463 // indirect github.com/cespare/xxhash/v2 v2.2.0 // indirect github.com/cheekybits/genny v1.0.0 // indirect github.com/cognusion/go-cache-lru v0.0.0-20170419142635-f73e2280ecea // indirect @@ -132,7 +132,7 @@ require ( github.com/prometheus/common v0.44.0 // indirect github.com/prometheus/procfs v0.12.0 // indirect github.com/refraction-networking/gotapdance v1.7.4 // indirect - github.com/refraction-networking/utls v1.3.3 // indirect + github.com/refraction-networking/utls v1.5.3 // indirect github.com/sergeyfrolov/bsbuffer v0.0.0-20180903213811-94e85abb8507 // indirect github.com/sirupsen/logrus v1.9.3 // indirect github.com/spf13/cobra v1.7.0 @@ -142,10 +142,10 @@ require ( github.com/templexxx/xorsimd v0.4.2 // indirect github.com/tjfoc/gmsm v1.4.1 // indirect github.com/wader/filtertransport v0.0.0-20200316221534-bdd9e61eee78 // indirect - github.com/xtaci/kcp-go/v5 v5.6.2 // indirect + github.com/xtaci/kcp-go/v5 v5.6.3 // indirect github.com/xtaci/smux v1.5.24 // indirect gitlab.com/yawning/bsaes.git v0.0.0-20190805113838-0a714cd429ec // indirect - golang.org/x/mod v0.12.0 // indirect + golang.org/x/mod v0.13.0 // indirect golang.org/x/term v0.13.0 // indirect golang.org/x/text v0.13.0 // indirect golang.org/x/tools v0.13.0 // indirect ``` We cannot upgrade: - gvisor because (a) we're fine with our pinned version used by netem and (b) the main branch of gvistor is not suitable for go-getting anyway - mint because this is the version Psiphon requires - utls because this is the version Psiphon requires - kcp-go because v.5.6.3 requires go1.21 - x/mod which seems to be needed by quic-go to generate mocks, so not upgrading it isn't a big deal.

Part of ooni/probe#2524

We changed the SNI, so let's change also the version number. Part of ooni/probe#2524.

Part of ooni/probe#2524

Concludes riseupvpn work for release 3.19. Reference issue: ooni/probe#2524.

bassosimone added the priority/medium label Aug 23, 2023

bassosimone self-assigned this Aug 23, 2023

bassosimone mentioned this issue Aug 23, 2023

chore: upgrade C dependencies ooni/probe-cli#1194

Merged

bassosimone added the releaseBlocker This issue blocks releasing label Oct 4, 2023

bassosimone mentioned this issue Oct 4, 2023

cli, engine: use go1.21 #2548

Closed

bassosimone added a commit to ooni/probe-cli that referenced this issue Oct 4, 2023

chore: upgrade to Psiphon-Labs/psiphon-tunnel-core@3f91b1b

939b672

See ooni/probe#2524

bassosimone mentioned this issue Oct 4, 2023

chore: upgrade to Psiphon-Labs/psiphon-tunnel-core@3f91b1b ooni/probe-cli#1338

Merged

bassosimone added a commit to ooni/probe-cli that referenced this issue Oct 4, 2023

chore: upgrade to Psiphon-Labs/psiphon-tunnel-core@3f91b1b (#1338)

24ce473

See ooni/probe#2524

bassosimone added a commit to ooni/probe-cli that referenced this issue Oct 4, 2023

chore: upgrade C dependencies

431120d

Part of ooni/probe#2524

bassosimone mentioned this issue Oct 4, 2023

chore: upgrade C dependencies ooni/probe-cli#1339

Merged

bassosimone added a commit to ooni/oohttp that referenced this issue Oct 4, 2023

chore: prepare syncing with go1.20.8

926efc6

See ooni/probe#2524

bassosimone added a commit to ooni/oohttp that referenced this issue Oct 4, 2023

chore: update to go1.20.8 (#47)

dbb36a6

Part of ooni/probe#2524

bassosimone added a commit to ooni/oohttp that referenced this issue Oct 4, 2023

chore: update examples to use go1.20.8

d5bed05

Part of ooni/probe#2524

bassosimone mentioned this issue Oct 4, 2023

chore: update examples to use go1.20.8 ooni/oohttp#48

Merged

bassosimone added a commit to ooni/oocrypto that referenced this issue Oct 4, 2023

chore: update to go1.20.8 (#21)

2dfb6ff

Part of ooni/probe#2524

bassosimone added a commit to ooni/probe-cli that referenced this issue Oct 4, 2023

chore: use go1.20.8

50e74d2

Part of ooni/probe#2524

bassosimone mentioned this issue Oct 4, 2023

chore: use go1.20.8 ooni/probe-cli#1340

Merged

bassosimone added a commit to ooni/historical-geoip that referenced this issue Oct 4, 2023

chore: update go dependencies

bc122eb

See ooni/probe#2524

bassosimone mentioned this issue Oct 4, 2023

chore: update go dependencies ooni/historical-geoip#7

Merged

bassosimone added a commit to ooni/historical-geoip that referenced this issue Oct 4, 2023

chore: update go dependencies (#7)

45b3c9a

See ooni/probe#2524

bassosimone added a commit to ooni/probe-cli that referenced this issue Oct 4, 2023

chore: use go1.20.8 (#1340)

e69ba88

Part of ooni/probe#2524

bassosimone added a commit to ooni/probe-assets that referenced this issue Oct 4, 2023

chore: use 2023-10 database

8bc10a7

See ooni/probe#2524

bassosimone added a commit to ooni/probe-cli that referenced this issue Oct 4, 2023

chore: use 2023-10 geoip database

11ecd67

Part of ooni/probe#2524

bassosimone mentioned this issue Oct 4, 2023

chore: use 2023-10 geoip database ooni/probe-cli#1341

Merged

bassosimone added a commit to ooni/probe-cli that referenced this issue Oct 4, 2023

chore: use 2023-10 geoip database (#1341)

9dedb7a

Part of ooni/probe#2524

bassosimone added a commit to ooni/probe-cli that referenced this issue Oct 11, 2023

[backport] chore(ooniprobe): add riseupvpn to experimental test suite

0eabe59

This diff backports #1365 to the release/3.19 branch. Concludes riseupvpn work for release 3.19. Reference issue: ooni/probe#2524.

bassosimone added a commit to ooni/probe-cli that referenced this issue Oct 11, 2023

chore: we're now hacking on v3.19.0-alpha.3

05e3716

Part of ooni/probe#2524

bassosimone mentioned this issue Oct 12, 2023

ios: replace ooni/go-libtor with scripts compiling tor #2564

Closed

bassosimone added a commit to ooni/probe-cli that referenced this issue Oct 12, 2023

chore: we're now hacking on v3.19.0-alpha.4

53cad17

Part of ooni/probe#2524

bassosimone mentioned this issue Oct 12, 2023

ios: test tor and torsf for the 3.19 cycle #2566

Closed

aanorbel mentioned this issue Oct 16, 2023

chore: use probe-cli v3.19.0 ooni/probe-ios#534

Merged

This was referenced Oct 17, 2023

android: pin to oonimkall using engine@3.19 #2571

Closed

ios: pin to oonimkall using engine@3.19 #2572

Closed

desktop: pin to cli@3.19 #2573

Open

debian: publish cli@3.19 packages #2574

Closed

cli: notify iThena about 3.19 release #2575

Closed

bassosimone closed this as completed Oct 17, 2023

aanorbel added a commit to ooni/probe-android that referenced this issue Oct 20, 2023

chore: use probe-cli v3.19.0 (#622)

a9d8054

Part of ooni/probe#2524 ## Proposed Changes - Enable `echcheck` ooni/probe#2567. - Add support for appending errors

Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024

chore: upgrade to Psiphon-Labs/psiphon-tunnel-core@3f91b1b (ooni#1338)

4acfedc

See ooni/probe#2524

Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024

chore: use go1.20.8 (ooni#1340)

576d130

Part of ooni/probe#2524

Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024

chore: use 2023-10 geoip database (ooni#1341)

be7b768

Part of ooni/probe#2524

Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024

chore: update the user agent we use (ooni#1344)

937f7e2

While there, make sure there are no changes in STUN servers. Part of ooni/probe#2524

Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024

chore: upgrade C dependencies (ooni#1339)

99b6d88

This diff updates C dependencies to the latest version. Part of ooni/probe#2524.

Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024

chore: upgrade the NDK version (ooni#1345)

a2d7a2f

Part of ooni/probe#2524

Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024

chore: we're now hacking on v3.20.0-alpha (ooni#1356)

5d4435d

Part of ooni/probe#2524

Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024

fix(torsf): bump the version number (ooni#1357)

f352783

We changed the SNI, so let's change also the version number. Part of ooni/probe#2524.

Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024

chore: use go1.20.10 (ooni#1362)

6069871

Part of ooni/probe#2524

Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024

chore(ooniprobe): add riseupvpn to experimental test suite (ooni#1365)

736e5e6

Concludes riseupvpn work for release 3.19. Reference issue: ooni/probe#2524.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cli: release 3.19 #2524

cli: release 3.19 #2524

bassosimone commented Aug 23, 2023 •

edited

jefferyto commented Sep 22, 2023

bassosimone commented Oct 4, 2023

bassosimone commented Oct 17, 2023

cli: release 3.19 #2524

cli: release 3.19 #2524

Comments

bassosimone commented Aug 23, 2023 • edited

jefferyto commented Sep 22, 2023

bassosimone commented Oct 4, 2023

bassosimone commented Oct 17, 2023

bassosimone commented Aug 23, 2023 •

edited