Merge branch 'master' into issue/1019

Conflicts:
	experiment/urlgetter/urlgetter.go

.github/workflows/shorttests.yml

@@ -10,7 +10,7 @@ jobs:
         with:
           go-version: "1.14"
       - uses: actions/checkout@v2
-      - run: go test -short -race -tags shaping -coverprofile=probe-engine.cov ./...
+      - run: go test -short -race -tags PSIPHON_DISABLE_QUIC,shaping -coverprofile=probe-engine.cov ./...
       - uses: shogo82148/actions-goveralls@v1
         with:
           path-to-profile: probe-engine.cov

CODE_OF_CONDUCT.md

@@ -0,0 +1,255 @@
+# OONI Code of Conduct
+
+
+Statement of Intent:
+
+OONI is committed to fostering an inclusive environment and
+community. OONI is a place where people should feel safe to engage, share their
+point of view, and participate.
+
+This code of conduct applies to OONI as a whole. It is intended to
+provide guidelines for contributors. Employees and contractors of OONI
+are also subject to company policies and procedures. Those
+people should feel free to contact HR with questions or concerns.
+
+This code of conduct is not exhaustive or complete. It is an ongoing effort to
+summarize our shared understanding. We want to provide a welcoming, safe
+environment, so we can work together to pursue powerful solutions. We reserve
+the right to deviate from strictly enforcing this code. Any deviations must
+produce an outcome which is fairer, and aligned with our values. We understand
+that keeping a living document relevant and “patched” involves sustained
+effort.
+
+0. Summary: Don't be a jerk. Be awesome instead.
+
+The OONI community should be a good place where people are glad to be.
+
+  DO: Be kind, thoughtful, and considerate.
+
+  DO: Make OONI a place where people are happy and comfortable.
+
+  DO: Remember: We are all contributing; we are all learning. Nobody was born
+    an expert.
+
+  DO: Yield the floor. Listen. Make sure everyone gets heard.
+
+  DON'T: Insult, harass, intimidate, or be a jerk.
+
+  DON'T: Treat honest mistakes as an excuse to hassle people. Mistakes are for
+    learning.
+
+  DON'T: Hunt for ways to uphold the letter of this code while violating its
+    spirit.
+
+  AND DO: Eagerly email: team@openobservatory.org with
+    questions or concerns.
+
+1. Purpose
+
+A primary goal of the OONI community is to be inclusive of many different
+contributors. We want to include contributors from the most varied and diverse
+backgrounds possible. As such, we are committed to providing a friendly, safe
+and welcoming environment for all, regardless of their experience, gender
+identity or expression, sexual orientation, family, relationships, ability
+(whether bodily or mental), personal appearance, socioeconomic status, body
+size, race, ethnicity, age, religion, nationality, or membership in a
+disadvantaged and/or underrepresented group.
+
+A safe environment means one free from abuse, whether isolated or systemic. We
+explicitly acknowledge that tolerating abuse is a security problem. Allowing
+abusers and malicious people to disrupt our community puts our software,
+developers, and users at risk.
+
+This code of conduct outlines our expectations for all those who participate in
+our community, as well as the consequences for unacceptable behavior.
+
+We invite all those who participate in OONI to help us create safe
+and positive experiences for everyone.
+
+2. Be your best self
+
+The following behaviors are expected and requested of all community members:
+
+  * Participate in an honest and active way. In doing so, you contribute to
+    the health and longevity of this community.
+
+  * Exercise consideration and respect in your speech and actions.
+
+  * Attempt collaboration and dialog before engaging in conflict.
+
+  * Refrain from demeaning, discriminatory, or harassing behavior and speech.
+
+  * Be mindful of your surroundings and of your fellow participants. Alert
+    community leaders if you notice:
+
+    - a dangerous situation
+
+    - someone in distress
+
+    - violations of this code of conduct, even if they seem minor
+
+  * Remember that community event venues may be shared with members of the
+    public. Please be respectful to everyone using these locations.
+
+  * Respect the privacy of your fellow community members.
+
+3. Unacceptable behaviors
+
+The following behaviors are unacceptable within our community:
+
+  * Violence, sexual assault, threats of violence, or violent language
+    directed against another person, especially violence against a person or
+    group based on a protected characteristic. (Display of weapons may
+    constitute intimidation or a threat of violence.)
+
+  * Sexist, racist, homophobic, transphobic, ableist or otherwise
+    discriminatory jokes and language.
+
+  * Spontaneously posting or displaying sexually explicit or violent
+    material. (If it is necessary to share such material when working on
+    OONI's mission, do so with sensitivity. Be aware that many people don't
+    want to see it.)
+
+  * Posting or threatening to post other people’s personally identifying
+    information ("doxing") without their consent.
+
+  * Personal insults or attacks, particularly those related to:
+
+    - experience
+
+    - gender identity or expression
+
+    - sexual orientation
+
+    - family
+
+    - relationships
+
+    - ability (whether bodily or mental)
+
+    - personal appearance
+
+    - socioeconomic status
+
+    - body size
+
+    - race
+
+    - ethnicity
+
+    - age
+
+    - religion
+
+    - nationality
+
+    - membership in a disadvantaged and/or underrepresented group
+
+  * Inappropriate photography, audio recording, or recording of personal
+    information. You should have someone's consent before recording these
+    things, and before posting them publicly.
+
+  * Inappropriate physical contact. You should have someone’s consent before
+    touching them.
+
+  * Unwelcome sexual attention: this includes sexualized comments or jokes,
+    inappropriate touching, groping, and unwelcome sexual advances.
+
+  * Deliberate intimidation, stalking or following (online or in person).
+
+  * Deliberately undermining the spirit of this code while following the
+    letter.
+
+  * Sustained disruption of any community events, including talks,
+    presentations, and online conversations.
+
+  * Deliberately pushing against someone's stated boundaries.
+
+  * Advocating for, or encouraging, any of the above behavior.
+
+4. Unacceptable behavior has consequences
+
+We will not tolerate unacceptable behavior from any community member. We will
+not make exceptions for sponsors and those with decision-making authority.
+People in formal or informal leadership roles must model the highest standards
+of behavior.
+
+Anyone asked by another community member to stop unacceptable behavior is
+expected to comply immediately. Please don't step in on someone else's behalf
+without their consent.
+
+5. Where to go for help
+
+The OONI team can assist with intra-community conflict resolution. You can contact
+the whole OONI team:
+
+  - if you have questions or concerns about the code of conduct, or
+
+  - if you feel that you have witnessed a code of conduct violation
+
+However, if you feel that there is a conflict of interest with any team
+member, you may contact members individually. See: https://ooni.org/about/.
+
+6. What to do if you witness unacceptable behavior
+
+If you are subject to or witness unacceptable behavior, or have any other
+concerns, please notify the OONI team as soon as possible. You can
+contact the OONI team in person, or at
+team@openobservatory.org. Current team members are listed
+on the [about page](https://ooni.org/about/). The OONI team's incident response will vary on a
+case-by-case basis. We will make every effort to respond to the
+incident immediately. We will prioritize the safety of the person who
+has been harmed, or is at risk of harm. Person(s) who have been harmed or are
+at risk of harm can withdraw the incident report at any time. We will never do
+anything without the consent of the person who has been harmed or is at risk of
+harm, except in situations where there is a threat of imminent danger or harm
+to anyone.
+
+7. What the person reporting a violation can expect
+
+The OONI team prioritizes the safety and well-being of any person who
+feels that they have been harmed or may be in danger of being harmed. Anyone
+reporting an issue to the OONI team can expect that their report will
+be taken seriously. Initial reports can be taken in written or verbal form.
+The next steps in an incident response will vary on a case-by-case basis.
+
+8. How the OONI team responds to incidents
+
+If a community member engages in unacceptable behavior, the OONI team
+may take any action they deem appropriate, including but not limited to a
+warning, informal mediation, temporary ban or permanent expulsion from the
+community. 
+
+9. Scope of this document
+
+This code of conduct covers all community participants:
+
+  - paid and unpaid contributors
+
+  - sponsors
+
+  - other guests
+
+when interacting:
+
+  - in all online and in-person community venues
+
+  - in one-on-one communications that relate to community work
+
+This code of conduct and its related procedures also applies to unacceptable
+behavior occurring outside the scope of community activities when such behavior
+has the potential to adversely affect the safety and well-being of community
+members. As members of the OONI community, we support and follow this Code while
+we are working on OONI, and take care not to undermine it in the rest of our
+time.
+
+
+10. License and attribution
+
+
+This code of conduct is shared under a Creative Commons CC-BY-SA 4.0
+International license.
+
+This code of conduct is adapted from The Tor Project code of conduct and uses
+some language and framing from the [Citizen Code of Conduct](https://citizencodeofconduct.org),
+which is shared under a CC-BY-SA license.

experiment/urlgetter/configurer.go

@@ -43,6 +43,7 @@ func (c Configurer) NewConfiguration() (Configuration, error) {
 			CertPool:            c.Config.CertPool,
 			ContextByteCounting: true,
 			DialSaver:           c.Saver,
+			HTTP3Enabled:        c.Config.HTTP3Enabled,
 			HTTPSaver:           c.Saver,
 			Logger:              c.Logger,
 			ReadWriteSaver:      c.Saver,

experiment/urlgetter/urlgetter.go

@@ -24,6 +24,7 @@ type Config struct {
 	DNSHTTPHost       string         `ooni:"Force using specific HTTP Host header for DNS requests"`
 	DNSTLSServerName  string         `ooni:"Force TLS to using a specific SNI for encrypted DNS requests"`
 	FailOnHTTPError   bool           `ooni:"Fail HTTP request if status code is 400 or above"`
+	HTTP3Enabled      bool           `ooni:"Force http3"`
 	HTTPHost          string         `ooni:"Force using specific HTTP Host header"`
 	Method            string         `ooni:"Force HTTP method different than GET"`
 	NoFollowRedirects bool           `ooni:"Disable following redirects"`

go.mod

@@ -40,6 +40,7 @@ require (
 	github.com/hashicorp/golang-lru v0.5.4 // indirect
 	github.com/iancoleman/strcase v0.1.2
 	github.com/juju/ratelimit v1.0.2-0.20191002062651-f60b32039441 // indirect
+	github.com/lucas-clemente/quic-go v0.18.1
 	github.com/marusama/semaphore v0.0.0-20171214154724-565ffd8e868a // indirect
 	github.com/miekg/dns v1.1.34
 	github.com/montanaflynn/stats v0.6.3