Import of v0.30.0 (#1931)

CHANGES.md

@@ -3,12 +3,83 @@ Change log
 
 All notable changes to this program are documented in this file.
 
-0.29.1  (2021-04-09), `970ef713fe58`)
+0.30.0  (2021-09-16, `d372710b98a6`)
+------------------------------------
+
+### Known problems
+
+- _macOS 10.15 (Catalina) and later:_
+
+  Due to the requirement from Apple that all programs must be
+  notarized, geckodriver will not work on Catalina if you manually
+  download it through another notarized program, such as Firefox.
+
+  Whilst we are working on a repackaging fix for this problem, you can
+  find more details on how to work around this issue in the [macOS
+  notarization] section of the documentation.
+
+- _Android:_
+
+  For releases of Firefox 89.0 and earlier Marionette will only be enabled in
+  GeckoView based applications when the Firefox preference
+  `devtools.debugger.remote-enabled` is set to `true` via [`moz:firefoxOptions`].
+
+### Added
+
+- Support for WebDriver clients to opt in to WebDriver BiDi.
+
+  Introduced the new boolean capability [`webSocketUrl`] that can be used by
+  WebDriver clients to opt in to a bidirectional connection. A string capability
+  with the same name will be returned by [`NewSession`], which contains the
+  WebSocket URL of the newly created WebDriver session in the form of:
+  `ws://host:port/session/<session id>`.
+
+  When running on Android a port forward will be set on the host machine,
+  which is using the exact same port as on the device.
+
+  All the supported WebDriver BiDi commands depend on the version of
+  Firefox, and not geckodriver. The first commands will be shipped in
+  Firefox 94.
+
+- It's now possible to set additional preferences when a custom profile has been
+  specified. At the end of the session they will be removed.
+
+### Fixed
+
+- Improved Host header checks to reject requests not sent to a well-known
+  local hostname or IP, or the server-specified hostname.
+
+- Added validation that the `--host` argument resolves to a local IP address.
+
+- Limit the `--foreground` argument of Firefox to MacOS only.
+
+- Increased Marionette handshake timeout to not fail for slow connections.
+
+- `Marionette:Quit` is no longer sent twice during session deletion.
+
+- When deleting a session that was attached to an already running browser
+  instance, the browser is not getting closed anymore.
+
+- Android
+
+  - Starting Firefox on Android from a Windows based host will now succeed as
+    we are using the correct Unix path separator to construct on-device paths.
+
+  - Arguments as specified in [`moz:firefoxOptions`] are now used when starting
+    Firefox.
+
+  - Port forwards set for Marionette and the WebSocket server (WebDriver BiDi)
+    are now correctly removed when geckodriver exits.
+
+  - The test root folder is now removed when geckodriver exists.
+
+
+0.29.1  (2021-04-09, `970ef713fe58`)
 -------------------------------------
 
 ### Known problems
 
-- _macOS 10.15 (Catalina):_
+- _macOS 10.15 (Catalina) and later:_
 
   Due to the requirement from Apple that all programs must be
   notarized, geckodriver will not work on Catalina if you manually
@@ -21,9 +92,9 @@ All notable changes to this program are documented in this file.
 - _Android:_
 
   Marionette will only be enabled in GeckoView based applications when the
-  Firefox preference `devtools.debugger.remote-enabled` is set to `True` via
-  [`moz:firefoxOptions`]. This will be fixed in one of the upcoming Firefox
-  for Android releases.
+  Firefox preference `devtools.debugger.remote-enabled` is set to `true` via
+  [`moz:firefoxOptions`]. This will be fixed in the Firefox 90 release for
+  Android.
 
 ### Added
 
@@ -58,7 +129,7 @@ All notable changes to this program are documented in this file.
 
 ### Known problems
 
-- _macOS 10.15 (Catalina):_
+- _macOS 10.15 (Catalina) and later:_
 
   Due to the requirement from Apple that all programs must be
   notarized, geckodriver will not work on Catalina if you manually
@@ -71,7 +142,7 @@ All notable changes to this program are documented in this file.
 - _Android:_
 
   Marionette will only be enabled in GeckoView based applications when the
-  Firefox preference `devtools.debugger.remote-enabled` is set to `True` via
+  Firefox preference `devtools.debugger.remote-enabled` is set to `true` via
   [`moz:firefoxOptions`]. This will be fixed in one of the upcoming Firefox
   for Android releases.
 
@@ -84,7 +155,7 @@ All notable changes to this program are documented in this file.
 
 ### Added
 
-- Introduced the new boolean capability `moz:debuggerAddress` that can be used
+- Introduced the new boolean capability [`moz:debuggerAddress`] that can be used
   to opt-in to the experimental Chrome DevTools Protocol (CDP) implementation.
   A string capability with the same name will be returned by [`NewSession`],
   which contains the `host:port` combination of the HTTP server that can be
@@ -98,7 +169,7 @@ All notable changes to this program are documented in this file.
 
 ### Known problems
 
-- _macOS 10.15 (Catalina):_
+- _macOS 10.15 (Catalina) and later:_
 
   Due to the requirement from Apple that all programs must be
   notarized, geckodriver will not work on Catalina if you manually
@@ -111,7 +182,7 @@ All notable changes to this program are documented in this file.
 - _Android:_
 
   Marionette will only be enabled in GeckoView based applications when the
-  Firefox preference `devtools.debugger.remote-enabled` is set to `True` via
+  Firefox preference `devtools.debugger.remote-enabled` is set to `true` via
   [`moz:firefoxOptions`]. This will be fixed in one of the upcoming Firefox
   for Android releases.
 
@@ -160,7 +231,7 @@ All notable changes to this program are documented in this file.
 
 ### Known problems
 
-- _macOS 10.15 (Catalina):_
+- _macOS 10.15 (Catalina) and later:_
 
   Due to the requirement from Apple that all programs must be
   notarized, geckodriver will not work on Catalina if you manually
@@ -173,7 +244,7 @@ All notable changes to this program are documented in this file.
 - _Android:_
 
   Marionette will only be enabled in GeckoView based applications when the
-  Firefox preference `devtools.debugger.remote-enabled` is set to `True` via
+  Firefox preference `devtools.debugger.remote-enabled` is set to `true` via
   [`moz:firefoxOptions`]. This will be fixed in one of the upcoming Firefox
   for Android releases.
 
@@ -221,7 +292,7 @@ has changed to Firefox ≥60.
 
 ### Known problems
 
-- _macOS 10.15 (Catalina):_
+- _macOS 10.15 (Catalina) and later:_
 
   Due to the recent requirement from Apple that all programs must
   be notarized, geckodriver will not work on Catalina if you manually
@@ -240,7 +311,7 @@ has changed to Firefox ≥60.
 - _Android:_
 
   Marionette will only be enabled in GeckoView based applications when the
-  Firefox preference `devtools.debugger.remote-enabled` is set to `True` via
+  Firefox preference `devtools.debugger.remote-enabled` is set to `true` via
   [`moz:firefoxOptions`]. This will be fixed in one of the upcoming Firefox
   for Android releases.
 
@@ -427,8 +498,7 @@ with this particular release that we intend to release a fix for soon.
 
 - ARMv7 HF builds have been discontinued
 
-  We [announced](https://lists.mozilla.org/pipermail/tools-marionette/2018-September/000035.html)
-  back in September 2018 that we would stop building for ARM,
+  We announced back in September 2018 that we would stop building for ARM,
   but builds can be self-serviced by building from source.
 
   To cross-compile from another host system, you can use this command:
@@ -1491,6 +1561,7 @@ and greater.
 [README]: https://github.com/mozilla/geckodriver/blob/master/README.md
 [Browser Toolbox]: https://developer.mozilla.org/en-US/docs/Tools/Browser_Toolbox
 [WebDriver conformance]: https://wpt.fyi/results/webdriver/tests?label=experimental
+[`webSocketUrl`]: https://developer.mozilla.org/en-US/docs/Web/WebDriver/Capabilities/webSocketUrl
 [`moz:firefoxOptions`]: https://developer.mozilla.org/en-US/docs/Web/WebDriver/Capabilities/firefoxOptions
 [`moz:debuggerAddress`]: https://firefox-source-docs.mozilla.org/testing/geckodriver/Capabilities.html#moz-debuggeraddress
 [Microsoft Visual Studio redistributable runtime]: https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "geckodriver"
-version = "0.29.1"
+version = "0.30.0"
 description = "Proxy for using WebDriver clients to interact with Gecko-based browsers."
 keywords = ["webdriver", "w3c", "httpd", "mozilla", "firefox"]
 repository = "https://hg.mozilla.org/mozilla-central/file/tip/testing/geckodriver"
@@ -17,17 +17,17 @@ hyper = "0.13"
 lazy_static = "1.0"
 log = { version = "0.4", features = ["std"] }
 marionette = { path = "./marionette" }
-mozdevice = "0.3.2"
-mozprofile = "0.7.2"
-mozrunner = "0.12.1"
-mozversion = "0.4.1"
+mozdevice = "0.4.0"
+mozprofile = "0.7.3"
+mozrunner = "0.13.0"
+mozversion = "0.4.2"
 regex = { version="1.0", default-features = false, features = ["perf", "std"] }
 serde = "1.0"
 serde_derive = "1.0"
 serde_json = "1.0"
 serde_yaml = "0.8"
 uuid = { version = "0.8", features = ["v4"] }
-webdriver = "0.43.1"
+webdriver = "0.44.0"
 zip = { version = "0.4", default-features = false, features = ["deflate"] }
 
 [[bin]]

README.md

@@ -70,10 +70,7 @@ Contact
 -------
 
 The mailing list for geckodriver discussion is
-tools-marionette@lists.mozilla.org ([subscribe], [archive]).
+https://groups.google.com/a/mozilla.org/g/dev-webdriver.
 
-There is also a Matrix channel to talk about using and developing
-geckodriver on `#interop:mozilla.org <https://chat.mozilla.org/#/room/#interop:mozilla.org>`__
-
-[subscribe]: https://lists.mozilla.org/listinfo/tools-marionette
-[archive]: https://lists.mozilla.org/pipermail/tools-marionette/
+There is also an Element channel to talk about using and developing
+geckodriver on `#webdriver:mozilla.org <https://chat.mozilla.org/#/room/#webdriver:mozilla.org>`__

doc/ARM.md

@@ -1,7 +1,7 @@
 Self-serving an ARM build
 =========================
 
-Mozilla [announced the intent] to deprecate ARMv7 HF builds of
+Mozilla announced the intent to deprecate ARMv7 HF builds of
 geckodriver in September 2018.  This does not mean you can no longer
 use geckodriver on ARM systems, and this document explains how you
 can self-service a build for ARMv7 HF.
@@ -35,5 +35,4 @@ cross-compile ARMv7 from a Linux host system is as follows:
          % cd testing/geckodriver
          % cargo build --release --target armv7-unknown-linux-gnueabihf
 
-[announced the intent]: https://lists.mozilla.org/pipermail/tools-marionette/2018-September/000035.html
 [central]: https://hg.mozilla.org/mozilla-central/

doc/Patches.md

@@ -0,0 +1,28 @@
+Submitting patches
+==================
+
+You can submit patches by using [Phabricator]. Walk through its documentation
+in how to set it up, and uploading patches for review. Don't worry about which
+person to select for reviewing your code. It will be done automatically.
+
+Please also make sure to follow the [commit creation guidelines].
+
+Once you have contributed a couple of patches, we are happy to sponsor you in
+[becoming a Mozilla committer].  When you have been granted commit access
+level 1, you will have permission to use the [Firefox CI] to trigger your own
+“try runs” to test your changes. You can use the following [try preset] to run
+the most relevant tests:
+
+	mach try --preset geckodriver
+
+This preset will schedule geckodriver-related tests on various platforms. You can
+reduce the number of tasks by filtering on platforms (e.g. linux) or build type
+(e.g. opt):
+
+	mach try --preset geckodriver -xq "'linux 'opt"
+
+[Phabricator]: https://moz-conduit.readthedocs.io/en/latest/phabricator-user.html
+[commit creation guidelines]: https://mozilla-version-control-tools.readthedocs.io/en/latest/devguide/contributing.html?highlight=phabricator#submitting-patches-for-review
+[becoming a Mozilla committer]: https://www.mozilla.org/en-US/about/governance/policies/commit/
+[Firefox CI]: https://treeherder.mozilla.org/
+[try preset]: https://firefox-source-docs.mozilla.org/tools/try/presets.html

doc/Profiles.md

@@ -75,7 +75,7 @@ Firefox (1), and a set of recommended preferences set on startup (2).
 These can be perused here:
 
   1. [testing/geckodriver/src/prefs.rs](https://searchfox.org/mozilla-central/source/testing/geckodriver/src/prefs.rs)
-  2. [testing/marionette/components/marionette/marionette.js](https://searchfox.org/mozilla-central/source/testing/marionette/components/marionette.js)
+  2. [remote/components/marionette.js](https://searchfox.org/mozilla-central/source/remote/components/marionette.js)
 
 As mentioned, these are _recommended_ preferences, and any user-defined
 preferences in the [user.js file] or as part of the [`prefs` capability]
@@ -84,7 +84,9 @@ take precedence.  This means for example that the user can tweak
 starting the browser with a blank page.
 
 The recommended preferences set at runtime (see 2 above) may also
-be disabled entirely by setting `marionette.prefs.recommended`.
+be disabled entirely by setting `remote.prefs.recommended` starting with Firefox
+91. For older versions of Firefox, the preference to use was
+`marionette.prefs.recommended`.
 This may however cause geckodriver to not behave correctly according
 to the WebDriver standard, so it should be used with caution.
 

doc/Releasing.md

@@ -45,7 +45,8 @@ For each crate:
   1. Bump the version number in Cargo.toml
   2. Update the crate: `cargo update -p <crate name>`
   3. Commit the changes for the modified `Cargo.toml`, and `Cargo.lock`
-     (can be found in the repositories root folder)
+     (can be found in the repositories root folder). Use a commit message
+     like `Bug XYZ - [rust-<crate name>] Release version <version>.`
 
 
 Update the change log

doc/Support.md

@@ -22,6 +22,11 @@ and required versions of Selenium and Firefox:
     <th>max
   </tr>
  </thead>
+ <tr>
+  <td>0.30.0
+  <td>≥ 3.11 (3.14 Python)
+  <td>78 ESR
+  <td>n/a
  <tr>
   <td>0.29.1
   <td>≥ 3.11 (3.14 Python)

doc/index.rst

@@ -40,6 +40,7 @@ For developers
 
    Building.md
    Testing.md
+   Patches.md
    Releasing.md
    ARM.md
 
@@ -48,10 +49,7 @@ Communication
 =============
 
 The mailing list for geckodriver discussion is
-tools-marionette@lists.mozilla.org (`subscribe`_, `archive`_).
+https://groups.google.com/a/mozilla.org/g/dev-webdriver.
 
 If you prefer real-time chat, ask your questions
-on `#interop:mozilla.org <https://chat.mozilla.org/#/room/#interop:mozilla.org>`__.
-
-.. _subscribe: https://lists.mozilla.org/listinfo/tools-marionette
-.. _archive: https://lists.mozilla.org/pipermail/tools-marionette/
+on `#webdriver:mozilla.org <https://chat.mozilla.org/#/room/#webdriver:mozilla.org>`__.