Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass server config to WebDriver via a file instead of an env variable. #28834

Merged
merged 2 commits into from May 10, 2021

Conversation

@letitz
Copy link
Contributor

@letitz letitz commented May 5, 2021

Instead of serializing the whole config in an environment variable, I amended the code to dump the config as JSON in a temporary file (the tests are conveniently executed against a temporary directory) and pass the file path in the environment variable instead.

This should avoid failures due to the 16-bit length limit being exceeded for good. In other words, no more issues like #21705 (comment) and #12461 :) This also removes the need to special-case origin-policy subdomains.

This change was extracted from #28768 per @jgraham's suggestion.

@jgraham
jgraham approved these changes May 5, 2021
Copy link
Contributor

@jgraham jgraham left a comment

Thanks!

@jgraham
Copy link
Contributor

@jgraham jgraham commented May 5, 2021

Test failure is ./serve/test_serve.py:1:1: F401 'json' imported but unused

@foolip
Copy link
Member

@foolip foolip commented May 5, 2021

Hmm, "Azure Pipelines (tools/wpt/ tests: macOS + Python 3.6)" failed in a novel way:

Error: Download failed on Cask 'firefox-nightly' with message: bad argument (expected URI object or URI string)

https://raw.githubusercontent.com/Homebrew/homebrew-cask-versions/master/Casks/firefox-nightly.rb exists, but perhaps there was a network error. I'll close and reopen to see if it was a fluke.

We could probably install Firefox using ./wpt install instead here...

@foolip foolip closed this May 5, 2021
@foolip foolip reopened this May 5, 2021
@letitz
Copy link
Contributor Author

@letitz letitz commented May 6, 2021

Different failure this time in "Azure Pipelines (tools/wpt/ tests: macOS + Python 3.6)": a flaky manifest test, it seems.

manifest/tests/test_manifest.py:122:

<snip>

message = "Hypothesis test_manifest_to_json(s=[<Mock id='4408810848'>,\n <Mock id='4412119456'>,\n <Mock id='4412281184'>,\n <Mo...>,\n <Mock id='4418202064'>]) produces unreliable results: Falsified on the first call but did not on a subsequent one"

    def __flaky(self, message):
        if len(self.falsifying_examples) <= 1:
>           raise Flaky(message)
E           hypothesis.errors.Flaky: Hypothesis test_manifest_to_json(s=[<Mock id='4408810848'>,
E            <Mock id='4412119456'>,
E            <Mock id='4412281184'>,
E            <Mock id='4418804272'>,
E            <Mock id='4417926144'>,
E            <Mock id='4418202064'>]) produces unreliable results: Falsified on the first call but did not on a subsequent one

Trying to close and re-open once more.

@letitz letitz closed this May 6, 2021
@letitz letitz reopened this May 6, 2021
@foolip
Copy link
Member

@foolip foolip commented May 6, 2021

@letitz also try rebasing to be sure you're testing the latest CI configuration.

@letitz
Copy link
Contributor Author

@letitz letitz commented May 6, 2021

The wpt-chrome-dev-stability and wpt-firefox-nightly-stability tasks both timed out, but they were running mostly fine up to there. What do we do now?

@LukeZielinski LukeZielinski assigned foolip and unassigned LukeZielinski May 6, 2021
@letitz letitz force-pushed the letitz:wd-server-config branch from c429e7e to 647b88f May 6, 2021
@letitz
Copy link
Contributor Author

@letitz letitz commented May 6, 2021

Tried rebasing per @foolip's suggestion.

@foolip
Copy link
Member

@foolip foolip commented May 6, 2021

@letitz it looks like flake8 is still failing, did you accidental revert that fix?

@letitz
Copy link
Contributor Author

@letitz letitz commented May 10, 2021

Oh, I think I totally did. Sorry! Not very familiar with the PR workflow yet, especially around how to handle commits added to PRs by others. Fixing...

@foolip
foolip approved these changes May 10, 2021
Copy link
Member

@foolip foolip left a comment

CI is still running, but changes look good!

@foolip
Copy link
Member

@foolip foolip commented May 10, 2021

OK, stability jobs have timed out due to #7660 and wpt.fyi checks point to changes in test results due to #28925. I'll admin merge.

@foolip foolip merged commit cd82725 into web-platform-tests:master May 10, 2021
38 of 41 checks passed
38 of 41 checks passed
@github-actions
build-and-publish
Details
@github-actions
build-and-tag
Details
@community-tc-integration
sink-task Community-TC (pull_request)
Details
@community-tc-integration
wpt-chrome-dev-stability Community-TC (pull_request)
Details
@community-tc-integration
wpt-firefox-nightly-stability Community-TC (pull_request)
Details
@wpt-fyi
wpt.fyi - chrome[experimental] Chrome results
Details
@wpt-fyi
wpt.fyi - firefox[experimental] Firefox results
Details
@wpt-fyi
wpt.fyi - safari[experimental] Safari results
Details
@azure-pipelines
Azure Pipelines Build #20210510.56 succeeded
Details
@azure-pipelines
Azure Pipelines (./wpt test-jobs) ./wpt test-jobs succeeded
Details
@azure-pipelines
Azure Pipelines (affected tests without changes: Safari Technology Preview) affected tests without changes: Safari Technology Preview succeeded
Details
@azure-pipelines
Azure Pipelines (affected tests: Safari Technology Preview) affected tests: Safari Technology Preview succeeded
Details
@azure-pipelines
Azure Pipelines (infrastructure/ tests: macOS) infrastructure/ tests: macOS succeeded
Details
@azure-pipelines
Azure Pipelines (tools/ unittests: Windows + Python 3.6) tools/ unittests: Windows + Python 3.6 succeeded
Details
@azure-pipelines
Azure Pipelines (tools/ unittests: Windows + Python 3.9) tools/ unittests: Windows + Python 3.9 succeeded
Details
@azure-pipelines
Azure Pipelines (tools/ unittests: macOS + Python 3.6) tools/ unittests: macOS + Python 3.6 succeeded
Details
@azure-pipelines
Azure Pipelines (tools/ unittests: macOS + Python 3.9) tools/ unittests: macOS + Python 3.9 succeeded
Details
@azure-pipelines
Azure Pipelines (tools/wpt/ tests: Windows + Python 3.6) tools/wpt/ tests: Windows + Python 3.6 succeeded
Details
@azure-pipelines
Azure Pipelines (tools/wpt/ tests: Windows + Python 3.9) tools/wpt/ tests: Windows + Python 3.9 succeeded
Details
@azure-pipelines
Azure Pipelines (tools/wpt/ tests: macOS + Python 3.6) tools/wpt/ tests: macOS + Python 3.6 succeeded
Details
@azure-pipelines
Azure Pipelines (tools/wpt/ tests: macOS + Python 3.9) tools/wpt/ tests: macOS + Python 3.9 succeeded
Details
@azure-pipelines
Azure Pipelines (tools/wptrunner/ unittests: Windows + Python 3.6) tools/wptrunner/ unittests: Windows + Python 3.6 succeeded
Details
@azure-pipelines
Azure Pipelines (tools/wptrunner/ unittests: Windows + Python 3.9) tools/wptrunner/ unittests: Windows + Python 3.9 succeeded
Details
@azure-pipelines
Azure Pipelines (tools/wptrunner/ unittests: macOS + Python 3.6) tools/wptrunner/ unittests: macOS + Python 3.6 succeeded
Details
@azure-pipelines
Azure Pipelines (tools/wptrunner/ unittests: macOS + Python 3.9) tools/wptrunner/ unittests: macOS + Python 3.9 succeeded
Details
@azure-pipelines
Azure Pipelines (wpt.fyi hook: safari-preview-affected-tests) wpt.fyi hook: safari-preview-affected-tests succeeded
Details
@azure-pipelines
Azure Pipelines (wpt.fyi hook: safari-preview-affected-tests-without-changes) wpt.fyi hook: safari-preview-affected-tests-without-changes succeeded
Details
@community-tc-integration
download-firefox-nightly Community-TC (pull_request)
Details
@community-tc-integration
infrastructure/ tests Community-TC (pull_request)
Details
@community-tc-integration
lint Community-TC (pull_request)
Details
@community-tc-integration
resources/ tests (Python 3.6) Community-TC (pull_request)
Details
@community-tc-integration
resources/ tests (Python 3.9) Community-TC (pull_request)
Details
@community-tc-integration
tools/ integration tests (Python 3.6) Community-TC (pull_request)
Details
@community-tc-integration
tools/ integration tests (Python 3.9) Community-TC (pull_request)
Details
@community-tc-integration
tools/ unittests (Python 3.6) Community-TC (pull_request)
Details
@community-tc-integration
tools/ unittests (Python 3.9) Community-TC (pull_request)
Details
@community-tc-integration
wpt-chrome-dev-results Community-TC (pull_request)
Details
@community-tc-integration
wpt-chrome-dev-results-without-changes Community-TC (pull_request)
Details
@community-tc-integration
wpt-decision-task Community-TC (pull_request)
Details
@community-tc-integration
wpt-firefox-nightly-results Community-TC (pull_request)
Details
@community-tc-integration
wpt-firefox-nightly-results-without-changes Community-TC (pull_request)
Details
DamieFC pushed a commit to DamieFC/chromium that referenced this pull request May 12, 2021
This change unblocks importer, where webdriver tests are failing due to changes in web-platform-tests/wpt#28834

Change-Id: Ia07ed877aa808da41a49260900a390c87e9c5521
No-Export: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2890259
Reviewed-by: Jason Chase <chasej@chromium.org>
Reviewed-by: John Chen <johnchen@chromium.org>
Commit-Queue: Kyle Ju <kyleju@chromium.org>
Cr-Commit-Position: refs/heads/master@{#882198}
T3-M4 pushed a commit to bayandin/chromedriver that referenced this pull request May 13, 2021
This change unblocks importer, where webdriver tests are failing due to changes in web-platform-tests/wpt#28834

Change-Id: Ia07ed877aa808da41a49260900a390c87e9c5521
No-Export: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2890259
Reviewed-by: Jason Chase <chasej@chromium.org>
Reviewed-by: John Chen <johnchen@chromium.org>
Commit-Queue: Kyle Ju <kyleju@chromium.org>
Cr-Commit-Position: refs/heads/master@{#882198}
@letitz letitz deleted the letitz:wd-server-config branch May 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants