Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dotnet][rb][java][js][py] Automated Browser Version Update #13942

Merged
merged 2 commits into from
May 21, 2024

Conversation

selenium-ci
Copy link
Member

@selenium-ci selenium-ci commented May 15, 2024

This is an automated pull request to update pinned browsers and drivers

Merge after verify the new browser versions properly passing the tests and no bugs need to be filed

Copy link

PR Description updated to latest commit (4eb007f)

Copy link

PR Review 🔍

⏱️ Estimated effort to review [1-5]

2, because the changes are straightforward and involve only updates to URLs and hashes for browser versions. The structure and logic of the code remain unchanged.

🧪 Relevant tests

No

⚡ Possible issues

Possible Bug: Ensure that the new browser versions and drivers are compatible with existing tests and functionalities. This should be verified as suggested in the PR description.

🔒 Security concerns

No

Copy link

PR Code Suggestions ✨

CategorySuggestion                                                                                                                                    Score
Enhancement
Implement automated verification of URLs for browser archives to ensure their availability and correctness

Consider adding a verification step to check the availability and correctness of the URLs
for all updated browser versions periodically. This can be automated as part of a CI/CD
pipeline to catch any broken links or incorrect updates early.

common/repositories.bzl [185]

-url = "https://msedgedriver.azureedge.net/124.0.2478.105/edgedriver_mac64.zip"
+url = "https://msedgedriver.azureedge.net/124.0.2478.105/edgedriver_mac64.zip"  # URL verified as part of automated checks
 
Suggestion importance[1-10]: 7

Why: Adding automated verification for URLs in a CI/CD pipeline is a valuable enhancement for maintaining the reliability and correctness of external dependencies.

7
Possible issue
Verify the integrity and correctness of the URL for the linux_firefox archive

Consider verifying the integrity of the URL used for the linux_firefox archive. The URL
format seems to have changed from previous versions, which might indicate a potential typo
or error. Ensure that the URL points to the correct version and file.

common/repositories.bzl [14]

-url = "https://ftp.mozilla.org/pub/firefox/releases/126.0/linux-x86_64/en-US/firefox-126.0.tar.bz2"
+url = "https://ftp.mozilla.org/pub/firefox/releases/126.0/linux-x86_64/en-US/firefox-126.0.tar.bz2"  # Verified correct URL
 
Suggestion importance[1-10]: 3

Why: The suggestion to verify the URL integrity is valid but not critical as the URL change follows a predictable version update pattern.

3
Security
Re-verify the sha256 hash for the mac_firefox archive to ensure its correctness

Ensure that the sha256 hash for the mac_firefox archive is correct by re-verifying the
hash value. This is crucial to maintain the integrity and security of the downloaded file.

common/repositories.bzl [37]

-sha256 = "7fdf67bd49d4e9f3a2f34aadc5d03b7d9558d153c63a4f212c08100237589685"
+sha256 = "7fdf67bd49d4e9f3a2f34aadc5d03b7d9558d153c63a4f212c08100237589685"  # Re-verified and confirmed correct
 
Suggestion importance[1-10]: 3

Why: While ensuring the correctness of the sha256 hash is important for security, the suggestion does not point out any specific issue with the current hash.

3

Copy link

codiumai-pr-agent-pro bot commented May 15, 2024

CI Failure Feedback 🧐

(Checks updated until commit 72a45e4)

Action: Ruby / Local Tests (edge, windows) / Local Tests (edge, windows)

Failed stage: Run Bazel [❌]

Failed test name: Selenium::WebDriver::BiDi

Failure summary:

The action failed because the test //rb/spec/integration/selenium/webdriver:bidi-edge failed. The
specific reasons for the failure are:

  • Selenium::WebDriver::BiDi gets session status failed because it was expected to raise an error due
    to a pending test guard, but no error was raised.
  • Selenium::WebDriver::BiDi does not close BiDi session if at least one window is opened failed
    because it was expected to raise an error due to a pending test guard, but no error was raised.
  • Selenium::WebDriver::BiDi closes BiDi session if last window is closed failed because it was
    expected to raise an error due to a pending test guard, but no error was raised.

  • Relevant error logs:
    1:  ##[group]Operating System
    2:  Microsoft Windows Server 2022
    ...
    
    776:  �[32m[1,875 / 1,887]�[0m 11 / 27 tests;�[0m [Sched] Testing //rb/spec/integration/selenium/webdriver:element-edge; 30s ... (4 actions, 2 running)
    777:  �[32m[1,876 / 1,887]�[0m 12 / 27 tests;�[0m [Sched] Testing //rb/spec/integration/selenium/webdriver:element-edge; 31s ... (4 actions, 1 running)
    778:  �[32m[1,876 / 1,887]�[0m 12 / 27 tests;�[0m [Sched] Testing //rb/spec/integration/selenium/webdriver:element-edge; 41s ... (4 actions, 1 running)
    779:  �[32m[1,876 / 1,887]�[0m 12 / 27 tests;�[0m [Sched] Testing //rb/spec/integration/selenium/webdriver:element-edge; 43s ... (4 actions, 1 running)
    780:  �[32m[1,876 / 1,887]�[0m 12 / 27 tests;�[0m [Sched] Testing //rb/spec/integration/selenium/webdriver:takes_screenshot-edge; 26s ... (4 actions, 2 running)
    781:  �[32m[1,877 / 1,887]�[0m 13 / 27 tests;�[0m [Sched] Testing //rb/spec/integration/selenium/webdriver:takes_screenshot-edge; 28s ... (4 actions, 1 running)
    782:  �[32m[1,877 / 1,887]�[0m 13 / 27 tests;�[0m [Sched] Testing //rb/spec/integration/selenium/webdriver:takes_screenshot-edge; 38s ... (4 actions, 1 running)
    783:  �[32m[1,877 / 1,887]�[0m 13 / 27 tests;�[0m [Sched] Testing //rb/spec/integration/selenium/webdriver:takes_screenshot-edge; 40s ... (4 actions, 1 running)
    784:  �[32m[1,877 / 1,887]�[0m 13 / 27 tests;�[0m [Sched] Testing //rb/spec/integration/selenium/webdriver:error-edge; 21s ... (4 actions, 2 running)
    785:  �[32m[1,878 / 1,887]�[0m 14 / 27 tests;�[0m [Sched] Testing //rb/spec/integration/selenium/webdriver:error-edge; 22s ... (4 actions, 1 running)
    ...
    
    826:  �[32m[1,890 / 1,891]�[0m 26 / 27 tests;�[0m Testing //rb/spec/integration/selenium/webdriver:bidi-edge; 16s local, disk-cache
    827:  �[31m�[1mFAIL: �[0m//rb/spec/integration/selenium/webdriver:bidi-edge (see D:/_bazel/execroot/_main/bazel-out/x64_windows-fastbuild/testlogs/rb/spec/integration/selenium/webdriver/bidi-edge/test_attempts/attempt_1.log)
    828:  �[32m[1,890 / 1,891]�[0m 26 / 27 tests;�[0m Testing //rb/spec/integration/selenium/webdriver:bidi-edge; 17s local, disk-cache
    829:  �[32m[1,890 / 1,891]�[0m 26 / 27 tests;�[0m Testing //rb/spec/integration/selenium/webdriver:bidi-edge; 31s local, disk-cache
    830:  �[31m�[1mFAIL: �[0m//rb/spec/integration/selenium/webdriver:bidi-edge (see D:/_bazel/execroot/_main/bazel-out/x64_windows-fastbuild/testlogs/rb/spec/integration/selenium/webdriver/bidi-edge/test_attempts/attempt_2.log)
    831:  �[32m[1,890 / 1,891]�[0m 26 / 27 tests;�[0m Testing //rb/spec/integration/selenium/webdriver:bidi-edge; 33s local, disk-cache
    832:  �[32m[1,890 / 1,891]�[0m 26 / 27 tests;�[0m Testing //rb/spec/integration/selenium/webdriver:bidi-edge; 48s local, disk-cache
    833:  �[31m�[1mFAIL: �[0m//rb/spec/integration/selenium/webdriver:bidi-edge (see D:/_bazel/execroot/_main/bazel-out/x64_windows-fastbuild/testlogs/rb/spec/integration/selenium/webdriver/bidi-edge/test.log)
    834:  �[31m�[1mFAILED: �[0m//rb/spec/integration/selenium/webdriver:bidi-edge (Summary)
    ...
    
    840:  Running Ruby specs:
    841:  browser: edge
    842:  driver: edge
    843:  version: 125.0.2535.51
    844:  platform: windows
    845:  ci: github
    846:  ruby: ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x64-mingw32]
    847:  Selenium::WebDriver::BiDi
    848:  gets session status (FAILED - 1)
    849:  can navigate and listen to errors
    850:  does not close BiDi session if at least one window is opened (FAILED - 2)
    851:  closes BiDi session if last window is closed (FAILED - 3)
    852:  Failures:
    853:  1) Selenium::WebDriver::BiDi gets session status FIXED
    854:  Expected pending 'Test guarded; https://bugs.chromium.org/p/chromedriver/issues/detail?id=4676' to fail. No error was raised.
    855:  # ./rb/spec/integration/selenium/webdriver/bidi_spec.rb:33
    856:  2) Selenium::WebDriver::BiDi does not close BiDi session if at least one window is opened FIXED
    857:  Expected pending 'Test guarded; https://bugs.chromium.org/p/chromedriver/issues/detail?id=4676' to fail. No error was raised.
    858:  # ./rb/spec/integration/selenium/webdriver/bidi_spec.rb:64
    859:  3) Selenium::WebDriver::BiDi closes BiDi session if last window is closed FIXED
    860:  Expected pending 'Test guarded; https://bugs.chromium.org/p/chromedriver/issues/detail?id=4676' to fail. No error was raised.
    861:  # ./rb/spec/integration/selenium/webdriver/bidi_spec.rb:82
    862:  Finished in 14.41 seconds (files took 0.58224 seconds to load)
    863:  4 examples, 3 failures
    864:  Failed examples:
    ...
    
    870:  Running Ruby specs:
    871:  browser: edge
    872:  driver: edge
    873:  version: 125.0.2535.51
    874:  platform: windows
    875:  ci: github
    876:  ruby: ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x64-mingw32]
    877:  Selenium::WebDriver::BiDi
    878:  gets session status (FAILED - 1)
    879:  can navigate and listen to errors
    880:  does not close BiDi session if at least one window is opened (FAILED - 2)
    881:  closes BiDi session if last window is closed (FAILED - 3)
    882:  Failures:
    883:  1) Selenium::WebDriver::BiDi gets session status FIXED
    884:  Expected pending 'Test guarded; https://bugs.chromium.org/p/chromedriver/issues/detail?id=4676' to fail. No error was raised.
    885:  # ./rb/spec/integration/selenium/webdriver/bidi_spec.rb:33
    886:  2) Selenium::WebDriver::BiDi does not close BiDi session if at least one window is opened FIXED
    887:  Expected pending 'Test guarded; https://bugs.chromium.org/p/chromedriver/issues/detail?id=4676' to fail. No error was raised.
    888:  # ./rb/spec/integration/selenium/webdriver/bidi_spec.rb:64
    889:  3) Selenium::WebDriver::BiDi closes BiDi session if last window is closed FIXED
    890:  Expected pending 'Test guarded; https://bugs.chromium.org/p/chromedriver/issues/detail?id=4676' to fail. No error was raised.
    891:  # ./rb/spec/integration/selenium/webdriver/bidi_spec.rb:82
    892:  Finished in 13.28 seconds (files took 0.57144 seconds to load)
    893:  4 examples, 3 failures
    894:  Failed examples:
    ...
    
    900:  Running Ruby specs:
    901:  browser: edge
    902:  driver: edge
    903:  version: 125.0.2535.51
    904:  platform: windows
    905:  ci: github
    906:  ruby: ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x64-mingw32]
    907:  Selenium::WebDriver::BiDi
    908:  gets session status (FAILED - 1)
    909:  can navigate and listen to errors
    910:  does not close BiDi session if at least one window is opened (FAILED - 2)
    911:  closes BiDi session if last window is closed (FAILED - 3)
    912:  Failures:
    913:  1) Selenium::WebDriver::BiDi gets session status FIXED
    914:  Expected pending 'Test guarded; https://bugs.chromium.org/p/chromedriver/issues/detail?id=4676' to fail. No error was raised.
    915:  # ./rb/spec/integration/selenium/webdriver/bidi_spec.rb:33
    916:  2) Selenium::WebDriver::BiDi does not close BiDi session if at least one window is opened FIXED
    917:  Expected pending 'Test guarded; https://bugs.chromium.org/p/chromedriver/issues/detail?id=4676' to fail. No error was raised.
    918:  # ./rb/spec/integration/selenium/webdriver/bidi_spec.rb:64
    919:  3) Selenium::WebDriver::BiDi closes BiDi session if last window is closed FIXED
    920:  Expected pending 'Test guarded; https://bugs.chromium.org/p/chromedriver/issues/detail?id=4676' to fail. No error was raised.
    921:  # ./rb/spec/integration/selenium/webdriver/bidi_spec.rb:82
    922:  Finished in 14.46 seconds (files took 0.5928 seconds to load)
    923:  4 examples, 3 failures
    924:  Failed examples:
    925:  rspec ./rb/spec/integration/selenium/webdriver/bidi_spec.rb:33 # Selenium::WebDriver::BiDi gets session status
    926:  rspec ./rb/spec/integration/selenium/webdriver/bidi_spec.rb:64 # Selenium::WebDriver::BiDi does not close BiDi session if at least one window is opened
    927:  rspec ./rb/spec/integration/selenium/webdriver/bidi_spec.rb:82 # Selenium::WebDriver::BiDi closes BiDi session if last window is closed
    928:  ================================================================================
    929:  �[32mINFO: �[0mFound 27 test targets...
    930:  �[32mINFO: �[0mElapsed time: 1043.327s, Critical Path: 407.93s
    931:  �[32mINFO: �[0m1891 processes: 226 disk cache hit, 837 internal, 81 local, 747 worker.
    932:  �[32mINFO: �[0mBuild completed, 1 test FAILED, 1891 total actions
    933:  //rb/spec/integration/selenium/webdriver:action_builder-edge             �[0m�[32mPASSED�[0m in 18.8s
    934:  //rb/spec/integration/selenium/webdriver:devtools-edge                   �[0m�[32mPASSED�[0m in 163.2s
    935:  //rb/spec/integration/selenium/webdriver:driver-edge                     �[0m�[32mPASSED�[0m in 26.1s
    936:  //rb/spec/integration/selenium/webdriver:element-edge                    �[0m�[32mPASSED�[0m in 18.0s
    937:  //rb/spec/integration/selenium/webdriver:error-edge                      �[0m�[32mPASSED�[0m in 5.8s
    ...
    
    951:  //rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge      �[0m�[32mPASSED�[0m in 33.9s
    952:  //rb/spec/integration/selenium/webdriver/bidi:log_inspector-edge         �[0m�[32mPASSED�[0m in 52.6s
    953:  //rb/spec/integration/selenium/webdriver/edge:driver-edge                �[0m�[32mPASSED�[0m in 23.2s
    954:  //rb/spec/integration/selenium/webdriver/edge:options-edge               �[0m�[32mPASSED�[0m in 12.8s
    955:  //rb/spec/integration/selenium/webdriver/edge:profile-edge               �[0m�[32mPASSED�[0m in 5.2s
    956:  //rb/spec/integration/selenium/webdriver/edge:service-edge               �[0m�[32mPASSED�[0m in 10.8s
    957:  //rb/spec/integration/selenium/webdriver/remote:driver-edge              �[0m�[32mPASSED�[0m in 5.1s
    958:  //rb/spec/integration/selenium/webdriver/remote:element-edge             �[0m�[32mPASSED�[0m in 9.4s
    959:  //rb/spec/integration/selenium/webdriver:bidi-edge                       �[0m�[31m�[1mFAILED�[0m in 3 out of 3 in 16.1s
    960:  Stats over 3 runs: max = 16.1s, min = 14.9s, avg = 15.7s, dev = 0.6s
    961:  D:/_bazel/execroot/_main/bazel-out/x64_windows-fastbuild/testlogs/rb/spec/integration/selenium/webdriver/bidi-edge/test.log
    962:  D:/_bazel/execroot/_main/bazel-out/x64_windows-fastbuild/testlogs/rb/spec/integration/selenium/webdriver/bidi-edge/test_attempts/attempt_1.log
    963:  D:/_bazel/execroot/_main/bazel-out/x64_windows-fastbuild/testlogs/rb/spec/integration/selenium/webdriver/bidi-edge/test_attempts/attempt_2.log
    964:  Executed 27 out of 27 tests: 26 tests pass and �[0m�[31m�[1m1 fails locally�[0m.
    965:  There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
    966:  �[0m
    967:  ##[error]Process completed with exit code 1.
    

    ✨ CI feedback usage guide:

    The CI feedback tool (/checks) automatically triggers when a PR has a failed check.
    The tool analyzes the failed checks and provides several feedbacks:

    • Failed stage
    • Failed test name
    • Failure summary
    • Relevant error logs

    In addition to being automatically triggered, the tool can also be invoked manually by commenting on a PR:

    /checks "https://github.com/{repo_name}/actions/runs/{run_number}/job/{job_number}"
    

    where {repo_name} is the name of the repository, {run_number} is the run number of the failed check, and {job_number} is the job number of the failed check.

    Configuration options

    • enable_auto_checks_feedback - if set to true, the tool will automatically provide feedback when a check is failed. Default is true.
    • excluded_checks_list - a list of checks to exclude from the feedback, for example: ["check1", "check2"]. Default is an empty list.
    • enable_help_text - if set to true, the tool will provide a help message with the feedback. Default is true.
    • persistent_comment - if set to true, the tool will overwrite a previous checks comment with the new feedback. Default is true.
    • final_update_message - if persistent_comment is true and updating a previous checks message, the tool will also create a new message: "Persistent checks updated to latest commit". Default is true.

    See more information about the checks tool in the docs.

    @selenium-ci selenium-ci force-pushed the pinned-browser-updates branch 4 times, most recently from a15fa83 to 2b5d746 Compare May 19, 2024 00:36
    Copy link

    codecov bot commented May 21, 2024

    Codecov Report

    All modified and coverable lines are covered by tests ✅

    Project coverage is 58.72%. Comparing base (57f8398) to head (2d6744b).
    Report is 355 commits behind head on trunk.

    Current head 2d6744b differs from pull request most recent head 72a45e4

    Please upload reports for the commit 72a45e4 to get more accurate results.

    Additional details and impacted files
    @@            Coverage Diff             @@
    ##            trunk   #13942      +/-   ##
    ==========================================
    + Coverage   58.48%   58.72%   +0.23%     
    ==========================================
      Files          86       86              
      Lines        5270     5298      +28     
      Branches      220      227       +7     
    ==========================================
    + Hits         3082     3111      +29     
    + Misses       1968     1960       -8     
    - Partials      220      227       +7     

    ☔ View full report in Codecov by Sentry.
    📢 Have feedback on the report? Share it here.

    @diemol diemol merged commit 9ef5f25 into trunk May 21, 2024
    36 of 40 checks passed
    @diemol diemol deleted the pinned-browser-updates branch May 21, 2024 10:27
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    None yet

    2 participants