Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[🐛 Bug]: [rb] rspec spec/unit aborts with ruby3.2 #11498

Closed
mtasaka opened this issue Dec 31, 2022 · 9 comments
Closed

[🐛 Bug]: [rb] rspec spec/unit aborts with ruby3.2 #11498

mtasaka opened this issue Dec 31, 2022 · 9 comments

Comments

@mtasaka
Copy link

mtasaka commented Dec 31, 2022

What happened?

With selenium git head ( 21fe296 ), rspec test suite
"aborts" (raises expection) with ruby3.2.0.

The below log shows that rspec test suite

  • finished executing "uses provided args" test in spec/unit/selenium/webdriver/chrome/service_spec.rb
  • then trying to execute next test "uses args when passed in as a Hash" in the same file (service_spec.rb)
    and then raises exception internally.

How can we reproduce the issue?

`$ rspec -f d spec/unit`

Relevant log output

[mockbuild@ rb]$ ruby -v
ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux]
[mockbuild@3 rb]$ rspec -f d spec/unit

Selenium::Server
  raises an error if the jar file does not exist
  uses the given jar file and port (FAILED - 1)
  waits for the server process by default
  adds additional args
  adds additional JAVA options args
  downloads the specified version from the selenium site
  gets a server instance and downloads the specified version
  automatically repairs http_proxy settings that do not start with http://
  only downloads a jar if it is not present in the current directory
  should know what the latest version available is
  raises Selenium::Server::Error if the server is not launched within the timeout
  sets options after instantiation

Selenium::WebDriver::Chrome::Driver
  does not require any parameters
  accepts provided Options as sole parameter
  does not accept Options of the wrong class
  does not allow both Options and Capabilities
  with :capabilities
    accepts value as a Symbol
    accepts Capabilities.chrome
    accepts constructed Capabilities with Snake Case as Symbols
    accepts constructed Capabilities with Camel Case as Symbols
    accepts constructed Capabilities with Camel Case as Strings
    when value is an Array
      with Options instance
      with Options instance with profile
      with Capabilities instance
      with Options instance and an instance of a custom object responding to #as_json
      with Options instance, Capabilities instance and instance of a custom object responding to #as_json

Selenium::WebDriver::Chrome::Options
  #initialize
    accepts defined parameters
  accessors
    adds a command-line argument
    adds an extension
    sets the binary path
    adds a preference
    add an emulated device by name
    adds local state
    adds a switch to exclude
    adds performance logging preferences
    adds a window type
  #add_extension
    adds an extension
    raises error when the extension file is missing
    raises error when the extension file is not .crx
  #add_encoded_extension
    adds an encoded extension
  #add_argument
    adds a command-line argument
  #headless!
    should add necessary command-line arguments
  #add_option
    adds an option with ordered pairs
    adds an option with Hash
  #add_preference
    adds a preference
  #add_emulation
    add an emulated device by name
    adds emulated device metrics
    adds emulated user agent
  #enable_android
    adds default android settings
    accepts parameters
  #as_json
    returns empty options by default
    should raise error when w3c is false
    should raise error when w3c is true
    returns added options
    converts profile
    returns a JSON hash

Selenium::WebDriver::Chrome::Profile
  should set and get preference paths
  reads existing prefs
  writes out prefs

Selenium::WebDriver::Chrome::Service
  #new
    uses default path and port
    uses provided path and port
    allows #driver_path= with String value
    allows #driver_path= with Proc value
    does not create args by default
    uses provided args
/usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2274:in `tty?': closed stream (IOError)
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2274:in `output_to_tty?'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:927:in `color_enabled?'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/syntax_highlighter.rb:45:in `implementation'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/syntax_highlighter.rb:13:in `highlight'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/exception_presenter.rb:232:in `read_failed_lines'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/exception_presenter.rb:166:in `failure_slash_error_lines'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/exception_presenter.rb:153:in `block in failure_lines'
	from <internal:kernel>:90:in `tap'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/exception_presenter.rb:152:in `failure_lines'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/exception_presenter.rb:34:in `colorized_message_lines'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/exception_presenter.rb:257:in `formatted_message_and_backtrace'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/exception_presenter.rb:88:in `fully_formatted_lines'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/exception_presenter.rb:80:in `fully_formatted'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/notifications.rb:200:in `fully_formatted'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/notifications.rb:114:in `block in fully_formatted_failed_examples'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/notifications.rb:113:in `each'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/notifications.rb:113:in `each_with_index'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/notifications.rb:113:in `fully_formatted_failed_examples'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/base_text_formatter.rb:32:in `dump_failures'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:209:in `block in notify'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:208:in `each'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:208:in `notify'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:178:in `block in finish'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:194:in `close_after'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:174:in `finish'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:76:in `ensure in report'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:76:in `report'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:115:in `run_specs'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:89:in `run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:71:in `run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:45:in `invoke'
	from /usr/share/gems/gems/rspec-core-3.12.0/exe/rspec:4:in `<top (required)>'
	from /usr/bin/rspec:25:in `load'
	from /usr/bin/rspec:25:in `<main>'
/usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/output_wrapper.rb:24:in `write': closed stream (IOError)
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/output_wrapper.rb:24:in `puts'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/output_wrapper.rb:24:in `block (2 levels) in <class:OutputWrapper>'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/documentation_formatter.rb:26:in `example_group_started'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:209:in `block in notify'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:208:in `each'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:208:in `notify'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:118:in `example_group_started'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:659:in `block in for_filtered_examples'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:658:in `each'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:658:in `for_filtered_examples'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:614:in `rescue in run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:610:in `run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:121:in `block (3 levels) in run_specs'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:121:in `map'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:121:in `block (2 levels) in run_specs'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2070:in `with_suite_hooks'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:116:in `block in run_specs'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:74:in `report'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:115:in `run_specs'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:89:in `run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:71:in `run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:45:in `invoke'
	from /usr/share/gems/gems/rspec-core-3.12.0/exe/rspec:4:in `<top (required)>'
	from /usr/bin/rspec:25:in `load'
	from /usr/bin/rspec:25:in `<main>'
/usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2274:in `tty?': closed stream (IOError)
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2274:in `output_to_tty?'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:927:in `color_enabled?'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/console_codes.rb:67:in `wrap'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/documentation_formatter.rb:86:in `failure_output'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/documentation_formatter.rb:51:in `example_failed'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:209:in `block in notify'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:208:in `each'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:208:in `notify'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:145:in `example_failed'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:484:in `finish'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:442:in `fail_with_exception'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:614:in `block in run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:656:in `each'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:656:in `for_filtered_examples'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:614:in `rescue in run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:610:in `run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:608:in `block in run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:608:in `map'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:608:in `run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:121:in `block (3 levels) in run_specs'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:121:in `map'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:121:in `block (2 levels) in run_specs'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2070:in `with_suite_hooks'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:116:in `block in run_specs'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:74:in `report'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:115:in `run_specs'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:89:in `run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:71:in `run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:45:in `invoke'
	from /usr/share/gems/gems/rspec-core-3.12.0/exe/rspec:4:in `<top (required)>'
	from /usr/bin/rspec:25:in `load'
	from /usr/bin/rspec:25:in `<main>'
/usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2274:in `tty?': closed stream (IOError)
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2274:in `output_to_tty?'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:927:in `color_enabled?'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/console_codes.rb:67:in `wrap'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/documentation_formatter.rb:76:in `passed_output'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/formatters/documentation_formatter.rb:36:in `example_passed'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:209:in `block in notify'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:208:in `each'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:208:in `notify'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:139:in `example_passed'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:493:in `finish'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example.rb:293:in `run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:646:in `block in run_examples'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:642:in `map'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:642:in `run_examples'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:607:in `run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:608:in `block in run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:608:in `map'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/example_group.rb:608:in `run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:121:in `block (3 levels) in run_specs'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:121:in `map'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:121:in `block (2 levels) in run_specs'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2070:in `with_suite_hooks'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:116:in `block in run_specs'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:74:in `report'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:115:in `run_specs'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:89:in `run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:71:in `run'
	from /usr/share/gems/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:45:in `invoke'
	from /usr/share/gems/gems/rspec-core-3.12.0/exe/rspec:4:in `<top (required)>'
	from /usr/bin/rspec:25:in `load'
	from /usr/bin/rspec:25:in `<main>'

Operating System

Fedora 38 x86_64

Selenium version

ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux]

What are the browser(s) and version(s) where you see this issue?

No relevant

What are the browser driver(s) and version(s) where you see this issue?

No relevant

Are you using Selenium Grid?

No response

@github-actions
Copy link

@mtasaka, thank you for creating this issue. We will troubleshoot it as soon as we can.


Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

@mtasaka
Copy link
Author

mtasaka commented Dec 31, 2022

So:

mtasaka added a commit to mtasaka/selenium that referenced this issue Dec 31, 2022
…reopen

In selenium rb rspec test suite, rspec mocks `File.exist?` method partway.
With ruby3.2, this may confuse `::Logger.new` using `$stdout` as output because of
`IO` class being changed to accept `#path` method.
Then it may close `$stdout` with `::Logger.reopen` via `have_deprecated` rspec
matcher defined in selenium internally.

So when using have_deprecated matcher defined inside selenium, instead of using
Logger.reopen, redirect stdout directly, using `Tempfile` and reopen `$stdout`
directly.

Also, as said above, with ruby 3.2 `IO` also accepts `#path` method, so
`IO#path.exists?` is now valid on ruby3.2. This affects 'uses the given jar file and port'
testsuite in spec/unit/selenium/server_spec.rb: `File` class must be mocked
so that `#exist?` method accepts also '<STDOUT>` argument.

Closes SeleniumHQ#11498 .
@titusfortner
Copy link
Member

Thanks for bringing this to our attention and walking through all of this.

Couldn't we address this by just initializing the logger at the beginning of the suite so that when it is created nothing is mocked?

@mtasaka
Copy link
Author

mtasaka commented Jan 1, 2023

Well, first I tried so like adding

Selenium::WebDriver.logger.level = :info

to rb/spec/unit/selenium/webdriver/spec_helper.rb, for example, but it didn't work.

After all, :have_deprecated matcher once switch log output to another one, then switch back to $stdout at the end, so when mocking is happening, when returning back from :have_deprecated matcher, log output is in inconsistent state. So next time :have_deprecated matcher is called, this exception happens after all.

@titusfortner
Copy link
Member

The second problem is that the logger specs set it to nil, completely unnecessarily at this point, and the next call to logger has issues.

Just initializing it in the helper

Selenium::WebDriver.logger

And removing the line in the logger spec and it should work as desired without messing with the matchers

@mtasaka
Copy link
Author

mtasaka commented Jan 1, 2023

I may be misunderstanding what you commented, however if you just mean:

$ git diff 10a06007b70ae1a049ef34d5bb05b09696aab7cb
diff --git a/rb/spec/unit/selenium/webdriver/spec_helper.rb b/rb/spec/unit/selenium/webdriver/spec_helper.rb
index dc76372af0..afd6b060da 100644
--- a/rb/spec/unit/selenium/webdriver/spec_helper.rb
+++ b/rb/spec/unit/selenium/webdriver/spec_helper.rb
@@ -48,3 +48,5 @@ RSpec.configure do |c|
 
   c.filter_run focus: true if ENV['focus']
 end
+
+Selenium::WebDriver.logger

Then this is already what I commented and this does not work. Note that (again may be I am misunderstanding what you commented but) Selenium::WebDriver.logger is always initialized with $stdout.

@titusfortner
Copy link
Member

Yeah, I'm saying this line is the other troublemaker that keeps things from working properly: https://github.com/SeleniumHQ/selenium/blob/trunk/rb/spec/unit/selenium/webdriver/common/logger_spec.rb#L32

and it is completely unnecessary because we aren't even referencing Selenium::WebDriver.logger in the spec at this point.

@titusfortner
Copy link
Member

fixed here - 590cfbb

@mtasaka
Copy link
Author

mtasaka commented Jan 2, 2023

Yes, confirmed, thank you.

@github-actions github-actions bot locked and limited conversation to collaborators Dec 9, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants