Release 1.1.7 introduced failure on `require 'concurrent/atomic/ruby_thread_local_var'`

[emilyst]

Release of concurrent-ruby 1.1.7 appears to have introduced a regression when requiring 'concurrent/atomic/ruby_thread_local_var' directly.

* Operating system:                mac
* Ruby implementation:             Ruby
* `concurrent-ruby` version:       1.1.7
* `concurrent-ruby-ext` installed: no
* `concurrent-ruby-edge` used:     no

Below is a snippet which demonstrates the current behavior (in 1.1.7) and the expected behavior (in 1.1.6).

emilyst@MBP15-16-500-WQ123 ~ % gem install concurrent-ruby -v 1.1.7
Fetching concurrent-ruby-1.1.7.gem
Successfully installed concurrent-ruby-1.1.7
Building YARD (yri) index for concurrent-ruby-1.1.7...
Done installing documentation for concurrent-ruby after 2 seconds
1 gem installed
emilyst@MBP15-16-500-WQ123 ~ % pry
[1] pry(main)> require 'concurrent/atomic/ruby_thread_local_var'
NoMethodError: undefined method `on_cruby?' for Concurrent:Module
from /Users/emilyst/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/concurrent-ruby-1.1.7/lib/concurrent-ruby/concurrent/atomic/ruby_thread_local_var.rb:39:in `<class:RubyThreadLocalVar>'
Caused by LoadError: cannot load such file -- concurrent/atomic/ruby_thread_local_var
from /Users/emilyst/.rbenv/versions/2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
[2] pry(main)>
emilyst@MBP15-16-500-WQ123 ~ % gem uninstall concurrent-ruby -v 1.1.7
Successfully uninstalled concurrent-ruby-1.1.7
emilyst@MBP15-16-500-WQ123 ~ % gem install concurrent-ruby -v 1.1.6
Successfully installed concurrent-ruby-1.1.6
Building YARD (yri) index for concurrent-ruby-1.1.6...
Done installing documentation for concurrent-ruby after 1 seconds
1 gem installed
emilyst@MBP15-16-500-WQ123 ~ % pry
[1] pry(main)> require 'concurrent/atomic/ruby_thread_local_var'
=> true
[2] pry(main)>

Thanks in advance!

[pitr-ch]

Hi, this is not really a supported use case. Officially only require "concurrent" is supported since we cannot track the dependencies between files in CI to make sure partial requires work. However, I'd be happy to merge a PR which fixes this. Please let me know if you would find time to contribute it, thanks.

[emilyst]

Hi, this is not really a supported use case. Officially only require "concurrent" is supported since we cannot track the dependencies between files in CI to make sure partial requires work. However, I'd be happy to merge a PR which fixes this. Please let me know if you would find time to contribute it, thanks.

Thanks for the reply. I suspected something like this would be the case. We have already worked around the issue at work by using require "concurrent". The only bad part is that it means that we drag in the entire library into memory just to get the RubyThreadLocalVar stuff.

I suspect you're right that this is a use case not worthwhile for you to support, so I'm happy to let things be for now. I just thought you (and others) might like to be aware. Thanks!

[Prajjwal]

Hi,

It would indeed be a pain to track dependencies between files in CI, but wouldn't that be completely eliminated by using a module loader? Wanting to load only the stuff you want in memory is a common enough use case, and many gems do this already.

I've only glanced at the codebase so I'm not sure how much work this would be, but I'm open to working on it if a maintainer feels like its possible.

[pitr-ch]

@Prajjwal thanks you for your interest! What do you mean by module loader, Zeitwerk?

[Prajjwal]

@pitr-ch Yes, Zeitwerk is probably what I'd try first.

• It is quite opinionated about directory structure and class naming, although there's ways to hook into it and override things if you want. This is both a pro and con - on one hand you don't have to think about how to structure anymore, and on the other you might have to restructure the project to conform to it.
• Classes and modules are lazy loaded automatically, which is a huge win imo. You end up with only what you need in memory.

I'd understand if you're skeptical about introducing a fairly large dependency, so its worth mentioning that many Gems (including the venerable Rack) use Module#autoload, which is basically a manual way of lazy loading stuff. There's some talk of deprecation, but personally I think it is fine. Zeitwerk also uses autoload under the hood.

I have a hunch that autoload might be way less work for this codebase, but it's worth mentioning that it doesn't solve the original problem, which is broken partial dependencies which will have to be tracked and expressed the same as with require.

Another thing worth thinking about is whether all this is portable across implementations.

I'll throw something together this weekend as a poc.

[pitr-ch]

Thank you for your interest @Prajjwal !

There is bunch of related issues:

• maybe Create a clear require interface #714
• has to be fixed RubyMine integration fails due to custom gem require path #873
• has to be doable later Merge edge into main gem #713
• fix directory structure, classes and their namespaces do not match directory structure

One important thing to think about is that we cannot break backward compatibility. Even though only require "concurrent" is officially supported people do require just parts of the library and it would be good either have some transition or

Zeitwerk is not off the table. I've investigated it as well. The problem is that concurrent-ruby supports far older Ruby's then Zeitwerk. There would have to be some eager loading fallback added. Or it should be scheduled with some old ruby support drop.

I would very much appreciate any help with this. However, this a complex problem, maybe we should have a chat to make sure I'll share with you everything I can. Please DM me on https://twitter.com/pitr_ch and lets schedule something.

[Prajjwal]

Hey @pitr-ch .

We could start by just autoloading the top level components and see if that gets anywhere. I've messaged you on twitter (handle: prajjwalsin). Let's have a chat soon.

[eregon]

#934 has been implemented, so one can require specific classes of concurrent-ruby in 1.2.0+.

Re the original issue, requiring implementation paths like concurrent/atomic/ruby_thread_local_var which don't correspond to a class in the documentation is unsupported (similar to #986 in that regard).