Introduce config.autoload_lib

[fxn]

Introduction

This patch introduces API to easily autoload and eager load from lib:

# config/application.rb
config.autoload_lib(ignore: %w(assets tasks generators))

Some history

The lib directory was in the autoload paths by default in the early versions of Rails. However, lib stores different kinds of files, and eager loading in production proved to be an issue. Some files in there are not meant to be autoloaded or eager loaded. Think generators, for example.

Because of that, lib was removed from the default autoload paths in Rails 3.

app/lib

Since lib didn't have a good solution, the idea of using app/lib started to emerge.

New app subdirectories are added to the autoload and eager load paths automatically, and a posteriori you put there only regular code (not assets or tasks, for example), it worked out of the box.

However, I have a few remarks about app/lib:

• While app/models and app/controllers mean something, app/lib means nothing. They are at the same level, but naming is not quite consistent.
• lib is meant for things that are kind of tangential to the application core. What's in there feels better located in lib than under app, for me.
• Regardless, Rails programmers should be able to autoload from lib.
• In most projects I have consulted for, lib is added to the autoload paths by hand anyway. People want to autoload from lib, which is understandable. If we can, this use case should have first-class support from the framework.

We have more options now

Zeitwerk is able to ignore files and directories, so now Rails programmers can autoload and eager load from lib with more control. For example:

# config/application.rb
module MyApp
  class Application < Rails::Application
    lib = root.join("lib")

    config.autoload_paths << lib
    config.eager_load_paths << lib

    Rails.autoloaders.main.ignore(
      lib.join("assets"),
      lib.join("tasks"),
      lib.join("generators")
    )

    ...
  end
end

Technically, that is possible since Rails 6, but a better API is possible.

config.autoload_lib(ignore:)

In API design, exceptional use cases may justify exceptional support. You design for the common case, and let the edge case be edge.

In this case, I believe lib deserves ad-hoc API that allows users to do exactly that in one shot:

# config/application.rb
config.autoload_lib(ignore: %w(assets tasks generators))

Values passed to ignore are relative to lib because it does not make sense to ignore anything outside lib using this API, so you enforce that implicitly. As a bonus, the call is short and concise.

Why no default arguments?

I wondered if those three should be a default, but look at this:

# config/application.rb
config.autoload_lib

Is it obvious that there are directories being ignored? No, you open the file and see "autoload lib". That would be a misleading API. I believe a default makes sense when it is a no-brainer for clients, but in this case I don't think it is.

In this case, I believe it is better to be explicit and require that you think about what to ignore. Documentation already takes you there by example.

Why not generated for new applications?

I believe we could eventually include that line in newly generated applications. However, I don't want to rush this, let's have the API out and see. If it works well, perhaps in a future version of Rails.

Why not available for engines?

The lib directory in engines is generally speaking more similar to the lib directory of a gem than to the lib directory of an application. Having lib reloaded in engines is more tricky. By now, I prefer to let people that know what they do configure the autoloaders by hand.

[chaadow]

@fxn Maybe by calling this method, we can also remove lib from Ruby's load path

rails/railties/lib/rails/application.rb

Line 387 in 5c5b78d

def self.add_lib_to_load_path!(root) # :nodoc:

**

only if this option is set to false

[fxn]

@chaadow good call. I think this one is tricky:

• People may expect lib to be in $LOAD_PATH because it is generally there in Ruby projects.
• Rails issues a require to load generators on demand. lib/generators is idiomatic and you'd expect that to work as it did until now.

Let me think about it. Even if we special-case it, it'd need some docs and tests, so the comment was useful in either case, thanks!

[fxn]

In the end, I opted for documenting and adding test coverage for it (#48596). Thanks!