Provider and booter overhaul, including class-backed provider sources

[timriley]

After renaming "bootable component" to "provider" in #200, this PR completely overhauls the Provider and Booter structure and capabilities.

The biggest change is that it backs the register_provider block-based DSL with real classes implementing each provider's behavior, via Dry::System::Provider::Source. This "Provider Source" is defined by the block-based DSL in two ways:

• When you're defining a standalone provider inside your application, a new subclass of Provider::Source is created, then the block passed to Dry::System::Container.register_provider is used to define the body of that class. Specifically, it will define real instance methods for each of the lifecycle steps: #prepare, #start, and #stop.
• When you're defining a provider that uses an external provider source, then the provider source (via Dry::System.register_provider_source) will have already created and defined a subclass of Provider::Source with instance methods for its specific behavior, and then the block passed to .register_provider is then evaluated in the context of the instance of that source, allowing you to provide config (via the built-in Provider::Source#configure) or define lifecycle step callbacks (via the built-in Provider::Source#before and #after).

The block DSL evaluation context has become much simpler in this change, too. There's just two clear contexts, which are hopefully quite clear now that we have class-backed provider sources:

• When defining a source, self is the Provider::Source subclass
• When defining a source, inside the body of the prepare, start, stop blocks, self is the Provider::Source subclass instance
• Likewise using an external source, self is the Provider::Source subclass instance

For the latter two blocks, we no longer need block arguments to access the provider container or the target container. These are available as methods that are already available on self in each of the block evaluation contexts (#provider_container/#container and #target_container/#target). For the initial block, access to the target container is intentionally limited because any taget container access should be done inside the latter two block contexts anyway. This should make the providers easier to reason about because there's now just one way to access the provider/target containers.

As the final step of provider setup, a Dry::System::Provider is initialized, and is provided its specific Provider::Source subclass. Provider takes care of all the standard behavior for each Provider, such as setting up the provider container, which it passes to the Source (among other things) when it initializes it. Provider also takes care of all the provider lifecycle, making sure that e.g. lifecycle steps aren't run when they don't need to be, and that we merge components from the provider container into the target container after each lifecycle step.

With Provider taking full responsibility for the standard Provider behavior and lifecycle, this has allowed the (former) Booter to become much simpler: it's now focused on the loading of the provider files themselves, and then registering, storing, and providing access to the Provider instances. The rest of its behavior is just simple delegations to those Provider instances. I've renamed this to ProviderRegistrar to reflect its new behavior and fit better with the overall set of names we have now.

---

One really nice benefit of the new class-based provider sources is that we can now create our own source classes directly, rather than having to use the DSL only. This can be really helpful for complex providers:

module Test
  class LoggerProvider < Dry::System::Provider::Source
    def prepare
      require "logger"
    end

    def start
      logger = complex_custom_logic_to_build_the_logger(config)
      register(:logger, logger)
    end

    private

    def complex_custom_logic_to_build_the_logger(config)
      # ...
    end
  end
end

Test::Container.register_provider(:logger, source: Test::LoggerProvider)

Better yet, class-based provider sources allow us to share state between the lifecycle steps! Previously the only way we could do this was by having to register components in the provider container. For example:

module Test
  class LoggerProvider < Dry::System::Provider::Source
    def prepare
      @internal_thing = "something we may want to set up initially but we still need inside start"
    end

    def start
      # Look, I can access @internal_thing here, because this is just a PORO!
    end

    private

    def complex_custom_logic_to_build_the_logger(config)
      # ...
    end
  end
end

Test::Container.register_provider(:logger, source: Test::LoggerProvider)

But it doesn't stop there, because this also takes external provider sources to the next level. With class-based sources we can now have our external sources provide their own methods to be used when those sources are used when registering a provider. I've used this approach to straighten out our first-party settings provider source, which previously hijacked the provider settings object (which was intended only for providers to configure their sources) to load settings from env. This always twisted my brain whenever I had to think about it. Now, it's much more straightforward. See the source class:

module Dry
  module System
    module ProviderSources
      module Settings
        class Source < Dry::System::Provider::Source
          def prepare
            require "dry/system/provider_sources/settings/settings"
          end

          def start
            config = settings.load(target.root, target.config.env).config
            register(:settings, config)
          end

          def settings(&block)
            # Save the block and evaluate it lazily to allow a provider with this source
            # to `require` any necessary files for the block to evaluate correctly (e.g.
            # requiring an app-specific types module for setting constructors)
            if block
              @settings_block = block
            elsif @settings_class
              @settings_class
            elsif @settings_block
              @settings_class = Class.new(ProviderSources::Settings::Settings, &@settings_block)
            end
          end
        end
      end
    end
  end
end

Dry::System.register_provider_source(
  :settings,
  group: :dry_system,
  source: Dry::System::ProviderSources::Settings::Source
)

You'll see here that it defines its own #settings instance method, which can then be used when the provider source is used to register a provider:

register_provider(:settings, from: :dry_system) do
  before(:prepare) do
    target_container.require_from_root "types"
  end

  settings do
    setting :database_url, constructor: SettingsTest::Types::String.constrained(filled: true)
    setting :session_secret, constructor: SettingsTest::Types::String.constrained(filled: true)
  end
end

This all just works now because of those updated block-based DSL evaluation contexts, with the register_provider context above being the instance of the provider source, which lets us provide instance-level settings in a natural way, while still having those settings available to the lifecycle methods of the provider source (because they too are instance methods of the same object), which lets us take objects provided from the register_provider block and use it from within the source behavior.

I could imagine this being really helpful for complex provider sources. For example, if we wanted to provide one for rom-rb and we wanted to provide lots of targeted hooks for users to supply their own config or behaviour, in our provider source class we could set up as many instance methods as we want for this.

Finally, some other notable changes made in this PR:

• The default provider source class-level settings and instance-level config are now powered by dry-configurable rather than us building a custom Dry::Struct. This feels much more natural and more in-keeping with our overall ecosystem.
• Renamed the previous "source provider" concept to "provider source", since it fits much more naturally with the class structures and naming schemes we have in place in the code in this PR
• Renamed "dry/system/components" to "dry/system/provider_sources" (and adjusted the class namespacing to match)
• Renamed the group for our provider sources to :dry_system (from :system) in order to set a healthier precedent for third party use (i.e. use your own gem name for your group to ensure no conflicts).

These changes should all be fully backwards compatible, with a suite of deprecation shims and warnings in place.

[timriley]

Thanks for the feedback so far, folks 😄

@solnic, I applied all your suggestions!

I'll merge this one tomorrow. Happy for continued feedback post-merge, too, of course!

[flash-gordon]

side note, if we drop 2.6 we can delegate with ...

[timriley]

I'd love to do this. I'm going to make a GH issue for this and will PR it separately: #203

[timriley]

Thanks for the thorough feedback, @flash-gordon! ❤️