Autogeneration of MyApp::Controller and MyApp::View is magic

[janko]

I would like that autogeneration of MyApp::Controller is removed. This feature suprised me when I found out about it. What happened in my development was that I wanted to make a MyApp::Controller, and then I got warnings of already initialized constant, and some really weird errors (because I tried to make MyApp::Controller a class). There is

unless application_module.const_defined?('Controller')
  # generate MyApp::Controller
end

But it didn't work for me, because I didn't require "my_app/controller" before the app loading (I did it in individual controllers).

Another downside of this approach is that now controllers (and views) which "inherit" from MyApp::Controller (MyApp::View) can't be tested in isolation, because they require MyApp::Application to be loaded (so that it generates the "application" controller). I really liked Lotus' idea of isolated unit-testing, that I can just require the components I need.

I know there is the problem of "inheriting" some configuration from Lotus::Application#configuration, like #handle_exceptions, #format etc. But in my opinion these should have stayed only in Lotus::Controller, for me it's such a perfect idea to configure individual components (I original thought Lotus encouraged that), and not the global Application class itself.

What do you think about this?

[jodosha]

@janko-m TL;DR Lotus offers tools to avoid monolithic applications and to test components in isolation.

Slices

It allows to run multiple applications in the same Ruby process, to encourage isolation of macro functionalities.

Imagine an user facing application, a JSON API and an administration backend.
I've seen a lot of cases like this where the extraction (into a separated service) of the single "slice" was hard because of the tight coupling between them. With Lotus you can build the project planning and developing those "slices" starting from day 0.

Framework duplication

Because of this specific need, you want to keep your configuration separated. For instance, the default response type of your JSON api is application/json, but this isn't the case of the web application that your users will see.

To avoid conflicts between configurations, Lotus has a technique called "framework duplication". It allows to (thread) safely duplicate the components and the configurations of each framework, so they can coexist in the same Ruby process without friction. That's why if you have Web < Lotus::Application and JsonApi < Lotus::Application, you have to include Web::Action and JsonApi::Action.

Testing in isolation

For convenience sake, we configure the behavior of the application with Lotus::Application.configure. The application is able is able to communicate with the several frameworks and to set the proper values. If you use default_format :json, it will set the corresponding value into Lotus::Controller::Configuration#default_format. In other words, you don't have to configure the single frameworks by hand.

# Tedious
router = Lotus::Router.new do
  # ...
end

Lotus::Controller.configure do
  default_format :json
end

Lotus::View.configure do
  root __root__ + '/app/templates'
end

# This is new ;)
Lotus::Assets.configure do
  prefix '/myapp'
end

vs

# Convenient
module Bookshelf
  class Application < Lotus::Application
    configure do
      default_format :json
      templates 'app/templates'

      routes do
        # ...
      end
    end
  end
end

Now, forget for a second the framework duplication. Imagine that you still include the old Lotus::Action, instead of Bookshelf::Action. Because the configuration for Lotus::Action happens in the application definition, you still need to load it to test your actions.

Lotus helps you with this. When you do Bookshelf::Application.load! it preloads and configure the frameworks, without loading all the application classes. In other words, it ignores all your controllers, views, presenters, etc..

In your testing suite you can do:

# spec/spec_helper.rb
RSpec.configure do |config|
  # You load the application environment once
  config.before :suite do
    Bookshelf::Application.load!
  end
end

# spec/controllers/home/index_spec.rb
require 'spec/spec_helper'
require 'path/to/home/index'

describe Bookshelf::Controllers::Home::Index do
  # ...
end

If you run that single spec with bundle exec rspec spec/controllers/home/index_spec.rb, you only have the application configuration and the single controller. Which I think it's a good level of isolation.

Conclusion

I hope all the explanations above are exhaustive on why things are designed like that in Lotus. Closing this. Feel free to ask other questions.

/cc @jeremyf

[janko]

Thank you very much for the detailed explanation. I really appreciate that you care so much that everybody who wants to get involved understands the main concepts. I will try to tag on to your main points.

Framework duplication

To avoid conflicts between configurations, Lotus has a technique called "framework duplication". It allows to (thread) safely duplicate the components and the configurations of each framework, so they can coexist in the same Ruby process without friction. That's why if you have Web < Lotus::Application and JsonApi < Lotus::Application, you have to include Web::Action and JsonApi::Action.

Yes, this is what Lotus::Controller.duplicate does, right? How could the conflicts happen between configurations? Because if I configure Lotus::Controller itself, and include it in both of my microservices, then I expect those 2 microservices to both inherit that configuration (I shouldn't be suprised). I'm explicit about it, so the only conflicts that can happen are by my mistake (which I won't make if I know what I'm doing).

Testing in isolation

For convenience sake, we configure the behavior of the application with Lotus::Application.configure. The application is able is able to communicate with the several frameworks and to set the proper values. If you use default_format :json, it will set the corresponding value into Lotus::Controller::Configuration#default_format. In other words, you don't have to configure the single frameworks by hand.

That's a bummer, because this was for me one of main beauties of Lotus, that I configure each framework separately, that Lotus::Application only needs to do very little work gluing the frameworks together. Because, now for every new Lotus::Controller configuration, you need to remember to decide whether to add it to Lotus::Application also (and how do you decide that?). And the same for Lotus::View, Lotus::Assets and so on.

I really liked the explicitness, and it isn't tedious at all. I have the following structure for controllers (generalize it to views and others):

# my_app.rb
require "lotus"

module MyApp
  class Application < Lotus::Application
    load_paths << ["my_app/controllers"]
    routes do
      # ...
    end
  end
end

# my_app/controller.rb
require "lotus/controller"

module MyApp
  # this line would be if `Lotus::Application` wouldn't create it
  Controller = Lotus::Controller.duplicate(self) do
    # configuration
  end
end

# my_app/controllers/home.rb
require "my_app/controller"

module MyApp
  module Controllers
    module Home
      include MyApp::Controller
    end
  end
end

Now, forget for a second the framework duplication. Imagine that you still include the old Lotus::Action, instead of Bookshelf::Action. Because the configuration for Lotus::Action happens in the application definition, you still need to load it to test your actions.

You see, I configured the controller outside of my_app.rb, so I don't need to require it in the unit tests. And it feels so natural to configure my controller in MyApp::Controller instead of MyApp::Application. I originally did the above structure, and then it took me 4 hours to figure out that Lotus does it automagically, and that it's the reason why I was getting weird errors.

If you run that single spec with bundle exec rspec spec/controllers/home/index_spec.rb, you only have the application configuration and the single controller. Which I think it's a good level of isolation.

Yes, I agree, at the end it is testing in isolation. But it still strongly doesn't feel right that you need to require MyApp::Application in your controller tests, instead just having to require the controller you're testing. And Lotus is so close to not having to do that, because you made a lot of great decisions early on.

[jodosha]

@janko-m You're welcome.

Without framework duplication there will be race conditions.
Look at this simplified example:

# json_api.rb
Lotus::Controller.configure do
  default_format :json
end

# web.rb
Lotus::Controller.configure do
  default_format :html
end

# At this point Lotus::Controller.configuration.default_format is :html
# When you load the code for both the applications, and then inject the configuration, they will both see `:html`.

module JsonApi
  class MyAction
    include Lotus::Action
  end
end

module Web
  class MyAction
    include Lotus::Action
  end
end

JsonApi::MyAction.configuration.default_format # => :html WRONG
Web::MyAction.configuration.default_format # => :html OK

Speaking of convenience, you may not find tedious to configure all the frameworks separately, but they are a lot of people that are finding the current usage of Lotus a bit verbose. So why not streamline what can be easily achieved?

In conclusion, if you find those conventions not adhering to your point of view, Lotus is also a DIY framework. You can always pick the separate components and make them to work together. Don't get me wrong, I don't want to be rude here, just communicating the power of having reusable gems. 😉

[janko]

Yes, that is a race condition, but it's an obvious one, because you're configuring a Lotus::Controller, which you include in both apps, so of course it will be the same one :). I think every user will know that they have to create JsonApi::Controller and Web::Controller, and configure those ones if they don't want the configuration to be shared.

About convenience, doesn't it feel wrong (tedious) for you to have to add each configuration from each framework to Lotus::Application? I think it will often be out of sync (for example, currently #modules and #format are missing in Lotus::Configuration), so users will "have to" configure individual frameworks for those things. I hope you see here that my wish is to make Lotus easier to develop, and I think it doesn't trade users' convenience at all.

I just wanted to show you the advantages of my point of view about this problem, because I'm so sure it's the right way, but I can stop now :). Thank you for listening me out; you're right, maybe I can just use the frameworks separately.

[jodosha]

Yes, that is a race condition, but it's an obvious one

How to solve this problem then? 😉

I think it will often be out of sync (for example, currently #modules and #format are missing in Lotus::Configuration)

Good catch, it's because the configuration API for the single frameworks is still under high development. So I'm slowly porting them.

I have another idea in mind to avoid the porting at all:

module Bookshelf
  class Application < ::Lotus::Application
    configure do
      controller.default_format :json
    end
  end
end

That syntax will talk to Bookshelf::Controller.configuration.default_format directly. No sync between frameworks and Lotus and it clarifies which framework we are configuring. Not sure if this will land in the framework at all.

[jeremyf]

From the outside looking in (on this conversation) @jodosha is extending an olive branch to those requesting a less verbose means of configuring Lotus. Its a higher level of abstraction/obfuscation; but is something Rails "hostages" may be needing.

And @janko-m you are advocating for a more explicit and verbose process for configuration. Perhaps you might be willing to craft a guide/application that uses your preferred process. This is something that would fit great in Lotus::Docs.

I would love to see guides/examples of both.

[janko]

How to solve this problem then? 😉

I'm not sure I understand what you mean, but isn't this the solution?

module JsonApi
  Controller = Lotus::Controller.duplicate(self) do
    default_format :json
  end
end

module Web
  Controller = Lotus::Controller.duplicate(self) do
    default_format :html
  end
end

I have another idea in mind to avoid the porting at all:

This solution looks much better 👍

@jeremyf Good advice, I will definitely submit an example application, because I came to this by working on a real application.

[jodosha]

@janko-m But that creates the JsonApi::Controller and Web::Controller duplication as well. I assumed that you didn't liked it.

[janko]

No, I do like it :), I just don't think that Lotus::Application should do it by itself, because I found it suprising when I first found out about it (I first created MyApp::Controller, and then I was getting warnings of already initialized constant, because Lotus::Application also did it).

[twe4ked]

This thread is gold and should be linked somewhere :)