Implementing Routing Concerns · 0dd2472 · rails/rails

crankharder · August 21, 2012

While this is cool, and abstracted well, do we really need it?

We already have methods:

YayApp::Application.routes.draw do
  def add_posts
    resources :posts, :only => [:create, :destroy]
  end

  resources :events do
    add_posts
  end
end

coreyhaines · August 21, 2012

Is there an actual problem this is solving?

evanphx · August 21, 2012

Why is this a module? I see it being included a single place and it clearly is heavily dependent on being included into this exact class because it has a data dependency on @concerns. I'd imagine it's not designed to be used anywhere else, so why bother?

tenderlove · August 21, 2012

Yes, can we make this not a module please! :-)

coreyhaines · August 21, 2012

This has the danger of promoting bloated routes, as inappropriately large routes are added via a concern. This seems like a misunderstanding of what is meant by duplication.

timoschilling · August 21, 2012

@crankharder DSL is better and more cleaner Code than defining a Method!

carlosantoniodasilva · August 21, 2012

I think it's just following the pattern already implemented in this mapper file, where a group of related methods are put together in a module, such as HttpMethods, Scoping, or Resources.

About @concerns, maybe the same could be said about the @scope variable that's being used everywhere inside the mapper (across different modules I think)? I believe it's initialized down there just to avoid the defined? call.

At the end, I think the module is not meant to be used anywhere else, it's just a way of grouping similar things together, so whatever you say :).

coreyhaines · August 21, 2012

I agree the same could be said about all the instance variables there. Leaking the abstraction across is a reasonable flag that there is a problem here. You could certainly get away without a defined? call. Why not rely on the abstraction issue here to write the code a bit better.

rkh · August 21, 2012

You really like the word "concern", don't you?

coreyhaines · August 21, 2012

But then, I think this is a horrible addition to the router that doesn't appear to be actually solving a problem, as well as poor code.

carlosantoniodasilva · August 21, 2012

I think it's not leaking if the modules are not meant to be used elsewhere, or not? In any case, please feel free to suggest changes or put up a pull request improving the code, we'll be glad to help as possible.

ps: I'm not talking about the feature itself, but the mapper implementation, since I've never had this requirement so far

mboeh · August 21, 2012

@timoschilling It's certainly more code.

bjeanes · August 21, 2012

I wonder if taking advantage of Ruby's open classes is a better approach for these types of issues. Instead of using different modules (which imply shared reusability) that are functionally and conceptually coupled together, one could instead have multiple files that re-open the same class to add in extra behavior. You get the same net effect of grouping methods but without the intent ambiguity that module's bring as baggage.

This isn't thought out and was just a spur of the moment idea, so it probably has flaws — but hell, maybe it doesn't.

Thoughts?

coreyhaines · August 21, 2012

I'd like to come back to @crankharder's comment. How in the world is this better than using an actual Ruby mechanism: methods? Vague statements about DSL is better and cleaner code are hardly answers as to what makes this better. It actually is more opaque than just saying

def has_comments
    resources :comments
end

resources :videos do
  has_comments
end

I'm missing how renaming this to concerns adds value.

coreyhaines · August 21, 2012

@carlosantoniodasilva I'd like to suggest the change that this be replaced by a convention of enacting reuse through Ruby's standard mechanism: methods. This feature does not appear to add anything other than renaming "method" to "concern" through a series of indirections.

carlosantoniodasilva · August 21, 2012

Alright, fair enough. May I ask you to use the Ruby on Rails Core mailing list to ask for feedback on this, linking this commit and the related comments, so we can gather more feedback there. Thanks.

coreyhaines · August 21, 2012

Sent. Thanks.

yfeldblum · August 21, 2012

We write languages for a problem space such that solutions may, in that language, be readable, concise, and elegant.

Then we write solutions to our problems, and we write our solutions in these tailored languages. That way, our solutions turn out to be the most readable, concise, and elegant that they can possibly be.

That is why there exists a routing DSL to begin with, and why it was improved for Rails 3.

ernie · August 21, 2012

I can't get behind a change like this. This isn't even syntactic sugar. It's more like syntactic aspartame. It might slim down your code, but it's also a carcinogen.

This pattern of adding abstractions that aren't self-explanatory just adds to the specialized dialect needed to read Rails code, when anyone who can read Ruby would understand methods... it's... concerning.

coreyhaines · August 21, 2012

@yfeldblum This is hardly a case of what you are talking about. The routing DSL has a certain elegance to it. It deals with items in the domain of routing: resources, map, etc. This, however, takes a separate idea "concerns" and shoe-horns it in. There is no "concern" concept in the domain of routing.

dball · August 21, 2012

I don't like the change. Even the name "concern" suggests that there really isn't a general routing concept being reified.

pixeltrix · August 21, 2012

I'm not a big fan of this but I'm not so caremad that I'd want it removed, however it's basically syntactic sugar for this:

commentable = lambda do
  resources :comments
end

resources :posts do
  nested &commentable
end

resources :articles do
  nested &commentable
end

In fact using lambdas is more flexible as they can be passed to any of the mapper methods like namespace - see this Stack Overflow question.

bogdan · August 22, 2012

I also dislike this change. I would probably be more happy with code example by @pixeltrix rather than concern in my routes file because I can understand how it works without reading documentation.

codeodor · August 22, 2012

I've seen one project recently where I think this may have come in handy:

There were a ton of "concerns" that were spread across varying degrees of nested-ness, in which some of the final routes were not the same, but should have been. In this case, I think the mess would have been around anyway because there was so little care or thought put into it I'm doubtful concerns would have been used.

@yfeldblum mentions

We write languages for a problem space such that solutions may, in that language, be readable, concise, and elegant.

That's a laudable goal, but I don't know if this gets us there. "Concern" in my opinion is a terrible name for it. I understand it from an aspect-oriented programming jargon standpoint, but I don't think that's what it should be called here. On the other hand, I don't have a better name to offer.

Most of the examples show :concerns as an option. But can it be called inside the block too? I ask because I think that would be a more readable place for it.

On the surface of it, I kind of like the idea. But I'm not sure if it's worth the risk in additional effort the maintainers will endure. Since I'm not one, I have no bone to pick either way, but I'd be interested to hear contributors' opinions on this aspect.

coreyhaines · August 22, 2012

+1 for @pixeltrix example

@codeodor Often times, when we find a need for something like this, it really should be an indication that there is a design issue with the routes. This seems to be a case where a helper is created to mask a flaw with highly nested routes.

dmathieu · August 22, 2012

I concur with all the coming here. This adds an overkill feature. It was fine as a gem to me, and should have stayed that way.

ernie · August 22, 2012

This seems to me like one of those things that should remain a gem. As @pixeltrix and others have demonstrated, this can be achieved using plain Ruby, today, in a readable way -- more importantly, in a way that is immediately approachable by someone who understands Ruby. No need to go check the Rails docs the first time the user encounters this syntax, since it's just a lambda or method definition.

If a team wants to use it, however, more power to them, use the gem.

dhh · August 22, 2012

Lots of things in Rails can be achieved by using "plain Ruby". That's not the standard of which I decide whether to add a feature to Rails or not. Before/after code is the standard.

Here's the before using methods:

BCX::Application.routes.draw do
  def commentable do
    resources :comments
  end

  def trashable do
    post :trash, :restore, on: :member
  end

  def image_attachable do
    resources :image_attachments, only: :index
  end

  resources :calendar_events do
    commentable
    get :past, on: :collection
  end

  resources :messages, :forwards, :uploads, :documents, :todos do
    commentable
  end

  resources :projects, concerns: :trashable, defaults: { bucket_type: 'project' } do
    resources :messages, :uploads, :comments do
      trashable
      image_attachable

      resources :forwards do
        trashable
        image_attachable
        get :content, on: :member
      end
    end
  end
end

Here's the after using concerns:

BCX::Application.routes.draw do
  concern :commentable do
    resources :comments
  end

  concern :trashable do
    post :trash, :restore, on: :member
  end

  concern :image_attachable do
    resources :image_attachments, only: :index
  end

  resources :calendar_events, concerns: :commentable do
    get :past, on: :collection
  end

  resources :messages, :forwards, :uploads, :documents, :todos, concerns: :commentable

  resources :projects, concerns: :trashable, defaults: { bucket_type: 'project' } do
    resources :messages, :uploads, :comments, concerns: [:trashable, :image_attachable]

    resources :forwards, concerns: [:trashable, :image_attachable] do
      get :content, on: :member
    end
  end
end

Benefits:

Defining methods inside the draw DSL is a visual wart that doesn't fit with the rest of the flow.
The method calls do not signify what's going on when just looking at the call itself. What's this call going to do? The concern clearly marks this as "extra, shared routes being mixed into the resource".
The use and the word concern is consistent with its use for models and controllers. Concerns for resources are about sharing roles, like Trashable, Commentable, and ImageAttachable. In fact, we have controller or model concerns with the very same name in the Basecamp code base. Thus parity is established, the domain language is reused, and its imminently clear that there's a line to be traced throughout the code base.
The concern syntax allows you to declare multiple on a single line. This is especially nice when you have 2, 3, or more, like we do in the Basecamp code base.

Feel free to suggest alternative implementations, but the feature stays and so does the name.

dhh · August 22, 2012

Also, if you in general don't like the word concern, you're going to be in for a bad time. We're adding a default place for model and controller concerns to live in the directory structure shortly.

I find concerns to be a wonderful addition to the domain language and a key building block for making beautiful Rails applications. So we will spread that beauty far and wide.

coreyhaines · August 22, 2012

And conversation is shut down. Shame.

And, yes, if this is the direction Rails is going, and with a history of working with teams trying to maintain their applications after use of these "features", I agree that we're going to be in for a bad time. Thanks for taking part in the conversation, David.

dhh · August 22, 2012

Rails has never had much care for "what if this was abused!". We make a framework for people to aspire to do their best work. Not to shy away from great features, just because there might be someone, somewhere who could abuse it.

rkh · August 22, 2012

Also, if you in general don't like the word concern, you're going to be in for a bad time. We're adding a default place for model and controller concerns to live in the directory structure shortly.

I find concerns to be a wonderful addition to the domain language and a key building block for making beautiful Rails applications. So we will spread that beauty far and wide.

That's why I think naming this feature a concern, too, might be confusing. It's better than base, though.

dhh · August 22, 2012

So "the direction Rails is going" is entirely unchanged from 2003, in terms of the philosophical underpinnings. You're of course free to dislike individual features and if you want to keep talking about that here, that's fine too. But maybe leave the drama violins on the shelve.

coreyhaines · August 22, 2012

On Wed, Aug 22, 2012 at 8:17 AM, David Heinemeier Hansson < notifications@github.com> wrote: Rails has never had much care for "what if this was abused!". We make a framework for people to aspire to do their best work. Not to shy away from great features, just because there might be someone, somewhere who could abuse it.

It isn't a matter of "what if this was abused!" as much as a question of "is this leading people towards a reasonable design." There are some great features in Rails that contribute towards productivity. This isn't one of them.

…

dhh · August 22, 2012

@rkh, given the usage that I've put this feature through, I've found it to be illuminating. It draws that connection between what's going on in the domain of the controllers/models straight to the resources that serve as a the gateway to them.

dhh · August 22, 2012

@coreyhaines, that's obviously where we disagree. I think this leads to a much better design. It's beautiful, it's dry, and it fits into the larger story arc of concerns in other parts of the code. I hope people start using concerns to trace those same lines through their domain model as they adopt Rails 4. Let dozen of concerns bloom!

tomash · August 22, 2012

I like the consistency it introduces with modules that extend ActiveSupport::Concern. But the example shown by @dhh is not much shorter and I don't see much added value over just using subresource-adding methods. Is anybody really working with a codebase where using the new concerns syntax would shorten the routing file significantly? (as in: having large amount of orthogonal concerns)

radar · August 22, 2012

I disagree with this also. lambdas and methods are a much better way that doesn't require any more code within Rails itself to have this feature.

mariovisic · August 22, 2012

I also disagree. Quickly scanning the two BCX example routes. I can immediately see what the first one is doing, the methods are highlighted and I can see where they are being included. In the second example the concerns blend in very easily with the other routes and it takes me longer to scan and see how the routes are working. New users would also have to be familiar with what the concern method does.

Although a shorter routes file is nice, using methods is much easier to read and reduces the need for extra code to handle this.

coreyhaines · August 22, 2012

@dhh True, this is a fundamental disagreement. I work with teams that are struggling to keep a level of productivity up in large part due to heavy reliance on these sorts of features (for example, currently cleaning up a codebase that is mired in ActiveSupport::Concern for no good reason at all, when including a module was what they wanted). I wouldn't say they've abused these features, they just relied on them. Then the codebase becomes rigid and adding new features is more difficult as their ability to get any feedback on the effect of the changes starts to drop significantly. As more parts of the system become heavily coupled to the other parts, I've seen more than one codebase calcify.

I appreciate that your specific use of this feature may have convinced you it is useful, and that is a wonderful reason to write a gem for it; that's one of the great things about the Rails ecosystem. There are a lot of us who appreciated the apparent shift in Rails 3 towards a more modular design, rather than taking specific use cases and baking them into the core. For people who run into the case where they have a tremendous number of repeated routes, they can choose to use this gem.

I think our fundamental disagreement stems from the fact that I, based on my work with various teams, consider Rails a nice set of conventions, a fantastic routing system and a very useful ORM. I believe (based on comments you've made, but I could be wrong) that you consider it more all-encompassing than that. The teams I've worked with who have treated it as the whole environment see their productivity significantly drop due to the high coupling between the different parts, starting after a number of months, then dramatically after year one, in general. As I sit with teams, I don't find they've "abused" the features, they simply used them. This experience is what has caused a lot of us to start talking about alternate ways to write long-term maintainable applications with Rails.

As we've both said, there is a fundamental disagreement between us based on our own personal experiences with building Rails applications and working with teams who are building their businesses with Rails-based applications. As with everything, people's mileage may vary depending on how they choose to build.

yfeldblum · August 22, 2012

Methods may happen to work now. But is it actually part of the API that they should work? IE, that self inside a scope or resources block should be the same as self outside the block, or should delegate unrecognized messages to self outside the block?

ernie · August 22, 2012

@dhh I don't think, given the general tone of your response, that any alternative provided will receive serious consideration. That being said, I'm going to post one, anyway -- it doesn't really address the spirit of the assertions that @coreyhaines, @pixeltrix and others are making, but it does at least make some lemonade out of the given lemons.

dhh · August 22, 2012

Ernie, When I said implementation, I meant the internals of the feature (say whether it uses a module or whatever). But feel free to propose another API too. If you can come up with something that satisfies the list of benefits outlined above and is an improvement on the before/after, I'm all ears. Replaying existing arguments from this thread that have already failed to convince me would be a waste of time, though. But as long as its new and original arguments or benefits, bring it on! On Aug 22, 2012, at 9:06, Ernie Miller <notifications@github.com> wrote:

…

dhh · August 22, 2012

Corey, taking specific use cases and baking them into Rails is what Rails is! It's a collection of solutions to specific use cases encountered by me and the rest of core and other contributors. I don't work on any code for Rails that does not take offset in a specific use case in my code base. Never have. That's why Rails is good. It wasn't made by a bunch of consultants sitting around and imagining abstract use cases that their clients might have. There are plenty of alternatives that follow that model. Look at any framework backed by a consulting firm and you'll find that style.

…

On Aug 22, 2012, at 8:56, Corey Haines <notifications@github.com> wrote: @dhh True, this is a fundamental disagreement. I work with teams that are struggling to keep a level of productivity up in large part due to heavy reliance on these sorts of features (for example, currently cleaning up a codebase that is mired in ActiveSupport::Concern for no good reason at all, when including a module was what they wanted). I wouldn't say they've abused these features, they just relied on them. Then the codebase becomes rigid and adding new features is more difficult as their ability to get any feedback on the effect of the changes starts to drop significantly. As more parts of the system become heavily coupled to the other parts, I've seen more than one codebase calcify. I appreciate that your specific use of this feature may have convinced you it is useful, and that is a wonderful reason to write a gem for it; that's one of the great things about the Rails ecosystem. There are a lot of us who appreciated the apparent shift in Rails 3 towards a more modular design, rather than taking specific use cases and baking them into the core. For people who run into the case where they have a tremendous number of repeated routes, they can choose to use this gem. I think our fundamental disagreement stems from the fact that I, based on my work with various teams, consider Rails a nice set of conventions, a fantastic routing system and a very useful ORM. I believe (based on comments you've made, but I could be wrong) that you consider it more all-encompassing than that. The teams I've worked with who have treated it as the whole environment see their productivity significantly drop due to the high coupling between the different parts, starting after a number of months, then dramatically after year one, in general. As I sit with teams, I don't find they've "abused" the features, they simply used them. This experience is what has caused a lot of us to start talking about alternate ways to write long-term maintainable applications with Rails. As we've both said, there is a fundamental disagreement between us based on our own personal experiences with building Rails applications and working with teams who are building their businesses with Rails-based applications. As with everything, people's mileage may vary depending on how they choose to build. — Reply to this email directly or view it on GitHub.

ernie · August 22, 2012

David,

In keeping with your overall feeling about concerns' utility in the routes file, PR #7422 is a slight modification that at least provides for some code separation benefits as a result.

dmathieu · August 22, 2012

The general problem of this, with the ruby implementation looks very similar to memoizable, which has been deprecated in favor of ruby's internal memoization.
If things like this can get removed because ruby allows to do the same very easily, why include concerns at all ?

dhh · August 22, 2012

As mentioned on that pull request, thanks for putting time into actually making a code argument. It fails my before/after filter, but I do appreciate the effort.

…

rrouse · August 22, 2012

@dhh Feedback on why it fails? Could help refine it a bit more.

dhh · August 22, 2012

It's more code, it's uglier, and it doesn't bring any tangible benefits to the routing DSL. (the benefits of "it is it's own class!" is not compelling to me).

…

ernie · August 22, 2012

Reposing since the conversation is happening here:

@dhh At a minimum, it allows us to shorten the routes file considerably by moving concerns out into their own separate files (not shown here).

That would seem to pass the basic before/after test you're proposing. leading to a header in the routes file that read like:

  concerns :commentable, Commentable
  concerns :reviewable, ReviewableConcern.new(some: 'initializaton params')

With a minor modification, it could also eliminate the need for the second parameter altogether, via the same classifying logic as is used elsewhere.

Beyond this, there is strong preference in the Ruby community for implementations that allow duck-typed objects to be swapped in where appropriate.

Consider the differences between CarrierWave and Paperclip, for instance. The former allows for much more idiomatic separation of code, reducing the amount of noise we have to read through in order to parse the basics, and points us to a specific location to find out more should we so choose.

It's less about simplifying the writing of the code (though some would argue that anything that allows us to break things into a separate object if we so want will be a win) and more about a further "improvement" to readability, if we're going to go down this route. (pun not intended, but awesome)

dhh · August 22, 2012

Reposted the reply as well:

But if you look at the actual use cases this was extracted from, they don't justify an external class. They're too small. They're usually just a single line or two. Moving them into their own classes and putting those classes outside of the routing.rb file doesn't clarify things in my mind.

This has similarities to the "actions should be classes" debate. I don't think they should. There's not enough there to justify it and it makes things harder to follow.

Again, though, I very much appreciate this level of debate, Ernie. It's focused around real code alternatives, so it's concrete. Not abstract, hand-wavy "I've seen teams use this wrong!" kind of arguments. Thank you for that.

steveklabnik · August 22, 2012

I don't particularly care about the 'accepts a class' bit, but Validations now work this way: custom class, symbol that references a method name, or a block.

dhh · August 22, 2012

I think that's actually overkill for most cases when you write your own validations. I even think it's overkill for most of the built-in validations, but that's a separate debate. In this case the value of showing the 1 or 2 lines with the rest of the routes is worth more than any benefits of encapsulating this in a class.

…

ernie · August 22, 2012

@dhh Maybe -- my gut feeling is that since this doesn't break the previous style of usage, at all, it's a net positive.

I don't have time to write up additional code examples, but I'm thinking that developers of engines may find it useful.

At the very least, it allows us to intelligently select the controller (and therefore, associated view files, and on down the line) that handles the comment functionality, without having to use a separate concern, reducing the working vocabulary of the application reflected in the routes file to only the level of detail necessary.

coreyhaines · August 22, 2012

On Wed, Aug 22, 2012 at 9:32 AM, David Heinemeier Hansson < notifications@github.com> wrote: Corey, taking specific use cases and baking them into Rails is what Rails is! It's a collection of solutions to specific use cases encountered by me and the rest of core and other contributors.

I agree. The parts of Rails that make it very useful are the parts that cover a range of cases in a nice fashion. In this specific case, it doesn't have that same beauty.

I don't work on any code for Rails that does not take offset in a specific use case in my code base. Never have. That's why Rails is good. It wasn't made by a bunch of consultants sitting around and imagining abstract use cases that their clients might have. There are plenty of alternatives that follow that model. Look at any framework backed by a consulting firm and you'll find that style.

Strawman Consultants? I'll just pass by that. As I agreed, creating solutions for commonly encountered use cases is very powerful. That is something that has made Rails awesome. The beauty of continuing to modularize things, as is common in the Rails ecosystem, is that people can pick and choose the solutions when they encounter specific use cases. This spreading of concerns through the core seems like a wonderful candidate for this: something you can add to your Rails system if that is the style you prefer. Providing a more modular way, then putting reasonable arguments for it seems like what we've become accustomed to. If people agree with your view that this makes their code more maintainable (based on their definition of maintainable), then they can use it. There is a natural argument that people don't have to use this. As I've seen with other things, for example extensive lifecycle hook usage, when there is a feature that is in core, people will assume it is "best practice." Having it as an extension to Rails provides a good structure for people to spend time considering their usage.

…

ernie · August 22, 2012

And being able to intelligently select an appropriate controller for a specific functionality would reduce the likelihood of developers creating ridiculous filter-soup in their controllers.

ernie · August 22, 2012

It's also important to note that by accepting a callable, we aren't limited to only classes, but any object responding to call, generated by any means we deem appropriate. I used a class that had a class method in the test only because that was the most expedient way to show the functionality.

coreyhaines · August 22, 2012

On Wed, Aug 22, 2012 at 9:53 AM, David Heinemeier Hansson < notifications@github.com> wrote: Again, though, I very much appreciate this level of debate, Ernie. It's focused around real code alternatives, so it's concrete. Not abstract, hand-wavy "I've seen teams use this wrong!" kind of arguments. Thank you for that.

And with that, I'll bow out of the conversation. It appears the previous comments about "consultants" weren't just undirected strawman arguments. As I've said before, our experiences with both building Rails applications and working with teams on improving the maintainability of their Rails applications lead us to disagree. That happens. I'll just move back to the background and continue to work with people to code around these sorts of things.

…

fxn · August 22, 2012

@coreyhaines "I have seen teams misusing features" is not an argument against adding features in my view. Sometimes people misuse stuff, sometimes people just do not know what does Rails offer. That is normal, and it is the consultant's job to help teams tune their Rails skills.

New features have to be discussed by their own merits.

steveklabnik · August 22, 2012

Corey's argument is not "I've seen teams mis-use this feature" it's that "A team that uses this feature ends up regretting it.":

As I sit with teams, I don't find they've "abused" the features, they simply used them.

nateklaiber · August 22, 2012

@steveklabnik - Either way, what @fxn stated still stands: Sometimes people mis-use stuff. Isn't that the responsibility of the developer using the framework, to understand what the framework offers and then make the best decisions - architecturally or otherwise?

I can bet that even if it was designed as @coreyhaines would prefer that people would still mis-use it. There is no silver bullet.

steveklabnik · August 22, 2012

Sure, but since we're not talking about people mis-using a feature, then it's irrelevant.

revans · August 22, 2012

After reading all of this, I fail to see an argument as to why this needs to be in rails core instead of a gem? Is it "I say so" argument or is there a valid argument that this needs to be in rails?

fxn · August 22, 2012

@steveklabnik Corey mentions misuses of AS::Concern as an example to depict what he means. It is not just that people use the features of Rails, of course people use them! Or do you build Rails applications to write CGI scripts?

nateklaiber · August 22, 2012

@steveklabnik - the regret of teams using a feature is because they mis-used it in the first place. I have examples of this, too. It is relevant according to @coreyhaines. The documentation/guides describe how the feature works - not prescribe how a team or individual should use it. That's up to the individual or team to do their own research. Rails can't continue to be built with safety nets everywhere because people might mis-use it and then later regret it.

foca · August 22, 2012

@nateklaiber What Corey proposes however is not doing anything and leaving things as-is, since you could use method or lambdas right now for this.

New features have to be discussed by their own merits.

@fnx Agreed.

In this particular case, however, I don't see how this adds anything more than (slight) bloat to the framework and a DSL for the sake of having a DSL. This adds complexity, even if it isn't much compared to the framework as a whole. It means newcomers have to learn "one more thing", and adds more code to maintain and debug.

That said, I'm not the one calling the shots here, and it's clear that no matter how many arguments people come up with against the feature, the feature will stay. So let's just move on and do productive stuff :)

dhh · August 22, 2012

"The beauty of continuing to modularize things, as is common in the Rails ecosystem, is that people can pick and choose the solutions when they encounter specific use cases" The default Rails stack is not intended to be an empty plate that forces people to a buffet of choices that are all equal and just based on individual taste. Rails is omakase and I'm the chief chef. By default, I'll serve a menu that I think is best. You're free to say, no, I'd rather not have that unagi, but that doesn't mean I won't serve that to the rest of the patrons. Staying in the metaphor, it's also BYOB. So feel free to bring your Budweiser RSpecs (or any other gem, template language, whatever). Just because we have delicious sake on the table doesn't mean you have to drink it.

…

On Aug 22, 2012, at 10:01 AM, Corey Haines <notifications@github.com> wrote: > On Wed, Aug 22, 2012 at 9:32 AM, David Heinemeier Hansson < > notifications@github.com> wrote: > Corey, taking specific use cases and baking them into Rails is what Rails > is! It's a collection of solutions to specific use cases encountered by me > and the rest of core and other contributors. I agree. The parts of Rails that make it very useful are the parts that cover a range of cases in a nice fashion. In this specific case, it doesn't have that same beauty. > I don't work on any code for Rails that does not take offset in a specific > use case in my code base. Never have. That's why Rails is good. It wasn't > made by a bunch of consultants sitting around and imagining abstract use > cases that their clients might have. > There are plenty of alternatives that follow that model. Look at any > framework backed by a consulting firm and you'll find that style. > Strawman Consultants? I'll just pass by that. As I agreed, creating solutions for commonly encountered use cases is very powerful. That is something that has made Rails awesome. The beauty of continuing to modularize things, as is common in the Rails ecosystem, is that people can pick and choose the solutions when they encounter specific use cases. This spreading of concerns through the core seems like a wonderful candidate for this: something you can add to your Rails system if that is the style you prefer. Providing a more modular way, then putting reasonable arguments for it seems like what we've become accustomed to. If people agree with your view that this makes their code more maintainable (based on their definition of maintainable), then they can use it. There is a natural argument that people don't have to use this. As I've seen with other things, for example extensive lifecycle hook usage, when there is a feature that is in core, people will assume it is "best practice." Having it as an extension to Rails provides a good structure for people to spend time considering their usage. -Corey — Reply to this email directly or view it on GitHub.

ernie · August 22, 2012

@nateklaiber I think there's some room for a distinction to be made between "features" and "tasty treats that happen to be sitting on a mousetrap." I'm pretty sure @coreyhaines was talking about the latter.

ernie · August 22, 2012

Obviously, I'm sitting on both sides of the fence, here. I'm pretty much resigned to seeing this stay in core (though I'd prefer it remain a gem), but as such, I'd sure like to see something (#7422) that would allow for slightly improved separation of concerns (hah!).

neerajdotname · August 22, 2012

In this case @dhh had a pain and he solved it in BCX using the given technique . While others are thinking it will cause more problems and they might be right. However they have not used this new feature while @ddh put it to use and he liked it.

So as Jason will say "give it 5 minutes" http://37signals.com/svn/posts/3124-give-it-five-minutes

nateklaiber · August 22, 2012

@foca - Isn't that a slippery slope, though? Why use Rails at all then - if you could just use Ruby method or lambdas? In this case, it seems to come down to taste. You could argue that things added from Rails 1.X until now have been adding slight bloat and complexity. It's an evolution of the framework.

@ernie - I agree that I am both sides of the fence, too. Personally, I am more leaning towards it being a gem.

Can you protect developers from "tasty treats that happen to be sitting on a mousetrap" - that is an individual developer thing. It's a level of professionalism and knowledge. People can mess up plain Ruby, too, based on their mis-understandings. That argument just doesn't sit well with me as it's not something a framework or language will ever solve.

revans · August 22, 2012

@neerajdotname I don't think anyone is arguing that @dhh didn't solve a problem by using this. It's more of a discussion as to why this needs to be in rails-core.

Unless I missed it, I haven't see a real reason as to why it needs to be in core other than "I said so".

dhh · August 22, 2012

Robert, that's the reason most things are in Rails (all the way back to Rails 0.5 from '03). If I stumble across a pattern that I consider to be a generally useful pattern that I would use in most applications, most of the time, it's going into Rails. I've posted the before/after code above and the benefits that I perceive. Those are the reasons.

…

neerajdotname · August 22, 2012

@revans I think @dhh nicely described why it needs to be in core. The main argument is that rails is a collection of best practices. If a particular way of doing thing is on the fringe and it should not be mainstream then it should not be in core. Case in point is ActiveResource. It used to be core but now it is a plugin.

On the flip side strong_parameters started as a plugin but it is a better solution than attr_accessible so attr_accessible is moving to plugin and strong_parameters is coming from plugin to core.

Same argument can be made about asset pipeline. When it landed there was a strong voice that it should be left as plugin. However there were enough good things in it to bring it to core.

neerajdotname · August 22, 2012

I think we should give credit to rails core team for not only bringing new features to core but also for retiring features that do not justify to be in core. Many things like ActiveResource , RJS, rails api, attr_accessible have been removed from core. Yes rails api code was merged to master and it was removed before the release.

So just because something is in master do not think it will never go out of core. As I said ( or as Jason said ) give it 5 minutes and if it is not worthy its weight it will go out.

And just like coffeescript debate you do not have to use it. Rails is a collection of features. Use the features that you like.

revans · August 22, 2012

@dhh I understand that. I've been a long time user of Rails and have enjoyed those benefits.

Other than your use case, I am not aware (admittedly, I haven't gone looking either) of others. So for the reason as to why this needs to be in core, hasn't been apparent to me. If it's as simple as a pattern you like, I get it. Not sure I agree it needs to be in core, but it's your baby.

dburry · August 22, 2012

The concepts of code reuse, DRY-ness, and describing such succinctly and clearly, are all good ones. This change encourages everyone to do more of these, and to do it better. So adding concerns to the router is a good thing.

Sure people have always been able to do it in slightly longer more verbose ways, and they still can if they choose. I'm speaking of defining methods here, if you have a large app with a lot of modular pieces, those "few lines" longer can really add up. I for one will be switching to concerns because then I know I'm safe in not stepping on any previously-defined methods any longer (I'm losing enough hair as it is without debugging that)...

Also the way things can nest now has encouraged more code reuse in rails 3, compared to rails 2. This concerns thing is just more evolution of the same.

mkremer · August 22, 2012

I can see why @dhh wants this in the routing DSL, managing a routes file using purely DSL is just more consistent and pleasant. Sure you can use Ruby code to manage your routes, but to me it just doesn't feel as the way it was intended.

detomastah · August 22, 2012

I agree with @revans, shouldn't it be a gem?

peterc · August 22, 2012

Defining methods inside the draw DSL is a visual wart that doesn't fit with the rest of the flow.

That's true. By using 1.9 lambda syntax, though, it could start to look nicer perhaps. E.g.: commentable = ->{ resources :comments } .. and you could then support both methods of inclusion, whether through a concerns: commentable argument or with the @pixeltrix approach.

ernie · August 23, 2012

Noticed a flaw in the method alternative that was suggested initially. Calling a method defined in the top of a route set will generate resources from the context the method was defined in -- in other words, it will not know about its current mapper when adding resources, or any scoping.

Also fixed a flaw in my alternative implementation that suffered from a similar issue -- it wasn't caught in the existing tests, which weren't checking for context.

With #7422, we'd have the option of doing:

concern :commentable do
  resources :comments
end

def commentable(mapper)
  mapper.resources :comments
end
concern :commentable, method(:commentable)

or, for something that just shouldn't be in the routes file:

# Separate file
class PurchasableConcern
  RETURNABLES = Departments.map(&:name) - %w(pets electronics)

  def self.call(mapper)
    mapper.resources :purchases
    mapper.resources :receipts
    mapper.resources :returns if RETURNABLES.include?(mapper.current_scope[:controller])
  end
end

# routes.rb
concern :purchasable, PurchasableConcern

We gain the ability to behave differently based on the scope we're in using all fo these options, but I wouldn't think it would make sense except in the last case, since the idea would be to make the routes more readable by saying that something was something-able, and not delve into the specific implementation unless someone wanted to go looking for it.

Disclaimer: none of this should be construed to indicate that I think this is a sound idea at the fundamental level, but if we're gonna do it, then we should try to at least get some code separation options out of it.

dhh · August 26, 2012

I like the look of this. Still need to have some proper use cases to justify this, though. Any luck on those? On Aug 23, 2012, at 14:34, Ernie Miller <notifications@github.com> wrote:

…

yfeldblum · August 26, 2012

resources :toys do
  concern :purchasable, PurchasableConcern.new(include_returns: true)
end

resources :snacks do
  concern :purchasable, PurchasableConcern.new(include_returns: false)
end

ernie · August 26, 2012

@yfeldblum concerns would be defined outside the resource that uses them, then reused, so not quite like your example shows. That being said, it'd be trivial to allow concerns to accept options, and I like the idea of passing in options (telling) instead of the callable asking about the current scope of the mapper. Callables really shouldn't be asking about the mapper's scope as much as reacting to options like returnable: true. Updating PR to reflect that.

ernie · August 26, 2012

OK, updated #7422 accordingly. Added better documentation, as well. I think the documentation outlines the use case pretty well, so including here:

concern - Define a routing concern using a name.

Concerns may be defined inline, using a block, or handled by
another object, by passing that object as the second parameter.

The concern object, if supplied, should respond to call,
which will receive two parameters:

The current mapper
A hash of options which the concern object may use

Options may also be used by concerns defined in a block by accepting
a block parameter. So, using a block, you might do something as
simple as limit the actions available on certain resources, passing
standard resource options through the concern:

concern :commentable do |options|
  resources :comments, options
end

resources :posts, concerns: :commentable
resources :archived_posts do
  # Don't allow comments on archived posts
  concerns :commentable, only: [:index, :show]
end

Or, using a callable object, you might implement something more
specific to your application, which would be out of place in your
routes file.

# purchasable.rb
class Purchasable
  def initialize(defaults = {})
    @defaults = defaults
  end

  def call(mapper, options = {})
    options = @defaults.merge(options)
    mapper.resources :purchases
    mapper.resources :receipts
    mapper.resources :returns if options[:returnable]
  end
end

# routes.rb
concern :purchasable, Purchasable.new(returnable: true)

resources :toys, concerns: :purchasable
resources :electronics, concerns: :purchasable
resources :pets do
  concerns :purchasable, returnable: false
end

Any routing helpers can be used inside a concern. If using a
callable, they're accessible from the Mapper that's passed to
call.

ernie · August 26, 2012

Additionally (and @josevalim would be able to better confirm/deny this than me) it seems as a change like this would enable much of Devise's mapping code (https://github.com/plataformatec/devise/blob/master/lib/devise/mapping.rb) to be handled by an Authenticatable concern against a users resource.

If not directly applicable to Devise, it would certainly be a worthwhile hook for other authentication implementations.

To me:

concern :authenticatable, MyAuthenticationThing.new(some: 'opts')

resources :users
  concerns :authenticatable, authentication_methods: [:token, :password]
end

would be preferable to using macros to deliver similar functionality.

radar · August 27, 2012

My only... worry... about @ernie's implementation is the location of the concern file. I don't think that dumping it in the lib would be a great idea, much like dumping anything in lib is not a great idea.

What would the convention be for these concern files? lib/concerns/purchasable.rb? config/routing/purchasable.rb?

ernie · August 27, 2012

If it were my app, it'd be either in lib/routing/ or config/routing/ Of course, there's always app/concerns/routing if what @dhh was saying about the thread of concerns through the BCX codebase is a pattern to follow -- maybe all kinds of concerns should be in one place, and easy to correlate. In my mind, at least getting to the point of allowing for code separation so that we can have the discussion about where they should live (and aforementioned alternatives for gems to provide a more standard way of using their routing features) will be a win. On Aug 27, 2012, at 4:46 AM, Ryan Bigg <notifications@github.com> wrote: My only... worry... about @ernie <https://github.com/ernie>'s implementation is the location of the concern file. I don't think that dumping it in the lib would be a great idea, much like dumping *anything*in lib is not a great idea. What would the convention be for these concern files? lib/concerns/purchasable.rb? config/routing/purchasable.rb? — Reply to this email directly or view it on GitHub<https://github.com/rails/rails/commit/0dd24728a088fcb4ae616bb5d62734aca5276b1b#commitcomment-1768403>.

revans · August 27, 2012

I think config/routing or config/routes makes the most sense as a home for routing concerns.

dhh · August 27, 2012

We are still exploring here, so a default space created by default is not really on the menu for the time being. Before we put too much further work into this, I'd recommend searching through your own real apps and see if you can find some compelling use cases that we can do some before/after code comparisons on. If none exist yet, let's turn this into a plugin that extends the basic functionality while we search for those cases.

…

yfeldblum · August 27, 2012

I suggest moving routes to app/routes if possible. After all, they aren't really config: they are actually the top-level interface to your application, mapping HTTP requests to the classes and methods that handle these requests.

There is also certainly a need in larger applications to split up routes files in general, because they can grow to hundreds of lines or more with multiple groups of routes that are internally related but which do not relate strongly to each other. This need applies to the concept of routing concerns, but it's also a general need even for routes files which don't have any concerns.

dhh · August 27, 2012

Jay, we had an earlier commit to master that allowed you to split up routes.rb into multiple files. I ran it in the new Basecamp for quite a while but found that it actually just made things harder to follow. It was from that I started to look at concerns to ensure that there was no wasted repetition.

…

justinko · August 27, 2012

I suggest moving routes to app/routes if possible.

yfeldblum · August 27, 2012

@dhh In the case of BCX, it seems like the routing there is more amenable to concerns. But there are other applications where there is just a lot of routing and not very much commonality at all (and where attempting to impose commonality is dumb). Redmine's config/routes.rb, for example, has about 300 lines of routing code. I think a good DSL for that case would hit the spot for those large applications with lots of routing but not much commonality.

MyApp::Application.routes.draw do

  # includes routes from app/routes/storefront_routes.rb which has 200 lines
  draw_routes "storefront"

  # includes routes from app/routes/admin_routes.rb which has 150 lines
  draw_routes "admin"

  # includes routes from app/routes/helpdesk_routes.rb which has 50 lines
  draw_routes "helpdesk"

end

dhh · August 27, 2012

Yup, we had this exact setup in master. @wycats implemented it. Feel free to go back and find it and start up a plugin with it. When I ran it, it turned out not to be helpful. But it could have been that I wasn't segregating things correctly. So I'm open to the idea of seeing more usage in the wild, but as a plugin first.

…

ernie · August 29, 2012

@dhh OK, here's a situation, extracted from a slightly-modified case in an old app.

When you're doing things that involve social interactions, you end up with tons of comments, abuse flags, subscriptions, likes, and so on. A bunch of these are actually more or less toggleable on/off switches for users, and could probably inherit from the same controller. A preference of mine is to avoid trying to iterate through params looking for a likely suspect when it comes to a polymorphic relationship's nested resources, and instead specify the base, directly, so that I can find the "-able" without putting undue trust in user-submitted params.

This leads to a pattern like:

Some::Application.routes.draw do
  resources :posts do
    resources :likes, base: 'post'
    resources :flags, base: 'post'
    resources :subscriptions, base: 'post'
    resources :comments, base: 'post' do
      resources :likes, base: 'comment'
      resources :flags, base: 'comment'
    end
  end

  resources :photos do
    resources :likes, base: 'photo'
    resources :flags, base: 'photo'
    resources :comments, base: 'photo' do
      resources :likes, base: 'comment'
      resources :flags, base: 'comment'
    end
  end
end

Which only gets more unmanageable over time. With my patch, this is the after:

class SociallyInteractable
  def initialize(*resource_list)
    @resource_list = resource_list
  end

  def call(mapper, options)
    filtered_resources(options).each do |res|
      mapper.instance_eval do
        resources res, base: @scope[:controller].singularize do
          if res == :comments
            concerns :socially_interactable, only: [:likes, :flags]
          end
        end
      end
    end
  end

  def filtered_resources(options)
    if options[:except]
      @resource_list - Array(options[:except])
    elsif options[:only]
      @resource_list & Array(options[:only])
    else
      @resource_list
    end
  end
end

Some::Application.routes.draw do
  concern :socially_interactable,
          SociallyInteractable.new(:likes, :flags, :subscriptions, :comments)

  resources :posts do
    concerns :socially_interactable
  end

  resources :photos do
    concerns :socially_interactable, except: :subscriptions
  end
end

dhh · August 29, 2012

That's reasonably compelling. Thank you.

Anyone else have some similar use cases they could post based on this? I'm close to convinced that this is +1.

dhh · August 29, 2012

Btw, @ernie, I'm curious as to what insight lead you to switch from "This isn't even syntactic sugar. It's more like syntactic aspartame. It might slim down your code, but it's also a carcinogen" to extending and improving the feature. It seems like the SociallyInteractable extraction you just based your extension off does indeed materially cut down on the complexity of the sample routes.rb file. I take it that you see this as a good thing, ye?

(I'm not being I'm-told-you-so, I'm genuinely interested in learning what argument or insight made you change your position. We have more stuff coming in the vain of concerns for Rails 4, so I'd like to present those changes with the most effective and persuasive arguments.)

ernie · August 29, 2012

@dhh I'm of the strong opinion that these types of changes should do what they can to:

Allow for idioms that have become second-nature to Rubyists (in this case, duck-typing with #call, in a similar vein as Rack)
Enable and encourage developers to extract reusable code into easily-testable components. Writing a unit test for the logic being used in SociallyInteractable, above, is simple. The fact that my first implementation (and the method suggestions in this thread) were actually fundamentally broken when it came to nested resources but were not caught by the test suite nor any of those giving them +1s is an indication, to me, that this stuff can be tricky to test, without simply testing each and every path in a routes file. Better to be able to test the logic used to generate the paths, if possible.
Related to the first two, really. Where possible, I'd love to see us continue the trend toward modularity we saw in Rails 3, with Rails 4. One way to continue to move in this direction is to adhere to SOLID principles as we build features. I believe that my patch moves us further in that direction, defining a clear API that one must adhere to if one wants to behave as a concern, and then allowing objects to be substituted, Liskov-style, according to that API.

As I see it, the previous implementation did not address those goals, and for no good reason, as far as I could tell.

I can't speak for @coreyhaines, and I know he has since checked out of this thread, but I think that would probably summarize at least part of his position, no doubt less eloquently as he would, himself.

dhh · August 29, 2012

1) That's hardly worthy of the venom spewed at the beginning of the thread. It's a completely uncontroversial request to extend the feature. 2) If you see the original before/after code, there was no logic there that warranted separate testing (that would be different from how you'd integration test any of your other routes). There was no logic, it was all just declarative. 3) Again, you're referring to your own feature extension. The original feature was entirely declarative. No logic, no conditionals. Simply a different way to declare existing routes. Anyway, I understand how it's hard to back down from an extreme position like "This isn't even syntactic sugar. It's more like syntactic aspartame. It might slim down your code, but it's also a carcinogen", even as you work to extend the basics of that feature. So I'm not going to push on that lever further. Thanks for your code contributions.

…

ernie · August 29, 2012

@dhh I didn't really think I was spewing venom, for my part, but I was passionate. Clearly, my words were offensive, so I apologize -- particularly if that offense has led to a steeper-than-necessary climb to merge.

My comment was directly related to a concern about the previous implementation, which seemed like the addition of indirection using a less idiomatic means that would be trickier to understand, at first glance. I wouldn't have known that the concern was a declaration, since the convention of other blocks within the routes.rb DSL is to define routes on the spot.

In the previous implementation, the reader is unlikely to know what it is that they are seeing, at first glance. Is the concern a top-level entity like a mapping? Is it a declaration of something else? When does the code get invoked?

My PR was an attempt to partially right that perceived (correctly or incorrectly) wrong, by allowing us to use a syntax when declaring a concern that is more obviously a statement that we are assigning a name to a reusable component.

I think (hope) that the adjustments I am suggesting would make these things more clear, to more people.

evanphx · August 21, 2012