Lock down new ImplicitRender behavior for 5.0 RC

[chancancode]

1. Conceptually revert Allow default_render to take a block to customize behavior when there's no template #20276

   The feature was implemented for the responders gem. In the end,
   they did not need that feature, and have found a better fix (see
   Support rails5 heartcombo/responders#131).

   ImplicitRender is the place where Rails specifies our default
   policies for the case where the user did not explicitly tell us
   what to render, essentially describing a set of heuristics. If
   the gem (or the user) knows exactly what they want, they could
   just perform the correct render to avoid falling through to
   here, as responders did (the user called respond_with).

   Reverting the patch allows us to avoid exploding the complexity
   and defining “the fallback for a fallback” policies.

2. respond_to and templates are considered exhaustive enumerations

   If the user specified a list of formats/variants in a respond_to
   block, anything that is not explicitly included should result
   in an UnknownFormat error (which is then caught upstream to
   mean “406 Not Acceptable” by default). This is already how it
   works before this commit.

   Same goes for templates – if the user defined a set of templates
   (usually in the file system), that set is now considered exhaustive,
   which means that “missing” templates are considered UnknownFormat
   errors (406).

3. To keep API endpoints simple, the implicit render behavior for
   actions with no templates defined at all (regardless of formats,
   locales, variants, etc) are defaulted to “204 No Content”. This
   is a strictly narrower version of the feature landed in Controller actions should default to head :ok if no template exists #19036 and
   when a template is missing for the default render, do head no_content instead #19377.

4. To avoid confusion when interacting in the browser, these actions
   will raise a TemplateMissing error for “interactive” requests
   instead. (The precise definition of “interactive” requests might
   change – the spirit here is to give helpful messages and avoid
   confusions.)

Closes #20666, #23062, #23077, #23564

[Godfrey Chan, Jon Moss, Mike Clark, Matthew Draper, Kasper Timm Hansen]

---

TODO:

• Finish porting the new tests in the other PRs
• Write helpful error messages for the two "no templates" scenario

[kaspth]

Some of the docs felt a bit hard to read to me. Took a stab at rephrasing them.

Something feels funny about assumed exhaustive too, but ¯_(ツ)_/¯.

Here's the originals:

# Handles the implicit behavior for a controller action when it did not
# explicitly call +render+ or otherwise indicate what the response  should be,
# such as by calling +respond_to+, +redirect+ and +head+.
#
# If this controller action has any associated templates, it tries to render
# the most appropiate template for the current action, taking into account of
# the action name, format, locales, variants etc.
#
# The list of available templates is assumed to be exhausative – if a suitable
# template cannot be found (for example, when the client requested the XML
# format when the only templates in HTML and JSON formats are available), an
# <tt>ActionController::UnknownFormat</tt> error will be raised.
#
# On the other hand, when there are no templates associated with the controller
# action at all, it tries to determine if the current request is "interactive"
# (i.e. came from a real browser). If so, it raises the <tt>ActionView::MissingTemplate</tt>
# error in order to display a helpful error message. Otherwise, it assumes the
# request is an API request and renders a "204 No Content" response.
#
# For API controllers, the implicit render is always "204 No Content" without
# performing any template lookups.

[maclover7]

Taking the baton from @kaspth, and doing some additions to this.

r? @maclover7

[maclover7]

Pushed up a commit with some changes. Feel free to squash down -- only pushed up a separate commit so that way others can review.

Passing the baton back to @chancancode for the finale 😄

r? @chancancode

[kaspth]

to due

[kaspth]

repond

[chancancode]

Merging to unblock the release! We have some ideas for improvements (shortening the error messages and move the details into the HTML page like routing errors, bring back the "searched in..." info, add an config to turn off the "interactive request warning", fine-tune the browser-detection, perhaps express defualt_render in terms of respond_to, etc), but the basic behavior is better than what we have on master. Feel free to open PRs to improve this further

[chancancode]

Thanks everyone for the hard work!!!

[kaspth]

❤️

[maclover7]

🎉

[claudiob]

Should I try adding some new lines to the error message in the following page?

[claudiob]

On a separate note… I think this message is very explicit regarding the edge cases, but it does not convey exactly what to do in the case that will occur 99% of the times.

The old message used to clearly state:

1. you are missing a file
2. the file should have one of these names and be in one of these folders

I think we can try to improve the current message and make it more accessible to newcomers, also given that this is part of the Rails guides:

The final part of this message tells us where Rails has looked for the templates. Templates within a basic Rails application like this are kept in a single location, but in more complex applications it could be many different paths.

The simplest template that would work in this case would be one located at app/views/articles/new.html.erb. The extension of this file name is important: the first extension is the format of the template, and the second extension is the handler that will be used. Rails is attempting to find a template called articles/new within app/views for the application. The format for this template can only be html and the handler must be one of erb, builder or coffee. Because you want to create a new HTML form, you will be using the ERB language which is designed to embed Ruby in HTML.

Therefore the file should be called articles/new.html.erb and needs to be located inside the app/views directory of the application.

[kaspth]

@claudiob it's been rephrased because you aren't necessarily missing a file. If you expect a 204, then you don't need to do anything.

[claudiob]

@kaspth I understand and I agree with this PR, don't get me wrong.

I just think that if you do miss a file (probably still the most common case), we are not telling you anymore where to put that file and which filenames are accepted.

[kaspth]

Yeah, agreed. And it's one @chancancode's suggestions in #23827 (comment).

[chancancode]

@claudiob please continue to improve it!

The case to keep in mind is that..

1. def update; end is now a sanctioned/idiomatic way to write head :no_content
2. If you are planning on relying on that behavior, everything might be fine until you decided to debug something in the browser – you might be thinking "this endpoint was working fine since always, was it the gem I just installed? should I restart the server??? what's happening????"

So I think somehow conveying the fact that "this might be okay, actually" is important. But I like @matthewd's suggestion of putting less in the exception message (and probably bring back the info we had before) and put more in the specialized HTML error page.

By the way, since the browser detection might not be perfect (an API explore extension in chrome or a BrokenKit.framework on iOS might send some browser-like headers), it is probably good to have a way to turn it off and mention that on the error page too. ("If you are expected the 204, everything is cool, don't worry. You can also turn this off via... if you like.")

[clarkware]

Thanks to @chancancode for all the work on this!

FWIW, I strongly agree with @claudiob that the message would be improved by addressing the most common case of missing the file. The current message puts a lot of emphasis on the least common case. In fact, given the interactive_browser_request? conditional, I'm not clear on why you'd ever get this message when building an API endpoint as the message suggests. Perhaps I'm missing some nuances.

[jeremy]

Noticing that this generally applies to GET requests from the browser.

We have some internal API controllers that take multipart/form-data params or raw request bodies and don't return a response body. To use implicit responses, we'd need to force a fake request.format since it'd fall back to the HTML default, otherwise, and we'd get a missing template error.

Think we can get the best of both worlds by pruning the "is a developer viewing this in the browser and expecting a response?" check down to non-XHR HTML GETs. Non-XHR PUT/POST/DELETE are typically redirects to an HTML GET.

[matthewd]

@jeremy I worry that'll mean "I forgot to do the redirect" becomes "my browser is pretending I'm not pressing the button" 😕

What Accept is being sent in that situation? No header whatsoever? It definitely seems safe to narrow "browser" down to an explicit request for at least one of text/html or */*.

If we really wanted to get creative, presence of cookies might be worth considering.

[jeremy]

Agree. I wager that forgetting a redirect is fairly uncommon and, hopefully, easier to diagnose. The form will submit and you'll get a white page of death in response.

Checking for user-agent details gets tricky since more sophisticated API clients may appear browser-like in all those respects. Then a request could spookily seem to work with some clients but not others.