Exceptions in ERB templates not rendering correctly depending on surrounding context

[zarqman]

This is an extension of #48173 and the resulting PRs #48184 and #48294.

While those PRs address a failure to identify the source location in a template with spaces in front of the errant method, characters afterward can still cause DebugExceptions to error.

Steps to reproduce

Modify actionview/test/fixtures/test/unparseable_runtime_error.html.erb to:

  <% if true %>
    <%= method_that_does_not_exist %>a
  <% end %>

Note the added a at the end of the middle line. It seems a variety of characters (possibly all) in that location will cause the issue.

Expected behavior

DebugExceptions renders its normal error page.

Actual behavior

A generic 500 error type exception page is rendered to the browser.

Debugging the cause for the error shows NoMethodError: undefined method bytesize' for nil:NilClass`.

Backtrace

~/.rvm/gems/ruby-3.2.2@rails71/bundler/gems/rails-66099147482e/activesupport/lib/active_support/core_ext/erb/util.rb:169:in `tokenize'
~/.rvm/gems/ruby-3.2.2@rails71/bundler/gems/rails-66099147482e/actionview/lib/action_view/template/handlers/erb.rb:45:in `translate_location'
~/.rvm/gems/ruby-3.2.2@rails71/bundler/gems/rails-66099147482e/actionview/lib/action_view/template.rb:182:in `translate_location'
~/.rvm/gems/ruby-3.2.2@rails71/bundler/gems/rails-66099147482e/actionpack/lib/action_dispatch/middleware/exception_wrapper.rb:258:in `spot'
~/.rvm/gems/ruby-3.2.2@rails71/bundler/gems/rails-66099147482e/actionpack/lib/action_dispatch/middleware/exception_wrapper.rb:302:in `extract_source'
~/.rvm/gems/ruby-3.2.2@rails71/bundler/gems/rails-66099147482e/actionpack/lib/action_dispatch/middleware/exception_wrapper.rb:206:in `block in source_extracts'
~/.rvm/gems/ruby-3.2.2@rails71/bundler/gems/rails-66099147482e/actionpack/lib/action_dispatch/middleware/exception_wrapper.rb:205:in `map'
~/.rvm/gems/ruby-3.2.2@rails71/bundler/gems/rails-66099147482e/actionpack/lib/action_dispatch/middleware/exception_wrapper.rb:205:in `source_extracts'
~/.rvm/gems/ruby-3.2.2@rails71/bundler/gems/rails-66099147482e/actionpack/lib/action_dispatch/middleware/debug_exceptions.rb:125:in `create_template'
~/.rvm/gems/ruby-3.2.2@rails71/bundler/gems/rails-66099147482e/actionpack/lib/action_dispatch/middleware/debug_exceptions.rb:77:in `render_for_browser_request'
~/.rvm/gems/ruby-3.2.2@rails71/bundler/gems/rails-66099147482e/actionpack/lib/action_dispatch/middleware/debug_exceptions.rb:69:in `render_exception'
~/.rvm/gems/ruby-3.2.2@rails71/bundler/gems/rails-66099147482e/actionpack/lib/action_dispatch/middleware/debug_exceptions.rb:44:in `rescue in call'
~/.rvm/gems/ruby-3.2.2@rails71/bundler/gems/rails-66099147482e/actionpack/lib/action_dispatch/middleware/debug_exceptions.rb:28:in `call'
~/.rvm/gems/ruby-3.2.2@rails71/bundler/gems/rails-66099147482e/actionpack/lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'

System configuration

Rails version: main

Ruby version: 3.2.2