Fix browser performance issues with configurable maximum variable size #360

tobypinder · 2017-01-20T11:49:56Z

Based on @bquorning's comment on #334, this will allow the user to set a maximum payload size for variable.inspect, above which the variable is filtered out of the error page. This solves many performance problems associated with the browser being slow to render error pages, the CPU requirements for the error page skyrocketing etc.

Many variable types end up being huge like Rails collections being inspected (and therefore queried for) before they are paginated. I've set the default to 100,000 characters, but the default can be modified to unlimited by setting BetterErrors.maximum_variable_inspect_size to nil, if we think compatibility is more important than a friendly default.

Keen to work to get this PR merged, there should be test coverage and documentation but feedback is appreciated!

…tion on #334.

tobypinder · 2017-01-20T11:54:30Z

Hmm Travis seems unhappy for reasons unrelated to this PR...

…ases wherre inspection causes exceptions.

nirvdrum · 2017-03-23T13:59:30Z

The Gemfile needs to lock "rake" down to "0.9.6" (or something in that series). Or the Rakefile needs to be updated to work with newer versions of rake.

bquorning · 2017-03-23T14:01:30Z

lib/better_errors/version.rb

@@ -1,3 +1,3 @@
 module BetterErrors
-  VERSION = "2.1.1"
+  VERSION = "2.1.1.1"


I think you shouldn’t change the VERSION in a pull request. If e.g. v2.2.0 is released before this PR gets merged, you’d need to change it.

Apologies, can fix.

tobypinder · 2017-03-23T14:15:40Z

OK, have cleaned up the changeset to remove my gem forking stuff, version number and gemspec should be unaffected by this pull request.

bquorning · 2017-03-23T14:18:05Z

README.md

+# This will stop BetterErrors from trying to render larger objects, which can cause
+# slow loading times and browser performance problems.
+# default value: 100_000
+BetterErrors.maximum_variable_inspect_size = 100_000


Perhaps specify the unit. This is 100_000 characters, right? Or 100_000 bytes?

At the moment it's purely ~~bytes~~ character length (CGI.escapeHTML(obj.inspect).length), it's a very naive approach to get things moving. Happy to look at modifying this if we think there is a better approach, and I'll certainly add this caveat to documentation.

bquorning · 2017-03-23T14:20:36Z

lib/better_errors.rb

@@ -139,6 +144,7 @@ def self.default_editor
  require "binding_of_caller"
  require "better_errors/exception_extension"
  BetterErrors.binding_of_caller_available = true
+  BetterErrors.maximum_variable_inspect_size ||= 100_000


If binding_of_caller is not available (raises LoadError), this line won’t be evaluated. I’m not sure where the best location for setting default values would be. Perhaps inside the class definition? Perhaps right after this block?

bquorning · 2017-03-23T14:23:46Z

lib/better_errors/error_page.rb

+      value = CGI.escapeHTML(obj.inspect)
+
+      if !BetterErrors.maximum_variable_inspect_size.nil? && value.length > BetterErrors.maximum_variable_inspect_size
+        value = "<span class='unsupported'>(object too large - modify #{CGI.escapeHTML(obj.class.to_s)}#inspect or raise BetterErrors.maximum_variable_inspect_size)</span>"


Instead of modifying value here, you could add an else, e.g.

if length > limit "oops" else value end end

bquorning · 2017-03-23T14:24:15Z

spec/better_errors/error_page_spec.rb

+        html.should_not include "object too large"
+      end
+
+


Too many blank lines here :-)

bquorning · 2017-03-23T14:26:10Z

spec/better_errors/error_page_spec.rb

@@ -88,5 +88,35 @@ def initialize(message = nil)
        error_page.exception_message.should_not =~ /\A\n\n/
      end
    end
+
+    context 'with an inspect size limit set' do
+      before { BetterErrors.maximum_variable_inspect_size = 50_000 }


Seeing that you override this limit in the last example, the context description is a bit misleading. Would be too verbose if we set the limit inside each example?

… setup.

tobypinder · 2017-03-23T15:06:37Z

I have implemented suggested changes (great feedback btw) and retested manually with and without binding_of_caller installed. I suspect @nirvdrum's suggestion is for fixing CI? I've left it out of this branch for now, can add [ci skip] to future commits if you want me to stop hammering Travis accidentally 😎

bquorning · 2017-03-23T20:54:44Z

I’ve opened #364 and #365 as two approaches for fixing the Travis build.

DamienMetzger · 2017-06-09T13:49:16Z

Works for me, thank you !

bquorning · 2017-08-01T17:40:42Z

Hey @tobypinder, master is green again. Would you mind rebasing on master – and perhaps squashing your commit history a bit?

bquorning · 2017-08-01T17:44:02Z

lib/better_errors.rb

@@ -143,4 +148,6 @@ def self.default_editor
  BetterErrors.binding_of_caller_available = false
 end

+BetterErrors.maximum_variable_inspect_size ||= 100_000


I’m wondering if setting the default value should be moved to line 53, right after setting the default for ignored_instance_variables?

I realize that the value of binding_of_caller_available is being set down here, but that is probably just to avoid having requires inside the module code.

bquorning · 2017-08-01T17:45:22Z

spec/better_errors/error_page_spec.rb

+    it "hides variables with inspects that are above the inspect size threshold" do
+      BetterErrors.maximum_variable_inspect_size = 50_000
+
+      content = 'A' * (BetterErrors.maximum_variable_inspect_size)


Parentheses are not necessary here.

bquorning · 2017-08-01T17:45:28Z

spec/better_errors/error_page_spec.rb

+    it "shows variables with large inspects if max inspect size is disabled" do
+      BetterErrors.maximum_variable_inspect_size = nil
+
+      content = 'A' * (50_000)


Parentheses are not necessary here.

simi · 2017-08-01T17:46:10Z

spec/better_errors/error_page_spec.rb

+    it "shows variables with large inspects if max inspect size is disabled" do
+      BetterErrors.maximum_variable_inspect_size = nil
+
+      content = 'A' * (50_000)


Will be nice to test if this is over default (so default is overridden by nil).

RobinDaugherty · 2017-08-01T19:38:53Z

The project has been converted to RSpec 3's "expectation" syntax, so the specs need a slight rewording.

If you're not familiar with the new syntax, you can use transpec to perform the conversion for you.

tobypinder · 2017-08-02T08:01:01Z

@RobinDaugherty Happy to take a look at these within the next day or so.

bquorning · 2017-08-23T11:52:00Z

I changed the RSpec syntax and addressed the other PR comments as well – and rebased on master. Pushed the changes to a branch on my fork: master...bquorning:quieter_output

Let me know if I should open a new pull request.

tobypinder · 2017-08-23T15:35:47Z

@bquorning Looks great, sorry I couldn't get around to it!

bquorning · 2017-08-23T19:30:12Z

@tobypinder Would you update this PR with my commits, or should I open a new pull request?

tobypinder · 2017-08-24T08:38:39Z

@bquorning probably better for you to open a new PR if that's easier? Happy to close this one.

bquorning · 2017-08-24T08:42:33Z

probably better for you to open a new PR

It’s at #402. This one should automatically be closes when 402 is merged.

Configurably limit maximum variable size based on @bquorning's sugges…

da67deb

…tion on #334.

Release temporary gem.

f9cb84b

tobypinder changed the title ~~Configurably limit maximum variable size~~ Fix browser performance issues with configurable maximum variable size Jan 30, 2017

tobypinder added 3 commits March 15, 2017 15:28

Bump version so I can fix embarrassing binding.pry issue.

b032c3b

Merge branch 'master' into quieter_output

de55b55

Remove embarrassing code-execution-halting binding.pry statement in c…

f29ad9d

…ases wherre inspection causes exceptions.

bquorning reviewed Mar 23, 2017

View reviewed changes

tobypinder added 2 commits March 23, 2017 14:09

Revert version number change for pull request.

6e2a31c

Revert gemspec for pull request.

62571ef

bquorning reviewed Mar 23, 2017

View reviewed changes

tobypinder added 4 commits March 23, 2017 14:45

Spec formatting and more explicit maximum size config.

fc6a067

Document units and usage considerations for this configuration option.

b30e8ee

Tidier conditional.

4898884

Variable inspect size config should not be inside binding_of_caller…

ead97f8

… setup.

tobypinder mentioned this pull request Apr 21, 2017

Better errors page significantly slower with Puma 3.x #341

Closed

RobinDaugherty mentioned this pull request Jul 29, 2017

'variables' apiCall is failling for root page #359

Closed

RobinDaugherty added this to the v2.3 milestone Jul 29, 2017

Merge branch 'charliesome/master' into quieter_output

70c50e4

RobinDaugherty modified the milestones: v2.3, v2.5 Jul 31, 2017

bquorning mentioned this pull request Jul 31, 2017

"Loading" indicator while variables are being fetched #382

Open

bquorning reviewed Aug 1, 2017

View reviewed changes

simi reviewed Aug 1, 2017

View reviewed changes

bquorning mentioned this pull request Aug 24, 2017

Fix browser performance issues with configurable maximum variable size #402

Merged

RobinDaugherty closed this in #402 Aug 25, 2017

RobinDaugherty modified the milestones: v2.4, v2.5 Aug 29, 2017

RobinDaugherty mentioned this pull request Aug 29, 2017

Not working with Rails 4.1.0 #243

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix browser performance issues with configurable maximum variable size #360

Fix browser performance issues with configurable maximum variable size #360

tobypinder commented Jan 20, 2017

tobypinder commented Jan 20, 2017

nirvdrum commented Mar 23, 2017

bquorning Mar 23, 2017 •

edited

Loading

tobypinder Mar 23, 2017

tobypinder commented Mar 23, 2017

bquorning Mar 23, 2017

tobypinder Mar 23, 2017 •

edited

Loading

bquorning Mar 23, 2017

bquorning Mar 23, 2017

bquorning Mar 23, 2017

bquorning Mar 23, 2017

tobypinder commented Mar 23, 2017

bquorning commented Mar 23, 2017

DamienMetzger commented Jun 9, 2017

bquorning commented Aug 1, 2017

bquorning Aug 1, 2017

bquorning Aug 1, 2017

bquorning Aug 1, 2017

simi Aug 1, 2017

RobinDaugherty commented Aug 1, 2017

tobypinder commented Aug 2, 2017

bquorning commented Aug 23, 2017

tobypinder commented Aug 23, 2017

bquorning commented Aug 23, 2017 •

edited

Loading

tobypinder commented Aug 24, 2017

bquorning commented Aug 24, 2017

Fix browser performance issues with configurable maximum variable size #360

Fix browser performance issues with configurable maximum variable size #360

Conversation

tobypinder commented Jan 20, 2017

tobypinder commented Jan 20, 2017

nirvdrum commented Mar 23, 2017

bquorning Mar 23, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tobypinder commented Mar 23, 2017

Choose a reason for hiding this comment

tobypinder Mar 23, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tobypinder commented Mar 23, 2017

bquorning commented Mar 23, 2017

DamienMetzger commented Jun 9, 2017

bquorning commented Aug 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RobinDaugherty commented Aug 1, 2017

tobypinder commented Aug 2, 2017

bquorning commented Aug 23, 2017

tobypinder commented Aug 23, 2017

bquorning commented Aug 23, 2017 • edited Loading

tobypinder commented Aug 24, 2017

bquorning commented Aug 24, 2017

bquorning Mar 23, 2017 •

edited

Loading

tobypinder Mar 23, 2017 •

edited

Loading

bquorning commented Aug 23, 2017 •

edited

Loading