Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large size of live dumps in 2.6 #344

Closed
adrianbj opened this issue Feb 20, 2019 · 18 comments
Closed

Large size of live dumps in 2.6 #344

adrianbj opened this issue Feb 20, 2019 · 18 comments

Comments

@adrianbj
Copy link
Contributor

@adrianbj adrianbj commented Feb 20, 2019

Version: 2.6

Bug Description

Not sure if this is somehow specific to my implementation, but I've noticed that the size of a live dump in 2.6 is higher than 2.5. Here are some numbers to show what I mean. The tables below show the same dump once vs 100 times.

ONCE

Live ON Live OFF
2.5 3.8 K 204 K
2.6 44 K 204 K

100 TIMES

Live ON Live OFF
2.5 276 K Out of memory
2.6 4.6 M Out of memory

So as you can see, the size of the live dump in 2.6 is significantly greater than it is in 2.5.

Let me know if I can provide any additional info.

Thanks.

@adrianbj adrianbj changed the title Large size of dumps in 2.6 Large size of live dumps in 2.6 Feb 20, 2019
@dg

This comment has been minimized.

Copy link
Member

@dg dg commented Feb 21, 2019

I suppose you have a lot of dumps in Bar panels, right?

@adrianbj

This comment has been minimized.

Copy link
Contributor Author

@adrianbj adrianbj commented Feb 21, 2019

Well, the first example is 1 dump of a ProcessWire page object and the second example is 100 dumps of the same object.

The key thing is that in 2.6 the same dump is 12 to 16 times larger than it was in 2.5, so just wondering if you might know why and if it can possible be improved.

@dg

This comment has been minimized.

Copy link
Member

@dg dg commented Feb 21, 2019

I think I know why. I'll post here possible solution tomorrow, ok?

@adrianbj

This comment has been minimized.

Copy link
Contributor Author

@adrianbj adrianbj commented Feb 21, 2019

Tomorrow would awesome. It's late for you :)

@adrianbj

This comment has been minimized.

Copy link
Contributor Author

@adrianbj adrianbj commented Feb 22, 2019

Not really sure if it's related but I am noticing that the maxDepth limit mostly doesn't seem to apply to barDump() now that they are "live", so maybe that's why things are larger? But also, it seems like sometimes the limit is applied and I do get the [ ... ] indicator. Right now I am not really sure of the why/how this happens, but in complex/deep objects, it definitely seems to come into play. Let me know if you can't reproduce and I'll see what I can do about explaining / demonstrating it better.

BTW, is the plan that the maxDepth setting is actually no longer needed with barDumps?

And also, what is the plan for regular dumps? Do you plan on making these work with LIVE?

Thanks again for your efforts with this - it's really awesome!

@dg

This comment has been minimized.

Copy link
Member

@dg dg commented Feb 22, 2019

It actually changed a lot how live dump works in 2.6 a before 2.6. It has been unfinished for five years (#61, #62).

Now it works this way:

  • every dump have encoded collapsed parts in JavaScipt (HTML representation is created when you click to open)
  • dump with [Dumper::LIVE => true], is dump, that is completely encoded in JavaScript. It's good for places that are not visible after the page is loaded. So for places where you have to click to see them. (Bar panels, arguments in BlueScreen, etc)

Every dump generates two data, "dump data" (structure with pointers to snapshot) stored in HTML attribute data-tracy-dump, and "snapshot". Before 2.6 there was only one common snapshot stored in <script>Tracy.Debug.init(..., snapshot)</script>, since 2.6 there are a lot of snapshots stored in HTML attributes data-tracy-snapshot.

Everytime some object or array is changed, dump must create a new new snapshot.

Before 2.6 there was only one snapshot, so live-dumping (or JavaScript dumping) was not applicable to common dump(). Since 2.6 every dump() creates a new snapshot, so now it is possible that every collapsed data is in JavaScript.

On the other hand, when every dump() creates own snapshot, the size of page is bigger.

But there is way to tell to dump() „do not create new snapshot, because it is not needed, objects are still the same“. It is done using Dumper::SNAPSHOT option, see https://github.com/nette/tracy/blob/master/examples/dump-snapshot.php.

This is used for whole BlueScreen, not yet for Bar panels. Thats why now is output bigger.

@dg

This comment has been minimized.

Copy link
Member

@dg dg commented Feb 22, 2019

There are several ways to solve this problem. Thinking about how best way… Once I have it, I'll write you here.

@adrianbj

This comment has been minimized.

Copy link
Contributor Author

@adrianbj adrianbj commented Feb 22, 2019

@dg - thanks for the great explanation!

Regarding the maxDepth not being respected, I noticed that it is actually to do with live and the fact that it doesn't recognize recursion so the dump contain infinite levels if you expand an object that goes recursive.

Compare this:
image

to:
image

Note in the second screenshot, live is on and the page under filesManager is available to expand and here you can see that it just keeps going and going:
image

I expect this is not actually extra data being stored, but I still think it would be cleaner if these were marked as recursive.

Thanks.

@adrianbj

This comment has been minimized.

Copy link
Contributor Author

@adrianbj adrianbj commented Feb 26, 2019

@dg - just wanted to say thanks for the recursion fix - that's very helpful.

@adrianbj adrianbj closed this Feb 26, 2019
@adrianbj adrianbj reopened this Feb 26, 2019
dg added a commit that referenced this issue Feb 28, 2019
@dg

This comment has been minimized.

Copy link
Member

@dg dg commented Feb 28, 2019

This should be fixed.

@adrianbj

This comment has been minimized.

Copy link
Contributor Author

@adrianbj adrianbj commented Feb 28, 2019

@dg - just testing the new 2.6 version.

If I understand correctly, LIVE is no longer used, but LAZY is the default. Everything works for me with LAZY = true, but dumps are just as big as they were in the old version with LIVE = true, so I am not really seeing any effective change in size, but the time is much slower.

Let me know if I can provide more info to help.

@dg

This comment has been minimized.

Copy link
Member

@dg dg commented Feb 28, 2019

Try it with LIVE => true.

@adrianbj

This comment has been minimized.

Copy link
Contributor Author

@adrianbj adrianbj commented Feb 28, 2019

Yeah, I tried that - it results in a tiny dump, but I it is causing problems with my tabbed dump output where I output the debugInfo version along with the full object. Using LIVE with the latest version makes the Full Object version return the same content as the debugInfo version. This was not a problem before your latest commit and is not an issue with the latest version when LIVE is false.

image

OT, but I also noticed that passing array('maxDepth' => 10) as the 3rd argument no longer seems to work.

@dg

This comment has been minimized.

Copy link
Member

@dg dg commented Feb 28, 2019

I think that array('maxDepth' => 10) never worked. It should be Dumper::DEPTH => 10

@adrianbj

This comment has been minimized.

Copy link
Contributor Author

@adrianbj adrianbj commented Feb 28, 2019

I think that array('maxDepth' => 10) never worked. It should be Dumper::DEPTH => 10

Sorry about that - my wrapper around Tracy works a little differently. I was testing a direct bdump() call and was using my syntax.

So nevermind about that, but please let me know your thoughts on the other issue - thanks.

@dg

This comment has been minimized.

Copy link
Member

@dg dg commented Feb 28, 2019

LIVE => true had a little different meaning in 2.6.0 which caused an increase in page size. In 2.6-dev I reverted its meaning. (And added new LAZY => true which means what the LIVE => true meant in 2.6.0.)

So for barDump you can use LAZY => true or nothing as before, but not LIVE => true.

@adrianbj

This comment has been minimized.

Copy link
Contributor Author

@adrianbj adrianbj commented Feb 28, 2019

Ok, that makes sense, although it's a shame because the LIVE option does seem to work nicely if I am not dumping multiple tabs - the size is tiny, but I expect there are other issues I haven't noticed yet.

Curious then - is there a valid use case for LIVE in any calls outside of the core or will there always be problems?

@dg

This comment has been minimized.

Copy link
Member

@dg dg commented Feb 28, 2019

It works this way:

  • LAZY => true dumps variable as small JS structure (useful in bar panels, collapsed panels, etc)
  • LAZY => false dumps variable as big HTML structure (useful when output is written to the file, etc)
  • LAZY => null dumps variable as mix of JS (for collapsed parts) and HTML (for non-collapsed parts) (default)
  • LIVE => true dumps variable as JS structure within one snapshot. Which means that those variables (more precisely objects) must not change. Output is super small, but changes are ignored

Try

// changed object is not detected, because is part of snapshot
$obj->x = 'CHANGED!';
echo Dumper::toHtml($obj, [Dumper::SNAPSHOT => &$snapshot]);

So LIVE is only applicable if I know 100% that the objects I am dumping will not change. So it is useful for arguments listing in BlueScreen and in some Bar panels. On the contrary LAZY => true is applicable anytime.

dg added a commit that referenced this issue Feb 28, 2019
dg added a commit that referenced this issue Feb 28, 2019
dg added a commit that referenced this issue Feb 28, 2019
dg added a commit that referenced this issue Feb 28, 2019
dg added a commit that referenced this issue Feb 28, 2019
dg added a commit that referenced this issue Mar 1, 2019
dg added a commit that referenced this issue Mar 1, 2019
@adrianbj adrianbj closed this Mar 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.