Performance & Memory Optimisations #529

Closed
balupton opened this Issue Jun 20, 2013 · 3 comments

Comments

Projects
None yet
2 participants
@balupton
Member

balupton commented Jun 20, 2013

Memory

Issues

  • #545 memory usage optimisations discussion
  • #527 stream not buffer - risks stuffing up content reading in general
  • #276 raw directory - no risks - completed
  • #528 forget content after it's needed - risks stuffing up references inside dynamic documents
  • #629 render documents as we need them
  • #788 Get rid of Backbone Models for native javascript objects

Notes

  • watch memory and decrease parallel tasks is another option (requires #528 to be effective) but is silly compared to just doing #527
  • memory optimisations are essential for deployment to dotcloud - https://www.dotcloud.com/pricing.html

Performance

Issues

  • #508 performance optimisations discussion
  • #336 Keep references on which documents reference which
  • bevry/query-engine#24 specific (instead of generic) change listeners
  • #535 suspend collections - closed due to negligible performance impact
  • #538 remove all traces of synchronous file system calls - completed
  • #590 importer speed optimisations
  • #788 Get rid of Backbone Models for native javascript objects
  • implement a "no streams" option in caterpillar

Notes

  • performance improves with native streams over readable-stream inside caterpillar (until we drop 0.8 support, tough)

Want to back this issue? Place a bounty on it! We accept bounties via Bountysource.

@pflannery

This comment has been minimized.

Show comment
Hide comment
@pflannery

pflannery Nov 12, 2013

Contributor

Have been looking at docpad heap snapshots for the past day or so and found something I wanted to share with others.

In some circumstances coffee-script will instantiate your classes as "ctor" instead of the actual defined class name. (see here for more jashkenas/coffeescript#3236)

I have a very large docpad instance where there is around 16mb across 4644 ctor classes.
The problem with the ctor class is that it can actual refer to many different classes.

Here is a sample of class names in docpad heaps that like to lead a second life called "ctor"

Docpad
Task
TaskGroup
QueryCollection
EventSystem
...many more

To determine the real class behind the ctor class you can look at the ctor.constructor or the ctor.proto.constructor

So as a heads up when looking at a docpad heap snapshot make sure you also visit the ctor class. This will contain some very import class instances.

disclaimer: i'm not suggesting that ctor is introducing memory leaks. All I know is that it introduces confusion in the heap.

Contributor

pflannery commented Nov 12, 2013

Have been looking at docpad heap snapshots for the past day or so and found something I wanted to share with others.

In some circumstances coffee-script will instantiate your classes as "ctor" instead of the actual defined class name. (see here for more jashkenas/coffeescript#3236)

I have a very large docpad instance where there is around 16mb across 4644 ctor classes.
The problem with the ctor class is that it can actual refer to many different classes.

Here is a sample of class names in docpad heaps that like to lead a second life called "ctor"

Docpad
Task
TaskGroup
QueryCollection
EventSystem
...many more

To determine the real class behind the ctor class you can look at the ctor.constructor or the ctor.proto.constructor

So as a heads up when looking at a docpad heap snapshot make sure you also visit the ctor class. This will contain some very import class instances.

disclaimer: i'm not suggesting that ctor is introducing memory leaks. All I know is that it introduces confusion in the heap.

@pflannery

This comment has been minimized.

Show comment
Hide comment
@pflannery

pflannery Nov 12, 2013

Contributor

Another discovery I made today is that the current release of chrome has an issue when trying to compare nodejs heap snapshots. Fortunately it's working in Chrome Canary

Really useful when comparing your docpad changes ;)

Contributor

pflannery commented Nov 12, 2013

Another discovery I made today is that the current release of chrome has an issue when trying to compare nodejs heap snapshots. Fortunately it's working in Chrome Canary

Really useful when comparing your docpad changes ;)

@balupton

This comment has been minimized.

Show comment
Hide comment

@balupton balupton closed this Mar 21, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment