Performance & Memory Optimisations

[balupton]

Memory

Issues

• Discussion: Memory usage optimisations #545 memory usage optimisations discussion
• Stream content instead of buffering it #527 stream not buffer - risks stuffing up content reading in general
• Introduction of raw directory that is not parsed #276 raw directory - no risks - completed
• Forget content after it's needed #528 forget content after it's needed - risks stuffing up references inside dynamic documents
• Render documents as we need them #629 render documents as we need them
• Get rid of Backbone Models for native javascript objects #788 Get rid of Backbone Models for native javascript objects

Notes

• watch memory and decrease parallel tasks is another option (requires Forget content after it's needed #528 to be effective) but is silly compared to just doing Stream content instead of buffering it #527
• memory optimisations are essential for deployment to dotcloud - https://www.dotcloud.com/pricing.html

Performance

Issues

• DocPad got slower around v6.30, 6.31, 6.32 #508 performance optimisations discussion
• Keep references on which documents reference which #336 Keep references on which documents reference which
• listen to specific change listeners, rather than just the general one bevry/query-engine#24 specific (instead of generic) change listeners
• Fiddling with suspending collections #535 suspend collections - closed due to negligible performance impact
• Remove all traces of synchronous file system calls #538 remove all traces of synchronous file system calls - completed
• Discussion: Importer Speed Optimisations #590 importer speed optimisations
• Get rid of Backbone Models for native javascript objects #788 Get rid of Backbone Models for native javascript objects
• implement a "no streams" option in caterpillar

Notes

• performance improves with native streams over readable-stream inside caterpillar (until we drop 0.8 support, tough)

---

Want to back this issue? Place a bounty on it! We accept bounties via Bountysource.

[pflannery]

Have been looking at docpad heap snapshots for the past day or so and found something I wanted to share with others.

In some circumstances coffee-script will instantiate your classes as "ctor" instead of the actual defined class name. (see here for more jashkenas/coffeescript#3236)

I have a very large docpad instance where there is around 16mb across 4644 ctor classes.
The problem with the ctor class is that it can actual refer to many different classes.

Here is a sample of class names in docpad heaps that like to lead a second life called "ctor"

Docpad
Task
TaskGroup
QueryCollection
EventSystem
...many more

To determine the real class behind the ctor class you can look at the ctor.constructor or the ctor.proto.constructor

So as a heads up when looking at a docpad heap snapshot make sure you also visit the ctor class. This will contain some very import class instances.

disclaimer: i'm not suggesting that ctor is introducing memory leaks. All I know is that it introduces confusion in the heap.

[pflannery]

Another discovery I made today is that the current release of chrome has an issue when trying to compare nodejs heap snapshots. Fortunately it's working in Chrome Canary

Really useful when comparing your docpad changes ;)

[balupton]

Migrated to: https://discuss.bevry.me/t/docpad-performance-memory-details/583?u=balupton