New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using pre-rendering techniques [$25] #318

Closed
aggrosoft opened this Issue Nov 21, 2012 · 25 comments

Comments

Projects
None yet
10 participants
@aggrosoft
Contributor

aggrosoft commented Nov 21, 2012

Currently the _draw implementation of static canvas will always do a full render of each object - no matter what has been modified. The HTML5 Canvas best practices suggest to render an object only once if you have a lot of render passes and then just draw the bitmap representation if needed - this is equal to what flash does using the cacheAsBitmap property and it's display list .

Without caching:

// canvas, context are defined
function render() {
  drawMario(context);
  requestAnimationFrame(render);
}

Using caching:

var m_canvas = document.createElement('canvas');
m_canvas.width = 64;
m_canvas.height = 64;
var m_context = m_canvas.getContext(‘2d’);
drawMario(m_context);

function render() {
  context.drawImage(m_canvas, 0, 0);
  requestAnimationFrame(render);
}

For Fabric this would mean that the objects manage a property to tell if a redraw is really needed - this could be bound to the stateProperties. Then the static_canvas will use this property to call render or not. After rendering the bitmap will be cached and reused on the next calls.

Maybe I just did not inspect the code good enough, is fabric maybe already doing this?

See:

http://www.html5rocks.com/en/tutorials/canvas/performance/

There is a $25 open bounty on this issue. Add to the bounty at Bountysource.

@kangax

This comment has been minimized.

Show comment
Hide comment
@kangax

kangax Nov 24, 2012

Member

No, Fabric is not doing this kind of caching yet. We tried some optimizations once with complex SVG paths (replacing them with temporary images) but ran into few problems and eventually dropped it.

Avoiding rendering of objects that weren't modified could be a great optimization. There are some issues though, like the fact that we would need to mark objects as "dirty" somehow. For example, this wouldn't "work" anymore:

object.left = 100; // new value
canvas.renderAll(); // need to mark object dirty before rendering

although this could be "fixed" via set:

object.setLeft(100); // object is marked as dirty
canvas.renderAll(); // object is rendered at a new location

Would it be OK for objects to behave this way? Even though it's already recommended to use set/get methods to work with properties, I imagine that a lot of people still use plain properties.

On the other hand, we can always enable something like this via optional property on canvas! That way we could preserve backwards compatibility.

Member

kangax commented Nov 24, 2012

No, Fabric is not doing this kind of caching yet. We tried some optimizations once with complex SVG paths (replacing them with temporary images) but ran into few problems and eventually dropped it.

Avoiding rendering of objects that weren't modified could be a great optimization. There are some issues though, like the fact that we would need to mark objects as "dirty" somehow. For example, this wouldn't "work" anymore:

object.left = 100; // new value
canvas.renderAll(); // need to mark object dirty before rendering

although this could be "fixed" via set:

object.setLeft(100); // object is marked as dirty
canvas.renderAll(); // object is rendered at a new location

Would it be OK for objects to behave this way? Even though it's already recommended to use set/get methods to work with properties, I imagine that a lot of people still use plain properties.

On the other hand, we can always enable something like this via optional property on canvas! That way we could preserve backwards compatibility.

@aggrosoft

This comment has been minimized.

Show comment
Hide comment
@aggrosoft

aggrosoft Nov 24, 2012

Contributor

For backwards compatibility this should be an opt-in option on the canvas object, and refactoring the code to use the set methods is no problem from what I see. I think this will give a big improvement on the performance of canvas that hold a lot of objects, very complex objects and also it will do a good job for path objects and all transformations done on them.

Maybe fabric.Object should have a method 'setDirty()' as well to mark objects for re-rendering if you are running into unusual usage somehow.

All of this reminds me of what flex is doing with their framework implementation which keeps rendering cycles and all other computations down to a minimum set needed:

http://weblog.mrinalwadhwa.com/wp-content/uploads/2009/06/flex-4-component-lifecycle.pdf

Contributor

aggrosoft commented Nov 24, 2012

For backwards compatibility this should be an opt-in option on the canvas object, and refactoring the code to use the set methods is no problem from what I see. I think this will give a big improvement on the performance of canvas that hold a lot of objects, very complex objects and also it will do a good job for path objects and all transformations done on them.

Maybe fabric.Object should have a method 'setDirty()' as well to mark objects for re-rendering if you are running into unusual usage somehow.

All of this reminds me of what flex is doing with their framework implementation which keeps rendering cycles and all other computations down to a minimum set needed:

http://weblog.mrinalwadhwa.com/wp-content/uploads/2009/06/flex-4-component-lifecycle.pdf

@kangax

This comment has been minimized.

Show comment
Hide comment
@kangax

kangax Nov 24, 2012

Member

Feel free to take a stab at it! One problem I can think of is that we'll
need to take care of clearContext, since we can't just stop rendering
objects if they're dirty. They'll be erased forever since clearContext is
called on every renderAll.

On Sat, Nov 24, 2012 at 6:46 PM, aggrosoft notifications@github.com wrote:

For backwards compatibility this should be an opt-in option on the canvas
object, and refactoring the code to use the set methods is no problem from
what I see. I think this will give a big improvement on the performance of
canvas that hold a lot of objects, very complex objects and also it will do
a good job for path objects and all transformations done on them.

Maybe fabric.Object should have a method 'setDirty()' as well to mark
objects for re-rendering if you are running into unusual usage somehow.


Reply to this email directly or view it on GitHubhttps://github.com/kangax/fabric.js/issues/318#issuecomment-10680531.

Member

kangax commented Nov 24, 2012

Feel free to take a stab at it! One problem I can think of is that we'll
need to take care of clearContext, since we can't just stop rendering
objects if they're dirty. They'll be erased forever since clearContext is
called on every renderAll.

On Sat, Nov 24, 2012 at 6:46 PM, aggrosoft notifications@github.com wrote:

For backwards compatibility this should be an opt-in option on the canvas
object, and refactoring the code to use the set methods is no problem from
what I see. I think this will give a big improvement on the performance of
canvas that hold a lot of objects, very complex objects and also it will do
a good job for path objects and all transformations done on them.

Maybe fabric.Object should have a method 'setDirty()' as well to mark
objects for re-rendering if you are running into unusual usage somehow.


Reply to this email directly or view it on GitHubhttps://github.com/kangax/fabric.js/issues/318#issuecomment-10680531.

@alesdotio

This comment has been minimized.

Show comment
Hide comment
@alesdotio

alesdotio Jul 11, 2013

Any updates regarding this feature? Personally, I would really benefit from it, since I am using fabric for editing more 500+ objects.

It really starts to be a problem when you drag objects for example. Even though you are dragging only one object, fabric redraws all of them. I could easily live with manually marking objects that do not need re-drawing, and manually un-marking them when once ready.

alesdotio commented Jul 11, 2013

Any updates regarding this feature? Personally, I would really benefit from it, since I am using fabric for editing more 500+ objects.

It really starts to be a problem when you drag objects for example. Even though you are dragging only one object, fabric redraws all of them. I could easily live with manually marking objects that do not need re-drawing, and manually un-marking them when once ready.

@kangax

This comment has been minimized.

Show comment
Hide comment
@kangax

kangax Jul 11, 2013

Member

@alesdotio Didn't have a chance to do this yet, but also can't wait to get it done already.

Member

kangax commented Jul 11, 2013

@alesdotio Didn't have a chance to do this yet, but also can't wait to get it done already.

@jamespacileo

This comment has been minimized.

Show comment
Hide comment
@jamespacileo

jamespacileo Jul 13, 2013

@kangax this would be really useful.

I like EaselJS' implementation as it's abstracted away, KineticJS' implementation can be a bit tedious at times.
I don't mind if the cache needs to be manually invalidated by calling a method e.g. updateCache/invalidateCache. I don't think auto invalidating of cache would be necessary to start with.

There are plenty of use cases where this would be useful, especially mobile games where performance is very important.

I really like Fabric's API, if it had caching as well it would definitely be my go to library. :)

jamespacileo commented Jul 13, 2013

@kangax this would be really useful.

I like EaselJS' implementation as it's abstracted away, KineticJS' implementation can be a bit tedious at times.
I don't mind if the cache needs to be manually invalidated by calling a method e.g. updateCache/invalidateCache. I don't think auto invalidating of cache would be necessary to start with.

There are plenty of use cases where this would be useful, especially mobile games where performance is very important.

I really like Fabric's API, if it had caching as well it would definitely be my go to library. :)

@kangax

This comment has been minimized.

Show comment
Hide comment
@kangax

kangax Jul 15, 2013

Member

@jamespacileo I just changed a tag on this ticket from "possible feature" to "feature" :) Can you tell me more about Easel's implementation, or at least point to their docs/source? I'd be curious to get familiar with it.

Member

kangax commented Jul 15, 2013

@jamespacileo I just changed a tag on this ticket from "possible feature" to "feature" :) Can you tell me more about Easel's implementation, or at least point to their docs/source? I'd be curious to get familiar with it.

@jamespacileo

This comment has been minimized.

Show comment
Hide comment
@jamespacileo

jamespacileo Jul 16, 2013

@kangax thanks! I'll report back tomorrow! :)

jamespacileo commented Jul 16, 2013

@kangax thanks! I'll report back tomorrow! :)

@jamespacileo

This comment has been minimized.

Show comment
Hide comment
@jamespacileo

jamespacileo Jul 17, 2013

@kangax here is what EaselJS does

each Shape has the following methods cache uncache updateCache.

The cache method is used to mark a shape for caching, The content is rendered in an offscreen canvas. Created via createElement and not appended to DOM. The created canvas would fit the shape size times the scale.

uncache is used to invalidate and turn off caching.

updateCache is used to invalidate the cached data

Once a shape is marked for caching on the next redraw the shape will be drawn to it's own "offscreen" canvas object. On every subsequent redraw the shape will use the image drawn into the offscreen canvas object instead of drawing from scratch. If the cache needs updating updateCache would have to be called, on the following redraw the offscreen canvas is rewritten.

The only weakness is that the cache requires the user to manually supply x, y, width, height. FabricJS keeps track of these values so maybe they could be defaulted?

Quote from EaselJS docs:

Draws the display object into a new canvas, which is then used for subsequent draws. For complex content that does not change frequently (ex. a Container with many children that do not move, or a complex vector Shape), this can provide for much faster rendering because the content does not need to be re-rendered each tick. The cached display object can be moved, rotated, faded, etc freely, however if it's content changes, you must manually update the cache by calling updateCache() or cache() again. You must specify the cache area via the x, y, w, and h parameters. This defines the rectangle that will be rendered and cached using this display object's coordinates. For example if you defined a Shape that drew a circle at 0, 0 with a radius of 25, you could call myShape.cache(-25, -25, 50, 50) to cache the full shape.

Useful links:
shape.cache(x, y, width, height, [scale=1])
shape.uncache()
shape.updateCache()

Usage example

jamespacileo commented Jul 17, 2013

@kangax here is what EaselJS does

each Shape has the following methods cache uncache updateCache.

The cache method is used to mark a shape for caching, The content is rendered in an offscreen canvas. Created via createElement and not appended to DOM. The created canvas would fit the shape size times the scale.

uncache is used to invalidate and turn off caching.

updateCache is used to invalidate the cached data

Once a shape is marked for caching on the next redraw the shape will be drawn to it's own "offscreen" canvas object. On every subsequent redraw the shape will use the image drawn into the offscreen canvas object instead of drawing from scratch. If the cache needs updating updateCache would have to be called, on the following redraw the offscreen canvas is rewritten.

The only weakness is that the cache requires the user to manually supply x, y, width, height. FabricJS keeps track of these values so maybe they could be defaulted?

Quote from EaselJS docs:

Draws the display object into a new canvas, which is then used for subsequent draws. For complex content that does not change frequently (ex. a Container with many children that do not move, or a complex vector Shape), this can provide for much faster rendering because the content does not need to be re-rendered each tick. The cached display object can be moved, rotated, faded, etc freely, however if it's content changes, you must manually update the cache by calling updateCache() or cache() again. You must specify the cache area via the x, y, w, and h parameters. This defines the rectangle that will be rendered and cached using this display object's coordinates. For example if you defined a Shape that drew a circle at 0, 0 with a radius of 25, you could call myShape.cache(-25, -25, 50, 50) to cache the full shape.

Useful links:
shape.cache(x, y, width, height, [scale=1])
shape.uncache()
shape.updateCache()

Usage example

@kangax

This comment has been minimized.

Show comment
Hide comment
@kangax

kangax Jul 20, 2013

Member

@jamespacileo Thanks for explanation! This makes perfect sense, of course. Fun fact: we had this functionality back in the days. You can see the "stub" rendering block in an older version of all.js, for example, https://github.com/kangax/fabric.js/blob/a535b004e697f57394d85157e92596fad773d010/dist/all.js#L6623-L6633

The difficulties were with updating cached image when resizing a (vector) shape. At one point, I decided to drop it.

This was back in the days when we didn't even have fabric.StaticCanvas, which is kind of perfect for this case, since there's no user interaction with shapes.

Member

kangax commented Jul 20, 2013

@jamespacileo Thanks for explanation! This makes perfect sense, of course. Fun fact: we had this functionality back in the days. You can see the "stub" rendering block in an older version of all.js, for example, https://github.com/kangax/fabric.js/blob/a535b004e697f57394d85157e92596fad773d010/dist/all.js#L6623-L6633

The difficulties were with updating cached image when resizing a (vector) shape. At one point, I decided to drop it.

This was back in the days when we didn't even have fabric.StaticCanvas, which is kind of perfect for this case, since there's no user interaction with shapes.

@kangax

This comment has been minimized.

Show comment
Hide comment
@kangax

kangax Jul 20, 2013

Member

@jamespacileo this kind of caching, coupled with dirty-rectangle checking should provide tremendous speed improvements in Fabric

Member

kangax commented Jul 20, 2013

@jamespacileo this kind of caching, coupled with dirty-rectangle checking should provide tremendous speed improvements in Fabric

@coulix

This comment has been minimized.

Show comment
Hide comment
@coulix

coulix Jul 20, 2013

Contributor

Will this somehow, improves performance of pen tool on a large canvas ? > 3000px. I am looking for some technical trick, maybe having a dynamicly resizing canvas over topCanvas to receive and draw the pen mouvements.

Contributor

coulix commented Jul 20, 2013

Will this somehow, improves performance of pen tool on a large canvas ? > 3000px. I am looking for some technical trick, maybe having a dynamicly resizing canvas over topCanvas to receive and draw the pen mouvements.

@kangax

This comment has been minimized.

Show comment
Hide comment
@kangax

kangax Jul 20, 2013

Member

Dirty rectangle checking should improve performance in this case.

Sent from my iPhone

On Jul 20, 2013, at 13:12, coulix notifications@github.com wrote:

Will this somehow, improves performance of pen tool on a large canvas ? > 3000px. I am looking for some technical trick, maybe having a dynamicly resizing canvas over topCanvas to receive and draw the pen mouvements.


Reply to this email directly or view it on GitHub.

Member

kangax commented Jul 20, 2013

Dirty rectangle checking should improve performance in this case.

Sent from my iPhone

On Jul 20, 2013, at 13:12, coulix notifications@github.com wrote:

Will this somehow, improves performance of pen tool on a large canvas ? > 3000px. I am looking for some technical trick, maybe having a dynamicly resizing canvas over topCanvas to receive and draw the pen mouvements.


Reply to this email directly or view it on GitHub.

@jamespacileo

This comment has been minimized.

Show comment
Hide comment
@jamespacileo

jamespacileo Jul 20, 2013

@kangax awesome, great news! :)

Thanks for considering this. I'll be around, contribute more info if I can.

jamespacileo commented Jul 20, 2013

@kangax awesome, great news! :)

Thanks for considering this. I'll be around, contribute more info if I can.

@Samreay

This comment has been minimized.

Show comment
Hide comment
@Samreay

Samreay Apr 23, 2014

Hey guys. I'm looking to use Fabricjs for in a scientific computing context, and would be rendering 20,000 items to screen (three layers of graphs, each displaying astronomical spectra). Has there been any update on caching or dirty-rectangle checking for Fabric?

I have been able to find this: http://fabricjs.com/svg-caching/, however this is not the dirty update method discussed for this feature request.

Samreay commented Apr 23, 2014

Hey guys. I'm looking to use Fabricjs for in a scientific computing context, and would be rendering 20,000 items to screen (three layers of graphs, each displaying astronomical spectra). Has there been any update on caching or dirty-rectangle checking for Fabric?

I have been able to find this: http://fabricjs.com/svg-caching/, however this is not the dirty update method discussed for this feature request.

@kangax

This comment has been minimized.

Show comment
Hide comment
@kangax

kangax Apr 24, 2014

Member

@Samreay Unfortunately, we haven't gotten to these optimization techniques yet. However, if you don't need interaction, it's fairly easy to create 3 fabric canvases overlayed on top of each other.

Member

kangax commented Apr 24, 2014

@Samreay Unfortunately, we haven't gotten to these optimization techniques yet. However, if you don't need interaction, it's fairly easy to create 3 fabric canvases overlayed on top of each other.

@Samreay

This comment has been minimized.

Show comment
Hide comment
@Samreay

Samreay Apr 27, 2014

Hi kangax. Thanks for the update mate.

Samreay commented Apr 27, 2014

Hi kangax. Thanks for the update mate.

@matte00

This comment has been minimized.

Show comment
Hide comment
@matte00

matte00 Jun 4, 2014

Thank you kangax for your great library! Have you got news for this request?

matte00 commented Jun 4, 2014

Thank you kangax for your great library! Have you got news for this request?

@kangax

This comment has been minimized.

Show comment
Hide comment
@kangax

kangax Jun 5, 2014

Member

I will make sure to update this ticket once there's news

Member

kangax commented Jun 5, 2014

I will make sure to update this ticket once there's news

@taikar

This comment has been minimized.

Show comment
Hide comment
@taikar

taikar Apr 21, 2015

Is there any news on this request?

If not, if you any notes/plans on how you were planning on implementing this then I would be happy to contribute to the project!

taikar commented Apr 21, 2015

Is there any news on this request?

If not, if you any notes/plans on how you were planning on implementing this then I would be happy to contribute to the project!

@kangax

This comment has been minimized.

Show comment
Hide comment
@kangax

kangax Apr 22, 2015

Member

We'll be focusing on performance improvements in the upcoming release. Some things are already in effect — for example, we do itext cursor rendering on a separate canvas. Notes on implementing — hard to tell; these are the usual common techniques used in games, etc. so we just need to adapt them according to fabric's architecture. Any help is welcome of course!

Member

kangax commented Apr 22, 2015

We'll be focusing on performance improvements in the upcoming release. Some things are already in effect — for example, we do itext cursor rendering on a separate canvas. Notes on implementing — hard to tell; these are the usual common techniques used in games, etc. so we just need to adapt them according to fabric's architecture. Any help is welcome of course!

@jwmao

This comment has been minimized.

Show comment
Hide comment
@jwmao

jwmao Mar 7, 2016

@kangax , any updates regarding the performance improvements using cache. My application is having performance issues when rendering objects on large canvas. Especially on Chrome with hardware acceleration turned on. There are not a lot of objects on my canvas, my canvas size is 3050px X 3050px. The dragging is very slow and same for mouse clicks. I'm just wondering with caching, it can improve the overall performance.

jwmao commented Mar 7, 2016

@kangax , any updates regarding the performance improvements using cache. My application is having performance issues when rendering objects on large canvas. Especially on Chrome with hardware acceleration turned on. There are not a lot of objects on my canvas, my canvas size is 3050px X 3050px. The dragging is very slow and same for mouse clicks. I'm just wondering with caching, it can improve the overall performance.

@asturur

This comment has been minimized.

Show comment
Hide comment
@asturur

asturur Mar 8, 2016

Member

with a 9Mega pixel canvas your problem is probably the use of clearRect() for the full canvas.
you would benefit nearly nothing from object caching, probably more from dirty rects (another possible feature)

Member

asturur commented Mar 8, 2016

with a 9Mega pixel canvas your problem is probably the use of clearRect() for the full canvas.
you would benefit nearly nothing from object caching, probably more from dirty rects (another possible feature)

@asturur

This comment has been minimized.

Show comment
Hide comment
@asturur

asturur May 23, 2016

Member

I opened a PR to address this.
It is in usable state on most shapes, need some testing.
It does not scale yet.
Is a planned feature for 1.7.0, cache will be enabled on by default if it does not affect performance on small quantity of objects.

Member

asturur commented May 23, 2016

I opened a PR to address this.
It is in usable state on most shapes, need some testing.
It does not scale yet.
Is a planned feature for 1.7.0, cache will be enabled on by default if it does not affect performance on small quantity of objects.

@asturur asturur referenced this issue May 23, 2016

Closed

Object caching #2989

@asturur asturur added this to the 1.7.0 milestone Oct 3, 2016

@asturur

This comment has been minimized.

Show comment
Hide comment
@asturur

asturur Nov 13, 2016

Member

closed with #3317

Member

asturur commented Nov 13, 2016

closed with #3317

@asturur asturur closed this Nov 13, 2016

@asturur asturur reopened this Nov 13, 2016

@asturur asturur closed this Nov 13, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment