2d canvas / webgl #67

Open
forresto opened this Issue Jan 4, 2014 · 15 comments

3 participants

@forresto
The Grid member

Thinking about React.js treating the browser more like GPU / game engine programming ("repaint whole app, only changed parts get wet").

The SVG #66 branch is going to make our current design nicer to implement.

But planning for the future, I think we're going to end up doing everything in WebGL for the control and speed. How could React's concept of virtual DOM be structured with FBP logic and WebGL views?

Cons:

  • lack of css (?)
@forresto
The Grid member

Since the structure is built on an immediate mode metaphor, and renderers are just functions, it should be possible to build on the react/svg work to move to a canvas/webgl game engine when speed or aesthetics makes it attractive:

The main thing to test in canvas/webgl engines is if their hit-testing can differentiate overlapping curves. When I checked at least pixi could only hit-test rectangles and ellipses.

Paper.js looks like the winner for our use case. 🍎

@forresto forresto referenced this issue Jan 7, 2014
Closed

speedup #68

@forresto
The Grid member
@subtleGradient

The trick I've seen used with canvas is to draw the same objects with a solid aliased color into a separate canvas. Then when you click, get the pixel color on the hidden canvas and match it up to the real object using a lookup table of pixel color keys to real object values. Not sure if that's the same technique that shift eleven is using or not.

@forresto
The Grid member

Looks like a different technique, but seems cleaner, and faster. As long as we have < 16777216 edges 😉

Shift Eleven is keeping track of all paths and then writing them in reverse order on every click. With 10000 objects you start to notice.

💡 Thanks!

@forresto forresto added the jank label Feb 22, 2014
@forresto
The Grid member

idea

While animating (zoom, pan, drag node) switch to a 2d canvas view for 60fps goodness. Then release to rerender SVG.

Then we still get the text and click advantages of SVG. A simplified canvas renderer wouldn't need all the bells and whistles. Just general shapes and colors, a little more than the-graph-thumb. Canvas' setTransform() matrix should be enough to keep things aligned with the SVG's transform matrix.

Would need tests to see how jarring switching between would be.

@subtleGradient

Maybe always render using canvas with SVG layered on top for text and noodles. Then hide the SVG while zooming and panning.

@forresto forresto added a commit that referenced this issue Mar 4, 2014
@forresto forresto canvas for background grid #67 fb6a889
@forresto
The Grid member

WebGL implementation of SVG, focused on d3: http://pathgl.com/ (Missing text and curves, which is most of what we have. And there is still no WebGL on iOS.)

@forresto
The Grid member
@subtleGradient

MathBox2 looks almost ludicrously fantastic: http://acko.net/files/pres/siggraph-2014-bof/online.html

@subtleGradient

iOS 8 has WebGL enabled for Safari and WebViews by default!
http://caniuse.com/#feat=webgl

@subtleGradient

ShaderGraph 1 could link up snippets of GLSL into pipelines, by matching type signatures.

You could construct arbitrary directed acyclic graphs out of snippets, which got compiled into a single vertex and fragment shader. It's driven by code though, it's not a graphical UI.

ShaderGraph 2 fixes some bad design decisions, and adds one major change: callbacks. It turns the traditional data flow into something functional, allowing e.g. the getArrowGeometry shader to call getSample as much as it wants. The data flow is essentially redirected to a completely different part of the graph.

But callbacks can be graphs themselves. Any open inputs or outputs are bundled up into its type signature. This is what allows complicated behavior to be wrapped up and re-used in a completely modular fashion.

@forresto
The Grid member

Yeah, I emailed with @unconed about making a flowhub runtime for ShaderGraph 2. Looking forward to playing with it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment