-
-
Notifications
You must be signed in to change notification settings - Fork 67
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement real-time vector graphics rendering (e.g. SVG, Rive) #2924
Comments
Well, if you're making this claim it would be nice to know where you're getting this metric from. 👀 If this to be implemented, you have to make sure that users aren't confused why their vector assets suddenly cause way worse performance. |
godotengine/godot#44772 implements this for SVGs using the same rendering technology as MSDF fonts. Since it was implemented as a proof of concept, it's limited to monochrome shapes and its usability isn't very good right now. That said, it can surely be made more convenient to use by creating a dedicated node. Note that adding new dependencies to Godot is serious business, so Blend2D may not be accepted in core. This is especially the case if Blend2D turns out to be difficult to build or increases binary size significantly. |
My approach for doing real-time vector art rendering is step by step via godotengine/godot#49645. Stage 1: Implement thorvg. Notice the render commands in the documentation. There is a lot of concern about library size, so I will need to get consensus on that. Edited: Blend2D is not planning to implement SVG importing. blend2d/blend2d#107 (comment) See also:
Please see previous prototypes. Both "vector graphics" and "lottie" in Godot Engine has trialed two modes. 1. is a baked mode and the second is a converted to curves mode. There is performance difficulty via the runtime route, but need help to revisit via the Vulkan Thorvg backend option |
Thanks for the comments and considerations everyone. |
I finally have a working c++ version playing Rive files using Skia as the rendering engine. Not in Godot, just a standalone app at present. Rive is a vector format for animation (much more than just animation, actually) which would be an amazing addition to Godot. I hope that @fire or someone else is able to implement a thorvg Rive renderer in time! (see skia renderer for reference). |
Here is a Tweet I saw today, just to show that this is indeed a useful feature that many developers are looking for and can't currently find in any game engine. |
godotengine/godot#49645 is currently merged into master for ThorVG vector graphics support. Currently the svg module is using the standard importer design, but there's no real reason to be only offline importable. Here's an early preview on SVG to Mesh. Not expecting to finish SVG to mesh before Godot 4.0 release though. |
Density-independent graphics are Really useful as we're already on the way of having every screen be a HiDPI screen. Every mobile phone already has a very dense screen with at least 144dpi, and even many of the mid-range computer monitors are already denser than 72dpi even if they don't quite reach true HiDPI yet (e.g. my 1920×1080px monitor is 80.68dpi, which is rounded up to 96dpi by Godot's My current workaround as of Godot 3.4.4 is to use Distance Field shaders. |
These are currently the three ways to use vector graphics in Godot Engine 4.
Missing:
|
https://github.com/Giwayume/godot-svg Here is a proof of concept demonstrating that bezier curves can be drawn in shader code, using a SVG to mesh conversion approach. Curves are not tessellated as line segments, they are perfect at every resolution. Comparing to fire's example above, the circles in the eyes of the Godot logo in this technique use 12 tris each, instead of like 30. And they look perfect no matter how closely you look at the mesh. |
@Giwayume i can't check your godot project right now, but this sounds great in past projects i have recreated some vector shapes as shaders to get infinite smoothness, because your proof-of-concept'd feature isn't in godot. if this feature gets added, i can get these smooth shapes without shader code but from just importing my SVG files |
This project takes a generalized approach. The technique can theoretically render any SVG shape, current bugs aside. |
A year ago, I made a realtime vector+SVG importer plugin for Godot 3. It's not complete, but it's never going to be complete, so I just posted it here. It supports shapes, strokes (via stroke to fill conversion), winding rules, and basic gradients. I was also planning to implement mesh gradients, but never got around to it. Underneath, it works purely as a shader. It makes an approximation of the vector's bounding box, draws a rectangular quad around it, and uses a tracing algorithm in the fragment shader. I did hit quite a few difficulties:
Incidentally, vectors can actually give an advantage over extremely high resolution sprites, due to taking up less memory, especially on mobile platform. But I did test this using a browser engine with native vector support, not using Godot. Overall, new features in Godot 4 might make it easier to implement an efficient vector shape renderer. But a native vector shape object, even if it's only shape fill (with winding rule support), that runs in the render pipeline, can save a significant amount of effort. Implementing gradients, stroker, shape tweens, etc, on top of that, is much easier. So, IMO, import formats should be left to plugins, but a basic bezier curve shape fill with infinite smoothness and builtin editor should still be implemented in-engine. Preferably with quadric and arc support (although the prior can be accurately represented using cubic) |
It's cool to see calculating the fill rule per-pixel in a shader is feasible for simple shapes, though the performance looks very poor. On my laptop: EDIT: Also, the FPS absolutely tanks to single digits the more I zoom in. If you split the quad the shader is drawn across into a grid of quads, could take advantage of triangle culling. I actually can't get many SVGs to render in order to test more complex examples. I'd love to see improvements made to that project as theoretically it could be the best technique for rendering SVG animations that morph the path every frame. |
the SVG format was made for websites, not games SVG has many advanced features that help with making more detailed pictures at the cost of speed. you can make lite vector graphics with Godot features like Polygon2D and shaders and the |
I think to SlugFilter's point, what people mostly want out of the engine is a way to render shapes with smooth bezier curves, that can be colored, textured, or used with a shader. What I have presented is a concept where that shape is converted to a triangulated mesh, where each bezier command gets dedicated triangles and the calculation to draw the bezier curves in the shader is simple linear algebra. This is very performant at runtime, but requires some work on the CPU beforehand to generate the mesh from the SVG path commands. The approach SlugFilter shows could potentially be more performant for animating the I could envision Godot natively supporting a node that uses both techniques in an exchangeable manner, either automatically if it's feasible to detect that the developer is trying to animate the path commands, or through a boolean property on the node. |
@sosasees The issue with both While a complex format like SVG isn't built for speed, something like CanvasRenderingContext2D::bezierCurveTo is built for performance, and is used to effect in HTML5 2D games. Even something as tiny as adding a In fact, I might prefer this minimal approach, because, despite the extra effort, I'm more likely to do well making and using my own curve editor, then rely on one designed by someone else. @Giwayume The reason it's slow is because the fragment shader recalculates a lot of things a normal rasterizer would only need to calculate once. For example, if a bunch of path segments are completely above or below the current y, each fragment in the same y would need to make the same calculation to figure this out, even though it's the exact same calculation for all of them (I do winding counting by tracing a ray in the screen's x+ direction, so checks for the y value are largely uniform). Even though they do this in parallel, the GPU's number of cores is not infinite, and it takes resources that could be otherwise used for parallelizing for different y values. As I've said above, if I had a "row" shader, that allowed me to make a bunch of calculations for a given y value, and then pass the results as an array to the fragment shader, it could vastly improve performance. This is how software rasterizers can fill curves very quickly. Hybrid rasterizers can do extract horizontal slices in CPU, then let the GPU simply draw horizontal line commands. An alternative approach is to draw winding additively on a buffer, then rasterize each row left to right, summing up the winding as you go. The fragment shader's inability to run in any sort of sequence works against it, again, but this could be alleviated by storing the winding in a tree format, that would make it possible to look up the winding for a given fragment at Put more simply, my approach is NOT the best approach, it is the best approach given the limitations of working inside Godot 3. Browsers and programs like Inkscape, demonstrate rasterizing far more complex vectors, at significantly higher speed, due to using more effective rasterization methods that can't be implemented in pure GDScript or simple vertex-fragment shaders. As for subdividing the shader rect, triangle culling wouldn't help with performance, because the fragment shader doesn't run for out-of-screen pixels anyway. If anything, it would just create extra effort for the GPU, as fragments near the seams have to run twice. The reason performance dies as you zoom in is simply because the shader is ran for more fragments. It would be pretty easy to test in practice, though. Replace the draw_rect in fill.gd with a draw_mesh. The locations of the vertices in the source mesh don't really matter, only the UVs, since the vertex shader uses only the UVs to produce the actual vertex positions. |
i like to make my game vector graphics as Godot shaders, mainly because i can add motion effects that i can't with SVG i understand that many people don't want to write shaders, so i respect the effort of adding realtime SVG rendering to Godot — different from Godot's current SVG support, which justs imports SVG files as raster images (similar to 'Export PNG' in Inkscape) |
@sosasees The issue is that shaders alone, in their current specs/limitations, are insufficient to implement generic and efficient bezier shape rendering. You can see that both my approach, and @Giwayume's approach, completely buckle under certain conditions. This is due to limitations of what the Godot shader specs expose, and how slow GDScript is for large iterations or recursion. It's possible that Godot 4 exposes better APIs for making a more efficient shader. I haven't tested. It still couldn't possibly be as efficient as an engine-supported command. Incidentally, if you only want infinitely smooth corners, and don't need beziers specifically, you could make a shader that turns a triangle into a filled arc, then make a smooth shape out of those. But it would be impossible to work with artists who expect standard tools, and very difficult to make any line effects and animations. |
This is not what I'm seeing, because the entire shape is already taking up the entire size of the game window, running at about 50fps, then as I zoom in more it dips into the single digits. There might be something Godot is doing under the hood that you're not expecting. |
@Giwayume Are you zooming near the center, or the edge? From what I saw, the shape you're using is a circle, which is implemented as 4 arcs (the editor shows 2, but the engine pre-divides arcs to 90 degrees, since they are easier to work with mathematically). So a possible culprit is that the number of subdivisions necessary before a given fragment is determined to be outside the arc's approximate bounding box increases as you zoom. From the get go, the arc's bounding box approximation isn't as good as for beziers. And a good approximation is necessary for early-stopping on the iterations. |
Yep, native vector graphics support for Godot could bring on Flash-like popularity for the engine, as vector unlocks many new kinds of game and application possibilities. How many times do we need to see grainy raster graphics before we accept that resolutions & devices will vary widely? The future is scalable. The present is, too. Even if the implementation just rasterized to the right size texture at runtime, that'd be good enough for many use cases. |
How does this proposal tie in to the use of SVGs as |
This proposal is about rendering SVGs as polygons or SDFs, while your proposal is different entirely. Each approach has upsides and downsides:
|
@TechnoPorg Textures are necessarily raster. Any attempt to use SVG as a texture will therefore necessarily set it to a specific resolution. This can already be done directly today, by importing the SVG as a raster image. However, if you want the resolution to change in runtime, it is possible to do this by combining this suggestion with a viewport. i.e. rendering the vector to a viewport, and then using the viewport's texture as the texture. Since you still need to manually adjust the viewport's resolution, it is questionable how much of an advantage this gives. It is impossible for an SVG to act as a "true" texture at infinite resolution. At most, it can be done as a shader. But that would make it difficult to combine with other shaders, and also, as demonstrated in the above shader-based vector graphics implementation, is incredibly slow, putting way more stress on the GPU than can be managed in a game that must run in real time. |
One of the thorvg team is employed at https://lottiefiles.com/ which takes After Effect animated motion artwork and renders it. So both lottie (standard) and rive (proprietary) are approaches to render. A contributor to godot engine is also working on shader based rasterization of vector artwork. So there are a few approaches to rendering animated vector graphics without going through the raster movie route. |
@SlugFiller Those are excellent points that I had not considered, and I like the idea of the viewport solution. |
@TechnoPorg If the vector doesn't change, you don't need to call |
@fire Now that I'm done working on this, I took the time to see how Tove2D implements its path to mesh. It's... "interesting". It uses a library called PolyPartition, and performs a few steps:
All in all, the algorithm is horribly inefficient, and can fail on intersecting edges, e.g. in a simple pentagram. Worse yet, PolyPartition actually contains a different algorithm called "Triangulate_MONO" which is basically Bently-Ottmann, but without intersection checking. It would still fail on self-intersection, but would run at a fraction of the time. Incidentally, Godot already contains an ear-clipping triangulator, used by Polygon2D, so including a third party library for it would have been redundant. |
I've found that a constrained delaunay implementation works fairly well for generating a triangulation with holes. The idea being, run the fill rule in the center of each triangle generated to find the holes. |
I didn't look into this deeply. @SlugFiller thanks for summary. Have a SVG to mesh cat :D. This was generated by the Tove2D PolyPartition workflow. Hope to find a way to get that into Godot Engine properly. |
This comment was marked as resolved.
This comment was marked as resolved.
@Giwayume Delauney will not only ignore edge intersections, but the edges themselves, as it does not accept them as input. It can choose to split triangles using an edge that crosses one of the original edges, causing it to have an incorrect shape, regardless of which triangles you leave out. @fire If my patch gets merged, you can use the methods I expose on |
You need to look up what a "constrained" delaunay is, not a regular delaunay. From your description, it appears you don't understand what I'm talking about. A constrained delaunay can be passed a list of edges (the edges that define the outside of a shape, the outside of a hole), then work around those, assuming that those edges must be attached. I have actually implemented this, it works. |
@Giwayume Wiki says there's |
This comment was marked as off-topic.
This comment was marked as off-topic.
|
This comment was marked as resolved.
This comment was marked as resolved.
My original design was for thorvg to first export to pixel buffers. Once things are futher along we can export static and animations as vector buffers for godot engine. We don’t expose the thorvg api bare. |
@felaugmar The issue with that approach is that ThorVG, as it appears in Godot, uses a 100% software renderer. It's good enough for turning icons to bitmaps in the editor. It is not, however, sufficient for rendering vector animations, in real time, at any resolution expected of a modern game. Even a meshing method, with GPU draw, is so slow that you can only afford 50-80 sprites on screen before going below 60 FPS. And that's running on an Intel i7. A 100% software scanline renderer would be even slower. If a render-to-texture approach was to be used, something like Vello or at least Pathfinder, would be more appropriate, as it at least uses GPU acceleration to achieve 60 FPS on large scenes. They do carry other downsides, and would be a new 3rd party dependency if taken as-is. One thing that could be a benefit in combination with a |
since @SlugFiller's comment, it seems even more unlikely that real-time vector graphics rendering will be added to Godot. |
Cross-posting from godotengine/godot#75278 (comment) There's been a suggestion to instead of implementing this directly, to implement hooks to the rendering process which would allow implementing this as an addon/extension. To anyone following this topic: Input would be appreciated. |
To add to SlugFiller's comment, we discussed the relative PR and the general consensus from the maintainers is that this feature opens up a new usecase for Godot and we're not sure that it's possible, resource wise, to support this long-term, offer bugfixes and new features to the people that would start using Godot more like they'd use flash for making games before. Due to these concerns, the first step is to figure out if this can be made into a gdextension, and if not, what's missing in order to make it into a gdextension. (web support for gdextension is a goal of Godot so the current non-working state should not be ground for putting this into core) If the extension route is not possible, we'll evaluate whether this is a use-case we can afford to support or not. My own personal take (so take this as a one person opinion and not the rendering team's final stance) is that it's better for Godot to focus on properly supporting the feature set that it already offers and improve on that, and leave realtime meshing for vector rendering to either plugins or a specialized engine. Godot never set out to be a replacement for flash. I know this comment will displease all of you who would like to have these features, but I think it's important to set the expectations right for everyone participating. SlugFiller has asked for help in figuring out how to decouple the svg rendering from Godot's core rendering loop, so if anyone wants to help with that please do get in touch in rocket chat https://chat.godotengine.org/channel/rendering/ |
Hi @here |
Rive just announced they are working to implement their runtime into Godot. Fingers crossed it gets in sooner rather than later! https://twitter.com/guidorosso/status/1714737152633008500?t=Gf-8igKuoDWwUIXtEIEkHQ&s=19 |
Here's an update on that: |
Describe the project you are working on
Game development using vector art work. I'm a long-time (former) Flash/AIR/ActionScript3 developer, almost exclusively using vector art assets.
Describe the problem or limitation you are having in your project
Although vector art can be imported into Godot, it is rasterized on import and is a bitmap from that point and is not rendered in real time. Real-time vector art rendering has several benefits over bitmap art, which include:
Yes, there are negatives too - the main one being much slower to render than bitmaps, and lack of GPU acceleration. However, many game developers are willing to make this trade-off for the benefits that vector art can provide,
Describe the feature / enhancement and how it helps to overcome the problem or limitation
I propose a new Node2D descendent which supports real-time vector rendering using a TBD vector renderer.
Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams
The mechanism, if this is the way recommended by the core devs, would be basically to use the custom drawing feature of CanvasItem to update the node (draw a new texture) if the vector updates. Ideally a vector graphic asset would be assigned to the node.
If this enhancement will not be used often, can it be worked around with a few lines of script?
The work-around is not to use vector art, which is really not a workaround :)
Is there a reason why this should be core and not an add-on in the asset library?
If available, it should be a core feature and maintained as part of the product.
The text was updated successfully, but these errors were encountered: