This year I was really excited to have been selected by appleseed – an open source rendering engine designed for animation and visual effects – to participate in Google Summer of Code 2020 (GSoC).
My project focused on extending appleseed's stack of post-processing stages – which previously consisted of only two: a Render Stamp, to add text information, and a Color Map stage for visualizing relative luminance – with a good number of high-quality post-processing effects, aimed towards artists. Post effects are image filters and transforms, frequently used by artists to modify their final renders, making them more realistic or stylized.
I set out to implement four image effects as new post-processing stages – Vignette, Bloom, Tone Map, and Chromatic Aberration – alongside improvements to appleseed.studio, appleseed's standalone GUI lookdev application.
For further details, you can check out my original proposal, as well as my weekly progress reports.
Showcase of the interactive preview of effect changes, inside appleseed.studio
Pior to the beginning of GSoC, I started working on some small issues to get to know the project, and interacting with the community made me excited to work on appleseed. This led to my first notable contribution at the end of March, introducing a new vignette effect (although the implementation was quite straight-forward and not optimized).
| Original Coffe Maker scene | Moderate vignetting | Intense vignetting |
|---|---|---|
%20-%20coffee.png) |
%20-%20coffee.png) |
%20-%20coffee.png) |
Since I had already made some contributions to appleseed, during the Community Bonding period I focused on writing a new abstraction layer for post-processing stages, on top of which I would refactor the vignette effect I had implemented, and also build the new, more complex, effects.
My aim with it was twofold:
-
To make it really simple to separate the actual effect algorithm – i.e. what / how to process the image – from its execution schedule, and from exposing parameters to the user;
-
To seamlessly handle concurrency – for effects that allow it – by leaveraging appleseed's job system to run the effect on smaller portions of the image, in parallel, given the number of available threads.
I successfully finished this at the end of May, and after some really insightful suggestions from François Beaune and Kevin Mason to improve its simplicity, we got it merged in early June.
Thereon, I implemented the remaining effects using this "compression layer" I created, starting with bloom – which was the most challenging one, and required a lot of testing, comparisons between different implementation trade-offs, and profiling (all of which you can read more about on a post I made to the appleseed Users Forum) – and then, moving on to tone mapping and chromatic aberration.
Rendering with (bottom-left) and without (top-right) the bloom effect
Along the way, I also came across some bugs on appleseed.studio – while testing my post-processing effects implementations – and worked on enhancing effects preview to improve the user experience (which was only possible after a rendering finished, adding delays to the artistic process of iteratively testing changes).
These changes, however, are still pending review. Though, once a new release of appleseed ships with these brand-new effects, artists should be able to quickly preview what their scenes would look like in post – without having to leave appleseed.studio – and to easily tweak stage parameters while watching the changes they have live.
🟣 Merged: #2791*, #2785*, #2807*, #2806*, #2855*, #2865
🟢 Open: #2875, #2877, #2880, #2884, #2887
⚫ Draft: #2885
* contributions before the official GSoC start date
Full Pull Requests Timeline (click to expand)
- March*
- [🟣 #2791] Add Google AI's Turbo rainbow colormap
- [🟣 #2785] Tile highlights are now colored
- [🟣 #2807] Add Vignette post-processing stage
- [🟣 #2806] Ignore incandescence color in black-body mode
- April*
- [🟣 #2855] Fix double slider regression
- June
- [🟣 #2865] Refactor the vignette post-processing stage
- [🟢 #2875] Add Bloom post-processing stage
- [🟢 #2877] Fix Shift+F5 causing False Colors to be applied twice
- July
- [🟢 #2880] Fix false colors not being applied to all tiles on a final render
- [🟢 #2884] Add Tone Map post-processing stage
- August
As mentioned, the vignette effect is already merged into master (together with its improved implementation, which takes advantage of multi-threading).
Bloom and two fixes to appleseed.studio I made (#2877, #2880) have already received a first, overall, review.
Unfortunately, development around appleseed has slowed down a bit after the pandemic, so the pull requests for tone mapping and chromatic aberration have not yet started to be reviewed.
Hence, my changes for interactively updating appleseed.studio's viewport as effect parameters are modified – and previewing how the complete stack of post rendering stages make the render look like, when a rendering is paused / stopped – are still in draft, since getting the user experience right first requires users to experience the effects, so it should still undergo changes as other effects get merged into master.
.png)
Modified Spaceship scene, with details showing the original (left) and tone mapped images (right, with decreasing exposure values)
As pretty much everything in software, there are always ways in which things could be improved.
Below I list my three picks for "cherries on the cake" that could follow up on the work that has been done:
One of the main ways we can improve the creative process for artists is to provide an immediate connection with what they are creating.
Therefore, by running the effects on the GPU we can get huge speed boosts, which would enable them to be previewed in real-time on interactive rendering. The simplest way to do this would probably be to convert the effects to GLSL code – which should be straight-forward, given the logical division made to run them in parallel, from the get-go – another thing that might be worth looking into is Halide, a DSL on top of C++ for high-performance image processing.
Tone mapping is an often requested effect – since physically-based renderings work in High Dynamic Range (HDR), but most monitors can only display RGB values on the [0, 255] range, so we need to map the HDR color values into this Low Dynamic Range (LDR).
Here are some ways of making it even more useful inside appleseed.studio, decreasing the need for third-party DCC tools:
- Add tone curves for real cameras, as well as "user-defined looks", with LUTs
- Include a plotting widget in appleseed.studio's
Attribute Editorfor visualizing the tone mapping operator curve, and previewing changes to it - Let the user choose between tone mapping:
- Only luminance values (available for Reinhard curves)
- Each RGB channel separately (default for all other operators)
- Operating with
max(R, G, B)oraverage(R, G, B)values
Tone mapping operators comparison (on The Breakfast Room scene)
Keeping with the theme of fluidity and ease of use, here are some ideas to improve the user experience when experimenting with post effects in appleseed.studio:
- Allow previewing effects in downsized scales (for faster processing and greater responsiveness)
- Choose to toggle multiple post-processing stages on/off when a rendering isn't running
- Expose the option of saving (unfinished) renderings with the previewed effects
- Store effects in a separate layer, for non-destructive editing
.png)
appleseed.studio's Attribute Editor, used for changing post-processing stage parameters
I learned a lot during GSoC, not only technically, but also in the ways of approaching challenging problems, communicating ideas and deciding when to shift focus. I am super thrilled to see everything I worked on for the past few months be included in a future release, and hope to continue contributing to appleseed.
I would like to say a huge thank you to the community around appleseed, as there are so many knowledgeable people who have always been super kind, and helped me a lot.
Lastly, a special thank you to my mentor, Kevin Masson, for the many advices, and for pushing me to not be afraid to do the hard things first. This is something I will take with me going forward.