Frequently asked questions #12
Replies: 4 comments 5 replies
|
Have you tried other ordered dithering patterns? In particular, interleaved gradient noise. While it's not a self-similar fractal, if you look at this analysis by demofox, then the essential quality of ILG is its low discrepancy when looking at 3 by 3 blocks of pixels, even if those blocks overlap: ... which suggests that instead of subdividing a dot into a 2x2 patch of dots, maybe dividing them into a patch of 3x3 dots could work? |
|
No I haven't tried implementing other ordered dithering patterns, partially because I few patterns are self-similar in the required way, and partially because I'm not interested in doing the work either way. I don't see how IGN can be made self-similar, but if you believe it can be done, and think it would be interesting to try, I invite you to try it out yourself. |
No good deed goes unpunished, huh? Totally understand where you're coming from; to be clear I wasn't making a request, only sharing it in the odd chance that you were not familiar with ILG and whether this was something you might be interested in. Plus the suggestion didn't seem quite covered by the FAQ (the blue noise question came closest). Thank you for replying! |
|
Hi, I am very new to unity, I've installed the correct version to open the project. I can open the example tutorial files, but I have no idea how to begin the tutorial. Sorry to be a bother, could anyone point me to a link? |
|
This repository contains an example project, but not a tutorial. You can open the scene files in the Assets/Scenes/ folder. I recommend following some tutorials elsewhere about the basics of using Unity before looking more into the project here. |
|
Thank you for your time! Your work is truly inspiring! Could your dithering technique bring a sense of depth and value to drone formation sculptures? |
You'd have to ask people who do drone formation sculptures. But I doubt it. |
|
The sample project seems to be utilizing the legacy rendering pipeline. Any plans for Universal Render Pipeline support in the fture? |
|
Sorry, I don’t have plans for any future updates in general. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Please see this FAQ before asking new questions about the technique, and before making suggestions, thanks!
If you haven't yet seen the explainer video, that's a good idea to watch first, as the FAQ mostly doesn't cover all the things already explained there.
Other useful links:
❓What's the point?
Surface-Stable Fractal Dithering is a stylistic effect that can be used for novel aesthetics. It's not an optimization technique, even though the effects it's inspired by were originally implemented for optimization purposes, or to overcome hardware limitations.
What's unique about Surface-Stable Fractal Dithering compared to other dithering approaches is that the dots in the dither patterns stick to surfaces, and yet the dot sizes and spacing remain approximately constant on the screen, even as surfaces move closer by or further away. This is achieved by dynamically adding or removing dots as needed.
❓What exactly is meant by surface-stable?
Here is how I define surface-stable:
Conforming to the second constraint is in particular what is new in Surface-Stable Fractal Dithering.
Approaches that fade between different scales of a pattern, which is not self-similar in the required way, will typically see dots both appearing and disappearing when zooming in, which breaks the second constraint, or the dots shift and move around slightly while zooming in or out, which breaks the first constraint.
❓Could the technique be changed so the dots are circular in screen space and don't become squashed at shallow angles or at non-uniform scales?
There various ways this could be done, but without solving the related issue of dots clumping together, it would have some big downsides.
Surface shader approach: In addition to the dot radial gradients, each pixel in the 3D texture could also store a vector to the center of the nearest dot. This could be stored in the red and blue channels, with the gradient stored in the green channel. This vector would be in texel space, but the UV derivatives could be used to calculate an estimate of the dot center in screen space. Based on this, the surface shader could render a dot which would often be round in screen space. However, this approach can't account for uneven surface curvature, and of course it can't make dots extend beyond the surface geometry the dots come from, so they would still be deformed and clipped under various circumstances.
Fullscreen shader approach: The surface shader could be changed to render dots as single pixels only and a fullscreen shader could expand the pixels into dots of the desired size. This does not have the issues of the surface shader approach.
Fundamental brightness and spacing issue: Both of the approaches above will introduce a problem in pattern brightness. If each dot that was previously squashed is now rendered as a full screen-space disk, the result is a higher proportion of white pixels in the final pattern. This skews the brightness of the dither patterns, and quite heavily at shallow angles and other very anisotropic UV frequencies. This could be worked around by making the dots smaller, but then the dot sizes are no longer constant. It could alternatively be worked around by increasing the spacing between the dots, but at shallow angles this would have a big effect that would leave gaps between the dots (or lines formed by clumped together dots) that are so large that it would completely break the uniformity of the pattern density.
In other words, I don't see any robust solution (meaning without significant downsides) to making the dots circular in screen space without also solving the more complex problem of avoiding making the dots clump together under shallow angles and other anisotropic UV frequencies.
❓The dithering makes it obvious the skybox is fake. Could the dithering be screen-relative for just the skybox?
I think the reason the surface-stable dithering makes the skybox (in the demo video for example) look obviously fake is that the dots everywhere else aligns with surfaces, but on the skybox it doesn't, so the sky and the distant mountains (which are part of the skybox) look like they are part of a single flat surface.
The skybox is not actually a sphere, but a box. Yet it looks like a sphere because I made the dots be aligned in a spherical pattern, as they looked worse if they were aligned with the sides of the box. So it's the dot pattern that makes you think it has a specific shape. The issue could be fixed by using artist-painted UV coordinates for the skybox, where the UV patterns are painted to follow the things depicted in the skybox. If the skybox is created from a 3D scene, this could be automated by rendering out the UV coordinates.
❓Could you use world or object coordinates with triplanar mapping to avoid the issues with non-uniform UVs?
Not really. The biggest issue related to anisotropic UV frequencies is seeing surfaces at shallow angles, and triplanar mapping doesn't help with that. It's true in principle that it could help with non-uniformly scaled objects or UV-mapping in certain cases, but for surfaces at odd angles, patterns based on triplanar mapping are non-uniform too. It also doesn't work at all for animated skinned meshes. For my purposes, it would introduce many more problems than it solves.
❓Could the technique work in color?
Yes, coloring is now supported! See it demonstrated in the demo video, showing color RGB dithering, CMYK halftone, and more.
Surface-stable fractal dithering can be used for coloring, as long as the coloring can be expressed as multiple independent 1-bit dithering patterns (meaning patterns of only two values each), for example one pattern for each color channel. This can for example be used for a look similar to color printing, but in a surface-stable fractal way. The first implementation of this was the one here (with a halftone look) before I added support in the official repo too.
❓Could the technique work with a limited color palette such as 16 colors or 256 colors?
The technique could be made to work with a set number of values per color channels - for example 6 values per channel for a total of 216 colors - since this can be implemented in the form of multiple independent 1-bit dithering patterns. However, it cannot work with an arbitrary palette of hand-picked colors, as this cannot to my knowledge be implemented as multiple independent 1-bit dithering patterns.
The reason the technique can only support independent 1-bit dithering patterns - unlike traditional pixel-based dithering - is that surface-stable fractal dithering has a fundamental difference between the dots and the space between the dots. Due to the fractal nature of the pattern, the dots of a single color/value can appear at any coordinate of the pattern, leaving no room for additional dots of a different color/value.
❓Could the technique be changed to work with other regular patterns based on triangles, hexes, square root 2 ratio rectangles, or similar?
Yes, this should be relatively straightforward to implement for someone who so desires.
❓Could the technique be changed to work with Sierpinski triangles or other fractals?
Any fractal that has big solid parts or big empty parts is unsuitable, as it would not produce a constant dot density. Essentially the dots and shading would entirely disappear when zooming into one of the solid or empty parts. Most classic fractals have such empty or solid parts, including Sierpinski triangles.
The fractal-like properties needed for surface-stable fractal dithering do not look like classic fractals, but rather like more or less uniformly distributed dot patterns that have self-similar properties.
❓Could the technique be changed to work with a blue noise pattern?
Some people have suggested using blue noise for the pattern. But it is not straightforward to construct a blue noise pattern which is self-similar in a way that conforms to the second surface-stable constraint.
If we use only a single tiled square of blue noise pattern, it would require the dots in each quadrant of the pattern to perfectly line up with dots in the full pattern, when scaled up to cover the same area as the full pattern. See this post for more details.
❓Could the technique use different patterns to create a crosshatch or stippling or woodcut look?
It would not be quite the same technique, but yes this is generally possible. See the page about implementations and experiments by others for a bit more info on this.
❓Could the technique basically be implemented as texture mipmaps?
No. In a mipmap, each successive map is lower resolution than the previous. Mipmaps cannot be used to produce fractal-like detail that keeps being crisp at any zoom level.
❓But could you replace your UV frequency code with the function that determines mip levels in mipmaps?
No. The function that determines mip levels is based on UV frequency too, but I need different info from it than mipmaps do, so I can't reuse the function that is used for mipmap purposes.
❓Could the issue with anisotropic UV frequencies be solved with anisotropic texture filtering?
No. Anisotropic just means "not the same in all directions". Two concepts both having "anisotropic" in the name does not imply that one is a solution for the other, or that they share the same solution.
❓ Could this technique be used to optimize ______ (some aspect of computer graphics)?
In all variations I've seen of this question so far, the answer is no, and it probably always will be. Surface-Stable Fractal Dithering is a stylistic effect that can be used for novel aesthetics. It's not an optimization technique, even though the effects it's inspired by were originally implemented for optimization purposes, or to overcome hardware limitations.
If you consider dither use cases where the dithering should ideally be invisible (and they still exist today; for example for semi-transparent effects in some games), then Surface-Stable Fractal Dithering will be inferior to classic dithering methods, since with Surface-Stable Fractal Dithering the dots have to be larger (at least a diameter of two pixels instead of one) to avoid excessive moiré patterns.
❓Does it work in VR?
I've now tested it, and yes it works, with some caveats.
Naturally, dot densities change when moving one's head (or whole body) forward, or in other directions, but this doesn't feel any more weird in VR than on a flat screen. I'd say it feels quite natural, even.
If Radial Compensation is turned off, dot densities also change when just rotating one's head, which is slightly distracting. This is expected, as this is what Radial Compensation is meant to fix. However, when Radial Compensation is turned on, all surfaces shimmer in an unpleasant way due to the dot densities not being quite the same in the left and right eye. To fix this, the Radial Compensation calculation would need to be changed to calculate angles relative to a head forward vector starting from a point in between the two eyes. I won't be pursuing such a fix myself, but someone familiar with how to handle stereoscopic rendering in shaders could probably implement it without too much trouble.
What does feel more weird in VR is the fact that the dot sizes in world space are wildly inconsistent. With stereoscopic vision, we don't have much feeling for how large something is "on the screen", just how large it is in the world. Distant surfaces appear as if they have much larger dots than close surfaces (because they do), to the point that it can kind of feel like inconsistent art styles. In this sense, fractal dithering may not be a very good fit for VR.
Shadows also look more strange in VR. When a surface is not darker in shadow, but just have fewer white dots, that does not convince our instinctive perception that it's a shadow. In 2D mediums, we can perceive the dither shading as separate from the depicted content, but in VR the visual system interprets things more literally, and the dithered shadow feels like a fake shadow painted onto the surface. Maybe it could still work for some VR games, but it definitely feels weirder in VR than on a flat screen.
I tested in PC VR and don't know if the shaders are performant enough for running directly on all-in-one headsets such as Quest 2 or Quest 3.
All reactions