Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Multipass shaders (buffer A ...) #30

Open
wants to merge 35 commits into
base: main
Choose a base branch
from
Open

Conversation

Vipitis
Copy link
Collaborator

@Vipitis Vipitis commented May 9, 2024

part of #4

approximately 17.5% of public Shadertoys are multipass. Multipass allows up to 4 buffers (A through D) to be rendered as a texture. These can also be used to store data and enable quite some more experiences.

Some of the challenges include timing as well as cross inputs.
Buffer passes can seemingly take the exact same inputs as the main "Image" renderpass, including other buffers (and themselves?)

This PR starts to bloat a little and contains some refactor for the whole channel input concept... still in flux

Instead, will try to implement BufferTexture as a ShadertoyChannel subclass so it can hold for example the sampler settings.
Additionally, there will likely be a RenderPass base class and subclasses for Image, Buffer(a-d) and later cube and sound.
So the main Shadertoy class contains several render passes, and all of these get their inputs(channels) attached.
I even started to try and sketch it out - but will have to sleep through this for a few more days... my concepts change every day but I need to just try and work on the ideas for a bit.
image

The render order should be Buffer A through D and then Image. So you can keep temporal data, by using itself has an input.

TODOs:

  • Refactor common code to ShadertoyChannel base class
  • additional tests cases for inferred input types, empty channels
  • dynamic headers
  • test coverage for examples in readme!
  • Working buffer
  • working multiple buffers!
  • tests for buffers
  • multipass examples
  • (maybe) some debug mode where you can render the buffers to canvas? (you can use RenderDoc with "capture child processes")

@Vipitis
Copy link
Collaborator Author

Vipitis commented May 24, 2024

Little update:
It seems to sorta work. But a bunch of stuff is still broken:

  • missing sampler filters fail to recreate some behavior, example: https://www.shadertoy.com/view/MfyXzV
  • having unsupported channels break the layout, example: https://www.shadertoy.com/view/ssjyWc (layout fixed by detecting channels in the common code, shader still broken, fixed in float32)
  • resizing the window breaks the nbytes of the previous frame - so maybe there needs to be a hook to the on_resize function to fill the new space (similar to behaviour when going fullscreen on the website)

new breaking examples found, that might be unrelated to this PR, but I will note them down for later reference:

@Vipitis
Copy link
Collaborator Author

Vipitis commented Jun 7, 2024

I think I finally fixed the compatibility issue. There is some small visual issues which look like precision problems to me (not sure yet). And the performance is horrible it seems... Please let me know if you find any shaders that are broken (not due to missing features, wgpu bugs)

Will work on tests, examples and documentation to hopefully get this ready for next week.

E: found this one seemingly broken: >wgpu-shadertoy https://www.shadertoy.com/view/tsKXR3
detailed example of precision of this alpha channel is different: https://www.shadertoy.com/view/wsjSDt

@Vipitis Vipitis marked this pull request as ready for review June 21, 2024 19:13
@Vipitis
Copy link
Collaborator Author

Vipitis commented Jun 21, 2024

I think this is finally ready for review - and I welcome some feedback.

  • This PR refactors the whole inputs/channels to be easily extendable with the missing channel types.
  • Multipass shaders are quite complex, but I learned quite a lot to get this running, but I think my implementation is what is minimally required for wgpu (with the redoing the sampler and pipeline).
  • resizing is really janky, since you download the texture and pad it with numpy only to upload it again - but I wanted to mirror the website (this includes breaking quite a few shaders)

@Vipitis Vipitis changed the title [WIP] Support Multipass shaders (buffer A ...) Support Multipass shaders (buffer A ...) Jun 24, 2024
@property
def renderpass(self): # -> BufferRenderPass:
if self._renderpass is None:
self._renderpass = self.parent.main.buffers[self.buffer_idx]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this causes issues when the channel buffer class references a buffer_idx that has not been given an actual BufferPass for the main class. This can happen via the API (example: https://www.shadertoy.com/view/4XcXWM) too.
non existing Buffers should probably sample the black texture, but we can't replace this class in place - need to detect this earlier

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant