New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nannou v0.9 - Vulkan graphics, simpler event API and much more #240

Open
wants to merge 51 commits into
base: master
from

Conversation

Projects
3 participants
@mitchmindtree
Copy link
Member

mitchmindtree commented Jan 2, 2019

This is easily the biggest update for nannou so far!

OpenGL -> Vulkan

The switch from OpenGL to Vulkan for cross-platform graphics is the main story here. You can find more details at #208 and #216, but the primary reasons for switching to Vulkan are:

  • Gaining support for compiling and running languages other than GLSL (e.g. RLSL, HLSL) on the GPU via Vulkan's low-level SPIR-V representation.
  • Access to finer-grained control over the GPU for more thorough optimisation if necessary.
  • No longer having to deal with OpenGL bugs resulting from inconsistent implementations across different devices and versions.
  • Switching to a more modern, future-proof, cross-platform graphics standard.

Both High-level and Low-level Access

While the existing high-level graphics APIs should behave almost exactly the same (apart from smoother graphics and slightly better performance), this PR also allows for much lower level access to the graphics stack than was previously available. In other words, nannou now allows for fully custom graphics pipeline creation, access to each window's unique swapchain, custom vulkan instance creation, custom device selection and loads more. To demonstrate this low level access, @JoshuaBatty and I have started a new series of vk_*.rs examples in the examples/vulkan/ directory. These demonstrate how to work with basic geometry, images, 3D, first-person cameras, compute shaders and more. We'll continue adding to these over time - requests are very welcome!

GLSL

Note that GLSL is still well supported so you can still reach into your existing bag of shader tricks! Note however that currently only the most recent versions of GLSL are supported, so you may need to update older versions. Funnily enough, this might make it easier to run GLSL 450 code on macOS as I believe support for more recent versions of OpenGL and GLSL from Apple have been lacking.

MacOS

For folks wondering about how macOS is supported, nannou uses vulkano under the hood which links to MoltenVK on macOS for translating Vulkan to Apple's Metal API. We're hoping that by the time this is merged, @freesig's work on automating the MoltenVK installation process at build time will have landed. In theory this means that you should be able to run cargo build like normal and everything should Just Work™ (you might have to restart your terminal), though don't be surprised if there are hiccups as this is all still very new.

Tradeoffs

All that said, this change will mean that nannou may no longer be compatible on some older systems that don't yet have Vulkan support. Nannou aims to foster the next generation of creative coding and with our small team's limited capacity this unfortunately means we must draw a line in the sand at some point. If you're curious about the level of support for Vulkan on your system's GPU you can search for it here to find detailed reports. If you're not sure what GPU you have or it's not listed in that database, the site will allow you to download the tool for generating your own report.

If nannou was working for you in 0.8 but stopped in 0.9 we would love for you to leave an issue and let us know. We'll do our best to address each issue, though keep in mind that our ability will be limited in cases where Vulkan driver support is lacking.

Simpler Event API

One glaring issue with nannou's current event API is the way that user's are expected to handle events via the event function. While the event type is thorough and allows for accessing loads of hardware, application and window events, handling the Event type itself could be quite verbose and confronting for new users who might be unfamiliar with the fancy pattern matching required.

This update allows for using functions to handle events in a way that will be much more familiar to users coming from Processing, OF and Cinder and reduces the amount of language-learning users have to go through before they can start being creative.

For a closer look at these changes check out #221 and in particular the changes to the examples in the diff. The new all_functions.rs example demonstrates all of the new event functions that may be registered with apps and windows in one place.


For details on the other small tweaks, patches, fixes and features check out the CHANGELOG diff and the commit messages below.

Merging is currently pending:

mitchmindtree added some commits Nov 29, 2018

[WIP] Switch from OpenGL to Vulkan
This switches the windowed-graphics API from OpenGL (via glium) to
Vulkan (via vulkano). You can read more about the motivations, thoughts
and discussion behind this switch at #208 and #108.

There are still a few items left to complete before this PR is ready:

- [ ] Fix bug where view seems to be vertically flipped (probably in
  vertex mapping process).
- [ ] Fix bug where clearing an image that is then multisampled does not
  work. vulkano-rs/vulkano#1123
- [ ] Add depth attachment to draw renderpass.
- [ ] Fix bug where alpha channels always seem opaque.
- [ ] Update old loop modes for changes in the application loop.
- [ ] Merge conrod vulkan backend and switch to it.
- [ ] Merge and publish vulkano-rs/vulkano#1117 or related fix so we can
  switch to crates.io dep.

Closes #208.
Closes #108.
Update examples for new app builder methods.
Also removes the use of `Frame::clear_all` in favour of `Frame::clear`
as a `Frame` now only represents the frame for a single window.
Update vertex transform for change from GL to vulkan coordinates
In Vulkan, the *y* axis increases in the downwards direction rather than
the upwards direction like in OpenGL.
Add new `gpu` module. Add custom Vulkan Instance and debug support.
The new `gpu` module contains items related to interfacing with one or
more GPUs via Vulkan.

The `App` builder now allows for specifying a custom Vulkan instance and
also allows for easily setting up a debug callback for receiving
messages from validation layers..
Add depth attachment to `draw` module for supporting 3D
Still need to:

- Allow for custom depth formats
- Validate depth format
Update `Wait` and `Rate` loop modes for switch to Vulkan
This involved refactoring the `RefreshSync` run loop into a set of
modular functions for easier re-use between loop modes.

Note that while each loop mode is now supported, the `Wait` and `Rate`
present modes are not yet optimal. If using the `Wait` or `Rate` we
should attempt to use the `Mailbox` present mode with triple-buffering
in order to avoid blocking when acquiring an image. This is different to
the `RefreshSync` mode as the `RefreshSync` mode is *driven* by the rate
at which swapchain images can be retrieved for each window.
Allow Swapchain customisation. Recreate Swapchain on LoopMode change.
This allows for specifying a set of custom swapchain parameters during
the window building process.

The swapchain will now be recreated each time the `LoopMode` is changed
at runtime. It will attempt to use optimal `PresentMode` and image count
for the new swapchain unless the user has chosen a specific value for
either of these parameters.

The `loop_mode` example has been updated with some short docs along with
the addition of the new loop mode. The window title now displays the
current `LoopMode`.
@JoshuaBatty

This comment has been minimized.

Copy link
Collaborator

JoshuaBatty commented on 5211c3a Dec 3, 2018

Epic work dude!

mitchmindtree and others added some commits Dec 4, 2018

Integrate new conrod_vulkano backend into `ui` module.
The `simple_ui.rs` example works!

There is an issue on my machine where, once I added MSAA, the `Ui`
drawing now clears over the top of the existing window contents. I think
this is just another case of vulkano-rs/vulkano#1123 and probably only
affects linux machines running the intel mesa drivers.
Share render pass images between framebuffers in `draw` Renderer.
Previously a unique set of images was created for each framebuffer
unnecessarily. Now, a single set of `RenderPassImages` is created only
when dimensions have changed and these are shared between each of the
swapchain image framebuffers.
Switch render pass based on draw Renderer background value
This should allow the `draw` renderer to draw on top of existing
graphics in the case that a background was not specified.
Tweak and rustfmt examples ready for vulkan branch
Switches vulkano to freesigs branch that includes two patches.
Switches vulkano-shaders to the master vulkano branch for `#include`
support.
Merge pull request #216 from mitchmindtree/vulkan
[WIP] Switch from OpenGL to Vulkan
Add window-specific view and event user function support
An `all_functions.rs` example has been added that shows all functions
that can be registered with the app and model.

All other examples have been updated to take advantage of the new event
functions.

Functions that update the model, including the existing App `event`
function, now provide the model via `&mut` rather than by value to be
returned.
Merge pull request #221 from mitchmindtree/user_functions
Add window-specific view and event user function support. Add app update user function.
Merge pull request #222 from mitchmindtree/vk_examples
Add simple vulkan debug example

mitchmindtree added some commits Dec 14, 2018

Explicitly set git vulkano deps for receiving nannou patches downstream
Still need a [replace] so that vulkano-win and vulkano-shaders versions
of vulkano also uses the git version so that we don't get type and trait
mismatches.
Merge pull request #226 from mitchmindtree/v0.9
Bump version in Cargo.toml to 0.9. Explicitly set vulkano git deps for receiving nannou patches downstream.
Add `vk_teapot.rs` example
This should act as a nice simple example for getting started with 3D,
and will also be a useful reference for testing the 3D component of the
`draw` API.

I'm thinking it might be nice to follow this up with a
`vk_teapot_camera.rs` example which is basically the exact same example
but allows you to move around a virtual camera using WASD and orient it
using the mouse.

Another example I'd like to make is a `vk_teapot_matrices.rs` example
which is the exact same but with a GUI overlay that allows you to tweak
the world, view and projection matrices directly in order to gain an
intuition for how they work.
Merge pull request #228 from mitchmindtree/vk_teapot
Add `vk_teapot.rs` example
Merge pull request #229 from mitchmindtree/v0.9
Add `Window::grab_cursor` and `Window::hide_cursor` methods
Add `vk_teapot_camera.rs` example
The same as the `vk_teapot.rs` example but demonstrates how to create
and use a first person camera.

The W, A, S, D, Q and E keys move the camera.
The mouse orients the pitch and yaw of the camera.
The Spacebar key toggles camera interaction.
Merge pull request #230 from mitchmindtree/vk_teapot_camera
Add `vk_teapot_camera.rs` example
Create a `SwapchainFramebuffers` type
This significantly simplifies the process of creating and maintaining
swapchain image framebuffers, currently probably the most tedious part
of using raw vulkan with nannou. See the docs for the new
`window::SwapchainFramebuffers` type and its `update` method for more
details on how it works under the hood.

This significantly simplifies the `ui` and `draw` backends, as well as
the raw vulkan examples as a result. The easiest way to see the impact
of this PR is probably to check the diff for the vk examples.

cc @JoshuaBatty should be a good start on this simplification process!
Merge pull request #231 from mitchmindtree/swapchain_framebuffers
Create a `SwapchainFramebuffers` type
Merge pull request #232 from mitchmindtree/v0.9
Remove glium from Cargo.toml
Merge pull request #235 from mitchmindtree/v0.9
Fix ordering of Rect corners for triangulation
Merge pull request #237 from mitchmindtree/v0.9
Switch conrod dependency from git to crates.io v0.62
Change `RefreshSync` loop mode to wait on previous frame presentation
This should fix the issue where the GPU would lag behind the CPU on
macos.

For **v0.9**.
Merge pull request #239 from mitchmindtree/future_wait
Change `RefreshSync` loop mode to wait on previous frame presentation

@mitchmindtree mitchmindtree added this to In progress in v0.9 via automation Jan 2, 2019

@freesig

This comment has been minimized.

Copy link
Collaborator

freesig commented Jan 2, 2019

Awesome stuff!!
I might suggest waiting for the next release of the lunarg SDK because otherwise we might have to distribute a build of MoltenVK for this to actually work. Looks like they usually release monthly and we are already over due for one. Hopefully there's a release while we are waiting for the moltenvk_deps to merge.

mitchmindtree added some commits Jan 26, 2019

Recreate window swapchains if user model requires loop change
Previously this check was missing, so if a user changed the loop mode
after they created a window, the swapchain would not always match
accordingly until the loop mode changed again. Now, we check for a
change to the loop mode after the user model function is called and
recreate window swapchains if necessary.
Merge pull request #249 from mitchmindtree/model_loop_mode_check
Recreate window swapchains if user model requires loop mode change
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment