Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bootstrapping Vulkan support #234

Merged
merged 85 commits into from
Nov 29, 2020
Merged

Bootstrapping Vulkan support #234

merged 85 commits into from
Nov 29, 2020

Conversation

mosra
Copy link
Owner

@mosra mosra commented Mar 2, 2018

This currently depends on #233, so I'm setting base branch to that instead of master so the changes are visible clearer. I'm also working on both at the same time. #233 is merged now.

Things to do:

  • Initial library bootstrap (WITH_VK)
  • Find module updates
    • Require CMake 3.7 not needed
    • Require C++14 for dependencies of this lib rather not, unless really needed by some feature
  • TARGET_VK variable
  • @fn_vk etc Doxygen aliases, update Coding Style for them
  • Header generation using flextGL
    • some types are parsed with trailing whitespace, remove that
    • the global enums have a different order every time they are generated
    • the vk.xml file is not sorted in type dependency order, submit patch upstream (or hotfix in flextGL?)
    • don't export the flextVkInitInstance() / flextVkInitDevice() function (done just because there was no Instance/Device wrapper yet)
    • avoid global function pointers, fill a struct with them instead
    • add another flextVkInitDevice() for populating device-dependent pointers into another struct
    • clean up and submit the patches upstream (Vulkan support flextgl#15)
    • update the Developers Guide with generating Vulkan headers
  • ShaderTools library -- 13dfb64
    • New AbstractShaderConverter plugin interface
    • Integrate glslang for GLSL validation and GLSL->SPIR-V compilation -- mosra/magnum-plugins@66a03a4
    • Integrate SPIRV-Tools (dis)assembler -- mosra/magnum-plugins@6b0680f
    • Integrate SPIRV-Tools linker for putting multiple stages together in a single file (heh, i thought glslang/shaderc did that, but no) postponed
    • Integrate SPIRV-Tools optimizer for SPIR-V optimization
    • Integrate SPIRV-Tools validator -- mosra/magnum-plugins@308df5b
    • others? glsl-optimizer, shaderc, glsl-validator(?), DirectXShaderCompiler) postponed
    • GLSL minifier (whitespace and comment removal, at least), this could be used in Magnum itself to help trimming down the size of the Shaders library postponed
    • Introspection for SPIR-V (querying attribute, uniform, ... names and bindings of an imported shader bytecode)
    • Specialization constants, specialization constant folding (because apparently even for a disabled branch all uniforms and inputs need to exist, sigh)
  • Initial Instance wrapper
  • Device property queries
    • Ability to disable even device extensions that don't need enabling (such as VK_KHR_driver_properties so we can the codepath for drivers that don't have it) postponed
  • Initial Device wrapper
  • Device selection using --magnum-device
  • Enabling validation layers etc using --magnum-gpu-validation postponed
  • Queue setup
    • Convenient queue retrieval using "out" params in addQueues()
  • Command pool creation, command buffer allocation
    • Command buffer submit, reset postponed
  • Memory allocation, mapping,
    • Flushing postponed
    • Usage docs
    • Dedicated strong type for "memory type bits" as it's otherwise too easy to mix up with size when allocating postponed
    • Support KHR_dedicated_allocation since it makes sense for large data and render targets (and we don't have anything else now anyway) postponed
    • Generic API for hooking up non-trivial memory allocators like the AMD one postponed, needs a whole new abstraction on top
  • Image wrapper
    • Usage docs
    • (implicitly convertible) image view wrapper, ideally as simple as GL texture / framebuffer wrappers for the common case postponed
    • "Preset" tags for linear mappable / framebuffer images? postponed
  • Buffer wrapper
    • Usage docs
    • Buffer view wrapper postponed
  • Pipeline + shader setup postponed
  • Vulkan-enabled alternative of the GL MeshTools::compile() postponed
  • KHR_surface for Linux, Windows, Android and macOS postponed
  • Initial magnum-vk-info utility
  • Initial Vulkan function / limit mapping page
  • Initial Vulkan extension support page
  • Initial Vulkan wrapping docs, focusing on interoperability with raw code
  • Vulkan-specific projection matrix creation. Would need to be moved out from Math::Matrix[34]::*Projection() to some API-specific header. postponed
  • Initial base class for Vk tests, BUILD_VK_TESTS option
    • Decide what to set up in the Vulkan tester and what to leave up to the tests (will probably become clearer once we implement a few)
    • Setting up debug layers? Leak detection? later maybe
    • Setting up queues? later maybe
    • Ability to override the chosen device on command-line? -- yes, already done
  • Test w/ SwiftShader on the CIs
    • Make a Windows, Mac and Linux build in magnum-ci
    • Make it working on Windows as well -- currently it fails with VK_ERROR_INITIALIZATION_FAILED (does it require some stuff being added to registry? ugh)
  • make our own FindVulkan.cmake that finds just the library but correctly
    • Make it work with MoltenVK (libMoltenVk instead of libvulkan)
    • Make it work on MinGW (to avoid -DVulkan_LIBRARY=${MINGW_PREFIX}/lib/libvulkan.dll.a .. maybe it just needs to explicitly list libvulkan in addition to vulkan?
  • Building on Travis CI against a minimal stub (can't be bothered to build some random driver there just yet)
    • Why is it looking for OpenGL even though everything else is disabled? -- not anymore since 89d9883
  • Building on AppVeyor the same way -- d578d4d
  • Integrating an initial Vulkan Triangle example -- mosra/magnum-examples@41c78ef
    • Gradually rewrite it using the new, less error-prone and less verbose wrappers postponed
  • Basic translation of Magnum enums to raw Vulkan enums (will get deprecated/removed later once we have our own types there) -- b0a1719
  • Conversion of basic Vulkan types into equivalent Magnum types (vectors, ranges, colors...) -- 3281a27

Cc: @Squareys

@mosra mosra added this to the 2018.0b milestone Mar 2, 2018
@mosra mosra self-assigned this Mar 2, 2018
@mosra mosra added this to TODO in Vulkan via automation Mar 2, 2018
@mosra mosra moved this from TODO to In Progress in Vulkan Mar 2, 2018
@mosra mosra force-pushed the vulkan branch 3 times, most recently from a8d67a5 to 88cbdf6 Compare March 2, 2018 23:01
@mosra mosra mentioned this pull request Mar 2, 2018
27 tasks
@mosra mosra force-pushed the split branch 2 times, most recently from c792bc9 to e425f37 Compare March 6, 2018 23:12
@mosra mosra force-pushed the split branch 2 times, most recently from 3917fe9 to 3c3105b Compare March 28, 2018 19:55
@mosra mosra force-pushed the split branch 13 times, most recently from acbbe8b to adef5d9 Compare April 21, 2018 20:10
@mosra mosra force-pushed the split branch 5 times, most recently from be06426 to aaf39a5 Compare April 28, 2018 22:43
This avoids allocating a potentially large array in case just the first
device is needed. Originally I did this in a hope to avoid a stall + CPU
power management issues due to some bad shit in the AMD driver, but it
seems that enumerating even just one device still makes it stall. Sigh.
Today I spent six hours wrongly convincing myself that it's a driver bug
when vkGetPhysicalDeviceProperties2() is null on a 1.1 instance for a
1.0 physical device. It's not a bug, it's me not reading specs
carefully.

This commit thus basically moves all Instance-level extension-dependent
state to DeviceProperties, because it's actually device-dependent. Which
makes the DeviceProperties class quite heavy and thus it's good it was
readied to be transferred all the way to a Device instance a few commits
back -- I don't really want to do all the dispatch, string processing,
sorting and other mess more times than strictly necessary.

In addition, DeviceProperties::apiVersion() got renamed to version() and
a new isVersionSupported() API got added, mirroring what's on Device
itself; plus thanks to the chicken-and-egg problem of having to call
vkGetPhysicalDeviceProperties() twice, the device version and other
things can now be retrieved in a slightly more efficient way.
For some reason it wants me to allocate 16 bytes more. Why can't that be
stored somewhere else, I wonder?

Hm, and for this I implemented VK_KHR_driver_properties only to discover
that the info is not queryable if we run the tests with
KHR_get_physical_device_properties2 disabled. Sigh.
Will be used for mapping and in the future possibly for memory slices /
views.
More convenient to use since one doesn't have to explicitly store a
DeviceProperties instance to call pickQueueFamily() on it and then move
it to DeviceCreateInfo to keep it efficient.

This required a slight redesign of how DeviceProperties are stored in
DeviceCreateInfo, now we need the instance to be always valid (but it
can get never used). The wrap() API isn't doing any extra work so this
won't add any inefficiency.
Nope, it's 2020 and there's still no way to link to &-qualified
functions. Feature that is in C++ since 2011.
Makes it possible to write Vk::Instance instance{{argc, argv}} which is
a good tradeoff between passing no arguments at all and doing the fully
verbose thing.
Basically the same as with Image, but easier.
Again basically the same as with Image.
This was easy ... except for the fun with taking ownership of the
binary. And it led to me inventing CORRADE_INTERNAL_ASSERT_EXPRESSION().
Thus it has no place in the general overview docs of Vk::Instance and
Vk::Device. Better to put it into the VUlkan wrapping docs, where it
also makes sense to have both the device and instance side together.
Realized usefulness of such an utility while writing a documentation
snippet.
It's a new extension in Vulkan 1.2.
In Vulkan there's no mess like with GL/GLES/WebGL. Well, yet.
@mosra mosra merged commit 518abbb into master Nov 29, 2020
Vulkan automation moved this from In Progress to Done Nov 29, 2020
@mosra
Copy link
Owner Author

mosra commented Nov 29, 2020

Now that the initial design got pretty stable, the overhead from branch switching and rebasing overweighted the advantages of being able to experiment, so I'm merging. Future development will happen directly on the master branch.

@mosra mosra deleted the vulkan branch November 29, 2020 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Vulkan
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

3 participants