Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spir-V pipeline improvements #7659

Merged
merged 4 commits into from May 16, 2023
Merged

Spir-V pipeline improvements #7659

merged 4 commits into from May 16, 2023

Conversation

Jhonnyg
Copy link
Contributor

@Jhonnyg Jhonnyg commented May 15, 2023 •
edited

Improved the SPIR-V shader pipeline:

  • Better error feedback when shader inputs and outputs mismatch during the build process
  • Removed redundancy when producing shader resource bindings since we now run optimisation passes on the SPIR-V source
  • Output name hashes for shader bindings instead of doing it in runtime

Fixes #5674

PR checklist

  • Code
    • Add engine and/or editor unit tests.
    • New and changed code follows the overall code style of existing code
    • Add comments where needed
  • Documentation
    • Make sure that API documentation is updated in code comments
    • Make sure that manuals are updated (in github.com/defold/doc)
  • Prepare pull request and affected issue for automatic release notes generator
    • Pull request - Write a message that explains what this pull request does. What was the problem? How was it solved? What are the changes to APIs or the new APIs introduced? This message will be used in the generated release notes. Make sure it is well written and understandable for a user of Defold.
    • Pull request - Write a pull request title that in a sentence summarises what the pull request does. Do not include "Issue-1234 ..." in the title. This text will be used in the generated release notes.
    • Pull request - Link the pull request to the issue(s) it is closing. Use on of the approved closing keywords.
    • Affected issue - Assign the issue to a project. Do not assign the pull request to a project if there is an issue which the pull request closes.
    • Affected issue - Assign the "breaking change" label to the issue if introducing a breaking change.
    • Affected issue - Assign the "skip release notes" is the issue should not be included in the generated release notes.

@Jhonnyg Jhonnyg added bug Something is not working as expected engine Issues related to the Defold engine vulkan labels May 16, 2023
@Jhonnyg Jhonnyg requested a review from JCash May 16, 2023 07:24
Jhonnyg
Jhonnyg commented May 16, 2023
View reviewed changes
com.dynamo.cr/com.dynamo.cr.bob/src/com/dynamo/bob/pipeline/ShaderCompilerHelpers.java
@@ -290,7 +291,6 @@ static public SPIRVCompileResult compileGLSLToSPIRV(String shaderSource, ES2ToES
// Set (1) |- Binding(0) |_ U4
// |_ U5
//
HashMap<Integer,SetEntry> setBindingMap = new HashMap<Integer,SetEntry>();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not needed any longer since we run the optimization pass. (I should remove the comment above as well)

Jhonnyg
Jhonnyg commented May 16, 2023
View reviewed changes
engine/gamesys/src/gamesys/resources/res_material.cpp
Comment on lines +213 to +218
if (!material)
{
dmResource::Release(params.m_Factory, (void*)resources.m_VertexProgram);
dmResource::Release(params.m_Factory, (void*)resources.m_FragmentProgram);
return dmResource::RESULT_DDF_ERROR;
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoids a hard crash when the program can't be created

Jhonnyg
Jhonnyg commented May 16, 2023
View reviewed changes
engine/graphics/proto/graphics/graphics_ddf.proto
Comment on lines +199 to +213
required uint64 name_hash = 2;
required ShaderDataType type = 3;
optional uint32 element_count = 4 [default=1];
optional uint32 set = 5 [default=0];
optional uint32 binding = 6 [default=0];
}

message Shader
{
required Language language = 1;
optional bytes source = 2;
repeated ResourceBinding uniforms = 4;
repeated ResourceBinding attributes = 5;
optional bool variant_texture_array = 6 [default = false];
repeated ResourceBinding inputs = 5;
repeated ResourceBinding outputs = 6;
optional bool variant_texture_array = 7 [default = false];
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the order a bit here, but it's build/engine only so should be fine. Editor uses bob for building so no changes should be needed for editor either

JCash
JCash previously approved these changes May 16, 2023
View reviewed changes
Copy link
Contributor

@JCash JCash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.
Although if possible, I'd like to move the input/output check to the pipeline.

engine/graphics/src/vulkan/graphics_vulkan.cpp Outdated
Comment on lines 2150 to 2162
if (ddf->m_Outputs.m_Count > 0)
{
shader->m_Outputs = new ShaderResourceBinding[ddf->m_Outputs.m_Count];
shader->m_OutputCount = ddf->m_Outputs.m_Count;

for (uint32_t i=0; i < ddf->m_Outputs.m_Count; i++)
{
ShaderResourceBinding& res = shader->m_Outputs[i];
res.m_Binding = ddf->m_Outputs[i].m_Binding;
res.m_Set = ddf->m_Outputs[i].m_Set;
res.m_Type = ddf->m_Outputs[i].m_Type;
res.m_NameHash = ddf->m_Outputs[i].m_NameHash;
res.m_Name = strdup(ddf->m_Outputs[i].m_Name);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it's worth it but these two segments could be put into a function. it's an edge case I think.

engine/graphics/src/vulkan/graphics_vulkan.cpp Outdated
Comment on lines 2246 to 2250
if (!input_found)
{
dmLogError("Input of fragment shader '%s' not written by vertex shader", fragment_module->m_Inputs[i].m_Name);
return false;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this check should be in the shader build pipeline?

At runtime we don't change the vertex/fragment programs, only the whole material. So this situation shouldn't come up at runtime?

engine/graphics/src/vulkan/graphics_vulkan.cpp Outdated
@@ -2250,11 +2278,11 @@ namespace dmGraphics
HashState64 program_hash;
dmHashInit64(&program_hash, false);

if (vertex_module->m_AttributeCount > 0)
if (vertex_module->m_InputCount > 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if-statement is redundant. The for-loop does the job.

@Jhonnyg Jhonnyg merged commit 73dd2b4 into dev May 16, 2023
19 checks passed
@Jhonnyg Jhonnyg deleted the spirv-shader-pipeline-improvements branch May 16, 2023 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JCash JCash JCash left review comments

Assignees
No one assigned
Labels
bug Something is not working as expected engine Issues related to the Defold engine vulkan
Projects
No open projects
1.4.7
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Vulkan] VK_ERROR_INITIALIZATION_FAILED
2 participants
@Jhonnyg @JCash