Adds Metal Shader Generation and Rendering Support #1258

Morteeza · 2023-02-24T19:09:45Z

Adds Metal Support to MaterialXShaderGen Backend
Adds Metal Support to MaterialX Rendering code for testing generated shaders
Adds Metal Support to MaterialXView Rendering Support (switchable between Metal and OpenGL, default is Metal)

Updating the repo

linux-foundation-easycla · 2023-02-24T19:09:50Z

The committers listed above are authorized under a signed CLA.

✅ login: Morteeza / name: Morteza Mostajab (e82661e, d78b4b3, 349f7ee)

Adds Metal Support to MaterialX Rendering code for testing generated shaders Adds Metal Support to MaterialXView Rendering Support (switchable between Metal and OpenGL, default is Metal)

kwokcb

Hi @Morteeza , this is a lot of work done here so congrats on getting this done!
I've left some minor points, and future points. For the PR itself, I think it would be good to address:

Target inheritance. Seems that the approach is to use GLSL code snippets as much as possible and tweaking them during code generation. Just making it inherit in the target definition is a small start.
I assume at some point we'd want to check the Python wrappers for this. One reason I am curious to expose this is I think you can use the metal command line to perform compile line compilation testing and we have a python script called generateShader.py which could add in the Metal generator to test with a supplied command line validator.
Along the lines of validation, I'm not sure how you perform unit tests for compilation? (I think I'm just missing something). We could keep what's there are allow an external program to perform validation as is done for OSL. I raise this in case this is seen as a better option ?
There are some comments about the derivation of interfaces, and I only want to bring up if we want to have
- a "pipeline" interface to extract all backend specifics as opposed to splitting the Viewer class directly.
- a "texture baker" based interface as there just seems to so much duplicate code there.

Just some "2-cent" stuff to think over :).

libraries/stdlib/genmsl/lib/mx_hsv.metal

libraries/stdlib/genmsl/stdlib_genmsl_impl.mtlx

libraries/stdlib/genmsl/stdlib_genmsl_cm_impl.mtlx

libraries/targets/genmsl.mtlx

resources/Lights/envmap_shader.mtlx

source/MaterialXRenderMsl/TextureBaker.h

source/MaterialXRenderMsl/TextureBaker.mm

source/MaterialXTest/MaterialXGenMsl/CompileMSL.mm

source/MaterialXTest/MaterialXRenderMsl/RenderMsl.mm

source/MaterialXView/ViewerMSL.mm

Refactorings to address comments on the review (TextureBaker, MetalTexture class, Math-Scalar Operators, MSLProgram, ...)

# Conflicts: # source/MaterialXRenderGlsl/TextureBaker.cpp

…naming

It causes linker errors for Visual Studio, but in any case exporting template classes is not helpful.

Morteeza · 2023-03-07T19:45:07Z

@kwokcb @jstone-lucasfilm This is second revision of the change. I appreciate it if you folks review it and provide me more comments.

Also, I would like to add MSL python bindings, shader graph editor, and render pipeline (viewer refactoring) as a future change.

Looking forward to your comments... and thanks in advance

Morteeza · 2023-03-08T15:33:50Z

Now, we have python bindings for Metal Shading Language generator. We can also use xcrun metal --language=metal as validator to compile and validate generated shaders.

Signed-off-by: Jonathan Stone <jstone@lucasfilm.com>

source/MaterialXRenderGlsl/GlslMaterial.h

source/MaterialXTest/MaterialXGenMsl/CompileMSL.h

…ew abstraction call RenderPipeline that takes care of rendering. This change is done with goal of reducing code duplication.

…ing MSL to Msl.

… handling

… with other parts of code Correct file inclusion typo

Morteeza · 2023-03-13T15:50:33Z

@kwokcb @jstone-lucasfilm If you review MaterialXView project files again, you can see that we don't have ViewerCommon, ViewerGL, and ViewerMetal files anymore; but instead I moved the code for rendering to MetalRenderPipeline and GLRenderPipeline class. Please let me know how is that change? We can revert it back or improve it.

Signed-off-by: Jonathan Stone <jstone@lucasfilm.com>

kwokcb

Looks great to me! Thanks @Morteeza
There is just one minor test documentation comment that can be looked at after.

python/Scripts/generateshader.py

source/MaterialXRenderGlsl/TextureBaker.h

source/MaterialXRenderMsl/MslRenderer.mm

source/MaterialXView/RenderPipeline.h

kwokcb

Sorry for the additional spam but looking at TextureBaker, it seems like the MSL one will replace the GLSL one on Mac. I left a comment for an option for the Python wrappers, but any existing users for this (which includes Usd) would be switching to a "new baker" on Mac so a possible cause concern for integration ?

python/Scripts/baketextures.py

python/Scripts/translateshader.py

kwokcb · 2023-03-14T14:20:34Z

source/MaterialXRenderMsl/MSLPipelineStateObject.h

+    /// The structure is populated by directly scanning the program so may not contain
+    /// some inputs listed on any associated HwShader as those inputs may have been
+    /// optimized out if they are unused.
+    struct MX_RENDERMSL_API Input


Hi @Morteeza, What do you think on this comment. Sorry for being nitpicky :).

…both baketextures and translateshader python scripts. The default is left to be "Metal"

Morteeza · 2023-03-14T15:39:49Z

Sorry for the additional spam but looking at TextureBaker, it seems like the MSL one will replace the GLSL one on Mac. I left a comment for an option for the Python wrappers, but any existing users for this (which includes Usd) would be switching to a "new baker" on Mac so a possible cause concern for integration ?

Thanks for the feedback. As I mentioned on previous comments I added the option to use glsl texture baker on Mac.

Morteeza · 2023-03-14T15:40:47Z

#1258 (comment)

@kwokcb Not nitpicking at all. The comment still holds. Did I get that correctly?

source/MaterialXRender/ImageHandler.h

source/MaterialXGenMsl/MslShaderGenerator.cpp

Addresses potential overflow risks of ImageSamplingKeyHasher Updates MaterialXTest document with relevant metal shading language support

This fixes a crash on pre-Big Sur systems where tiled pipeline is unsupported

Morteeza · 2023-03-15T16:03:57Z

Hi @pablode

Thanks again for checking out MaterialX Metal backend. Today I tried it on macOS 10.15.7 and fixed issues we were facing due to a slight change in API behaviour. It should work fine now, so we only fall back to OpenGL backend for MaterialXView for 10.14 and below, now.

pablode · 2023-03-15T16:42:53Z

@Morteeza Awesome - and I can confirm that it works! Coincidentally, I did some investigation too, and found some potential improvements (see PR in your repo).

@pablode

Metal backend improvements Thanks to @pablode

Morteeza · 2023-03-15T16:56:10Z

@Morteeza Awesome - and I can confirm that it works! Coincidentally, I did some investigation too, and found some potential improvements (see PR in your repo).

Awesome! Thanks for your improvements. I appreciate the effort. I merged all changes in.

This changelist addresses a minor warning flagged by PVS-Studio, where an evaluated expression was always false in the given context.

This changelist clarifies the parenthesis guards of the TEXTURE_NAME and SAMPLER_NAME macros.

jstone-lucasfilm

Great work on these new features, thanks @Morteeza!

Morteeza · 2023-03-16T18:56:12Z

Great work on these new features, thanks @Morteeza!

Thanks @jstone-lucasfilm for improvements and support 👍

…undation#1258) - Adds Metal Support to MaterialXShaderGen Backend - Adds Metal Support to MaterialX Rendering code for testing generated shaders - Adds Metal Support to MaterialXView Rendering Support (switchable between Metal and OpenGL, default is Metal)

Morteeza added 2 commits February 8, 2023 21:53

Merge pull request #1 from AcademySoftwareFoundation/main

e82661e

Updating the repo

Merge branch 'AcademySoftwareFoundation:main' into main

d78b4b3

Adds Metal Support to MaterialXShaderGen Backend

aa18bc9

Adds Metal Support to MaterialX Rendering code for testing generated shaders Adds Metal Support to MaterialXView Rendering Support (switchable between Metal and OpenGL, default is Metal)

Morteeza force-pushed the main branch from 75583c0 to aa18bc9 Compare February 27, 2023 16:55

kwokcb reviewed Mar 1, 2023

View reviewed changes

Morteeza added 5 commits March 7, 2023 15:37

Fixes a typo (emitVariableDeclarations)

8f59521

Refactorings to address comments on the review (TextureBaker, MetalTexture class, Math-Scalar Operators, MSLProgram, ...)

Merge branch 'main' of github.com:AcademySoftwareFoundation/MaterialX

3023220

# Conflicts: # source/MaterialXRenderGlsl/TextureBaker.cpp

Moving copyShader to Material interface

5b04863

Adopts ViewerGL and python texture baker binding with latest class re…

08e5014

…naming

Don't export TextureBaker since it is a template.

6f41c26

It causes linker errors for Visual Studio, but in any case exporting template classes is not helpful.

Morteeza requested a review from kwokcb March 7, 2023 20:34

Morteeza force-pushed the main branch 2 times, most recently from e2acf59 to ac037ce Compare March 8, 2023 16:14

Adds python bindings for Metal Shading Language generator

e351a13

Morteeza force-pushed the main branch from ac037ce to e351a13 Compare March 8, 2023 16:42

Morteeza and others added 3 commits March 9, 2023 07:23

Adds MaterialXRenderMsl python bindings

12588c3

Remove extra changelog file

39f1c6a

Signed-off-by: Jonathan Stone <jstone@lucasfilm.com>

Remove extra type suffix

5825bff

Signed-off-by: Jonathan Stone <jstone@lucasfilm.com>

jstone-lucasfilm reviewed Mar 11, 2023

View reviewed changes

source/MaterialXRenderGlsl/GlslMaterial.h Outdated Show resolved Hide resolved

jstone-lucasfilm reviewed Mar 11, 2023

View reviewed changes

source/MaterialXTest/MaterialXGenMsl/CompileMSL.h Outdated Show resolved Hide resolved

Morteeza added 3 commits March 13, 2023 15:17

Move MaterialXView's rendering codes out of Viewer class and define n…

4c35490

…ew abstraction call RenderPipeline that takes care of rendering. This change is done with goal of reducing code duplication.

Renaming file and function to match MaterialX code conventions: renam…

807b96c

…ing MSL to Msl.

Bring back OpenGL error checking and disabling SRGB mode on exception…

56a8eb8

… handling

Morteeza force-pushed the main branch from 91ef549 to 56a8eb8 Compare March 13, 2023 15:29

Morteeza added 2 commits March 13, 2023 15:39

Removing unused variables to resolve VS compilation errors

5c96767

Renaming Renderer's Material class to ShaderMaterial to be consistent…

067a397

… with other parts of code Correct file inclusion typo

jstone-lucasfilm added 2 commits March 13, 2023 17:50

Align script naming conventions

27202ba

Signed-off-by: Jonathan Stone <jstone@lucasfilm.com>

Minor style fix

0e13dde

Signed-off-by: Jonathan Stone <jstone@lucasfilm.com>

kwokcb approved these changes Mar 14, 2023

View reviewed changes

python/Scripts/generateshader.py Show resolved Hide resolved

source/MaterialXRenderGlsl/TextureBaker.h Show resolved Hide resolved

source/MaterialXRenderMsl/MslRenderer.mm Show resolved Hide resolved

source/MaterialXView/RenderPipeline.h Show resolved Hide resolved

kwokcb reviewed Mar 14, 2023

View reviewed changes

Adds the option to switch from msl backend to glsl backend on Mac in …

b3e86ee

…both baketextures and translateshader python scripts. The default is left to be "Metal"

jstone-lucasfilm reviewed Mar 14, 2023

View reviewed changes

source/MaterialXRender/ImageHandler.h Outdated Show resolved Hide resolved

jstone-lucasfilm reviewed Mar 14, 2023

View reviewed changes

source/MaterialXGenMsl/MslShaderGenerator.cpp Outdated Show resolved Hide resolved

Morteeza and others added 6 commits March 15, 2023 13:33

Merge branch 'AcademySoftwareFoundation:main' into main

eafaae9

Enables MacOS 10.15 rendering MaterialXView, and MaterialXTests

210ad24

Addresses potential overflow risks of ImageSamplingKeyHasher Updates MaterialXTest document with relevant metal shading language support

Remove unused GLSL line causing shader compilation error

147c148

Replace C++17 reinterpret_cast to fix C++11 AppleClang compile error

d5200ef

Initialize objc handles to nil

eceb082

This fixes a crash on pre-Big Sur systems where tiled pipeline is unsupported

Fix MacOS platform detection define

7aa8254

Avoid external executable for MacOS host version detection

ada2b46

Merge pull request #2 from pablode/metal-backend

21b5cae

Metal backend improvements Thanks to @pablode

jstone-lucasfilm added 2 commits March 15, 2023 21:06

Fix PVS-Studio warning

bd32c94

This changelist addresses a minor warning flagged by PVS-Studio, where an evaluated expression was always false in the given context.

Clarify generator macros

5a215fb

This changelist clarifies the parenthesis guards of the TEXTURE_NAME and SAMPLER_NAME macros.

jstone-lucasfilm force-pushed the main branch from 5ff0f22 to 5a215fb Compare March 16, 2023 05:15

jstone-lucasfilm added 3 commits March 16, 2023 10:43

Delete files for letter case update

9a87eee

Update letter case

8e0d80b

Update implementation filename for MSL

47b8bd9

jstone-lucasfilm approved these changes Mar 16, 2023

View reviewed changes

jstone-lucasfilm merged commit 6f8b3bd into AcademySoftwareFoundation:main Mar 16, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds Metal Shader Generation and Rendering Support #1258

Adds Metal Shader Generation and Rendering Support #1258

Morteeza commented Feb 24, 2023

linux-foundation-easycla bot commented Feb 24, 2023 •

edited

Loading

kwokcb left a comment •

edited

Loading

Morteeza commented Mar 7, 2023 •

edited

Loading

Morteeza commented Mar 8, 2023

Morteeza commented Mar 13, 2023

kwokcb left a comment

kwokcb left a comment

kwokcb Mar 14, 2023

Morteeza commented Mar 14, 2023

Morteeza commented Mar 14, 2023 •

edited

Loading

Morteeza commented Mar 15, 2023

pablode commented Mar 15, 2023

Morteeza commented Mar 15, 2023

jstone-lucasfilm left a comment

Morteeza commented Mar 16, 2023

Adds Metal Shader Generation and Rendering Support #1258

Adds Metal Shader Generation and Rendering Support #1258

Conversation

Morteeza commented Feb 24, 2023

linux-foundation-easycla bot commented Feb 24, 2023 • edited Loading

kwokcb left a comment • edited Loading

Choose a reason for hiding this comment

Morteeza commented Mar 7, 2023 • edited Loading

Morteeza commented Mar 8, 2023

Morteeza commented Mar 13, 2023

kwokcb left a comment

Choose a reason for hiding this comment

kwokcb left a comment

Choose a reason for hiding this comment

kwokcb Mar 14, 2023

Choose a reason for hiding this comment

Morteeza commented Mar 14, 2023

Morteeza commented Mar 14, 2023 • edited Loading

Morteeza commented Mar 15, 2023

pablode commented Mar 15, 2023

Morteeza commented Mar 15, 2023

jstone-lucasfilm left a comment

Choose a reason for hiding this comment

Morteeza commented Mar 16, 2023

linux-foundation-easycla bot commented Feb 24, 2023 •

edited

Loading

kwokcb left a comment •

edited

Loading

Morteeza commented Mar 7, 2023 •

edited

Loading

Morteeza commented Mar 14, 2023 •

edited

Loading