draft Metal implementation

[ygdrasil-io]

Not yet an official PR, but more a base of discussion to a proper metal implementation.
This is following the discussion here #1155.
Be open minded, everything here is only a proposal.

1. First step

Not sure I have enough knowledge in graphics pipeline to do a complete implementation, but at first I will try to cover shader generation and compilation, then a basic rendering on a MTKView.

2. About the implementation

From my point of view, Korge code is not necessarily easy to parse so I tried to improve readability with digression regarding the actual code. Feel free to argue and I will update the code according to this.

2-1. Projects namings and organizations :

I think that using names like korgw, korio, etc.. make it difficult to newcomer to understand the projects organization, so, i created new sub project like this

• graphics : just a directory, but can be a project including subproject and make it easy to import
  • foundation : could be named 'core' or whatever generic, could contains interface and common code regarding "AG" stuff
  • backend : just a directory
    • metal : actual metal implementation

2-2. Misc

• Kotest is used to implement test because of his smooth DSL but can be removed later.
• I use the “korge” base package instead of “com.soywiz”, not sure that it is relevant to use Java convention here.
• There are a lot of abbreviations in the code, can i rename them with their full name to increase readability ?

[ygdrasil-io]

@soywiz found this on the gl implementation, not sure to understand the point.

[soywiz]

This is probably a typo

[ygdrasil-io]

@soywiz one question so far, Metal shader must use structure as input/output of vertex and fragment functions, but the current implementation of VarType only support predefined type, should i change this behaviour or just tweak the metal shader generator ?

[ygdrasil-io]

to revert later

[soywiz]

did you have any kind of errors because of this? Would be nice to figure out the root cause so other people can benefit from it

[ygdrasil-io]

Using :

when enabling :

Could not find compile target android-30 for modules :kbignum, :kds, :klock, :klogger, :kmem, :korau, :korge, :korgw, :korim, :korinject, :korio, :korma, :korte, :korvi, :krypto, :luak, :graphics:foundation, :graphics:backend:metal On IntelliJ

[ygdrasil-io]

to revert later

[soywiz]

Thanks for the PR/proposal.

I'm open to the suggestions your propose, but I believe we should handle them independently and separately to this PR, to avoid the complexity of handling everything at once.

Let's use the forum for example:
https://github.com/korlibs/korge/discussions/categories/ideas
Create one for reworking the project structure and/or directory structure, and other for kotest, and feel free to create extra ones for that. After discussion we can figure out how to proceed on each topic independently.

Now let's focus in this PR about the metal shader generation. I believe since it can be added to the repository without affecting any production feature, it could be merged even if incomplete. Just an initial step, then a test assertEquals for now. Then we can iterate it.

Makes sense to you?

[soywiz]

@ygdrasil-io

I have created several discussion threads for the major changes you proposed. Can we discuss them now? Some of them will likely affect compatibility, so now that we are still in 4.0 alphas we can consider it. After 4.0 is published, those proposals would have to wait for the next major release:

• Proposal: Add Kotest to korlibs? #1247
• Proposal: Chaneg projects namings and organization #1248
• Proposal: Change the base package of projects #1249

If I missed something, please create a new discussion in the Ideas and discussions section.

[ygdrasil-io]

@soywiz small update

AG and shader generation support basic elements such as vertex, position, and color.
Vertex structure is not yet supported, so right now i need to send a buffer for each vertex data.

My next step are :

• Implementing basic uniform handling
• I may need to deconstruct the vertex buffers sent to AG to recreate a list of buffer that can be exploited in the shader program (performance may be poor, but it is acceptable for the first version).
• Remove kotest
• Move new classes/files to korge current package
• Understand how korge implement textures and try a basic implementation.

As there is no such things as small victory :

Using the code

    private val vertexShader = VertexShader {
        SET(DefaultShaders.v_Col, DefaultShaders.a_Col)
        SET(out, vec4(DefaultShaders.a_Pos, 1f.lit, 1f.lit))
    }
    private val fragmentShader = FragmentShader {
        SET(out, DefaultShaders.v_Col)
    }

    private val program = Program(vertexShader, fragmentShader)

    private val vertexData = AGVertexArrayObject(
        AGVertexData(
            layout = VertexLayout(DefaultShaders.a_Col),
            buffer = AGBuffer().upload(floatArrayOf(
                1f, 1f, 1f, 0.0f, // White
                1f, 0f, 0f, 0.0f, // Red
                0f, 1f, 0.0f, 0.0f, // Blue
                0f, 0.0f, 1f, 0.0f, // Green
            ))
        ),
        AGVertexData(
            layout = VertexLayout(DefaultShaders.a_Pos),
            buffer = AGBuffer().upload(floatArrayOf(
                0f, 0f,
                1f, 0f,
                1f, 1f,
                0f, 1f
            ))
        )
    )

    private val indices = AGBuffer().upload(shortArrayOf(0, 1, 2, 2, 3, 0))

    override fun drawOnView(view: MTKView) {
        if (ag == null) {
            ag = AGMetal(view)
        }

        val width = 1024//view.drawableSize.useContents { width }.toInt()
        val height = 768//view.drawableSize.useContents { height }.toInt()
        val frameBuffer = AGFrameBufferBase(false)
        val frameBufferInfo = AGFrameBufferInfo(0)
            .withSize(width, height)
            .withSamples(1)
            .withHasDepth(true)
            .withHasStencil(true)


        ag?.draw(
            frameBuffer,
            frameBufferInfo,
            vertexData,
            program,
            drawType = AGDrawType.TRIANGLES,
            vertexCount = 6, // Draw 2 triangles
            indices,
            indexType = AGIndexType.USHORT,
            drawOffset = 0, // This value can be != of 0 ?
            blending = AGBlending.NORMAL, // Pure guess
            uniforms = AGUniformValues(), // Not yet supported on shader generation         
            stencilRef = AGStencilReference.DEFAULT, // Pure guess
            stencilOpFunc = AGStencilOpFunc.DEFAULT, // Pure guess
            colorMask = AGColorMask.DEFAULT,// Pure guess
            depthAndFrontFace= AGDepthAndFrontFace.DEFAULT,// Pure guess
            scissor= AGScissor.FULL,// Pure guess
            cullFace= AGCullFace.NONE,// Pure guess
            instances = 1// Pure guess
        )
    }

[soywiz]

@ygdrasil-io

Nice!

I have a pending PR:
#1175

that will produce uniforms in a buffer you will be able to directly upload to the GPU and use indices for them. But it is stil a WIP

[soywiz]

@ygdrasil-io Okay so I believe we should merge something already, then add a feature flag to disable it for now so you can continue working on it but still refactors and stuff are having this into account.

we can add a feature flag here by using environment variables:

Environment[“ENABLE_METAL”]==“true”

then once ready we can add DISABLE_METAL

as long as ir compiles, and metal failing tests are disabled and opengl backend is working, it should be fine merging it

the only thing is please, even if in other packages, can you put stuff in korgw? I still have to check how to split in modules, because that would create extra artifacts of name I dont know and every extra artifact has to download like 8 files per target and that adds up like a lot of time for new users to get started. I have to experiment and check how to do it

[ygdrasil-io]

@soywiz
I added a new project "osx-metal-playground" to try the draft implementation and moved all the other files into korgw with the "com.soywiz" package. The only new test covering shader generation is passing (at least on my macbook with intel processor).

Anything else to change ?

[soywiz]

Nope. Let's merge it and iterate