Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove libcmt, use native ObjectiveC FFI #117

Merged
merged 32 commits into from
Mar 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
28f58ef
Import ObjectiveC.jll and load CoreGraphics.
maleadt Mar 3, 2023
f1e7352
Migrate MTLDevice and MTLSize.
maleadt Mar 3, 2023
8177b0c
Migrate MTLEvent.
maleadt Mar 4, 2023
b7a9fd4
Migrate MTLFence.
maleadt Mar 4, 2023
ca38e2b
Make the getproperties generator reusable, and make it check for nil.
maleadt Mar 4, 2023
cc9a162
Support actual types in the getproperties generator.
maleadt Mar 4, 2023
15942e4
Auto-generate setproperty method body too.
maleadt Mar 4, 2023
870ec1f
Auto-generate MTLFence properties.
maleadt Mar 4, 2023
0450da8
Remove MTLAbstractEvent, which doesn't exist.
maleadt Mar 4, 2023
8e3b5d4
Avoid property duplication by invoking parent get/setproperty methods.
maleadt Mar 4, 2023
9eddfe6
Migrate MTLResource and MTLBuffer.
maleadt Mar 6, 2023
8cbe2d4
Migrate MTLCompileOptions.
maleadt Mar 6, 2023
798e3c5
Port property generator to ObjectiveC.jl.
maleadt Mar 6, 2023
87bea6a
Migrate NSError.
maleadt Mar 6, 2023
b1c2543
Remove unused file.
maleadt Mar 6, 2023
fabf905
Adapt to strongly-typed id.
maleadt Mar 6, 2023
598ec4e
Migrate MTLLibrary.
maleadt Mar 6, 2023
828249b
Migrate command buffer, encoder and queue.
maleadt Mar 7, 2023
62a1ac9
Migrage MTLFunction and compute pipeline.
maleadt Mar 7, 2023
6b93db2
Migrate MTLHeap.
maleadt Mar 7, 2023
2623d23
Migrate and simplify remainder of compute command encoder.
maleadt Mar 7, 2023
da3734f
Remove unused MTLComputePipelineReflection.
maleadt Mar 7, 2023
7c10998
Migrate MTLBinaryArchive.
maleadt Mar 7, 2023
4d5e6e0
Migrate capture functionality.
maleadt Mar 7, 2023
7ada7bd
Use Objective-C blocks for command buffer handlers.
maleadt Mar 8, 2023
191bac1
Migrate MPS.
maleadt Mar 8, 2023
ae88836
Take off the training wheels.
maleadt Mar 8, 2023
9b57f07
cmt begone
maleadt Mar 8, 2023
bff4696
Update README.
maleadt Mar 8, 2023
bf50f3c
NFC clean-ups.
maleadt Mar 8, 2023
8c228d9
Disable a test that looks like a Metal bug.
maleadt Mar 8, 2023
d4e0d3c
Bump ObjectiveC.jl.
maleadt Mar 8, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 0 additions & 8 deletions .buildkite/pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,6 @@ steps:
- src
- lib
- examples
command: |
julia --project=deps deps/build_ci.jl
agents:
queue: "juliaecosystem"
os: "macos"
Expand All @@ -37,8 +35,6 @@ steps:
# - src
# - lib
# - examples
# command: |
# julia --project=deps deps/build_ci.jl
# agents:
# queue: "juliaecosystem"
# os: "macos"
Expand All @@ -58,8 +54,6 @@ steps:
# - src
# - lib
# - examples
# command: |
# julia --project=deps deps/build_ci.jl
# agents:
# queue: "juliaecosystem"
# os: "macos"
Expand All @@ -86,8 +80,6 @@ steps:
- src
- lib
- examples
command: |
julia --project=deps deps/build_ci.jl
env:
MTL_DEBUG_LAYER: '1'
MTL_SHADER_VALIDATION: '1'
Expand Down
14 changes: 7 additions & 7 deletions Manifest.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

julia_version = "1.8.5"
manifest_format = "2.0"
project_hash = "97c97a79809f4555c5aae0f6b4c9553a7330505e"
project_hash = "f5746c1a98e06b2349e32c5d2131234120f2cc34"

[[deps.Adapt]]
deps = ["LinearAlgebra", "Requires"]
Expand Down Expand Up @@ -143,6 +143,12 @@ version = "2022.2.1"
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"
version = "1.2.0"

[[deps.ObjectiveC]]
deps = ["CEnum"]
git-tree-sha1 = "5b3c98aacd5429e9c367e551134d9466e40ed6d5"
uuid = "e86c9b32-1129-44ac-8ea0-90d5bb39ded9"
version = "0.1.0"

[[deps.OpenBLAS_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"]
uuid = "4536629a-c528-5b80-bd46-f80d51c5b363"
Expand Down Expand Up @@ -228,12 +234,6 @@ deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"
version = "1.2.12+3"

[[deps.cmt_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl"]
git-tree-sha1 = "049e4965e08ffb1fe62f731c385b92cf62d34817"
uuid = "65323cdd-17ec-5719-9643-72016a7f97e3"
version = "0.2.0+0"

[[deps.libblastrampoline_jll]]
deps = ["Artifacts", "Libdl", "OpenBLAS_jll"]
uuid = "8e850b90-86db-534c-a0d3-1478176c7d93"
Expand Down
10 changes: 4 additions & 6 deletions Project.toml
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
name = "Metal"
uuid = "dde4c033-4e86-420c-a63e-0dd931031962"
authors = ["Filippo Vicentini <filippovicentini@gmail.com>",
"Tim Besard <tim.besard@gmail.com>",
"Max Hawkins <max.hawkins65@gmail.com>"]
authors = ["Filippo Vicentini <filippovicentini@gmail.com>", "Tim Besard <tim.besard@gmail.com>", "Max Hawkins <max.hawkins65@gmail.com>"]
version = "0.2.0"

[deps]
Expand All @@ -14,18 +12,18 @@ GPUCompiler = "61eb1bfa-7361-4325-ad38-22787b887f55"
LLVM = "929cbde3-209d-540e-8aea-75f648917ca0"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
Metal_LLVM_Tools_jll = "0418c028-ff8c-56b8-a53e-0f9676ed36fc"
ObjectiveC = "e86c9b32-1129-44ac-8ea0-90d5bb39ded9"
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
Reexport = "189a3867-3050-52da-a836-e630ba90ab69"
cmt_jll = "65323cdd-17ec-5719-9643-72016a7f97e3"

[compat]
Adapt = "3"
CEnum = "0.4"
ExprTools = "0.1"
GPUArrays = "8.4"
GPUCompiler = "0.16, 0.17"
Reexport = "1.0"
LLVM = "4.15"
Metal_LLVM_Tools_jll = "~0.3"
cmt_jll = "~0.2"
ObjectiveC = "0.1"
Reexport = "1.0"
vchuravy marked this conversation as resolved.
Show resolved Hide resolved
julia = "1.8"
61 changes: 15 additions & 46 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,61 +154,30 @@ may fail to start.

## Metal API wrapper

Finally, all of the above functionality is made possible by interfacing with
the Metal libraries through a [small C library](https://github.com/JuliaGPU/cmt)
that wraps the ObjectiveC APIs.

These low-level wrappers, along with some slightly higher-level Julia wrappers,
are available in the `MTL` submodule exported by Metal.jl. All wrapped C
functions and types start with the `mt` prefix, whereas the Julia wrappers are
prefixed with `Mtl`:
Finally, all of the above functionality is made possible by interfacing with the Metal
libraries through [ObjectiveC.jl](https://github.com/JuliaInterop/ObjectiveC.jl). We provide low-level objects and functions that map These
low-level API wrappers, along with some slightly higher-level Julia wrappers, are available
in the `MTL` submodule exported by Metal.jl:

```julia
julia> dev = MtlDevice(1)
MtlDevice:
name: Apple M1 Pro
lowpower: false
headless: false
removable: false
unified memory: true
registry id: 4294969448
transfer rate: 0
julia> dev = MTLDevice(1)
<AGXG13XDevice: 0x14c17f200>
name = Apple M1 Pro

julia> dev.name
"Apple M1 Pro"
NSString("Apple M1 Pro")
```


## Hacking

Metal.jl relies on two binary dependencies (provided as JLLs):

- [cmt](https://github.com/JuliaGPU/Metal.jl/tree/main/deps/cmt)
- [LLVM with an AIR back-end](https://github.com/JuliaGPU/llvm-metal)

Normally, these dependencies are built on
[Yggdrasil](https://github.com/JuliaPackaging/Yggdrasil/).
If you need to make changes to these dependencies, have a look at the
`build_cmt.jl` and `build_llvm.jl` scripts in the `deps/` folder. These
scripts build a local version of the dependency, and configure a local
preference such that any environment depending on the corresponding JLLs will
pick-up the modified version (i.e., do `julia --project` in a clone
of `Metal.jl`):

```
$ julia --project -e 'using Metal; @show MTL.cmt.libcmt'
MTL.libcmt = "/Users/tim/Julia/depot/artifacts/6adc0ed9a8370ff1e3bb8fbaf36e8519ee11fd96/lib/libcmt.dylib"

$ julia --project=deps deps/build_cmt.jl
...
[100%] Built target cmt

$ julia --project -e 'using Metal; @show MTL.cmt.libcmt'
MTL.libcmt = "/Users/tim/Julia/depot/scratchspaces/dde4c033-4e86-420c-a63e-0dd931031962/cmt/lib/libcmt.dylib"
```

These scripts are integrated with our CI, and will be triggered if
the `ci.build_cmt` or `ci.build_llvm` labels are set on a pull request.
Metal.jl relies on a custom [LLVM with an AIR
back-end](https://github.com/JuliaGPU/llvm-metal), provided as a JLL. Normally, this JLLis
built on [Yggdrasil](https://github.com/JuliaPackaging/Yggdrasil/). If you need to make
changes to the LLVM back-end, have a look at the `build_llvm.jl` in the `deps/` folder. This
scripts builds a local version of the LLVM back-end, and configures a local preference such
that any environment depending on the corresponding JLLs will pick-up the modified version
(i.e., do `julia --project` in a clone of `Metal.jl`).


## Acknowledgements
Expand Down
54 changes: 0 additions & 54 deletions deps/build_ci.jl

This file was deleted.

58 changes: 0 additions & 58 deletions deps/build_cmt.jl

This file was deleted.

88 changes: 0 additions & 88 deletions deps/cmt/CMakeLists.txt

This file was deleted.

Loading