Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FPU operator issues #165

Open
maleadt opened this issue Oct 27, 2023 · 3 comments
Open

FPU operator issues #165

maleadt opened this issue Oct 27, 2023 · 3 comments
Assignees
@wardvermeulen

Comments

@maleadt
Copy link
Member

maleadt commented Oct 27, 2023

While tuning, I encountered a couple of compilation errors that either should be fixed in the kernel, or generate a ConfigError.

GemmKernels.Config{(M = 4096, N = 4096, K = 4096), (M = 16, N = 128, K = 16), 4, (M = 16, K = 8), (M = 4, K = 1), (K = 16, N = 8), (K = 4, N = 1), (M = 16, N = 8), (M = 4, N = 1), (M = 8, N = 64, K = 16), (M = 8, N = 16, K = 16), GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.Zero{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.Padded{GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, 8}, GemmKernels.Layout.Padded{GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, 8}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Operator.FPUOp{8, 16, 16, 8, 4, 1, Float32, Float32}, true, true}()

Reason: unsupported call to an unknown function (call to julia.new_gc_frame)
Reason: unsupported call to an unknown function (call to julia.push_gc_frame)
Reason: unsupported dynamic function invocation (call to _totuple_err(T) @ Base tuple.jl:395)
Stacktrace:
 [1] _totuple
   @ ./tuple.jl:418
 [2] Tuple
   @ ./tuple.jl:391
 [3] load_b
   @ ~/Julia/pkg/GemmKernels/src/operator.jl:81
 [4] load_b
   @ ~/Julia/pkg/GemmKernels/src/operator.jl:15
 [5] macro expansion
   @ ~/Julia/pkg/GemmKernels/src/array.jl:69
 [6] matmul_singlestage
   @ ~/Julia/pkg/GemmKernels/src/kernel.jl:97
Reason: unsupported call to an unknown function (call to julia.get_gc_frame_slot)
Reason: unsupported call to an unknown function (call to jl_f__apply_iterate)
Stacktrace:
 [1] _totuple
   @ ./tuple.jl:420
 [2] Tuple
   @ ./tuple.jl:391
 [3] load_b
   @ ~/Julia/pkg/GemmKernels/src/operator.jl:81
 [4] load_b
   @ ~/Julia/pkg/GemmKernels/src/operator.jl:15
 [5] macro expansion
   @ ~/Julia/pkg/GemmKernels/src/array.jl:69
 [6] matmul_singlestage
   @ ~/Julia/pkg/GemmKernels/src/kernel.jl:97
Reason: unsupported dynamic function invocation (call to GemmKernels.Transform.Elementwise{typeof(identity)}(identity))
Stacktrace:
 [1] macro expansion
   @ ~/Julia/pkg/GemmKernels/src/array.jl:69
 [2] matmul_singlestage
   @ ~/Julia/pkg/GemmKernels/src/kernel.jl:97
Reason: unsupported dynamic function invocation (call to setindex)
Stacktrace:
 [1] macro expansion
   @ ~/Julia/pkg/GemmKernels/src/array.jl:69
 [2] matmul_singlestage
   @ ~/Julia/pkg/GemmKernels/src/kernel.jl:97
Reason: unsupported call to an unknown function (call to jl_f__apply_iterate)
Stacktrace:
 [1] setindex
   @ ./tuple.jl:59
 [2] setindex
   @ ~/Julia/pkg/GemmKernels/src/array.jl:41
 [3] macro expansion
   @ ~/Julia/pkg/GemmKernels/src/array.jl:69
 [4] load_b
   @ ~/Julia/pkg/GemmKernels/src/operator.jl:77
 [5] load_b
   @ ~/Julia/pkg/GemmKernels/src/operator.jl:15
 [6] macro expansion
   @ ~/Julia/pkg/GemmKernels/src/array.jl:69
 [7] matmul_singlestage
   @ ~/Julia/pkg/GemmKernels/src/kernel.jl:97
Reason: unsupported call through a literal pointer (call to ijl_alloc_array_1d)
Stacktrace:
  [1] Array
    @ ./boot.jl:475
  [2] _array_for
    @ ./array.jl:720
  [3] _collect
    @ ./array.jl:694
  [4] collect
    @ ./array.jl:692
  [5] _totuple
    @ ./tuple.jl:416
  [6] Tuple
    @ ./tuple.jl:391
  [7] load_b
    @ ~/Julia/pkg/GemmKernels/src/operator.jl:81
  [8] load_b
    @ ~/Julia/pkg/GemmKernels/src/operator.jl:15
  [9] macro expansion
    @ ~/Julia/pkg/GemmKernels/src/array.jl:69
 [10] matmul_singlestage
    @ ~/Julia/pkg/GemmKernels/src/kernel.jl:97
Reason: unsupported call to an unknown function (call to julia.pop_gc_frame)

Another:

GemmKernels.Config{(M = 4096, N = 4096, K = 4096), (M = 256, N = 32, K = 32), 8, (M = 128, K = 1), (M = 4, K = 1), (K = 32, N = 4), (K = 4, N = 1), (M = 128, N = 1), (M = 4, N = 1), (M = 64, N = 16, K = 4), (M = 16, N = 16, K = 4), GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.Zero{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.Padded{GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, 8}, GemmKernels.Layout.Padded{GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, 8}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Operator.FPUOp{16, 16, 4, 2, 16, 1, Float32, Float32}, true, true}()

Reason: unsupported call to an unknown function (call to julia.new_gc_frame)
Reason: unsupported call to an unknown function (call to julia.push_gc_frame)
Reason: unsupported call through a literal pointer (call to ijl_alloc_array_1d)
Stacktrace:
  [1] Array
    @ ./boot.jl:475
  [2] _array_for
    @ ./array.jl:720
  [3] _collect
    @ ./array.jl:694
  [4] collect
    @ ./array.jl:692
  [5] _totuple
    @ ./tuple.jl:416
  [6] Tuple
    @ ./tuple.jl:391
  [7] load_a
    @ ~/Julia/pkg/GemmKernels/src/operator.jl:64
  [8] load_a
    @ ~/Julia/pkg/GemmKernels/src/operator.jl:15
  [9] macro expansion
    @ ~/Julia/pkg/GemmKernels/src/array.jl:69
 [10] matmul_singlestage
    @ ~/Julia/pkg/GemmKernels/src/kernel.jl:89
Reason: unsupported dynamic function invocation (call to _totuple_err(T) @ Base tuple.jl:395)
Stacktrace:
 [1] _totuple
   @ ./tuple.jl:418
 [2] Tuple
   @ ./tuple.jl:391
 [3] load_a
   @ ~/Julia/pkg/GemmKernels/src/operator.jl:64
 [4] load_a
   @ ~/Julia/pkg/GemmKernels/src/operator.jl:15
 [5] macro expansion
   @ ~/Julia/pkg/GemmKernels/src/array.jl:69
 [6] matmul_singlestage
   @ ~/Julia/pkg/GemmKernels/src/kernel.jl:89
Reason: unsupported call to an unknown function (call to julia.get_gc_frame_slot)
Reason: unsupported call to an unknown function (call to jl_f__apply_iterate)
Stacktrace:
 [1] _totuple
   @ ./tuple.jl:420
 [2] Tuple
   @ ./tuple.jl:391
 [3] load_a
   @ ~/Julia/pkg/GemmKernels/src/operator.jl:64
 [4] load_a
   @ ~/Julia/pkg/GemmKernels/src/operator.jl:15
 [5] macro expansion
   @ ~/Julia/pkg/GemmKernels/src/array.jl:69
 [6] matmul_singlestage
   @ ~/Julia/pkg/GemmKernels/src/kernel.jl:89
Reason: unsupported dynamic function invocation (call to GemmKernels.Transform.Elementwise{typeof(identity)}(identity))
Stacktrace:
 [1] macro expansion
   @ ~/Julia/pkg/GemmKernels/src/array.jl:69
 [2] matmul_singlestage
   @ ~/Julia/pkg/GemmKernels/src/kernel.jl:89
Reason: unsupported dynamic function invocation (call to setindex)
Stacktrace:
 [1] macro expansion
   @ ~/Julia/pkg/GemmKernels/src/array.jl:69
 [2] matmul_singlestage
   @ ~/Julia/pkg/GemmKernels/src/kernel.jl:89
Reason: unsupported call to an unknown function (call to julia.pop_gc_frame)
@maleadt maleadt changed the title FPU operator: compilation errors FPU operator issues Oct 27, 2023
@maleadt
Copy link
Member Author

maleadt commented Oct 27, 2023

I also encountered a couple that perform wrong computations:

┌ Warning: Configuration produced invalid result: GemmKernels.Config{(M = 4096, N = 4096, K = 4096), (M = 32, N = 32, K = 8), 8, (M = 32, K = 4), (M = 4, K = 1), (K = 8, N = 16), (K = 4, N = 1), (M = 32, N = 4), (M = 4, N = 1), (M = 8, N = 16, K = 32), (M = 4, N = 8, K = 32), GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.Zero{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.Padded{GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, 8}, GemmKernels.Layout.Padded{GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, 8}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Operator.FPUOp{4, 8, 32, 4, 8, 1, Float32, Float32}, true, true}()`

┌ Warning: Configuration produced invalid result: GemmKernels.Config{(M = 4096, N = 4096, K = 4096), (M = 32, N = 16, K = 16), 8, (M = 32, K = 4), (M = 4, K = 1), (K = 16, N = 8), (K = 4, N = 1), (M = 32, N = 4), (M = 4, N = 1), (M = 8, N = 8, K = 32), (M = 8, N = 4, K = 32), GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.Zero{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.Padded{GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, 8}, GemmKernels.Layout.Padded{GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, 8}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Operator.FPUOp{8, 4, 32, 8, 4, 1, Float32, Float32}, true, true}()

┌ Warning: Configuration produced invalid result: GemmKernels.Config{(M = 4096, N = 4096, K = 4096), (M = 32, N = 32, K = 8), 4, (M = 32, K = 4), (M = 4, K = 1), (K = 8, N = 16), (K = 4, N = 1), (M = 32, N = 4), (M = 4, N = 1), (M = 16, N = 16, K = 32), (M = 16, N = 16, K = 32), GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.Zero{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.Padded{GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, 8}, GemmKernels.Layout.Padded{GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, 8}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Layout.UnsafeAlignedColMajor{Float32}, GemmKernels.Operator.FPUOp{16, 16, 32, 8, 4, 1, Float32, Float32}, true, true}()

@wardvermeulen
Copy link
Collaborator

The first two are apply_iterate issues (#99), which I did not account for in the ConfigErrors. I could easily add a check for them in the Config if you want?

e.g. for the first Config:

julia> bs = base_shape(conf.operator)
(M = 8, N = 16, K = 16, mb = 8, nb = 4, kb = 1)

julia> div(bs.N, bs.nb) * div(bs.K, bs.kb) >= 32
true

The wrong computations will require further investigation, although at a first glance the block shape K = 8 seems to small for the operator shape K = 32.

@wardvermeulen wardvermeulen mentioned this issue Nov 1, 2023
Merged
@wardvermeulen
Copy link
Collaborator

I added the fix for the invalid results in #167

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wardvermeulen wardvermeulen

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@maleadt @wardvermeulen