Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for BFloat16 #41075

Closed
DhairyaLGandhi opened this issue Jun 3, 2021 · 4 comments · Fixed by #51470
Closed

Add support for BFloat16 #41075

DhairyaLGandhi opened this issue Jun 3, 2021 · 4 comments · Fixed by #51470

Comments

@DhairyaLGandhi
Copy link

DhairyaLGandhi commented Jun 3, 2021

This came up on the ML call today. BFloat16 has basically taken over Float16 as a low precision floating point representation and is now used in accelerating a number of ML ops to use lower precision element types. LLVM has also introduced the BFloat16 primitive. @Keno mentioned adding this support in the language generically would be possible and useful. Opening this here for folks to chime in with what needs to happen for this to come to fruition. BFloat16s.jl is already quite old at this point too.

Also cc @maleadt for his thoughts.

@Keno
Copy link
Member

Keno commented Jun 3, 2021

Since LLVM has support now, we need to figure out how to make the BFloat16 type (https://github.com/JuliaMath/BFloat16s.jl/blob/master/src/bfloat16.jl#L8) properly map to its LLVM equivalent. We could either add it to Base and hack something in, or add some generic mechanism for primitive types to declare their LLVM mapping. That said, there should be nothing preventing us from at least hooking up CuDNN and similar libraries using the existing BFloat16s package, so I think for the ML use case, that might not be required.

@maleadt
Copy link
Member

maleadt commented Jun 4, 2021

CUDA.jl's CUBLAS-based GEMM already works with BFloat16s like that, https://github.com/JuliaGPU/CUDA.jl/blob/f9f21dd3f80be936c30c44b051ffaaf599eac87e/lib/cublas/wrappers.jl#L868-L874.

@pengtu
Copy link

pengtu commented Jul 10, 2023

@Keno @maleadt @DhairyaLGandhi : When can the work for adding the avx512-bfloat16 instruction generation for x86 start? Similar to the work for float16

@maleadt
Copy link
Member

maleadt commented Jul 18, 2023

That would first require a way to target LLVM's bfloat type, as Keno mentioned above. That work hasn't happened yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants