Skip to content

OpenCL dequant_mul_mat #1459

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
May 22, 2023
Merged

OpenCL dequant_mul_mat #1459

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
a7e3bee
Move back to C++ for OpenCL
0cc4m May 14, 2023
17e53db
Refactor OpenCL code to work more like the CUDA code, add missing fun…
0cc4m May 14, 2023
5f610c9
Fix bugs in dequant_mul_mat code
0cc4m May 14, 2023
8c7a7ce
Fix dequant_mul_mat kernel
0cc4m May 14, 2023
cb588e2
Add remaining dequant_mul_mat functions
0cc4m May 14, 2023
1968380
Fix CMakeLists.txt
0cc4m May 15, 2023
915d0d1
Generate dequant_mul_mat kernels from simple templates
0cc4m May 16, 2023
cda2d48
Fix error in convert f16 to f32 kernel call
0cc4m May 16, 2023
42e1a2b
Fix tensor load to device
0cc4m May 16, 2023
457eff9
Deduplicate dequant kernels
0cc4m May 18, 2023
e41a7ae
Fix convert_row_f16 kernel issue
0cc4m May 18, 2023
a1657d0
Add OpenCL compile options
0cc4m May 19, 2023
b6b3996
Use compile args for preprocessing constants
0cc4m May 21, 2023
18e9dd8
Explicitely set GEMM type
0cc4m May 21, 2023
4a55951
Only copy f16/f32 buffer if not already on GPU
0cc4m May 22, 2023
e1ee281
change to fprintf
SlyEcho May 22, 2023
4dfd4fe
Restore default platform + device selection by id behavior
0cc4m May 22, 2023
cb28080
Small compiler warning fixes
SlyEcho May 22, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,7 @@ if (LLAMA_CLBLAST)
if (CLBlast_FOUND)
message(STATUS "CLBlast found")

set(GGML_OPENCL_SOURCES ggml-opencl.c ggml-opencl.h)
set(GGML_OPENCL_SOURCES ggml-opencl.cpp ggml-opencl.h)

add_compile_definitions(GGML_USE_CLBLAST)

Expand Down
5 changes: 3 additions & 2 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,15 +138,16 @@ ggml-cuda.o: ggml-cuda.cu ggml-cuda.h
endif
ifdef LLAMA_CLBLAST
CFLAGS += -DGGML_USE_CLBLAST
CXXFLAGS += -DGGML_USE_CLBLAST
# Mac provides OpenCL as a framework
ifeq ($(UNAME_S),Darwin)
LDFLAGS += -lclblast -framework OpenCL
else
LDFLAGS += -lclblast -lOpenCL
endif
OBJS += ggml-opencl.o
ggml-opencl.o: ggml-opencl.c ggml-opencl.h
$(CC) $(CFLAGS) -c $< -o $@
ggml-opencl.o: ggml-opencl.cpp ggml-opencl.h
$(CXX) $(CXXFLAGS) -c $< -o $@
endif
ifneq ($(filter aarch64%,$(UNAME_M)),)
# Apple M1, M2, etc.
Expand Down
Loading