Issue: LibWhisper.swift Fails to Build Objective-C Module whisper

[DeepBhupatkar]

I'm encountering a build issue with the iOS Swift example for whisper.cpp. The project throws the following error:
Steps I followed:

1. Cloned the whisper.cpp repository.
2. Tried running the iOS Swift example located in examples/whisper.swiftui.

However, the build fails with the following errors:
Key Errors:
'whisper.h' file not found with
Multiple Cannot find in scope errors, such as:
Cannot find 'whisper_free' in scope
Cannot find 'whisper_full_default_params' in scope
Cannot find 'WHISPER_SAMPLING_GREEDY' in scope

Am I missing any steps? I followed all five steps from the SwiftUI project guide : https://github.com/ggerganov/whisper.cpp/tree/master/examples/whisper.swiftui, but despite trying multiple times, this issue keeps occurring. same thing happing in object-c example.

[AlanConstantino]

I had this issue happen to me too! It also happens on the swift ui app example as well.

The whisper.h file one seems to be an easy fix by just changing

import <whisper.h>

to

import "whisper.h"

Now I don't know if this is the actual "fix" for this but it the compiler seems to stop yelling at me after this change.

However when you go to compile it, it still causes issues. I think there is something wrong with the way Xcode links to the whisper package which is causing issue you're seeing

[DeepBhupatkar]

I made that change, but the issue persists, just like in the image you attached. However, with the help of someones other works and some custom build modifications, I was able to create a package that allows me to run whisper.cpp. I’m now planning to build a new SPM or pod for public release or use.

[pgorzelany]

Can you share the solution?

[zydeico]

Same issue here

[DeepBhupatkar]

Yes, sure! Here you go. This is the link to my repo for the Whisper.cpp project: https://github.com/DeepBhupatkar/swift-whisper.cpp-transcription (ignore the README). I'm still working on releasing the package through SPM and CocoaPods, but for now, you can use it from my example just copy the package and use it.

[AlanConstantino]

Yes, sure! Here you go. This is the link to my repo for the Whisper.cpp project: https://github.com/DeepBhupatkar/swift-whisper.cpp-transcription (ignore the README). I'm still working on releasing the package through SPM and CocoaPods, but for now, you can use it from my example just copy the package and use it.

Appreciate you broski!

[anka]

Yes, sure! Here you go. This is the link to my repo for the Whisper.cpp project: https://github.com/DeepBhupatkar/swift-whisper.cpp-transcription (ignore the README). I'm still working on releasing the package through SPM and CocoaPods, but for now, you can use it from my example just copy the package and use it.

@DeepBhupatkar Do you already have some experience with how well the transcription works with your example code? Did you add further optimizations regarding the whisper model parameters for running it on iOS?

[jo32]

Yes, sure! Here you go. This is the link to my repo for the Whisper.cpp project: https://github.com/DeepBhupatkar/swift-whisper.cpp-transcription (ignore the README). I'm still working on releasing the package through SPM and CocoaPods, but for now, you can use it from my example just copy the package and use it.

@DeepBhupatkar Can you list what you have done to make it work? This way, we can follow your procedure in case whisper.cpp is updated.

[tokyo4]

I had this issue happen to me too! It also happens on the swift ui app example as well.

The whisper.h file one seems to be an easy fix by just changing

import <whisper.h>

to

import "whisper.h"

Now I don't know if this is the actual "fix" for this but it the compiler seems to stop yelling at me after this change.

However when you go to compile it, it still causes issues. I think there is something wrong with the way Xcode links to the whisper package which is causing issue you're seeing

I also meet the same problem, did u already solve this problem?

i still don't know why

[lwb4]

It took quite a while but by copying the same approach @DeepBhupatkar took in his repo, I was able to get the Swift example to work.

Here is the patch:

diff --git a/Package.swift b/Package.swift
index 303ed0d..3b85875 100644
--- a/Package.swift
+++ b/Package.swift
@@ -14,6 +14,47 @@ let package = Package(
         .library(name: "whisper", targets: ["whisper"]),
     ],
     targets: [
-        .systemLibrary(name: "whisper", pkgConfig: "whisper"),
-    ]
+        .target(
+            name: "whisper",
+            path: ".",
+            exclude: [
+                "bindings",
+                "cmake",
+                "examples",
+                "models",
+                "samples",
+                "tests",
+                "CMakeLists.txt",
+                "Makefile",
+                "build"
+            ],
+            sources: [
+                "ggml/src/ggml-cpu/ggml-cpu.c",
+                "ggml/src/ggml-cpu/ggml-cpu.cpp",
+                "ggml/src/ggml-cpu/ggml-cpu-traits.cpp",
+                "ggml/src/ggml-threading.cpp",
+                "ggml/src/ggml-cpu/ggml-cpu-quants.c",
+                "ggml/src/ggml-backend-reg.cpp",
+                "ggml/src/ggml-backend.cpp",
+                "ggml/src/ggml.c",
+                "src/whisper.cpp",
+                "ggml/src/ggml-alloc.c",
+                "ggml/src/ggml-quants.c",
+                "ggml/src/ggml-metal/ggml-metal.m",
+            ],
+            resources: [.process("ggml/src/ggml-metal/ggml-metal.metal")],
+            publicHeadersPath: "spm-headers",
+            cSettings: [
+                .unsafeFlags(["-Wno-shorten-64-to-32", "-O3", "-DNDEBUG"]),
+                .define("GGML_USE_ACCELERATE"),
+                .unsafeFlags(["-fno-objc-arc"]),
+                .define("GGML_USE_METAL")
+            ],
+            linkerSettings: [
+                .linkedFramework("Accelerate")
+            ]
+            
+        )
+    ],
+    cxxLanguageStandard: .cxx17
 )
diff --git a/Sources/whisper/whisper.h b/Sources/whisper/whisper.h
index e503c8a..bf62952 100644
--- a/Sources/whisper/whisper.h
+++ b/Sources/whisper/whisper.h
@@ -1,4 +1,3 @@
 #pragma once
 
-#include <whisper.h>
-
+#include "whisper.h"
diff --git a/examples/whisper.swiftui/whisper.swiftui.demo/Resources/models/ggml-tiny.en.bin b/examples/whisper.swiftui/whisper.swiftui.demo/Resources/models/ggml-tiny.en.bin
new file mode 100644
index 0000000..9dd1a6b
Binary files /dev/null and b/examples/whisper.swiftui/whisper.swiftui.demo/Resources/models/ggml-tiny.en.bin differ
diff --git a/spm-headers/ggml-backend-impl.h b/spm-headers/ggml-backend-impl.h
new file mode 120000
index 0000000..10e864a
--- /dev/null
+++ b/spm-headers/ggml-backend-impl.h
@@ -0,0 +1 @@
+../ggml/src/ggml-backend-impl.h
\ No newline at end of file
diff --git a/spm-headers/ggml-common.h b/spm-headers/ggml-common.h
new file mode 120000
index 0000000..b9a3862
--- /dev/null
+++ b/spm-headers/ggml-common.h
@@ -0,0 +1 @@
+../ggml/src/ggml-common.h
\ No newline at end of file
diff --git a/spm-headers/ggml-cpu-impl.h b/spm-headers/ggml-cpu-impl.h
new file mode 120000
index 0000000..214b294
--- /dev/null
+++ b/spm-headers/ggml-cpu-impl.h
@@ -0,0 +1 @@
+../ggml/src/ggml-cpu/ggml-cpu-impl.h
\ No newline at end of file
diff --git a/spm-headers/ggml-impl.h b/spm-headers/ggml-impl.h
new file mode 120000
index 0000000..68c0d5c
--- /dev/null
+++ b/spm-headers/ggml-impl.h
@@ -0,0 +1 @@
+../ggml/src/ggml-impl.h
\ No newline at end of file
diff --git a/spm-headers/ggml-quants.h b/spm-headers/ggml-quants.h
new file mode 120000
index 0000000..fa470c4
--- /dev/null
+++ b/spm-headers/ggml-quants.h
@@ -0,0 +1 @@
+../ggml/src/ggml-quants.h
\ No newline at end of file
diff --git a/spm-headers/ggml-threading.h b/spm-headers/ggml-threading.h
new file mode 120000
index 0000000..307397c
--- /dev/null
+++ b/spm-headers/ggml-threading.h
@@ -0,0 +1 @@
+../ggml/src/ggml-threading.h
\ No newline at end of file
diff --git a/spm-headers/gguf.h b/spm-headers/gguf.h
new file mode 120000
index 0000000..66bfdd7
--- /dev/null
+++ b/spm-headers/gguf.h
@@ -0,0 +1 @@
+../ggml/include/gguf.h
\ No newline at end of file

The issue for me seems to be that the pkg-config based systemLibrary approach in the original Package.swift doesn't work on my computer. No clue why but it kept say "couldn't find pc file for whisper", even though build/whisper.pc was right there, and I kept adding it to PKG_CONFIG_PATH to no avail. Setting up the Package.swift to build everything from explicitly defined sources seems to work.

[tokyo4]

It took quite a while but by copying the same approach @DeepBhupatkar took in his repo, I was able to get the Swift example to work.

Here is the patch:

diff --git a/Package.swift b/Package.swift
index 303ed0d..3b85875 100644
--- a/Package.swift
+++ b/Package.swift
@@ -14,6 +14,47 @@ let package = Package(
         .library(name: "whisper", targets: ["whisper"]),
     ],
     targets: [
-        .systemLibrary(name: "whisper", pkgConfig: "whisper"),
-    ]
+        .target(
+            name: "whisper",
+            path: ".",
+            exclude: [
+                "bindings",
+                "cmake",
+                "examples",
+                "models",
+                "samples",
+                "tests",
+                "CMakeLists.txt",
+                "Makefile",
+                "build"
+            ],
+            sources: [
+                "ggml/src/ggml-cpu/ggml-cpu.c",
+                "ggml/src/ggml-cpu/ggml-cpu.cpp",
+                "ggml/src/ggml-cpu/ggml-cpu-traits.cpp",
+                "ggml/src/ggml-threading.cpp",
+                "ggml/src/ggml-cpu/ggml-cpu-quants.c",
+                "ggml/src/ggml-backend-reg.cpp",
+                "ggml/src/ggml-backend.cpp",
+                "ggml/src/ggml.c",
+                "src/whisper.cpp",
+                "ggml/src/ggml-alloc.c",
+                "ggml/src/ggml-quants.c",
+                "ggml/src/ggml-metal/ggml-metal.m",
+            ],
+            resources: [.process("ggml/src/ggml-metal/ggml-metal.metal")],
+            publicHeadersPath: "spm-headers",
+            cSettings: [
+                .unsafeFlags(["-Wno-shorten-64-to-32", "-O3", "-DNDEBUG"]),
+                .define("GGML_USE_ACCELERATE"),
+                .unsafeFlags(["-fno-objc-arc"]),
+                .define("GGML_USE_METAL")
+            ],
+            linkerSettings: [
+                .linkedFramework("Accelerate")
+            ]
+            
+        )
+    ],
+    cxxLanguageStandard: .cxx17
 )
diff --git a/Sources/whisper/whisper.h b/Sources/whisper/whisper.h
index e503c8a..bf62952 100644
--- a/Sources/whisper/whisper.h
+++ b/Sources/whisper/whisper.h
@@ -1,4 +1,3 @@
 #pragma once
 
-#include <whisper.h>
-
+#include "whisper.h"
diff --git a/examples/whisper.swiftui/whisper.swiftui.demo/Resources/models/ggml-tiny.en.bin b/examples/whisper.swiftui/whisper.swiftui.demo/Resources/models/ggml-tiny.en.bin
new file mode 100644
index 0000000..9dd1a6b
Binary files /dev/null and b/examples/whisper.swiftui/whisper.swiftui.demo/Resources/models/ggml-tiny.en.bin differ
diff --git a/spm-headers/ggml-backend-impl.h b/spm-headers/ggml-backend-impl.h
new file mode 120000
index 0000000..10e864a
--- /dev/null
+++ b/spm-headers/ggml-backend-impl.h
@@ -0,0 +1 @@
+../ggml/src/ggml-backend-impl.h
\ No newline at end of file
diff --git a/spm-headers/ggml-common.h b/spm-headers/ggml-common.h
new file mode 120000
index 0000000..b9a3862
--- /dev/null
+++ b/spm-headers/ggml-common.h
@@ -0,0 +1 @@
+../ggml/src/ggml-common.h
\ No newline at end of file
diff --git a/spm-headers/ggml-cpu-impl.h b/spm-headers/ggml-cpu-impl.h
new file mode 120000
index 0000000..214b294
--- /dev/null
+++ b/spm-headers/ggml-cpu-impl.h
@@ -0,0 +1 @@
+../ggml/src/ggml-cpu/ggml-cpu-impl.h
\ No newline at end of file
diff --git a/spm-headers/ggml-impl.h b/spm-headers/ggml-impl.h
new file mode 120000
index 0000000..68c0d5c
--- /dev/null
+++ b/spm-headers/ggml-impl.h
@@ -0,0 +1 @@
+../ggml/src/ggml-impl.h
\ No newline at end of file
diff --git a/spm-headers/ggml-quants.h b/spm-headers/ggml-quants.h
new file mode 120000
index 0000000..fa470c4
--- /dev/null
+++ b/spm-headers/ggml-quants.h
@@ -0,0 +1 @@
+../ggml/src/ggml-quants.h
\ No newline at end of file
diff --git a/spm-headers/ggml-threading.h b/spm-headers/ggml-threading.h
new file mode 120000
index 0000000..307397c
--- /dev/null
+++ b/spm-headers/ggml-threading.h
@@ -0,0 +1 @@
+../ggml/src/ggml-threading.h
\ No newline at end of file
diff --git a/spm-headers/gguf.h b/spm-headers/gguf.h
new file mode 120000
index 0000000..66bfdd7
--- /dev/null
+++ b/spm-headers/gguf.h
@@ -0,0 +1 @@
+../ggml/include/gguf.h
\ No newline at end of file

The issue for me seems to be that the pkg-config based systemLibrary approach in the original Package.swift doesn't work on my computer. No clue why but it kept say "couldn't find pc file for whisper", even though build/whisper.pc was right there, and I kept adding it to PKG_CONFIG_PATH to no avail. Setting up the Package.swift to build everything from explicitly defined sources seems to work.

I still can't run it normal, could you please show more?thx

[sampullman]

The OP's method (https://github.com/DeepBhupatkar/swift-whisper.cpp-transcription) is the only one I could get working for both phone and simulator, after trying to follow the build instructions in another issue. The main problem is XCode not finding headers when compiling the whisper.cpp package, and normalizing everything to flat directory fixes that. It's not a good long term solution though, very annoying to keep up to date.