If you encounter the error 'Couldn't resolve native function 'llama_context_default_params' on macOS ARM64, please refer to Issue #2 for details and updates. Contributions and insights are welcome.
llama_dart is an experimental Flutter plugin designed to seamlessly integrate the capabilities of llama.cpp into your Flutter or Dart apps. Please note that this package is currently in development and may not be fully functional.
Important: Native assets support in Flutter MacOS, iOS, and Linux is available on the master channel behind an experimental flag: flutter config --enable-native-assets and flutter create --template=package_ffi [package name]. See this comment for more details.
Here are the steps to run llama_dart:
- Switch to the master channel:
flutter channel master - Upgrade Flutter:
flutter upgrade - Enable native assets:
flutter config --enable-native-assets - Create a package using the
ffitemplate:flutter create --template=package_ffi llama_dart - Navigate to the project folder:
cd llama_dart - Get dependencies:
flutter pub get - Run the app:
flutter run - Generate bindings using Dart SDK (temporary workaround):
dart --enable-experiment=native-assets run ffigen --config ffigen.yaml
Please note that step 8 currently requires using the Dart SDK instead of Flutter until the Flutter team completes implementing the feature in the Flutter SDK.
llama_dart aims to support various use cases, including:
- Running the app with
flutter run - Debugging
- Hot reload (1)
- Hot restart (1)
- Release/profile builds
- Building debug/release/profile builds with
flutter build - Running tests with
flutter testordart test - Adding to an app (e.g.,
flutter build macos-frameworkfor MacOS) - Running with the Flutter tester:
flutter run -dflutter-tester
Please note that the plugin is still under development, and not all use cases may be fully supported at this time.
- MacOS
- Windows
- Linux
- Android
- iOS
Note: This list represents our intended platform support after addressing the initial issues on MacOS ARM64. Support for other platforms may be added in the future.
Stay tuned for updates! llama_dart is meant to make it easier to incorporate llama.cpp capabilities into your Flutter or Dart apps, and we're actively working on improving its functionality.
If you encounter any issues or have additional information that could help, please feel free to contribute to the discussion. Your insights and experiences are valuable in addressing this problem effectively.