Conversation
chmjkb
force-pushed
the
@chmjkb/text-embeddings-cpp-port
branch
from
0200048 to
2cfb189
Compare
chmjkb
force-pushed
the
@chmjkb/llm-cpp-port
branch
from
46d597e to
45938f3
Compare
msluszniak
reviewed
Jun 30, 2025
msluszniak
reviewed
Jun 30, 2025
msluszniak
reviewed
Jun 30, 2025
msluszniak
reviewed
Jun 30, 2025
msluszniak
reviewed
Jun 30, 2025
mkopcins
reviewed
Jun 30, 2025
| * SOFTWARE. | ||
| */ | ||
|
|
||
| // #include <executorch/extension/llm/sampler/sampler.h> |
Collaborator
Author
There was a problem hiding this comment.
I'm not exactly sure why that line was here in the first place. Below you can see a line:
#include "sampler.h"Which literally includes the same thing. Also, we can't access extension/llm/sampler/sampler.h since there is no such header in our code, so it wouldn't compile.
mkopcins
approved these changes
Jun 30, 2025
mkopcins
pushed a commit
that referenced
this pull request
Oct 15, 2025
## Description This PR ports the current LLM functionality to C++, getting rid of the `Runner` within the `ExecutorchLib` framework. I've also made some changes to the build system. `tokenizers-cpp` - Previously the library was linked in `ExecutorchLib` in XCode via a build script, now it is completely removed from the frameworks. - I've prebuiilt static libraries from `tokenizers-cpp` repo, which I uploaded to `common/ios/libs/tokenizers-cpp`, similarly to the pre-build ExecuTorch binaries. - The includes for tokenizers-cpp are now in `react-native-executorch/third-party/include/tokenizers-cpp/tokenizers_cpp.h` - Made some changes to the `libs` directory structure, please see the podspec for reference. - These headers are then included from the llama runner source code. - Since `tokenizers-cpp` for Android is pre-built with the `ExecuTorch` aar, I'm not making any changes here. This will need to be updated once we bump the ExecuTorch runtime and when we can safely get rid of the aar/jitpack setup. We can keep tokenizers-cpp as our submodule and then just reference it in Android's CMake. `runner` - The runner source along with headers was moved from `ExecuTorchLib` to `common/runner`, similarly to `ada` and is compiled on the fly when our library compiles. - In the current situation, I think that we will be soon able to get rid of the ET fork and the submodule. ### Type of change - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation update (improves or adds clarity to existing documentation) ### Tested on - [x] iOS - [x] Android ### Testing instructions <!-- Provide step-by-step instructions on how to test your changes. Include setup details if necessary. --> ### Screenshots <!-- Add screenshots here, if applicable --> ### Related issues <!-- Link related issues here using #issue-number --> ### Checklist - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation accordingly - [ ] My changes generate no new warnings ### Additional notes <!-- Include any additional information, assumptions, or context that reviewers might need to understand this PR. -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR ports the current LLM functionality to C++, getting rid of the
Runnerwithin theExecutorchLibframework. I've also made some changes to the build system.tokenizers-cppExecutorchLibin XCode via a build script, now it is completely removed from the frameworks.tokenizers-cpprepo, which I uploaded tocommon/ios/libs/tokenizers-cpp, similarly to the pre-build ExecuTorch binaries.react-native-executorch/third-party/include/tokenizers-cpp/tokenizers_cpp.hlibsdirectory structure, please see the podspec for reference.tokenizers-cppfor Android is pre-built with theExecuTorchaar, I'm not making any changes here. This will need to be updated once we bump the ExecuTorch runtime and when we can safely get rid of the aar/jitpack setup. We can keep tokenizers-cpp as our submodule and then just reference it in Android's CMake.runnerExecuTorchLibtocommon/runner, similarly toadaand is compiled on the fly when our library compiles.Type of change
Tested on
Testing instructions
Screenshots
Related issues
Checklist
Additional notes