Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated phi-3 mobile android example #420

Merged
merged 17 commits into from
May 14, 2024
Merged

Conversation

YUNQIUGUO
Copy link
Contributor

@YUNQIUGUO YUNQIUGUO commented Apr 30, 2024

This is based on the following two prs:

The GenAI lib used in the current version of this app is built from source code for android from onnxruntime-genai repo and uses the current main version of GenAI C API source code.

microsoft/onnxruntime-genai#227
microsoft/onnxruntime-genai#229

@YUNQIUGUO
Copy link
Contributor Author

YUNQIUGUO commented Apr 30, 2024

TODO: still a couple commits might be pushed to this branch for clean up/minor updates.

@pxz2016
Copy link

pxz2016 commented May 1, 2024

app crash when use chinese:
gotNextToken:
2024-05-01 20:28:37.476 9113-9185 GenAI ai.onnxruntime.genai.demo I gotNextToken: 你
2024-05-01 20:28:37.661 9113-9185 GenAI ai.onnxruntime.genai.demo I gotNextToken: 好
2024-05-01 20:28:37.848 9113-9185 GenAI ai.onnxruntime.genai.demo I gotNextToken: !
2024-05-01 20:28:38.036 9113-9185 time.genai.demo ai.onnxruntime.genai.demo A java_vm_ext.cc:594] JNI DETECTED ERROR IN APPLICATION: input is not valid Modified UTF-8: illegal continuation byte 0
java_vm_ext.cc:594] string: '?'
java_vm_ext.cc:594] input: '0xe5'
java_vm_ext.cc:594] in call to NewStringUTF
java_vm_ext.cc:594] from java.lang.String ai.onnxruntime.genai.demo.GenAIWrapper.run(long, long, java.lang.String, boolean)
2024-05-01 20:28:38.424 9113-9185 time.genai.demo ai.onnxruntime.genai.demo A runtime.cc:675] Runtime aborting...

@ithraasoft-dev
Copy link

Got this error

                                                                                               java.lang.RuntimeException: Unable to start activity ComponentInfo{ai.onnxruntime.genai.demo/ai.onnxruntime.genai.demo.MainActivity}: ai.onnxruntime.genai.demo.GenAIException: Deserialize tensor model.layers.29.attn.o_proj.MatMul.weight_Q4 failed.tensorprotoutils.cc:904 GetExtDataFromTensorProto External initializer: model.layers.29.attn.o_proj.MatMul.weight_Q4 offset: 2466656256 size to read: 4718592 given file_length: 2334412800 are out of bounds or can not be read in full.

@YUNQIUGUO
Copy link
Contributor Author

Got this error

                                                                                               java.lang.RuntimeException: Unable to start activity ComponentInfo{ai.onnxruntime.genai.demo/ai.onnxruntime.genai.demo.MainActivity}: ai.onnxruntime.genai.demo.GenAIException: Deserialize tensor model.layers.29.attn.o_proj.MatMul.weight_Q4 failed.tensorprotoutils.cc:904 GetExtDataFromTensorProto External initializer: model.layers.29.attn.o_proj.MatMul.weight_Q4 offset: 2466656256 size to read: 4718592 given file_length: 2334412800 are out of bounds or can not be read in full.

Are you trying this on an actual device or in android emulator?

YUNQIUGUO and others added 11 commits May 13, 2024 16:38
Co-authored-by: Scott McKay <skottmckay@gmail.com>
Co-authored-by: Scott McKay <skottmckay@gmail.com>
Co-authored-by: Scott McKay <skottmckay@gmail.com>
Co-authored-by: Scott McKay <skottmckay@gmail.com>
…/genai/demo/GenAIWrapper.java

Co-authored-by: Scott McKay <skottmckay@gmail.com>
…/genai/demo/GenAIWrapper.java

Co-authored-by: Scott McKay <skottmckay@gmail.com>
…/genai/demo/GenAIWrapper.java

Co-authored-by: Scott McKay <skottmckay@gmail.com>
@YUNQIUGUO YUNQIUGUO merged commit ff7ee24 into main May 14, 2024
18 of 26 checks passed
@YUNQIUGUO YUNQIUGUO deleted the yguo/add-phi3-android-example branch May 14, 2024 05:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants