Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WASI-NN] ggml: support grammar #3318

Merged
merged 2 commits into from
Apr 3, 2024
Merged

[WASI-NN] ggml: support grammar #3318

merged 2 commits into from
Apr 3, 2024

Conversation

dm4
Copy link
Collaborator

@dm4 dm4 commented Apr 2, 2024

Signed-off-by: dm4 <dm4@secondstate.io>
Copy link
Member

juntao commented Apr 2, 2024

Hello, I am a code review bot on flows.network. Here are my reviews of code commits in this PR.


Commit 1065aaccb5a8754648c90d5086995e48e454e415

This patch adds support for a grammar attribute in the Graph structure used by the WASI-NN GGML backend.

The following changes have been made:

  1. A new conditional block has been introduced in the parseMetadata function at plugins/wasi_nn/ggml.cpp. This block checks if a "grammar" attribute is present in the JSON document. If this key is detected and can be retrieved as a string, the Graph's Grammar attribute is set to this value. If an error happens during the retrieval of this key, a log message is output and ErrNo::InvalidArgument is returned.

  2. The setupGPTParam function has been modified to include the Graph's Grammar attribute into the GPTParams.sparams.grammar.

  3. In the load function, a default value for GraphRef.Grammar is fetched from SamplingDefault.grammar.

  4. The Graph structure at plugins/wasi_nn/ggml.h has been expanded with a std::string Grammar member.

Potential Problems:

  1. An assumption is made that the "grammar" attribute will be a string. If there is a chance this assumption does not always hold, additional error handling is required.
  2. No validation appears to be made for the actual contents of the "grammar" string. This might be an issue if certain grammar rules or structures are expected.
  3. There's no null or empty string check for the "grammar" string; this might introduce errors or unexpected behavior if handled improperly later.
  4. The parsing or loading of this new field has not been captured in any testing scenario. It would be beneficial to write tests to ensure this new field works as expected.
  5. There's no context provided as to what the Grammar string represents, there should ideally be some comments explaining its purpose to improve code readability.

Copy link

codecov bot commented Apr 2, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.28%. Comparing base (0c65d8d) to head (3b76f0d).
Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3318      +/-   ##
==========================================
- Coverage   77.29%   77.28%   -0.01%     
==========================================
  Files         185      185              
  Lines       25614    25614              
  Branches     5651     5651              
==========================================
- Hits        19799    19797       -2     
- Misses       4386     4388       +2     
  Partials     1429     1429              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dm4 dm4 marked this pull request as ready for review April 3, 2024 12:45
@dm4 dm4 requested a review from ibmibmibm as a code owner April 3, 2024 12:45
@dm4 dm4 merged commit db5b691 into master Apr 3, 2024
73 checks passed
@dm4 dm4 deleted the dm4/grammars branch April 3, 2024 13:36
@hydai hydai added the v-0.14.0 label Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c-Plugin An issue related to WasmEdge Plugin c-WASI-NN v-0.14.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants