Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.Net: Handlebars Planner Failing to Parse JSON Array #4721

Closed
rithvikb23 opened this issue Jan 24, 2024 · 4 comments
Closed

.Net: Handlebars Planner Failing to Parse JSON Array #4721

rithvikb23 opened this issue Jan 24, 2024 · 4 comments
Assignees
Labels
.NET Issue or Pull requests regarding .NET code

Comments

@rithvikb23
Copy link

I have an openapi plugin to get a list of people that hold a certain asset. When loading the openapi spec the handlebars planner is failing to parse the plan as the handlebars planner is incorrectly parsing the template. The openapi spec is rather complex with many fields and complex data types, but is still appropriately formatted and I have tested it in a swagger editor however the plan is failing. The planner is failing to parse the json array in step 2 below.

Here is the current step by step by the planner:
Goal: "What is my exposure to AAPL, id 123"

Handlebars Plan:
{{!-- Step 1: Set your necessary parameters --}}
{{set "id" "123"}}

{{!-- Step 2: Set your symbol(AAPL) in a fieldValues array --}}
{{set "fieldValues" (array "AAPL")}}

{{!-- Step 3: Set your applied filters --}}
{{set "appliedFiltersArray" (array (json{ "op": "EQLT", "field": "exposure_ss", "id": "1", "fieldValues": fieldValues}))}}

{{!-- Step 4: Invoke the PositionCrossReferencePlugin helper and store the result --}}
{{set "response" (PositionCrossReferencePlugin-getPositionCrossReference id=id faId=(array faId) appliedFilters=appliedFiltersArray)}}

Expected behavior
When loading this saved plan the issue is in step 2 with parsing the json array. I am given the error
Plan creation failed. Error: Reached unparseable token in expression: )}}
HandlebarsDotNet.HandlebarsParserException: Reached unparseable token in expression: )}}

  • OS: Windows
  • IDE: Visual Studio
  • Language: C#
  • Source: NuGet package version 1.1.0, pip package

Are json array handling in the handlebars planner currently not supported? Or is there a workaround?

@shawncal shawncal added .NET Issue or Pull requests regarding .NET code triage labels Jan 24, 2024
@github-actions github-actions bot changed the title Handlebars Planner Failing to Parse JSON Array .Net: Handlebars Planner Failing to Parse JSON Array Jan 24, 2024
@joslat
Copy link
Contributor

joslat commented Feb 13, 2024

@rithvikb23 did you find any workaround to this?

@gitri-ms gitri-ms assigned teresaqhoang and unassigned gitri-ms Feb 14, 2024
@teresaqhoang
Copy link
Contributor

teresaqhoang commented Feb 16, 2024

hi @rithvikb23 The error is a syntax error by the model here: json{ "op": "EQLT", "field": "exposure_ss", "id": "1", "fieldValues": fieldValues}. I wondering if adding an example directly to the json helper definition in the prompt will help.

We recently added more direction to better guide the model in creating initial variables. Can you pull the latest changes and see if your planner results have improved?

If not, please reply to this thread with:

  • openapi spec
  • The CreatePlan prompt template
  • Resulting plan

And I'll help troubleshoot

@teresaqhoang
Copy link
Contributor

Hi @rithvikb23 following up to see if you're still seeing this issue

github-merge-queue bot pushed a commit that referenced this issue Feb 26, 2024
… type handling (#5137)

### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Resolves 
- #4751 
- #4250 
- #4721

This PR contains fixes for template extraction from model results;
handling case where schema and parameter type can co-exist; + prompt
tuning

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
@teresaqhoang
Copy link
Contributor

Closing as stale - feel free to re-open if issue persists

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
.NET Issue or Pull requests regarding .NET code
Projects
Archived in project
Development

No branches or pull requests

6 participants