Add NCalc skills for language-based math problems #1464

lemillermicrosoft · 2023-06-13T23:43:17Z

Motivation and Context

This pull request adds two new skills for the Semantic Kernel project, using the NCalc library to evaluate mathematical expressions. The LanguageCalculatorSkill allows users to input natural language math problems and get the corresponding expression and result, while the SimpleCalculatorSkill evaluates simple arithmetic expressions. The skills use semantic functions to generate the expressions and evaluate them using the NCalc library. The skills also handle errors and edge cases gracefully.

Description

Add a new project NCalcSkills to the samples/dotnet folder, with a reference to the SemanticKernel project and the CoreCLR-NCalc package
Add a new class LanguageCalculatorSkill, which implements the ISkill interface and defines a Calculate function that takes a text input and returns a numerical output
The LanguageCalculatorSkill class uses a semantic function called _mathTranslator, which is initialized with a MathTranslatorPrompt constant that defines the task of translating a math problem into a NCalc expression, and some examples of input-output pairs
The Calculate function uses the _mathTranslator function to generate a NCalc expression from the text input, and then evaluates the expression using the NCalc.Evaluate method, which handles parameters and errors
Add a new class SimpleCalculatorSkill, which also implements the ISkill interface and defines a Calculate function that takes a simple arithmetic expression as input and returns a numerical output
The SimpleCalculatorSkill class uses a semantic function called _simpleCalculator, which evaluates the input expression using NCalc, without variables or advanced functions
Add some examples and documentation to the LanguageCalculatorSkill and SimpleCalculatorSkill classes, using the SKFunction, SKFunctionName, and SKFunctionInput attributes
Add the NCalcSkills project and the LanguageCalculatorSkill and SimpleCalculatorSkill classes to the SK-dotnet.sln solution file, and update the project GUIDs and dependencies accordingly

Contribution Checklist

The code builds clean without any errors or warnings
The PR follows SK Contribution Guidelines (https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
The code follows the .NET coding conventions (https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions) verified with dotnet format
All unit tests pass, and I have added new tests where possible
I didn't break anyone 😄

Co-authored-by: Almir Kazazic kaza@users.noreply.github.com

dmytrostruk

LGTM, left a couple of small comments, can be done in follow-up PRs.

dotnet/samples/NCalcSkills/LanguageCalculatorSkill.cs

This commit adds two new skills that use the NCalc library to evaluate mathematical expressions. The skills are: - LanguageCalculatorSkill: This skill uses a semantic function to translate natural language math problems into NCalc expressions, and then evaluates them using the NCalc engine. It supports more advanced math functions like sin/cosine/floor, and uses a regex to extract the expression from the semantic function output. - SimpleCalculatorSkill: This skill uses a simpler semantic function to evaluate basic math expressions like add, subtract, multiply and divide. It does not support variables or complex functions. The skills are defined in a separate project called NCalcSkills, which references the SemanticKernel project and the CoreCLR-NCalc package. The commit also updates the SK-dotnet solution file to include the NCalcSkills project.

This commit extracts the stop sequences for the math translator into a static field, to avoid creating a new array every time the SimpleCalculatorSkill constructor is called. This improves the performance and readability of the code.

This commit adds the CoreCLR-NCalc package as a dependency for the dotnet projects, which provides a library for evaluating mathematical expressions at runtime. This package is used by the NCalcSkills sample, which demonstrates how to create custom skills that use NCalc to perform calculations on input data. The NCalcSkills sample is moved from samples/dotnet/ncalc-skills to dotnet/samples/NCalcSkills, to follow the convention of other dotnet samples. The sample project is also updated to use the Directory.Packages.props file for managing the package versions, and to reference the SemanticKernel project using a relative path. The SK-dotnet.sln file is modified to reflect these changes and to fix the project GUIDs.

…kill The summary is: This commit fixes two issues in the LanguageCalculatorSkill class: - It corrects the output format of the expression template to include the word "problem" instead of a blank space. - It simplifies the exception handling logic by throwing an InvalidOperationException instead of creating a Task with an exception. This avoids unnecessary overhead and makes the code more consistent with the rest of the skill.

@kaza

### Motivation and Context This pull request adds a new extension for semantic planning using a stepwise approach. The StepwisePlanner extension allows users to create and execute plans that consist of a sequence of semantic and native functions, each with a goal and a set of inputs and outputs. The extension uses a semantic search engine to find relevant functions for each step, and a plan creation service to generate a plan that satisfies the user's ask. The extension also provides a system step function that executes the plan and returns the final answer and intermediate observations. The extension can be configured with various parameters, such as the relevancy threshold, the maximum number of relevant functions, the excluded and included functions and skills, and the maximum number of tokens, iterations, and time for the plan. Regarding #1472 ### Description - Add StepwisePlanner.cs, which registers the planner native functions and the system step function - Add StepwisePlannerConfig.cs, which defines the configuration options for the StepwisePlanner extension - Add SystemStep.cs, which represents a step in a Stepwise plan, with properties for the thought, action, action variables, observation, final answer, and original response - Add helper methods for formatting and validating function views, generating plan requests, and invoking the plan. - Add logging and error handling for the planner extension - Add unit tests for the planner extension and the native functions ### Related - Majority of work initially started from @kaza in #992 ### Changes in other PRs to merge separately - #1464 - #1465 - #1466 ### Contribution Checklist  - [x] The code builds clean without any errors or warnings - [x] The PR follows SK Contribution Guidelines (https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) - [x] The code follows the .NET coding conventions (https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions) verified with `dotnet format` - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Co-authored-by: Almir Kazazic <kaza@users.noreply.github.com> --------- Co-authored-by: Lee Miller <lemillermicrosoft@users.noreply.github.com>

@kaza

### Motivation and Context This pull request adds two new skills for the Semantic Kernel project, using the NCalc library to evaluate mathematical expressions. The LanguageCalculatorSkill allows users to input natural language math problems and get the corresponding expression and result, while the SimpleCalculatorSkill evaluates simple arithmetic expressions. The skills use semantic functions to generate the expressions and evaluate them using the NCalc library. The skills also handle errors and edge cases gracefully. ### Description - Add a new project NCalcSkills to the samples/dotnet folder, with a reference to the SemanticKernel project and the CoreCLR-NCalc package - Add a new class LanguageCalculatorSkill, which implements the ISkill interface and defines a Calculate function that takes a text input and returns a numerical output - The LanguageCalculatorSkill class uses a semantic function called _mathTranslator, which is initialized with a MathTranslatorPrompt constant that defines the task of translating a math problem into a NCalc expression, and some examples of input-output pairs - The Calculate function uses the _mathTranslator function to generate a NCalc expression from the text input, and then evaluates the expression using the NCalc.Evaluate method, which handles parameters and errors - Add a new class SimpleCalculatorSkill, which also implements the ISkill interface and defines a Calculate function that takes a simple arithmetic expression as input and returns a numerical output - The SimpleCalculatorSkill class uses a semantic function called _simpleCalculator, which evaluates the input expression using NCalc, without variables or advanced functions - Add some examples and documentation to the LanguageCalculatorSkill and SimpleCalculatorSkill classes, using the SKFunction, SKFunctionName, and SKFunctionInput attributes - Add the NCalcSkills project and the LanguageCalculatorSkill and SimpleCalculatorSkill classes to the SK-dotnet.sln solution file, and update the project GUIDs and dependencies accordingly ### Related - A piece taken from work from @kaza in microsoft#992 - In support of resolving microsoft#1472 ### Contribution Checklist  - [x] The code builds clean without any errors or warnings - [x] The PR follows SK Contribution Guidelines (https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) - [x] The code follows the .NET coding conventions (https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions) verified with `dotnet format` - [ ] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Co-authored-by: Almir Kazazic <kaza@users.noreply.github.com> --------- Co-authored-by: Lee Miller <lemillermicrosoft@users.noreply.github.com>

@kaza

### Motivation and Context This pull request adds a new extension for semantic planning using a stepwise approach. The StepwisePlanner extension allows users to create and execute plans that consist of a sequence of semantic and native functions, each with a goal and a set of inputs and outputs. The extension uses a semantic search engine to find relevant functions for each step, and a plan creation service to generate a plan that satisfies the user's ask. The extension also provides a system step function that executes the plan and returns the final answer and intermediate observations. The extension can be configured with various parameters, such as the relevancy threshold, the maximum number of relevant functions, the excluded and included functions and skills, and the maximum number of tokens, iterations, and time for the plan. Regarding microsoft#1472 ### Description - Add StepwisePlanner.cs, which registers the planner native functions and the system step function - Add StepwisePlannerConfig.cs, which defines the configuration options for the StepwisePlanner extension - Add SystemStep.cs, which represents a step in a Stepwise plan, with properties for the thought, action, action variables, observation, final answer, and original response - Add helper methods for formatting and validating function views, generating plan requests, and invoking the plan. - Add logging and error handling for the planner extension - Add unit tests for the planner extension and the native functions ### Related - Majority of work initially started from @kaza in microsoft#992 ### Changes in other PRs to merge separately - microsoft#1464 - microsoft#1465 - microsoft#1466 ### Contribution Checklist  - [x] The code builds clean without any errors or warnings - [x] The PR follows SK Contribution Guidelines (https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) - [x] The code follows the .NET coding conventions (https://learn.microsoft.com/dotnet/csharp/fundamentals/coding-style/coding-conventions) verified with `dotnet format` - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Co-authored-by: Almir Kazazic <kaza@users.noreply.github.com> --------- Co-authored-by: Lee Miller <lemillermicrosoft@users.noreply.github.com>

github-actions bot added .NET Issue or Pull requests regarding .NET code samples labels Jun 13, 2023

lemillermicrosoft force-pushed the 613_ncalc_skill branch from a861c22 to 2e61476 Compare June 14, 2023 01:29

lemillermicrosoft marked this pull request as ready for review June 14, 2023 01:33

lemillermicrosoft added the PR: ready for review All feedback addressed, ready for reviews label Jun 14, 2023

lemillermicrosoft mentioned this pull request Jun 14, 2023

Add StepwisePlanner extension for MRKL style planning #1468

Merged

5 tasks

lemillermicrosoft requested review from dmytrostruk, adrianwyatt and shawncal and removed request for dmytrostruk and adrianwyatt June 14, 2023 01:52

lemillermicrosoft self-assigned this Jun 14, 2023

lemillermicrosoft force-pushed the 613_ncalc_skill branch 4 times, most recently from 4137709 to f213d45 Compare June 16, 2023 21:19

lemillermicrosoft requested review from a team as code owners June 20, 2023 15:50

lemillermicrosoft force-pushed the 613_ncalc_skill branch from 0f2236b to 293af97 Compare June 21, 2023 02:19

github-actions bot removed .NET Issue or Pull requests regarding .NET code samples labels Jun 21, 2023

lemillermicrosoft force-pushed the 613_ncalc_skill branch 3 times, most recently from f8a2f37 to 1280a2f Compare June 21, 2023 19:23

dmytrostruk previously approved these changes Jun 22, 2023

View reviewed changes

dotnet/samples/NCalcSkills/LanguageCalculatorSkill.cs Outdated Show resolved Hide resolved

dotnet/samples/NCalcSkills/LanguageCalculatorSkill.cs Outdated Show resolved Hide resolved

dotnet/samples/NCalcSkills/LanguageCalculatorSkill.cs Show resolved Hide resolved

lemillermicrosoft added 3 commits June 22, 2023 08:58

🧮 Refactor stop sequences for math translator

f77b78d

This commit extracts the stop sequences for the math translator into a static field, to avoid creating a new array every time the SimpleCalculatorSkill constructor is called. This improves the performance and readability of the code.

lemillermicrosoft dismissed dmytrostruk’s stale review via 5ad6e88 June 22, 2023 16:04

lemillermicrosoft force-pushed the 613_ncalc_skill branch from 3c2f3a2 to 5ad6e88 Compare June 22, 2023 16:04

lemillermicrosoft enabled auto-merge June 22, 2023 16:07

dmytrostruk approved these changes Jun 22, 2023

View reviewed changes

lemillermicrosoft added this pull request to the merge queue Jun 22, 2023

Merged via the queue into microsoft:main with commit 74d5f8c Jun 22, 2023
10 checks passed

lemillermicrosoft deleted the 613_ncalc_skill branch June 22, 2023 16:12

lemillermicrosoft added this to the Sprint 33 milestone Jun 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add NCalc skills for language-based math problems #1464

Add NCalc skills for language-based math problems #1464

lemillermicrosoft commented Jun 13, 2023 •

edited

Loading

dmytrostruk left a comment

Add NCalc skills for language-based math problems #1464

Add NCalc skills for language-based math problems #1464

Conversation

lemillermicrosoft commented Jun 13, 2023 • edited Loading

Motivation and Context

Description

Related

Contribution Checklist

dmytrostruk left a comment

Choose a reason for hiding this comment

lemillermicrosoft commented Jun 13, 2023 •

edited

Loading