Add Support for Liquid Template Engine in Skills

[musicalmathmind]

Summary

The purpose of this feature proposition is to introduce support for the Liquid template engine in the Semantic Kernel framework, allowing developers to create dynamic and reusable prompts for skills. This will enhance the flexibility and maintainability of skills and make it easier for developers to create complex, context-aware AI capabilities.

Background

Currently, the Semantic Kernel framework allows developers to build skills using LLM AI prompts, native computer code, or a hybrid of both. However, constructing and maintaining dynamic prompts can become cumbersome and challenging as the complexity of skills increases. The Liquid template engine, created by Shopify, offers a powerful and flexible solution to this problem, enabling developers to create dynamic templates with control flow, iteration, and variable interpolation.

Proposed Solution

Integrate the Liquid template engine into the Semantic Kernel framework. There is an open source library called Fluid that can parse templates and generate render the output. This could be integrated into the Kernel natively, so that any skill could use liquid templates.

Benefits

• Flexibility: The Liquid template engine allows developers to create more dynamic and context-aware prompts by supporting conditional rendering, loops, and variable interpolation.
• Maintainability: By using templates, developers can reduce code duplication and make it easier to update and manage prompts as requirements change.
• Reusability: Liquid templates encourage the creation of reusable prompts that can be shared across different skills or skill versions. As more ChatGPT like "intelligences" come online, this would also make these somewhat "cross-platform".

I'm willing to take the first shot at this after some discussion!

I'm also aware that this has the potential to be a single Skill that other skills can put in the pipeline, would love to discuss the pros and cons of each approach (making this a skill vs integrating it into the Kernel).

[dluc]

hi @vardthomas - Fluid was one of the options we considered before implementing a custom language (which is described here https://github.com/microsoft/semantic-kernel/blob/main/docs/PROMPT_TEMPLATE_LANGUAGE.md). One of the main reason for our custom implementation is the simplicity, e.g. a prompt is pretty much plain text, with only basic references to variables and functions. The simplicity of natural language is one of the most visible advancement with GPT3 and GPT4, which allow to process unstructured data and "plans".

Not sure whether Fluid is a skill, unless you're thinking about using AI to generate Fluid templates, which is totally fine, but different from using Fluid as a template engine SK. I.e. SK template engine is part of the Function Definition layer, which works at a lower level.

I think there are a few ideas we could experiment using Fluid, without replacing the basic template language:

1. using Fluid as a serialization format for the planner, see the work in progress here Planner: Plan object model #255
2. using Fluid as an additional format for the prompts. In this case we'd have to find the right way to detect when a prompt is written in Fluid rather than TXT. I would start looking at this interface https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel/TemplateEngine/IPromptTemplateEngine.cs implementing an equivalent of https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel/TemplateEngine/PromptTemplateEngine.cs for Fluid e.g. FluidPromptTemplateEngine.cs. Might be just an impression, but this might not be viable.
3. using Fluid to define prompts via https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel/SemanticFunctions/IPromptTemplate.cs, providing an alternative to https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel/SemanticFunctions/PromptTemplate.cs e.g. FluidPromptTemplate.cs. This should be easy, but you'd have to try to see if there are blockers or edge cases.

@lemillermicrosoft FYI - once POM is ready, would be nice checking if a Plan can be serialized to Fluid.

[dluc]

thanks for the suggestion, however for the time being we'll continue using the current simple template language. Once we have a complete AST component, we should revisit this topic. And we always welcome PRs or extensions (e.g. new nugets) providing alternative implementations of the interfaces published under the Microsoft.SemanticKernel.Abstractions package.