v0.16.0

PromptingTools v0.16.0

Diff since v0.15.0

Added

• Added pretty-printing via PT.pprint that does NOT depend on Markdown and splits text to adjust to the width of the output terminal.
  It is useful in notebooks to add newlines.
• Added support annotations for RAGTools (see ?RAGTools.Experimental.annotate_support for more information) to highlight which parts of the generated answer come from the provided context versus the model's knowledge base. It's useful for transparency and debugging, especially in the context of AI-generated content. You can experience it if you run the output of airag through pretty printing (PT.pprint).
• Added utility distance_longest_common_subsequence to find the normalized distance between two strings (or a vector of strings). Always returns a number between 0-1, where 0 means the strings are identical and 1 means they are completely different. It's useful for comparing the similarity between the context provided to the model and the generated answer.
• Added a new documentation section "Extra Tools" to highlight key functionality in various modules, eg, the available text utilities, which were previously hard to discover.
• Extended documentation FAQ with tips on tackling rate limits and other common issues with OpenAI API.
• Extended documentation with all available prompt templates. See section "Prompt Templates" in the documentation.
• Added new RAG interface underneath airag in PromptingTools.RAGTools.Experimental. Each step now has a dedicated function and a type that can be customized to achieve arbitrary logic (via defining methods for your own types). airag is split into two main steps: retrieve and generate!. You can use them separately or together. See ?airag for more information.

Updated

• Renamed split_by_length text splitter to recursive_splitter to make it easier to discover and understand its purpose. split_by_length is still available as a deprecated alias.

Fixed

• Fixed a bug where LOCAL_SERVER default value was not getting picked up. Now, it defaults to http://localhost:10897/v1 if not set in the preferences, which is the address of the server started by Llama.jl.
• Fixed a bug in multi-line code annotation, which was assigning too optimistic scores to the generated code. Now the score of the chunk is the length-weighted score of the "top" source chunk divided by the full length of score tokens (much more robust and demanding).

Commits

Merged pull requests:

• Update docs to Vitepress (#88) (@svilupp)
• Add support annotations (#90) (@svilupp)
• Update Documentation (#91) (@svilupp)
• Add Prompt Templates to the Docs (#92) (@svilupp)
• fix typo on set_preferences! examples, fixes #93 (#94) (@ceferisbarov)
• Rag Interface Rewrite (#95) (@svilupp)

Closed issues:

• Wrong syntax in README (#93)

v0.15.0

PromptingTools v0.15.0

Diff since v0.14.0

Added

• Added experimental support for image generation with OpenAI DALL-E models, eg, msg = aiimage("A white cat on a car"). See ?aiimage for more details.

Commits

Merged pull requests:

• Add image generation with DALL-E 3 (#86) (@svilupp)
• Update Changelog (#87) (@svilupp)

v0.14.0

PromptingTools v0.14.0

Diff since v0.13.0

Added

• Added a new documentation section "How it works" to explain the inner workings of the package. It's a work in progress, but it should give you a good idea of what's happening under the hood.
• Improved template loading, so if you load your custom templates once with load_templates!("my/template/folder), it will remember your folder for all future re-loads.
• Added convenience function create_template to create templates on the fly without having to deal with PT.UserMessage etc. If you specify the keyword argument load_as = "MyName", the template will be immediately loaded to the template registry. See ?create_template for more information and examples.

Commits

Merged pull requests:

• Templating utilities (#84) (@svilupp)
• Improve create_template functionality (#85) (@svilupp)

v0.13.0

PromptingTools v0.13.0

Diff since v0.12.0

Added

• Added initial support for Google Gemini models for aigenerate (requires environment variable GOOGLE_API_KEY and package GoogleGenAI.jl to be loaded). It must be loaded explicitly as it's not yet registered.
• Added a utility to compare any two string sequences (and other iterators)length_longest_common_subsequence. It can be used to fuzzy match strings (eg, detecting context/sources in an AI-generated response or fuzzy matching AI response to some preset categories). See the docstring for more information ?length_longest_common_subsequence.
• Rewrite of aiclassify to classify into an arbitrary list of categories (including with descriptions). It's a quick and easy option for "routing" and similar use cases, as it exploits the logit bias trick and outputs only 1 token. Currently, only OpenAISchema is supported. See ?aiclassify for more information.
• Initial support for multiple completions in one request for OpenAI-compatible API servers. Set via API kwarg n=5 and it will request 5 completions in one request, saving the network communication time and paying the prompt tokens only once. It's useful for majority voting, diversity, or challenging agentic workflows.
• Added new fields to AIMessage and DataMessage types to simplify tracking in complex applications. Added fields:
  • cost - the cost of the query (summary per call, so count only once if you requested multiple completions in one call)
  • log_prob - summary log probability of the generated sequence, set API kwarg logprobs=true to receive it
  • run_id - ID of the AI API call
  • sample_id - ID of the sample in the batch if you requested multiple completions, otherwise sample_id==nothing (they will have the same run_id)
  • finish_reason - the reason why the AI stopped generating the sequence (eg, "stop", "length") to provide more visibility for the user
• Support for Fireworks.ai and Together.ai providers for fast and easy access to open-source models. Requires environment variables FIREWORKS_API_KEY and TOGETHER_API_KEY to be set, respectively. See the ?FireworksOpenAISchema and ?TogetherOpenAISchema for more information.
• Added an extra field to ChunkIndex object for RAG workloads to allow additional flexibility with metadata for each document chunk (assumed to be a vector of the same length as the document chunks).
• Added airetry function to PromptingTools.Experimental.AgentTools to allow "guided" automatic retries of the AI calls (eg, AIGenerate which is the "lazy" counterpart of aigenerate) if a given condition fails. It's useful for robustness and reliability in agentic workflows. You can provide conditions as functions and the same holds for feedback to the model as well. See a guessing game example in ?airetry.

Updated

• Updated names of endpoints and prices of Mistral.ai models as per the latest announcement and pricing. Eg, mistral-small -> mistral-small-latest. In addition, the latest Mistral model has been added mistral-large-latest (aliased as mistral-large and mistrall, same for the others). mistral-small-latest and mistral-large-latest now support function calling, which means they will work with aiextract (You need to explicitly provide tool_choice, see the docs ?aiextract).

Removed

• Removed package extension for GoogleGenAI.jl, as it's not yet registered. Users must load the code manually for now.

Commits

Merged pull requests:

• Add google api (#75) (@svilupp)
• Add LCS matching for strings (#76) (@svilupp)
• Aiclassify arbitrary choices (#78) (@svilupp)
• Multiple competions (n) (#79) (@svilupp)
• Add more APIs (#80) (@svilupp)
• Add airetry! (#82) (@svilupp)
• Remove GoogleGenAI (#83) (@svilupp)

v0.12.0

PromptingTools v0.12.0

Diff since v0.11.0

Added

• Added more specific kwargs in Experimental.RAGTools.airag to give more control over each type of AI call (ie, aiembed_kwargs, aigenerate_kwargs, aiextract_kwargs)
• Move up compat bounds for OpenAI.jl to 0.9

Fixed

• Fixed a bug where obtaining an API_KEY from ENV would get precompiled as well, causing an error if the ENV was not set at the time of precompilation. Now, we save the get(ENV...) into a separate variable to avoid being compiled away.

Commits

Merged pull requests:

• Update AIrag kwargs (#74) (@svilupp)

v0.11.0

PromptingTools v0.11.0

Diff since v0.10.0

Added

• Support for Databricks Foundation Models API. Requires two environment variables to be set: DATABRICKS_API_KEY and DATABRICKS_HOST (the part of the URL before /serving-endpoints/)
• Experimental support for API tools to enhance your LLM workflows: Experimental.APITools.create_websearch function which can execute and summarize a web search (incl. filtering on specific domains). It requires TAVILY_API_KEY to be set in the environment. Get your own key from Tavily - the free tier enables c. 1000 searches/month, which should be more than enough to get started.

Fixed

• Added an option to reduce the "batch size" for the embedding step in building the RAG index (build_index, get_embeddings). Set embedding_kwargs = (; target_batch_size_length=10_000, ntasks=1) if you're having some limit issues with your provider.
• Better error message if RAGTools are only partially imported (requires LinearAlgebra and SparseArrays to load the extension).

### Commits

Merged pull requests:

• Update to Codecov4 (#70) (@svilupp)
• Add Databricks API support (#71) (@svilupp)
• Embedding API (#72) (@svilupp)
• Add Tavily api (#73) (@svilupp)

v0.10.0

PromptingTools v0.10.0

Diff since v0.9.0

Added

• [BREAKING CHANGE] The default embedding model (MODEL_EMBEDDING) changes to "text-embedding-3-small" effectively immediately (lower cost, higher performance). The default chat model (MODEL_CHAT) will be changed by OpenAI to 0125 (from 0613) by mid-February. If you have older embeddings or rely on the exact chat model version, please set the model explicitly in your code or in your preferences.
• New OpenAI models added to the model registry (see the release notes).
  • "gpt4t" refers to whichever is the latest GPT-4 Turbo model ("gpt-4-0125-preview" at the time of writing)
  • "gpt3t" refers to the latest GPT-3.5 Turbo model version 0125, which is 25-50% cheaper and has updated knowledge (available from February 2024, you will get an error in the interim)
  • "gpt3" still refers to the general endpoint "gpt-3.5-turbo", which OpenAI will move to version 0125 by mid-February (ie, "gpt3t" will be the same as "gpt3" then. We have reflected the approximate cost in the model registry but note that it will be incorrect in the transition period)
  • "emb3small" refers to the small version of the new embedding model (dim=1536), which is 5x cheaper than Ada and promises higher quality
  • "emb3large" refers to the large version of the new embedding model (dim=3072), which is only 30% more expensive than Ada
• Improved AgentTools: added more information and specific methods to aicode_feedback and error_feedback to pass more targeted feedback/tips to the AIAgent
• Improved detection of which lines were the source of error during AICode evaluation + forcing the error details to be printed in AICode(...).stdout for downstream analysis.
• Improved detection of Base/Main method overrides in AICode evaluation (only warns about the fact), but you can use detect_base_main_overrides(code) for custom handling

Fixed

• Fixed typos in the documentation
• Fixed a bug when API keys set in ENV would not be picked up by the package (caused by inlining of the get(ENV,...) during precompilation)
• Fixed string interpolation to be correctly escaped when evaluating AICode

### Commits

Merged pull requests:

• Fix tests on model costs (#58) (@svilupp)
• Re-apply format (#59) (@svilupp)
• Add devcontainer.json (#60) (@svilupp)
• Fix API key getter with @noinline (#61) (@svilupp)
• Improve code feedback (#62) (@svilupp)
• Improve error capture + error lines capture (#63) (@svilupp)
• Escape fix in code loading (#64) (@svilupp)
• Detect Base method overrides (#65) (@svilupp)
• Tag v0.10 (#66) (@svilupp)

Closed issues:

• ERROR: ArgumentError: api_key cannot be empty (#57)

v0.9.0

PromptingTools v0.9.0

Diff since v0.8.1

### Added

• Split Experimental.RAGTools.build_index into smaller functions to easier sharing with other packages (get_chunks, get_embeddings, get_metadata)
• Added support for Cohere-based RAG re-ranking strategy (and introduced associated COHERE_API_KEY global variable and ENV variable)

### Commits

Merged pull requests:

• Refactor RAGTools (#56) (@svilupp)

v0.8.1

PromptingTools v0.8.1

Diff since v0.8.0

Fixed

• Fixed split_by_length to not mutate separators argument (appeared in RAG use cases where we repeatedly apply splits to different documents)

Commits

Merged pull requests:

• Fix separators in split_by_length (#55) (@svilupp)

v0.8.0

PromptingTools v0.8.0

Diff since v0.7.0

Added

• Initial support for Llama.jl and other local servers. Once your server is started, simply use model="local" to route your queries to the local server, eg, ai"Say hi!"local. Option to permanently set the LOCAL_SERVER (URL) added to preference management. See ?LocalServerOpenAISchema for more information.
• Added a new template StorytellerExplainSHAP (see the metadata)

Fixed

• Repeated calls to Ollama models were failing due to missing prompt_eval_count key in subsequent calls.

Commits

Merged pull requests:

• Fix typos (#49) (@pitmonticone)
• Add LocalServerOpenAISchema to support Llama.jl (#50) (@svilupp)
• Fix ollama repeated calls (#52) (@svilupp)
• New template and version update (#53) (@svilupp)

Closed issues:

• Ollama: repeated request with same prompt fails (#51)