v0.1.3.1: Example Flow Pane

Make it very easy to import example flows:

example-flows.mov

Other additions:

• Added a "Compare System Prompts" example, with one-shot versus zero-shot versus "threaten a fictional kitten" examples. (See Twitter thread that inspired this use case: https://twitter.com/ShriramKMurthi/status/1664978520131477505?s=20 )
• Made it possible to switch between GPT3.5 and GPT4 after initially adding one of them (removed separation between these types of models)
• Improved the 'add node' UI to look sleeker

v0.1.3: Model Settings (+ more models)

Proud to announce we now have model settings in ChainForge. 🥳
You can now compare across different versions of the same model, in addition to nicknaming models and choosing more specific models.

To install, do pip install chainforge --upgrade. Full changelog below.

More supported models 🤖

Along with model settings, we now have support for all OpenAI, Anthropic, Google PaLM (chat and text), Dalai-hosted models. For instance, you can now compare Llama.65B to PaLM text completions, if you were so inclined. For the full list, see models.py.

Here is comparing Google PaLM's text-bison to chat-bison for the same prompt:

Customizable model settings (and emojis! 😑)

Once you add a model to a PromptNode, now you can tap the 'settings' icon on a PromptNode to bring up a form with all settings for that base model. You can adjust the exact model used (for instance, text-bison-001 in PaLM, or Dalai-hosted llama.30B):

Temperature appears next to model names by default. For ease of reference, temperature is displayed on a sliding color scale from cyan #00ffff (coldest) to violet #ff00ff (lukewarm) to red #ff0000 (hottest). The percentage respects min and max temperature settings for individual models.

You can now also nickname models in PromptNodes. Names must be unique. Each nickname will appear elsewhere in Chainforge (e.g. in plots). You can also set the Emoji used. For instance, here is a comparison between two ChatGPT models at different temperatures, which I've renamed hotgpt and coldgpt with the emojis 🔥 and 🥶:

Note about importing previous flows

Unfortunately, this code rewrite involved a breaking change for how flows are imported and exported (.cforge file format). You may still be able to import old flows, but you need to re-populate each model list and re-query LLMs. I hope to avoid this, but in this case it was necessary to store model settings information and redo how the backend cache's responses.

Note about Dalai-hosted models

Currently, you cannot query multiple Dalai models/settings at once, since a locally run model can only take one request at a time. We're working on fixing this for the next minor release; for now, just choose one model at a time, and if you want more than one, add it to the list and re-query the prompt node (it will use the previously cache'd responses from the first Dalai model).

Encounter any bugs?

There was a lot to change for this release, and it's likely that at least one thing broke in the process that we haven't detected. If you encounter a bug or problem, open an Issue or respond to the Discussion about this release! 👍

Minor UI fixes to InspectNode variable headers and template variable Badges

This release contains two minor but important changes:

• Template variable hooks in PromptNode and TextFieldNode no longer auto-uppercase the template variable name by default:

• Selected variables in InspectNode now display up to 144 characters of their value (which is much more informative than the previous 12 characters in uppercase):

We are working on custom model settings for the next major release (0.1.3), so you can change the temperature and other settings of individual models. We are structuring the code to be easily extensible to add more (hopefully much more) models in the future, including user-specified models and settings forms (through react-json-schema).