Ideas for LLM-specific components

[shreyashankar]

Suppose a component state represents all the sub-parts of a prompt. We should have the following utilities:

• Automatically convert the sub-parts (or a subset of sub-parts) into a prompt
• Support typed states (where the user can define the types of the values). Then we may not need to use cloudpickle
• Incremental summarizer, e.g., if the state contains a list that only grows in size, we should have a data type of PartialSummary that contains the running summary so far + num items that have been summarized + outstanding items to summarize.
• List-type sub-parts that can monotonically increase in size (can use CRDTs) for this? But this is only for when we can do updates concurrently.