Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding a Return Format Checker Chain #2470

Closed

Conversation

girishkc24
Copy link

Problem:
LLMs can be given a scenario and be asked to make a decision based on the provided facts. This is especially helpful in agent based simulations where LLMs can simulate individual agents. To be able to use LLMs in these use cases, it's critical that they provide response actions that are not only valid, but formatted correctly. Given the large context window of these scenarios, it's often the case that the LLM provides an invalid or incorrectly formatted decision.

Solution:
This chain helps format and validate actions that LLMs provide in response to situations. It does so by breaking up the prompting into multiple stages

  1. Presenting the scenario and asking for the chosen action
  2. Checking the validity of the action, and, if invalid, asking for a valid action
  3. Formatting the action into the correct format
  4. Checking the format, and, if invalid, asking for the properly formatted action

It's been pretty useful in making sure that the agents in these ABSs are producing good feedback without having to resort to writing more deterministic return format validators or try-except blocks.

@dosubot dosubot bot added the 🤖:enhancement A large net-new component, integration, or chain. Use sparingly. The largest features label Jul 14, 2023
@leo-gan
Copy link
Collaborator

leo-gan commented Sep 13, 2023

@girishkc24 Hi , could you, please, resolve the merging issues and address the last comments (if needed)? After that ping me and I push this PR for the review. Thanks!

If this PR is not needed anymore, could you, please, let me know?

@GirishChandrasekar
Copy link

It's not needed anymore, thanks!

@leo-gan
Copy link
Collaborator

leo-gan commented Sep 13, 2023

Closing.
@girishkc24 Thanks for your thoughts and effort! Really appreciated!

@leo-gan leo-gan closed this Sep 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖:enhancement A large net-new component, integration, or chain. Use sparingly. The largest features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants