Skip to content

Commit

Permalink
Minor clean-up and update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
thekaranacharya committed Feb 26, 2024
1 parent 59698d8 commit 158371c
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 10 deletions.
31 changes: 22 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

## Description

This validator checks if an LLM-generated text is gibberish/non-sensical. It can validate either sentence by sentence or the entire text as a whole.
This validator validates the "cleanliness" of the text generated by a language model. It uses a pre-trained model to determine if the text is coherent and not gibberish. The validator can be used to filter out text that is not coherent or does not make sense.

## Requirements
- Dependencies: `nltk`, `transformers`, `torch`
Expand All @@ -32,19 +32,32 @@ In this example, we use the `gibberish_text` validator on any LLM generated text
from guardrails.hub import GibberishText
from guardrails import Guard

# Initialize Validator
val = GibberishText(threshold=0.75, validation_method="sentence", on_fail="fix")
# Use the Guard with the validator
guard = Guard().use(
GibberishText, threshold=0.5, validation_method="sentence", on_fail="exception"
)

# Setup Guard
guard = Guard.from_string(
validators=[val, ...],
# Test passing response
guard.validate(
"Azure is a cloud computing service created by Microsoft. It's a significant competitor to AWS."
)

# Pass LLM output through guard
guard.parse("Zoom is a great video conferencing tool. It's very easy to use.") # Pass
guard.parse("The quick brown fox jumps over the lazy dog. Fox fox fox fox fox. Floppyland gsdkd%$klsdml") # Fail
try:
# Test failing response
guard.validate(
"Floppyland love great coffee okay. Fox fox fox. Move to New York City."
)
except Exception as e:
print(e)
```
Output:
```console
Validation failed for field with errors: The following sentences in your response were found to be gibberish:

- Floppyland love great coffee okay.
- Fox fox fox.
```
**Note: See how only the first 2 sentences within the failing response are considered gibberish.**

## API Reference

Expand Down
2 changes: 1 addition & 1 deletion validator/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ def validate_each_sentence(
return FailResult(
metadata=metadata,
error_message=(
f"The following sentences in your response"
f"The following sentences in your response "
"were found to be gibberish:\n"
f"\n{unsupported_sentences}"
),
Expand Down

0 comments on commit 158371c

Please sign in to comment.