Support asynchronous API call of OpenAIBasedEvaluator.get_score()

[Alnusjaponica]

Motivation

The bottleneck of the OpenAI-based evaluator is IO time. This PR introduces an asynchronous API call option for each metric evaluation.

[Alnusjaponica]

I ran the following code in my local environment and get this result:

import time

from dotenv import load_dotenv
from langcheck.metrics.en import toxicity

load_dotenv()

generated_outputs = [
    "You look beautiful",
    "You look ugly",
    "You are a good person",
    "You are a bad person",
    "I have a pen",
    "I have an apple",
    "I have a pine apple",
    # "I have a gun", # Intentionally cause an error.
]

# Async
start = time.time()
toxicity_value = toxicity(
    generated_outputs,
    model_type="azure_openai",
    openai_args={"model": "gpt-4-turbo"},
    use_async=True,
)
print(f'Async call: {time.time() - start}[s]')

# No async
start = time.time()
toxicity_value = toxicity(
    generated_outputs,
    model_type="azure_openai",
    openai_args={"model": "gpt-4-turbo"},
    use_async=False,
)
print(f'No async call: {time.time() - start}[s]')

Async call: 27.088413953781128[s]
Intermediate assessments (1/2): 100%|████████████████| 7/7 [01:09<00:00,  9.96s/it]
Scores (2/2): 100%|█████████████████████████| 7/7 [00:17<00:00,  2.44s/it]
No async call: 86.82365489006042[s]

[liwii]

Nice! It worked well in my environment too!

Could you also edit the remaining metrics (e.g. Japanese toxicity) too?

[Alnusjaponica]

Sure, I overlooked them. I updated the code.

[Alnusjaponica]

@liwii I updated all the metric function which we can pass use_async options. PTAL.

[liwii]

I made a final check and added some missing use_async arguments. LGTM! Thanks for the work!!

The tests are failing due to the server that stores the Chinese tokenizers. Let me think about how to resolve that.

[liwii]

Seemingly the server is back, so let me merge this PR now. Thanks!!