EvaluationDataset in deepeval suddenly requires an OpenAi key since update

[kbarendrecht]

Describe the bug
EvaluationDataset in deepeval 1.4.7 or 1.4.8 suddenly requires an OpenAi key while 1.4.6 did not. I'm using a custom LLM (claude via bedrock) to do evaluations so I am not using OpenAi for anything. Deepeval 1.4.6 still works fine without an OpenAi key.

openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

To Reproduce
Steps to reproduce the behavior:
import EvaluationDataset in a python script without OpenAi env variables. it seems just het import breaks it already. (see traceback for more info)

Expected behavior
The deepeval python library should function without a OpenAi key.

Screenshots

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/docker/www/test/update_dataset.py", line 1, in <module>
    from deepeval.dataset import EvaluationDataset, Golden
  File "/home/docker/venv/lib/python3.12/site-packages/deepeval/dataset/__init__.py", line 1, in <module>
    from .dataset import EvaluationDataset
  File "/home/docker/venv/lib/python3.12/site-packages/deepeval/dataset/dataset.py", line 28, in <module>
    from deepeval.synthesizer.types import *
  File "/home/docker/venv/lib/python3.12/site-packages/deepeval/synthesizer/__init__.py", line 1, in <module>
    from .synthesizer import (
  File "/home/docker/venv/lib/python3.12/site-packages/deepeval/synthesizer/synthesizer.py", line 43, in <module>
    from deepeval.synthesizer.config import (
  File "/home/docker/venv/lib/python3.12/site-packages/deepeval/synthesizer/config.py", line 46, in <module>
    class ContextConstructionConfig:
  File "/home/docker/venv/lib/python3.12/site-packages/deepeval/synthesizer/config.py", line 48, in ContextConstructionConfig
    OpenAIEmbeddingModel()
  File "/home/docker/venv/lib/python3.12/site-packages/deepeval/models/openai_embedding_model.py", line 37, in __init__
    super().__init__(model_name)
  File "/home/docker/venv/lib/python3.12/site-packages/deepeval/models/base_model.py", line 107, in __init__
    self.model = self.load_model(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docker/venv/lib/python3.12/site-packages/deepeval/models/openai_embedding_model.py", line 76, in load_model
    return OpenAIEmbeddings(model=self.model_name, **self.kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docker/venv/lib/python3.12/site-packages/pydantic/main.py", line 212, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docker/venv/lib/python3.12/site-packages/langchain_openai/embeddings/base.py", line [33](https://github.com/ScientaNL/scienta/actions/runs/11662408084/job/32468698173#step:10:34)8, in validate_environment
    self.client = openai.OpenAI(**client_params, **sync_specific).embeddings  # type: ignore[arg-type]
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docker/venv/lib/python3.12/site-packages/openai/_client.py", line 105, in __init__
    raise OpenAIError(
openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

Additional context
In my setup I'm first running update_dataset to create a dataset.json which i then evaluate in test.py. Both the python files trigger this error.

[michieletto]

I have solved by removing line 28:

from deepeval.synthesizer.types import *

from deepval/dataset/dataset.py

[penguine-ip]

Hey @michieletto and @kbarendrecht thanks so much for this, I merged the PR and cut a new release for it in 1.4.9 :)

[kftam1994]

This error also happens to
from deepeval.synthesizer import Synthesizer
during the line
from deepeval.synthesizer.config import ( FiltrationConfig, EvolutionConfig, StylingConfig, ContextConstructionConfig, )
It is trying to initialise an embedding even though I am generating goldens from context.
Would it be fixed too? Thank you

[ms130]

@kftam1994 maybe you could put this in a new issue, if not already done so, to flag attention to it?

[penguine-ip]

Hey @kftam1994 can you please provide the full error message and where it is giving the error? I don't think the error will happen unless you intialize a ContextConstructionConfig without an embedder