-
Notifications
You must be signed in to change notification settings - Fork 14.7k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: add pytest-vcr for recording HTTP interactions in integration t…
…ests (#2445) Using `pytest-vcr` in integration tests has several benefits. Firstly, it removes the need to mock external services, as VCR records and replays HTTP interactions on the fly. Secondly, it simplifies the integration test setup by eliminating the need to set up and tear down external services in some cases. Finally, it allows for more reliable and deterministic integration tests by ensuring that HTTP interactions are always replayed with the same response. Overall, `pytest-vcr` is a valuable tool for simplifying integration test setup and improving their reliability This commit adds the `pytest-vcr` package as a dependency for integration tests in the `pyproject.toml` file. It also introduces two new fixtures in `tests/integration_tests/conftest.py` files for managing cassette directories and VCR configurations. In addition, the `tests/integration_tests/vectorstores/test_elasticsearch.py` file has been updated to use the `@pytest.mark.vcr` decorator for recording and replaying HTTP interactions. Finally, this commit removes the `documents` fixture from the `test_elasticsearch.py` file and replaces it with a new fixture defined in `tests/integration_tests/vectorstores/conftest.py` that yields a list of documents to use in any other tests. This also includes my second attempt to fix issue : #2386 Maybe related #2484
- Loading branch information
Showing
11 changed files
with
1,945 additions
and
16 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
# Readme tests(draft) | ||
|
||
## Integrations Tests | ||
|
||
### Prepare | ||
|
||
This repository contains functional tests for several search engines and databases. The | ||
tests aim to verify the correct behavior of the engines and databases according to their | ||
specifications and requirements. | ||
|
||
To run some integration tests, such as tests located in | ||
`tests/integration_tests/vectorstores/`, you will need to install the following | ||
software: | ||
|
||
- Docker | ||
- Python 3.8.1 or later | ||
|
||
We have optional group `test_integration` in the `pyproject.toml` file. This group | ||
should contain dependencies for the integration tests and can be installed using the | ||
command: | ||
|
||
```bash | ||
poetry install --with test_integration | ||
``` | ||
|
||
Any new dependencies should be added by running: | ||
|
||
```bash | ||
poetry add some_new_deps --group "test_integration" | ||
``` | ||
|
||
Before running any tests, you should start a specific Docker container that has all the | ||
necessary dependencies installed. For instance, we use the `elasticsearch.yml` container | ||
for `test_elasticsearch.py`: | ||
|
||
```bash | ||
cd tests/integration_tests/vectorstores/docker-compose | ||
docker-compose -f elasticsearch.yml up | ||
``` | ||
|
||
Additionally, it's important to note that some integration tests may require certain | ||
environment variables to be set, such as `OPENAI_API_KEY`. Be sure to set any required | ||
environment variables before running the tests to ensure they run correctly. | ||
|
||
### Recording HTTP interactions with pytest-vcr | ||
|
||
Some of the integration tests in this repository involve making HTTP requests to | ||
external services. To prevent these requests from being made every time the tests are | ||
run, we use pytest-vcr to record and replay HTTP interactions. | ||
|
||
When running tests in a CI/CD pipeline, you may not want to modify the existing | ||
cassettes. You can use the --vcr-record=none command-line option to disable recording | ||
new cassettes. Here's an example: | ||
|
||
```bash | ||
pytest tests/integration_tests/vectorstores/test_elasticsearch.py --vcr-record=none | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
import os | ||
|
||
import pytest | ||
|
||
# Getting the absolute path of the current file's directory | ||
ABS_PATH = os.path.dirname(os.path.abspath(__file__)) | ||
|
||
|
||
# This fixture returns a string containing the path to the cassette directory for the | ||
# current module | ||
@pytest.fixture(scope="module") | ||
def vcr_cassette_dir(request: pytest.FixtureRequest) -> str: | ||
return os.path.join( | ||
os.path.dirname(request.module.__file__), | ||
"cassettes", | ||
os.path.basename(request.module.__file__).replace(".py", ""), | ||
) | ||
|
||
|
||
# This fixture returns a dictionary containing filter_headers options | ||
# for replacing certain headers with dummy values during cassette playback | ||
# Specifically, it replaces the authorization header with a dummy value to | ||
# prevent sensitive data from being recorded in the cassette. | ||
@pytest.fixture(scope="module") | ||
def vcr_config() -> dict: | ||
return { | ||
"filter_headers": [ | ||
("authorization", "authorization-DUMMY"), | ||
("X-OpenAI-Client-User-Agent", "X-OpenAI-Client-User-Agent-DUMMY"), | ||
("User-Agent", "User-Agent-DUMMY"), | ||
], | ||
} |
Oops, something went wrong.