-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: high level integration testing framework #2258
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks great to me
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We'll want to add __pycache__
to gitignore too.
conn.autocommit = True | ||
yield conn |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should open an issue on the autocommit thing since it's unexpected that it wouldn't work without setting it to true.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is (obviously) a very high level pytest harness for running
process and system integration tests, and the CI and developer
experience (just) integration. There's nothing special about this
pytest development, and I expect that it can grow with us as needs
evolve.
There are a class of tests that are difficult to run in the current
testing enviornments that this harness should make easy and
trivial. Such as:
cases that run a glaredb binary.
cases that involve the orchestration of multiple glaredb binaries.
operations that require introspection or manipulation of the
filesystem.
This shouldn't replace any of the existing tests:
use rust unittests for as much of the internals as possible.
anything that involves the python bindings should be written (also
using pytest) in
bindings/python/tests
). This package shouldnever depend on the python bindings for build efficency reasons.
anything case that can be entirely expressed in SQL should be
written as a
sqllogictest
(SLT).anything that requires the test of a specific service level
dependency (as in for data-sources) should run in the SLT
integration tests.