Testing: Improve "Testcontainers for Python" implementation

[amotl]

Introduction

We are aiming to provide canonical "Testcontainers" implementations for Java and Python, per testcontainers-java and testcontainers-python.

• At Testcontainers for Java and CrateDB, we demonstrate how to run integration tests of Java applications with CrateDB, based on Testcontainers for Java, see Add CrateDB testcontainers testcontainers/testcontainers-java#6790.
• Testcontainers for Python crate/cratedb-examples#72 needs to be resolved by working through the backlog.

About

At the spots enumerated below, we added the first version of a corresponding Python implementation, originally conceived at daq-tools/lorrystream#47.

• Implementation: cratedb_toolkit/testing/testcontainers/cratedb.py
• pytest fixtures: tests/conftest.py
• test case: tests/testing/test_cratedb_sqlalchemy.py

Backlog

• Add documentation
• Apply database schema already when connecting #53
• Testing: Adapt "Testcontainers" implementation to unittest #58
• Currently, the adapter and test layer is being exercised using an SQLAlchemy connection and corresponding test case. It makes sense to also exercise and demonstrate a pure DBAPI-based variant of the same thing.
• It will be nice to have a modern test layer which forms a cluster, for both Java and Python. I think cr8 has it already?
• Cherry-pick CrateDB invocation options from cr8: '-Cdiscovery.initial_state_timeout=0', '-Cnetwork.host=127.0.0.1', '-Cudc.enabled=false', '-Ccluster.name=cr8-tests'
• Revisit downstream issues Testcontainers for Python crate/cratedb-examples#72 and [Testing] Demonstrate CrateDB test layers with parameterization crate/cratedb-examples#282.
• Upstream to testcontainers-python.

[amotl]

Together with @pilosus, we have been able to bring in a few advancements to the "Testcontainers for Python" adapter implementations we are running here. Thanks!

• Testing: Generalize KeepaliveContainer from CrateDBContainer #78
• Testing: Add Testcontainer for MongoDB with improved configurability
• Adapt testcontainers to be testing framework agnostic #82

[amotl]

It will be nice to have a modern test layer which forms a cluster, for both Java and Python.

If we would like to contribute the existing implementation early, I think it makes sense to think about another detail of "naming things" on this matter.

If we would like to have CrateDBContainer to be the one and only to be responsible to spin up single-node instances, while there will be future implementations permitting to run whole clusters, I think it can make sense to reflect that into a naming scheme like CrateDBSingleNodeContainer vs. CrateDBMultiNodeContainer.

However, while writing that down, it could also be a dual-class machinery of the existing CrateDBContainer and an additional CrateDBClusterManager, which, well, manages the former.

It is probably too early to think about this topic without getting hands-on with it, mainly starting by researching how others might be doing similar things like managing a fleet of containers on behalf of a single convenience entrypoint.