[improve][test] Bound container startup in all Testcontainers tests#103
Merged
david-streamlio merged 1 commit intoJul 10, 2026
Merged
Conversation
Add an explicit withStartupTimeout to every Testcontainers container started in @BeforeClass/@BeforeMethod so a stalled image pull or container start fails fast instead of hanging the CI job until its 45-minute limit. A @test(timeOut=...) only bounds the test method, not the container start in @BeforeClass/@BeforeMethod, so without withStartupTimeout a stuck start could cancel unrelated PRs' CI at the job limit (see apache#89; fixed for MQTT in apache#91). This applies the same bound to the remaining Testcontainers tests. N=3 for a single normal container; N=5 for the Debezium source tests (DB + Pulsar container) and for large/slow images (ClickHouse, LocalStack, Pulsar). Only the startup timeout is added; no test logic is changed.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR hardens Testcontainers-based connector integration tests by bounding container startup time, preventing unbounded hangs during @BeforeClass/@BeforeMethod container starts (e.g., stalled image pulls) from consuming the entire CI job timeout.
Changes:
- Added
.withStartupTimeout(Duration.ofMinutes(N))to all remaining Testcontainers usages in the affected test suites. - Added
import java.time.Duration;where needed to support the new startup timeout configuration. - Preserved existing wait strategies and container configuration while adding only the startup-time bounding.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| cassandra/src/test/java/org/apache/pulsar/io/cassandra/CassandraStringSinkTest.java | Adds a 3-minute startup timeout to the Cassandra container used in sink integration testing. |
| debezium/core/src/test/java/org/apache/pulsar/io/debezium/PulsarSchemaHistoryTest.java | Adds a 5-minute startup timeout to the Pulsar container used by schema history tests. |
| debezium/mariadb/src/test/java/org/apache/pulsar/io/debezium/mariadb/DebeziumMariaDbSourceTest.java | Adds 5-minute startup timeouts to both MariaDB and Pulsar containers in Debezium MariaDB source tests. |
| debezium/mongodb/src/test/java/org/apache/pulsar/io/debezium/mongodb/DebeziumMongoDbSourceTest.java | Adds 5-minute startup timeouts to both MongoDB and Pulsar containers in Debezium MongoDB source tests. |
| debezium/mysql/src/test/java/org/apache/pulsar/io/debezium/mysql/DebeziumMysqlSourceTest.java | Adds 5-minute startup timeouts to both MySQL and Pulsar containers in Debezium MySQL source tests. |
| debezium/postgres/src/test/java/org/apache/pulsar/io/debezium/postgres/DebeziumPostgresSourceTest.java | Adds 5-minute startup timeouts to both Postgres and Pulsar containers in Debezium Postgres source tests. |
| dynamodb/src/test/java/org/apache/pulsar/io/dynamodb/DynamoDBSourceIntegrationTest.java | Adds a 5-minute startup timeout to the LocalStack container used for DynamoDB integration tests. |
| influxdb/src/test/java/org/apache/pulsar/io/influxdb/v1/InfluxDBGenericRecordSinkIntegrationTest.java | Adds a 3-minute startup timeout to the InfluxDB v1 container used in sink integration tests. |
| influxdb/src/test/java/org/apache/pulsar/io/influxdb/v2/InfluxDBSinkIntegrationTest.java | Adds a 3-minute startup timeout to the InfluxDB v2 container used in sink integration tests. |
| jdbc/clickhouse/src/test/java/org/apache/pulsar/io/jdbc/ClickHouseJdbcSinkIntegrationTest.java | Adds a 5-minute startup timeout to the ClickHouse container (large/slow image) used in JDBC sink integration tests. |
| jdbc/mariadb/src/test/java/org/apache/pulsar/io/jdbc/MariadbJdbcSinkIntegrationTest.java | Adds a 3-minute startup timeout to the MariaDB container used in JDBC sink integration tests. |
| kinesis/src/test/java/org/apache/pulsar/io/kinesis/KinesisSinkTest.java | Adds a 5-minute startup timeout to the LocalStack container used for Kinesis integration testing. |
| mongo/src/test/java/org/apache/pulsar/io/mongodb/MongoSinkContainerTest.java | Adds a 3-minute startup timeout to the MongoDB container used in Mongo sink tests. |
| nsq/src/test/java/org/apache/pulsar/io/nsq/NSQSourceIntegrationTest.java | Adds 3-minute startup timeouts to both nsqlookupd and nsqd containers in NSQ integration tests. |
| rabbitmq/src/test/java/org/apache/pulsar/io/rabbitmq/RabbitMQBrokerManager.java | Adds a 3-minute startup timeout to the RabbitMQ container used by RabbitMQ tests. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
A Testcontainers container started in
@BeforeClass/@BeforeMethodwith nowithStartupTimeouthangs unbounded if the image pull or container start stalls.@Test(timeOut=...)does not help — it only bounds the test method, not the container start in@BeforeClass/@BeforeMethod. When a start stalls, the test hangs until the CI job's own 45-minute limit is reached, which has repeatedly cancelled unrelated PRs' CI runs (see #89, fixed for the MQTT sink in #91).This PR applies the same fix to the remaining Testcontainers-based tests so a stuck container start fails fast instead of consuming the whole CI job.
Modifications
Added
.withStartupTimeout(Duration.ofMinutes(N))to every container instance (andimport java.time.Duration;where missing). No test logic was changed — only the startup-timeout bound was added; all existing wait strategies, env, ports, network, etc. are preserved.Nvalues:cassandra/.../CassandraStringSinkTest.javadebezium/core/.../PulsarSchemaHistoryTest.java(Pulsar)debezium/mariadb/.../DebeziumMariaDbSourceTest.java(MariaDB + Pulsar)debezium/mongodb/.../DebeziumMongoDbSourceTest.java(Mongo + Pulsar)debezium/mysql/.../DebeziumMysqlSourceTest.java(MySQL + Pulsar)debezium/postgres/.../DebeziumPostgresSourceTest.java(Postgres + Pulsar)dynamodb/.../DynamoDBSourceIntegrationTest.java(LocalStack)influxdb/.../v1/InfluxDBGenericRecordSinkIntegrationTest.javainfluxdb/.../v2/InfluxDBSinkIntegrationTest.javajdbc/clickhouse/.../ClickHouseJdbcSinkIntegrationTest.java(large image)jdbc/mariadb/.../MariadbJdbcSinkIntegrationTest.javakinesis/.../KinesisSinkTest.java(LocalStack)mongo/.../MongoSinkContainerTest.javarabbitmq/.../RabbitMQBrokerManager.javansq/.../NSQSourceIntegrationTest.java(nsqlookupd + nsqd)All containers already had (or, via the specialized
*Containerclasses, provide) a wait strategy, so no new wait strategy was needed.Verifying
compileTestJavasucceeds for every touched module::cassandra :debezium:pulsar-io-debezium-core :debezium:pulsar-io-debezium-mariadb :debezium:pulsar-io-debezium-mongodb :debezium:pulsar-io-debezium-mysql :debezium:pulsar-io-debezium-postgres :dynamodb :influxdb :jdbc:pulsar-io-jdbc-clickhouse :jdbc:pulsar-io-jdbc-mariadb :kinesis :mongo :rabbitmq :nsq-> BUILD SUCCESSFUL.spotlessJavaCheckon all touched modules -> BUILD SUCCESSFUL (license/format gate).The two fast, reliable suites were run to confirm no regression:
:influxdb:test-> all passed, including the container-backedInfluxDBSinkIntegrationTest(1) andInfluxDBGenericRecordSinkIntegrationTest(1); 0 failures.:nsq:test-> all passed, including the container-backedNSQSourceIntegrationTest(1); 0 failures.The remaining container tests (Cassandra, Mongo, Kinesis, RabbitMQ, Debezium, ClickHouse) were not run locally, as they require amd64 emulation / are slow on arm64 — but the change only adds a startup-timeout bound and cannot alter a passing test's logic.