CouchbaseContainer does not actually wait until the query service is available #2993
Assignees
Comments
|
I have a feeling this might be a CB Java SDK issue now, I added extra checks (running some n1ql queries) to the CouchbaseContainer right after startup, the same checks I added to our integration tests pre test start. They pass from CouchbaseContainer but fail for about 1-4 seconds from the integrationt tests. I'll dig deeper into this, perhaps turn on client side debug in the CB Java SDK and try and see what is going on |
|
Yeah this appears to be an issue with the CB Java SDK. But out of the box this prevents the use of the CouchbaseContainer for N1QL queries... |
|
Created a CB support ticket #35272 to track this |
|
cc @daschl |
|
@aaronjwhiteside would it be possible for you to enable TRACE level and supply those logs? The reason is the error you are seeing means that the config the client got from the server does not include query, and right now I have a hard time figuring out the timeline of events. If you could send the trace over, it should include the raw configs the client gets from the server and maybe this helps figuring out what's going on. |
|
@daschl here you go client-trace.txt Thanks for looking into this! |
|
@daschl this second trace includes more details, like when the server actually decides to push the config containing the query service. |
|
@aaronjwhiteside thanks for sharing the log, I'll check it out and get back to you |
|
@aaronjwhiteside this is really interesting. 2 bucket configs arrive at the client, the first one does not have the query service enabled but the second one does. I'm not sure why this is the case, because the container code enables all services before it starts creating a bucket. Can you post your code and how you are using it so I can give it a try to reproduce? Also, which SDK version are you using? |
|
@daschl Java SDK 2.7.11, with Spring Data Couchbase 3.1.2 with modifications. I've also tried with CB 6.0.3, 6.5.0 and 6.5.1 there is no difference between them. Our setup is fairly custom/esoteric both in terms of TestContainers and the way we configure Spring Data Couchbase... so it's not easy to provide the code. I'll work on trying to provide a minimum test case that reproduces the issue.. |
|
@daschl Ok turns out to be really simple to reproduce.. <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>test-containers-couchbase-issue-2993</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>com.couchbase.client</groupId>
<artifactId>java-client</artifactId>
<version>2.7.11</version>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>couchbase</artifactId>
<version>1.14.3</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.30</version>
</dependency>
</dependencies>
</project>import com.couchbase.client.java.Bucket;
import com.couchbase.client.java.Cluster;
import com.couchbase.client.java.CouchbaseCluster;
import com.couchbase.client.java.env.CouchbaseEnvironment;
import com.couchbase.client.java.env.DefaultCouchbaseEnvironment;
import com.couchbase.client.java.query.N1qlQuery;
import org.testcontainers.couchbase.BucketDefinition;
import org.testcontainers.couchbase.CouchbaseContainer;
public class Example {
public static void main(final String...args) {
try (final CouchbaseContainer container = new CouchbaseContainer()
.withBucket(new BucketDefinition("bucket1")
.withPrimaryIndex(false))) {
container.start();
final CouchbaseEnvironment environment = DefaultCouchbaseEnvironment
.builder()
.bootstrapCarrierDirectPort(container.getBootstrapCarrierDirectPort())
.bootstrapHttpDirectPort(container.getBootstrapHttpDirectPort())
.build();
final Cluster cluster = CouchbaseCluster.create(environment, container.getHost())
.authenticate(container.getUsername(), container.getPassword());
final Bucket bucket = cluster.openBucket("bucket1");
bucket.query(N1qlQuery.simple("SELECT 1 FROM system:completed_requests WHERE true"));
}
}
}Yields... |
daschl
added a commit
to daschl/testcontainers-java
that referenced
this issue
Jul 17, 2020
This changeset adds a predicate to the wait strategy to make sure that it not only returns a 200, but also that every enabled service is actually already exposed in the config. Since we are polling the server during bootstrap here, not all of them might show up at the same time. Also, while not contributing to the fix we poll the terse bucket http config "b" instead of the verbose one "buckets" since it is a little more efficient on the server side and actually the config the client internally works with. fixes testcontainers#2993
|
@aaronjwhiteside pr #3003 should fix the issue. I think as a workaround in the meantime you can either sleep or re-enable the primary index building which will also "take some time" which increases the chance the config is in the state the client needs. |
daschl
added a commit
to daschl/testcontainers-java
that referenced
this issue
Jul 20, 2020
This changeset adds a predicate to the wait strategy to make sure that it not only returns a 200, but also that every enabled service is actually already exposed in the config. Since we are polling the server during bootstrap here, not all of them might show up at the same time. Also, while not contributing to the fix we poll the terse bucket http config "b" instead of the verbose one "buckets" since it is a little more efficient on the server side and actually the config the client internally works with. fixes testcontainers#2993
daschl
added a commit
to daschl/testcontainers-java
that referenced
this issue
Jul 20, 2020
This changeset adds a predicate to the wait strategy to make sure that it not only returns a 200, but also that every enabled service is actually already exposed in the config. Since we are polling the server during bootstrap here, not all of them might show up at the same time. Also, while not contributing to the fix we poll the terse bucket http config "b" instead of the verbose one "buckets" since it is a little more efficient on the server side and actually the config the client internally works with. fixes testcontainers#2993
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this is a mistake, please reply to this comment to keep it open. If there isn't one already, a PR to fix or at least reproduce the problem in a test case will always help us get back on track to tackle this. |
|
Keep alive! |
daschl
added a commit
to daschl/testcontainers-java
that referenced
this issue
Dec 16, 2020
This changeset adds a predicate to the wait strategy to make sure that it not only returns a 200, but also that every enabled service is actually already exposed in the config. Since we are polling the server during bootstrap here, not all of them might show up at the same time. Also, while not contributing to the fix we poll the terse bucket http config "b" instead of the verbose one "buckets" since it is a little more efficient on the server side and actually the config the client internally works with. fixes testcontainers#2993
bsideup
added a commit
that referenced
this issue
Feb 6, 2021
This changeset adds a predicate to the wait strategy to make sure that it not only returns a 200, but also that every enabled service is actually already exposed in the config. Since we are polling the server during bootstrap here, not all of them might show up at the same time. Also, while not contributing to the fix we poll the terse bucket http config "b" instead of the verbose one "buckets" since it is a little more efficient on the server side and actually the config the client internally works with. fixes #2993 Co-authored-by: Sergei Egorov <bsideup@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It seems that even though the
CouchbaseContainerchecks the health of the/admin/pingendpoint for the query service, the query service is not actually available to handle queries yet.If I
withReuse(true)this container, and run the test a second time, it succeeds because it now seems the query service has had enough time to initialize and become available to handle queries.The text was updated successfully, but these errors were encountered: