Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OAK-10280: Occasional failure to start docker container #969

Merged
merged 3 commits into from
Jun 9, 2023
Merged

OAK-10280: Occasional failure to start docker container #969

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5ae2e6c
OAK-10280: Occasional failure to start docker container
mreutegg Jun 8, 2023
3a9d407
OAK-10280: Occasional failure to start docker container
mreutegg Jun 8, 2023
f49e2ca
OAK-10280: Occasional failure to start docker container
mreutegg Jun 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
29 changes: 24 additions & 5 deletions ...c/test/java/org/apache/jackrabbit/oak/blob/cloud/azure/blobstorage/AzuriteDockerRule.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import java.net.URISyntaxException;
import java.security.InvalidKeyException;
import java.util.UUID;
import java.util.concurrent.atomic.AtomicReference;

import org.jetbrains.annotations.NotNull;
import org.junit.Assume;
Expand All @@ -36,23 +37,30 @@

public class AzuriteDockerRule implements TestRule {

private static final String IMAGE = "mcr.microsoft.com/azure-storage/azurite:3.15.0";
private static final String IMAGE = "mcr.microsoft.com/azure-storage/azurite:3.19.0";

public static final String ACCOUNT_KEY = "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==";
public static final String ACCOUNT_NAME = "devstoreaccount1";

private static final String CONTAINER_SUFFIX = UUID.randomUUID().toString().substring(0, 8);

private final DockerRule wrappedRule;

private static final AtomicReference<Exception> STARTUP_EXCEPTION = new AtomicReference<>();

public AzuriteDockerRule() {
wrappedRule = new DockerRule(ImmutableDockerConfig.builder()
.image(IMAGE)
.name("oak-test-azurite-" + CONTAINER_SUFFIX)
.ports("10000")
.addStartedListener(container -> {
container.waitForPort("10000/tcp");
container.waitForLog("Azurite Blob service is successfully listening at http://0.0.0.0:10000");
try {
container.waitForPort("10000/tcp");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to increase the timeout in waitForPort() to prevent tests to be ignored too often, if the origin of the issue is slow Jenkins nodes? The default timeout is 30s and it could be increased to 60s, for example .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if the problem is a slow Jenkins. 30 seconds already seems generous to me and the actual problem may be something else. Maybe this could be tackled in another JIRA ticket...

container.waitForLog("Azurite Blob service is successfully listening at http://0.0.0.0:10000");
} catch (IllegalStateException e) {
STARTUP_EXCEPTION.set(e);
throw e;
}
})
.addContainerConfigurer(builder -> builder.env("executable=blob"))
.alwaysRemoveContainer(true)
Expand Down Expand Up @@ -96,7 +104,18 @@ public Statement apply(Statement statement, Description description) {
Assume.assumeNoException(t);
}

return wrappedRule.apply(statement, description);
Statement base = wrappedRule.apply(statement, description);
return new Statement() {
@Override
public void evaluate() throws Throwable {
try {
base.evaluate();
} catch (Throwable e) {
Assume.assumeNoException(STARTUP_EXCEPTION.get());
throw e;
}
}
};
}

public int getMappedPort() {
Expand Down
106 changes: 0 additions & 106 deletions oak-run/src/test/java/org/apache/jackrabbit/oak/run/AzuriteDockerRule.java
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions oak-run/src/test/java/org/apache/jackrabbit/oak/run/DataStoreCopyCommandTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import com.microsoft.azure.storage.blob.SharedAccessBlobPolicy;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;
import org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AzuriteDockerRule;
import org.jetbrains.annotations.NotNull;
import org.junit.After;
import org.junit.Before;
Expand Down
7 changes: 7 additions & 0 deletions oak-segment-azure/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,13 @@
<artifactId>oak-blob</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.jackrabbit</groupId>
<artifactId>oak-blob-cloud-azure</artifactId>
<version>${project.version}</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
Expand Down
3 changes: 2 additions & 1 deletion ...azure/src/test/java/oak/apache/jackrabbit/oak/segment/azure/tool/SegmentCopyTestBase.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,13 @@
import com.microsoft.azure.storage.StorageException;
import com.microsoft.azure.storage.blob.SharedAccessBlobPermissions;
import com.microsoft.azure.storage.blob.SharedAccessBlobPolicy;

import org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AzuriteDockerRule;
import org.apache.jackrabbit.oak.commons.Buffer;
import org.apache.jackrabbit.oak.segment.SegmentCache;
import org.apache.jackrabbit.oak.segment.SegmentNodeStore;
import org.apache.jackrabbit.oak.segment.SegmentNodeStoreBuilders;
import org.apache.jackrabbit.oak.segment.azure.AzurePersistence;
import org.apache.jackrabbit.oak.segment.azure.AzuriteDockerRule;
import org.apache.jackrabbit.oak.segment.azure.tool.SegmentCopy;
import org.apache.jackrabbit.oak.segment.azure.tool.ToolUtils.SegmentStoreType;
import org.apache.jackrabbit.oak.segment.compaction.SegmentGCOptions.CompactorType;
Expand Down
1 change: 1 addition & 0 deletions ...-azure/src/test/java/org/apache/jackrabbit/oak/segment/azure/AzureArchiveManagerTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
import org.apache.jackrabbit.oak.api.CommitFailedException;
import org.apache.jackrabbit.oak.api.PropertyState;
import org.apache.jackrabbit.oak.api.Type;
import org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AzuriteDockerRule;
import org.apache.jackrabbit.oak.commons.Buffer;
import org.apache.jackrabbit.oak.segment.SegmentId;
import org.apache.jackrabbit.oak.segment.SegmentNodeStore;
Expand Down
2 changes: 2 additions & 0 deletions ...gment-azure/src/test/java/org/apache/jackrabbit/oak/segment/azure/AzureGCJournalTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@

import com.microsoft.azure.storage.StorageException;
import com.microsoft.azure.storage.blob.CloudBlobContainer;

import org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AzuriteDockerRule;
import org.apache.jackrabbit.oak.segment.spi.persistence.SegmentNodeStorePersistence;
import org.apache.jackrabbit.oak.segment.file.GcJournalTest;
import org.junit.Before;
Expand Down
1 change: 1 addition & 0 deletions ...ent-azure/src/test/java/org/apache/jackrabbit/oak/segment/azure/AzureJournalFileTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import com.microsoft.azure.storage.blob.ListBlobItem;
import java.util.stream.IntStream;
import org.apache.commons.lang3.time.StopWatch;
import org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AzuriteDockerRule;
import org.apache.jackrabbit.oak.segment.spi.persistence.JournalFileReader;
import org.apache.jackrabbit.oak.segment.spi.persistence.JournalFileWriter;
import org.jetbrains.annotations.NotNull;
Expand Down
2 changes: 2 additions & 0 deletions ...nt-azure/src/test/java/org/apache/jackrabbit/oak/segment/azure/AzureManifestFileTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@

import com.microsoft.azure.storage.StorageException;
import com.microsoft.azure.storage.blob.CloudBlobContainer;

import org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AzuriteDockerRule;
import org.apache.jackrabbit.oak.segment.spi.persistence.ManifestFile;
import org.junit.Before;
import org.junit.ClassRule;
Expand Down
1 change: 1 addition & 0 deletions ...ent-azure/src/test/java/org/apache/jackrabbit/oak/segment/azure/AzureReadSegmentTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import com.microsoft.azure.storage.blob.CloudBlobContainer;
import com.microsoft.azure.storage.blob.CloudBlobDirectory;

import org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AzuriteDockerRule;
import org.apache.jackrabbit.oak.commons.Buffer;
import org.apache.jackrabbit.oak.segment.SegmentId;
import org.apache.jackrabbit.oak.segment.SegmentNotFoundException;
Expand Down
2 changes: 2 additions & 0 deletions ...-azure/src/test/java/org/apache/jackrabbit/oak/segment/azure/AzureRepositoryLockTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
import com.microsoft.azure.storage.StorageException;
import com.microsoft.azure.storage.blob.CloudBlobContainer;
import com.microsoft.azure.storage.blob.CloudBlockBlob;

import org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AzuriteDockerRule;
import org.apache.jackrabbit.oak.segment.spi.persistence.RepositoryLock;
import org.junit.Before;
import org.junit.ClassRule;
Expand Down
2 changes: 2 additions & 0 deletions ...e/src/test/java/org/apache/jackrabbit/oak/segment/azure/AzureSegmentStoreServiceTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@
import java.time.Instant;
import java.util.*;
import java.util.stream.StreamSupport;

import org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AzuriteDockerRule;
import org.apache.jackrabbit.oak.segment.spi.persistence.SegmentNodeStorePersistence;
import org.apache.sling.testing.mock.osgi.junit.OsgiContext;
import org.jetbrains.annotations.NotNull;
Expand Down
2 changes: 2 additions & 0 deletions ...segment-azure/src/test/java/org/apache/jackrabbit/oak/segment/azure/AzureTarFileTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@

import com.microsoft.azure.storage.StorageException;
import com.microsoft.azure.storage.blob.CloudBlobContainer;

import org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AzuriteDockerRule;
import org.apache.jackrabbit.oak.segment.spi.monitor.FileStoreMonitorAdapter;
import org.apache.jackrabbit.oak.segment.spi.monitor.IOMonitorAdapter;
import org.apache.jackrabbit.oak.segment.file.tar.TarFileTest;
Expand Down
2 changes: 2 additions & 0 deletions ...egment-azure/src/test/java/org/apache/jackrabbit/oak/segment/azure/AzureTarFilesTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
package org.apache.jackrabbit.oak.segment.azure;

import com.microsoft.azure.storage.blob.CloudBlobContainer;

import org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AzuriteDockerRule;
import org.apache.jackrabbit.oak.segment.spi.monitor.FileStoreMonitorAdapter;
import org.apache.jackrabbit.oak.segment.spi.monitor.IOMonitorAdapter;
import org.apache.jackrabbit.oak.segment.file.tar.TarFiles;
Expand Down
2 changes: 2 additions & 0 deletions ...gment-azure/src/test/java/org/apache/jackrabbit/oak/segment/azure/AzureTarWriterTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
package org.apache.jackrabbit.oak.segment.azure;

import com.microsoft.azure.storage.blob.CloudBlobContainer;

import org.apache.jackrabbit.oak.blob.cloud.azure.blobstorage.AzuriteDockerRule;
import org.apache.jackrabbit.oak.segment.spi.monitor.IOMonitorAdapter;
import org.apache.jackrabbit.oak.segment.file.tar.TarWriterTest;
import org.apache.jackrabbit.oak.segment.spi.persistence.SegmentArchiveManager;
Expand Down