Use Cosmos DB emulator for the integration tests #1909

komamitsu · 2024-06-17T08:55:48Z

Description

Recent Cosmos DB integration tests are slow and unstable. We also need to manage Cosmos DB related resources for the integration tests. This PR uses the Azure Cosmos DB emulator instead using actual Cosmos DB.

Related issues and/or PRs

None

Changes made

In the GitHub Actions workflow:
- Launch the Cosmos DB emulator in windows-latest image
- Install TLS certification extracted from the emulator in the Windows VM
- Run the integration tests using the emulator
Reduce the concurrency of the integration test from 3 to 1 to avoid tentative errors as much as possible

Checklist

The following is a best-effort checklist. If any items in this checklist are not applicable to this PR or are dependent on other, unmerged PRs, please still mark the checkboxes after you have read and understood each item.

I have commented my code, particularly in hard-to-understand areas.
I have updated the documentation to reflect the changes.
Any remaining open issues linked to this PR are documented and up-to-date (Jira, GitHub, etc.).
Tests (unit, integration, etc.) have been added for the changes.
My changes generate no new warnings.
Any dependent changes in other PRs have been merged and published.

Additional notes (optional)

None

Release notes

N/A

komamitsu · 2024-06-17T09:01:47Z

.github/workflows/ci.yaml

-#       Delete all files modified more than 3 days ago with the ".out.log" file extension located in the "/home/azureuser/.gradle/daemon"
-#       folder hierarchy. These files accumulate over time and can end up using a lot of disk space
-        run : find /home/azureuser/.gradle/daemon -name "*.out.log" -type f -mtime +3 -exec rm -vf {} +
+          arguments: integrationTestCosmos -PjavaVersion=${{ env.JAVA_VERSION }} -PjavaVendor=${{ env.JAVA_VENDOR }} -PintegrationTestJavaRuntimeVersion=${{ env.INT_TEST_JAVA_RUNTIME_VERSION }} -PintegrationTestJavaRuntimeVendor=${{ env.INT_TEST_JAVA_RUNTIME_VENDOR }} -Dscalardb.cosmos.uri=https://localhost:8081/ -Dscalardb.cosmos.password=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw== -Dscalardb.cosmos.database_prefix=${{ env.db_prefix }}_ -Dfile.encoding=UTF-8


This password is a public value, so hard coded here.

-Dfile.encoding=UTF-8 is needed to address some encoding issues that occur due to the WM's default charset (Windows-1252)

komamitsu · 2024-06-17T09:02:36Z

core/build.gradle

    options {
        systemProperties(System.getProperties().findAll{it.key.toString().startsWith("scalardb")})
    }
-    maxParallelForks = 3


Concurrent operations seem to increase the possibility of tentative errors.

Ah, okay. How long does this integration test take? Is it shorter than before?

@brfrn169 About 26 minutes.

The test once passed with maxParallelForks = 3 and took 19 minutes. But the test often failed with the configuration...

Okay... Thank you. I wish it had solved the slow integration test problem as well. 😅

Using larger runners might make parallel execution feasible. It should be way more inexpensive than Cosmos DB, so it is worth trying. Let's discuss it later.

Torch3333 · 2024-06-18T00:21:47Z

.github/workflows/ci.yaml

      - name: Generate unique prefix using the epoch
        run: |
          echo "db_prefix=$(date +%s%3N)" >> $GITHUB_ENV


I think we can remove this since multiple integration test runners are not sharing the same CosmosDB instance now; there won't be database name conflicts without a prefix.
I guess we could also remove the handling of the scalardb.cosmos.database_prefix environment variable in the integration test code.

@Torch3333 Good catch! Thank you. I'll remove db_prefix related jobs and parameters from the workflow file.

For Java code, I think we may want to test with real Cosmo DB later. The implementation handling scalardb.cosmos.database_prefix might be useful for such a case. I don't have a strong opinion on it, though. What do you think?

Thanks.
Ok, keeping the implementation handling is fine.

@komamitsu @Torch3333 I think we can remove scalardb.cosmos.database_prefix. This parameter is useful only when using a shared Cosmos DB account. If we want to test with real Cosmos DB later, we can create a new dedicated Cosmos DB account.

@brfrn169 Thanks. Understood. I'll create a follow up PR for that.

@komamitsu Thank you!

Torch3333

LGTM, thank you.

brfrn169

LGTM other than what I commented. Thank you!

feeblefakie

LGTM! Thank you!

feeblefakie · 2024-06-18T02:18:29Z

core/build.gradle

    options {
        systemProperties(System.getProperties().findAll{it.key.toString().startsWith("scalardb")})
    }
-    maxParallelForks = 3


Using larger runners might make parallel execution feasible. It should be way more inexpensive than Cosmos DB, so it is worth trying. Let's discuss it later.

Use Cosmos DB emulator for IT

b47e4b7

komamitsu commented Jun 17, 2024

View reviewed changes

komamitsu requested review from Torch3333, feeblefakie and jnmt June 17, 2024 09:23

komamitsu self-assigned this Jun 17, 2024

komamitsu added the github_actions Pull requests that update GitHub Actions code label Jun 17, 2024

brfrn169 requested review from brfrn169 and removed request for jnmt June 17, 2024 16:55

brfrn169 added the improvement label Jun 17, 2024

Torch3333 reviewed Jun 18, 2024

View reviewed changes

Remove unnecessary jobs and parameters from CI

11f20a3

Torch3333 approved these changes Jun 18, 2024

View reviewed changes

brfrn169 approved these changes Jun 18, 2024

View reviewed changes

feeblefakie approved these changes Jun 18, 2024

View reviewed changes

feeblefakie merged commit 6d43a36 into master Jun 18, 2024

feeblefakie deleted the use-cosmos-db-emu-for-it branch June 18, 2024 02:20

feeblefakie pushed a commit that referenced this pull request Jun 18, 2024

Use Cosmos DB emulator for the integration tests (#1909)

43b25e6

feeblefakie mentioned this pull request Jun 18, 2024

Backport to branch(3.12) : Use Cosmos DB emulator for the integration tests #1911

Merged

feeblefakie pushed a commit that referenced this pull request Jun 18, 2024

Use Cosmos DB emulator for the integration tests (#1909)

3c584aa

feeblefakie mentioned this pull request Jun 18, 2024

Backport to branch(3.11) : Use Cosmos DB emulator for the integration tests #1912

Merged

feeblefakie pushed a commit that referenced this pull request Jun 18, 2024

Use Cosmos DB emulator for the integration tests (#1909)

2d48b10

feeblefakie mentioned this pull request Jun 18, 2024

Backport to branch(3.10) : Use Cosmos DB emulator for the integration tests #1913

Merged

feeblefakie pushed a commit that referenced this pull request Jun 18, 2024

Use Cosmos DB emulator for the integration tests (#1909)

b4df78a

feeblefakie mentioned this pull request Jun 18, 2024

Backport to branch(3.9) : Use Cosmos DB emulator for the integration tests #1914

Merged

feeblefakie pushed a commit that referenced this pull request Jun 18, 2024

Use Cosmos DB emulator for the integration tests (#1909)

7a2d51f

feeblefakie mentioned this pull request Jun 18, 2024

Backport to branch(3.8) : Use Cosmos DB emulator for the integration tests #1915

Merged

feeblefakie pushed a commit that referenced this pull request Jun 18, 2024

Use Cosmos DB emulator for the integration tests (#1909)

3dccd25

feeblefakie mentioned this pull request Jun 18, 2024

Backport to branch(3) : Use Cosmos DB emulator for the integration tests #1916

Merged

komamitsu mentioned this pull request Jun 18, 2024

Remove unnecessary scalardb.cosmos.database_prefix property #1918

Merged

6 tasks

komamitsu mentioned this pull request Jun 18, 2024

Reduce the number of concurrent DDLs from 10 to 1 for Cosmos DB in the integration tests #1920

Merged

6 tasks

Torch3333 mentioned this pull request Jun 19, 2024

Fix CosmosDB CI when running integration test with JDK11+ #1919

Merged

6 tasks

Use Cosmos DB emulator for the integration tests #1909

Use Cosmos DB emulator for the integration tests #1909

Uh oh!

Conversation

komamitsu commented Jun 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related issues and/or PRs

Changes made

Checklist

Additional notes (optional)

Release notes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Torch3333 Jun 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brfrn169 Jun 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Torch3333 left a comment

Choose a reason for hiding this comment

Uh oh!

brfrn169 left a comment

Choose a reason for hiding this comment

Uh oh!

feeblefakie left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

komamitsu commented Jun 17, 2024 •

edited

Loading

Torch3333 Jun 18, 2024 •

edited

Loading

brfrn169 Jun 18, 2024 •

edited

Loading