Skip to content

[CI] RepositoryS3ClientYamlTestSuiteIT class failing #117596

New issue
New issue
Closed
#117675
Closed
[CI] RepositoryS3ClientYamlTestSuiteIT class failing#117596
#117675
Assignees
ywangd
Labels
:Distributed Coordination/Snapshot/RestoreAnything directly related to the `_snapshot/*` APIs>test-failureTriaged test failures from CITeam:Distributed CoordinationMeta label for Distributed Coordination teamlow-riskAn open issue or test failure that is a low risk to future releases
@elasticsearchmachine

Description

@elasticsearchmachine
elasticsearchmachine
opened on Nov 26, 2024

Build Scans:

Reproduction Line:

./gradlew ":modules:repository-s3:yamlRestTest" --tests "org.elasticsearch.repositories.s3.RepositoryS3ClientYamlTestSuiteIT.test {yaml=repository_s3/40_repository_ec2_credentials/Get a non existing snapshot}" -Dtests.seed=FAA9DF498EC1EED4 -Dtests.locale=ro -Dtests.timezone=Europe/Helsinki -Druntime.java=17 -Dtests.fips.enabled=true

Applicable branches:
8.x

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: Failure at [repository_s3/40_repository_ec2_credentials:7]: expected [2xx] status code but api [snapshot.create_repository] returned [500 Internal Server Error] [---
error:
  root_cause:
  - type: "repository_verification_exception"
    reason: "[repository_ec2] path [ec2_base_path] is not accessible on master node"
    stack_trace: "org.elasticsearch.repositories.RepositoryVerificationException:\
      \ [repository_ec2] path [ec2_base_path] is not accessible on master node\nCaused\
      \ by: java.io.IOException: Unable to upload object [ec2_base_path/tests-X8eE8K6AQYesIdw10Xk6vw/master.dat]\
      \ using a single upload\n\tat org.elasticsearch.repositories.s3.S3BlobContainer.executeSingleUpload(S3BlobContainer.java:466)\n\
      \tat org.elasticsearch.repositories.s3.S3BlobContainer.lambda$writeBlob$1(S3BlobContainer.java:144)\n\
      \tat java.base/java.security.AccessController.doPrivileged(AccessController.java:569)\n\
      \tat org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedIOException(SocketAccess.java:38)\n\
      \tat org.elasticsearch.repositories.s3.S3BlobContainer.writeBlob(S3BlobContainer.java:142)\n\
      \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.blobstore.BlobContainer.writeBlob(BlobContainer.java:124)\n\
      \tat org.elasticsearch.repositories.s3.S3BlobContainer.writeBlobAtomic(S3BlobContainer.java:304)\n\
      \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:2205)\n\
      \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.repositories.RepositoriesService.lambda$validatePutRepositoryRequest$11(RepositoriesService.java:373)\n\
      \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.action.ActionRunnable$1.doRun(ActionRunnable.java:37)\n\
      \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1023)\n\
      \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)\n\
      \tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\
      \tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\
      \tat java.base/java.lang.Thread.run(Thread.java:833)\nCaused by: com.amazonaws.SdkClientException:\
      \ Unable to calculate a request signature: Unable to calculate a request signature:\
      \ Key size for HMAC must be at least 112 bits in approved mode: SHA-256/HMAC\n\
      \tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:109)\n\
      \tat com.amazonaws.auth.AWS4Signer.newSigningKey(AWS4Signer.java:639)\n\tat\
      \ com.amazonaws.auth.AWS4Signer.deriveSigningKey(AWS4Signer.java:404)\n\tat\
      \ com.amazonaws.auth.AWS4Signer.sign(AWS4Signer.java:253)\n\tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1320)\n\
      \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)\n\
      \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)\n\
      \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)\n\
      \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)\n\
      \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)\n\
      \tat com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)\n\
      \tat com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)\n\
      \tat com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)\n\
      \tat com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456)\n\
      \tat com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403)\n\
      \tat com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:421)\n\
      \tat com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:6531)\n\
      \tat com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1861)\n\
      \tat com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1821)\n\
      \tat org.elasticsearch.repositories.s3.S3BlobContainer.lambda$executeSingleUpload$16(S3BlobContainer.java:464)\n\
      \tat org.elasticsearch.repositories.s3.SocketAccess.lambda$doPrivilegedVoid$0(SocketAccess.java:47)\n\
      \tat java.base/java.security.AccessController.doPrivileged(AccessController.java:318)\n\
      \tat org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedVoid(SocketAccess.java:46)\n\
      \tat org.elasticsearch.repositories.s3.S3BlobContainer.executeSingleUpload(S3BlobContainer.java:464)\n\
      \t... 14 more\nCaused by: com.amazonaws.SdkClientException: Unable to calculate\
      \ a request signature: Key size for HMAC must be at least 112 bits in approved\
      \ mode: SHA-256/HMAC\n\tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:132)\n\
      \tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:105)\n\
      \t... 37 more\nCaused by: org.bouncycastle.crypto.IllegalKeyException: Key size\
      \ for HMAC must be at least 112 bits in approved mode: SHA-256/HMAC\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsSHS$MACOperatorFactory.createMAC(Unknown\
      \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsSHS$MACOperatorFactory.createMAC(Unknown\
      \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsMACOperatorFactory.createOutputMACCalculator(Unknown\
      \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsMACOperatorFactory.createOutputMACCalculator(Unknown\
      \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.jcajce.provider.BaseHMac.engineInit(Unknown\
      \ Source)\n\tat java.base/javax.crypto.Mac.init(Mac.java:432)\n\tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:127)\n\
      \t... 38 more\n"
  type: "repository_verification_exception"
  reason: "[repository_ec2] path [ec2_base_path] is not accessible on master node"
  caused_by:
    type: "i_o_exception"
    reason: "Unable to upload object [ec2_base_path/tests-X8eE8K6AQYesIdw10Xk6vw/master.dat]\
      \ using a single upload"
    caused_by:
      type: "sdk_client_exception"
      reason: "Unable to calculate a request signature: Unable to calculate a request\
        \ signature: Key size for HMAC must be at least 112 bits in approved mode:\
        \ SHA-256/HMAC"
      caused_by:
        type: "sdk_client_exception"
        reason: "Unable to calculate a request signature: Key size for HMAC must be\
          \ at least 112 bits in approved mode: SHA-256/HMAC"
        caused_by:
          type: "illegal_key_exception"
          reason: "Key size for HMAC must be at least 112 bits in approved mode: SHA-256/HMAC"
          stack_trace: "org.bouncycastle.crypto.IllegalKeyException: Key size for\
            \ HMAC must be at least 112 bits in approved mode: SHA-256/HMAC\n\tat\
            \ org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsSHS$MACOperatorFactory.createMAC(Unknown\
            \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsSHS$MACOperatorFactory.createMAC(Unknown\
            \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsMACOperatorFactory.createOutputMACCalculator(Unknown\
            \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsMACOperatorFactory.createOutputMACCalculator(Unknown\
            \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.jcajce.provider.BaseHMac.engineInit(Unknown\
            \ Source)\n\tat java.base/javax.crypto.Mac.init(Mac.java:432)\n\tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:127)\n\
            \tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:105)\n\
            \tat com.amazonaws.auth.AWS4Signer.newSigningKey(AWS4Signer.java:639)\n\
            \tat com.amazonaws.auth.AWS4Signer.deriveSigningKey(AWS4Signer.java:404)\n\
            \tat com.amazonaws.auth.AWS4Signer.sign(AWS4Signer.java:253)\n\tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1320)\n\
            \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)\n\
            \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)\n\
            \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)\n\
            \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)\n\
            \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)\n\
            \tat com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)\n\
            \tat com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)\n\
            \tat com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)\n\
            \tat com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456)\n\
            \tat com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403)\n\
            \tat com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:421)\n\
            \tat com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:6531)\n\
            \tat com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1861)\n\
            \tat com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1821)\n\
            \tat org.elasticsearch.repositories.s3.S3BlobContainer.lambda$executeSingleUpload$16(S3BlobContainer.java:464)\n\
            \tat org.elasticsearch.repositories.s3.SocketAccess.lambda$doPrivilegedVoid$0(SocketAccess.java:47)\n\
            \tat java.base/java.security.AccessController.doPrivileged(AccessController.java:318)\n\
            \tat org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedVoid(SocketAccess.java:46)\n\
            \tat org.elasticsearch.repositories.s3.S3BlobContainer.executeSingleUpload(S3BlobContainer.java:464)\n\
            \tat org.elasticsearch.repositories.s3.S3BlobContainer.lambda$writeBlob$1(S3BlobContainer.java:144)\n\
            \tat java.base/java.security.AccessController.doPrivileged(AccessController.java:569)\n\
            \tat org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedIOException(SocketAccess.java:38)\n\
            \tat org.elasticsearch.repositories.s3.S3BlobContainer.writeBlob(S3BlobContainer.java:142)\n\
            \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.blobstore.BlobContainer.writeBlob(BlobContainer.java:124)\n\
            \tat org.elasticsearch.repositories.s3.S3BlobContainer.writeBlobAtomic(S3BlobContainer.java:304)\n\
            \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:2205)\n\
            \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.repositories.RepositoriesService.lambda$validatePutRepositoryRequest$11(RepositoriesService.java:373)\n\
            \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.action.ActionRunnable$1.doRun(ActionRunnable.java:37)\n\
            \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1023)\n\
            \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)\n\
            \tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\
            \tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\
            \tat java.base/java.lang.Thread.run(Thread.java:833)\n"
        stack_trace: "com.amazonaws.SdkClientException: Unable to calculate a request\
          \ signature: Key size for HMAC must be at least 112 bits in approved mode:\
          \ SHA-256/HMAC\n\tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:132)\n\
          \tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:105)\n\
          \tat com.amazonaws.auth.AWS4Signer.newSigningKey(AWS4Signer.java:639)\n\t\
          at com.amazonaws.auth.AWS4Signer.deriveSigningKey(AWS4Signer.java:404)\n\
          \tat com.amazonaws.auth.AWS4Signer.sign(AWS4Signer.java:253)\n\tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1320)\n\
          \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)\n\
          \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)\n\
          \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)\n\
          \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)\n\
          \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)\n\
          \tat com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)\n\
          \tat com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)\n\
          \tat com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)\n\
          \tat com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456)\n\
          \tat com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403)\n\
          \tat com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:421)\n\
          \tat com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:6531)\n\
          \tat com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1861)\n\
          \tat com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1821)\n\
          \tat org.elasticsearch.repositories.s3.S3BlobContainer.lambda$executeSingleUpload$16(S3BlobContainer.java:464)\n\
          \tat org.elasticsearch.repositories.s3.SocketAccess.lambda$doPrivilegedVoid$0(SocketAccess.java:47)\n\
          \tat java.base/java.security.AccessController.doPrivileged(AccessController.java:318)\n\
          \tat org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedVoid(SocketAccess.java:46)\n\
          \tat org.elasticsearch.repositories.s3.S3BlobContainer.executeSingleUpload(S3BlobContainer.java:464)\n\
          \tat org.elasticsearch.repositories.s3.S3BlobContainer.lambda$writeBlob$1(S3BlobContainer.java:144)\n\
          \tat java.base/java.security.AccessController.doPrivileged(AccessController.java:569)\n\
          \tat org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedIOException(SocketAccess.java:38)\n\
          \tat org.elasticsearch.repositories.s3.S3BlobContainer.writeBlob(S3BlobContainer.java:142)\n\
          \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.blobstore.BlobContainer.writeBlob(BlobContainer.java:124)\n\
          \tat org.elasticsearch.repositories.s3.S3BlobContainer.writeBlobAtomic(S3BlobContainer.java:304)\n\
          \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:2205)\n\
          \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.repositories.RepositoriesService.lambda$validatePutRepositoryRequest$11(RepositoriesService.java:373)\n\
          \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.action.ActionRunnable$1.doRun(ActionRunnable.java:37)\n\
          \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1023)\n\
          \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)\n\
          \tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\
          \tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\
          \tat java.base/java.lang.Thread.run(Thread.java:833)\nCaused by: org.bouncycastle.crypto.IllegalKeyException:\
          \ Key size for HMAC must be at least 112 bits in approved mode: SHA-256/HMAC\n\
          \tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsSHS$MACOperatorFactory.createMAC(Unknown\
          \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsSHS$MACOperatorFactory.createMAC(Unknown\
          \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsMACOperatorFactory.createOutputMACCalculator(Unknown\
          \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsMACOperatorFactory.createOutputMACCalculator(Unknown\
          \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.jcajce.provider.BaseHMac.engineInit(Unknown\
          \ Source)\n\tat java.base/javax.crypto.Mac.init(Mac.java:432)\n\tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:127)\n\
          \t... 38 more\n"
      stack_trace: "com.amazonaws.SdkClientException: Unable to calculate a request\
        \ signature: Unable to calculate a request signature: Key size for HMAC must\
        \ be at least 112 bits in approved mode: SHA-256/HMAC\n\tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:109)\n\
        \tat com.amazonaws.auth.AWS4Signer.newSigningKey(AWS4Signer.java:639)\n\t\
        at com.amazonaws.auth.AWS4Signer.deriveSigningKey(AWS4Signer.java:404)\n\t\
        at com.amazonaws.auth.AWS4Signer.sign(AWS4Signer.java:253)\n\tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1320)\n\
        \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)\n\
        \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)\n\
        \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)\n\
        \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)\n\
        \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)\n\
        \tat com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)\n\
        \tat com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)\n\
        \tat com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)\n\
        \tat com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456)\n\
        \tat com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403)\n\
        \tat com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:421)\n\
        \tat com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:6531)\n\
        \tat com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1861)\n\
        \tat com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1821)\n\
        \tat org.elasticsearch.repositories.s3.S3BlobContainer.lambda$executeSingleUpload$16(S3BlobContainer.java:464)\n\
        \tat org.elasticsearch.repositories.s3.SocketAccess.lambda$doPrivilegedVoid$0(SocketAccess.java:47)\n\
        \tat java.base/java.security.AccessController.doPrivileged(AccessController.java:318)\n\
        \tat org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedVoid(SocketAccess.java:46)\n\
        \tat org.elasticsearch.repositories.s3.S3BlobContainer.executeSingleUpload(S3BlobContainer.java:464)\n\
        \tat org.elasticsearch.repositories.s3.S3BlobContainer.lambda$writeBlob$1(S3BlobContainer.java:144)\n\
        \tat java.base/java.security.AccessController.doPrivileged(AccessController.java:569)\n\
        \tat org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedIOException(SocketAccess.java:38)\n\
        \tat org.elasticsearch.repositories.s3.S3BlobContainer.writeBlob(S3BlobContainer.java:142)\n\
        \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.blobstore.BlobContainer.writeBlob(BlobContainer.java:124)\n\
        \tat org.elasticsearch.repositories.s3.S3BlobContainer.writeBlobAtomic(S3BlobContainer.java:304)\n\
        \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:2205)\n\
        \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.repositories.RepositoriesService.lambda$validatePutRepositoryRequest$11(RepositoriesService.java:373)\n\
        \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.action.ActionRunnable$1.doRun(ActionRunnable.java:37)\n\
        \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1023)\n\
        \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)\n\
        \tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\
        \tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\
        \tat java.base/java.lang.Thread.run(Thread.java:833)\nCaused by: com.amazonaws.SdkClientException:\
        \ Unable to calculate a request signature: Key size for HMAC must be at least\
        \ 112 bits in approved mode: SHA-256/HMAC\n\tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:132)\n\
        \tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:105)\n\
        \t... 37 more\nCaused by: org.bouncycastle.crypto.IllegalKeyException: Key\
        \ size for HMAC must be at least 112 bits in approved mode: SHA-256/HMAC\n\
        \tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsSHS$MACOperatorFactory.createMAC(Unknown\
        \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsSHS$MACOperatorFactory.createMAC(Unknown\
        \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsMACOperatorFactory.createOutputMACCalculator(Unknown\
        \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsMACOperatorFactory.createOutputMACCalculator(Unknown\
        \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.jcajce.provider.BaseHMac.engineInit(Unknown\
        \ Source)\n\tat java.base/javax.crypto.Mac.init(Mac.java:432)\n\tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:127)\n\
        \t... 38 more\n"
    stack_trace: "java.io.IOException: Unable to upload object [ec2_base_path/tests-X8eE8K6AQYesIdw10Xk6vw/master.dat]\
      \ using a single upload\n\tat org.elasticsearch.repositories.s3.S3BlobContainer.executeSingleUpload(S3BlobContainer.java:466)\n\
      \tat org.elasticsearch.repositories.s3.S3BlobContainer.lambda$writeBlob$1(S3BlobContainer.java:144)\n\
      \tat java.base/java.security.AccessController.doPrivileged(AccessController.java:569)\n\
      \tat org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedIOException(SocketAccess.java:38)\n\
      \tat org.elasticsearch.repositories.s3.S3BlobContainer.writeBlob(S3BlobContainer.java:142)\n\
      \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.blobstore.BlobContainer.writeBlob(BlobContainer.java:124)\n\
      \tat org.elasticsearch.repositories.s3.S3BlobContainer.writeBlobAtomic(S3BlobContainer.java:304)\n\
      \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:2205)\n\
      \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.repositories.RepositoriesService.lambda$validatePutRepositoryRequest$11(RepositoriesService.java:373)\n\
      \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.action.ActionRunnable$1.doRun(ActionRunnable.java:37)\n\
      \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1023)\n\
      \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)\n\
      \tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\
      \tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\
      \tat java.base/java.lang.Thread.run(Thread.java:833)\nCaused by: com.amazonaws.SdkClientException:\
      \ Unable to calculate a request signature: Unable to calculate a request signature:\
      \ Key size for HMAC must be at least 112 bits in approved mode: SHA-256/HMAC\n\
      \tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:109)\n\
      \tat com.amazonaws.auth.AWS4Signer.newSigningKey(AWS4Signer.java:639)\n\tat\
      \ com.amazonaws.auth.AWS4Signer.deriveSigningKey(AWS4Signer.java:404)\n\tat\
      \ com.amazonaws.auth.AWS4Signer.sign(AWS4Signer.java:253)\n\tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1320)\n\
      \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)\n\
      \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)\n\
      \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)\n\
      \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)\n\
      \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)\n\
      \tat com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)\n\
      \tat com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)\n\
      \tat com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)\n\
      \tat com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456)\n\
      \tat com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403)\n\
      \tat com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:421)\n\
      \tat com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:6531)\n\
      \tat com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1861)\n\
      \tat com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1821)\n\
      \tat org.elasticsearch.repositories.s3.S3BlobContainer.lambda$executeSingleUpload$16(S3BlobContainer.java:464)\n\
      \tat org.elasticsearch.repositories.s3.SocketAccess.lambda$doPrivilegedVoid$0(SocketAccess.java:47)\n\
      \tat java.base/java.security.AccessController.doPrivileged(AccessController.java:318)\n\
      \tat org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedVoid(SocketAccess.java:46)\n\
      \tat org.elasticsearch.repositories.s3.S3BlobContainer.executeSingleUpload(S3BlobContainer.java:464)\n\
      \t... 14 more\nCaused by: com.amazonaws.SdkClientException: Unable to calculate\
      \ a request signature: Key size for HMAC must be at least 112 bits in approved\
      \ mode: SHA-256/HMAC\n\tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:132)\n\
      \tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:105)\n\
      \t... 37 more\nCaused by: org.bouncycastle.crypto.IllegalKeyException: Key size\
      \ for HMAC must be at least 112 bits in approved mode: SHA-256/HMAC\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsSHS$MACOperatorFactory.createMAC(Unknown\
      \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsSHS$MACOperatorFactory.createMAC(Unknown\
      \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsMACOperatorFactory.createOutputMACCalculator(Unknown\
      \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsMACOperatorFactory.createOutputMACCalculator(Unknown\
      \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.jcajce.provider.BaseHMac.engineInit(Unknown\
      \ Source)\n\tat java.base/javax.crypto.Mac.init(Mac.java:432)\n\tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:127)\n\
      \t... 38 more\n"
  stack_trace: "org.elasticsearch.repositories.RepositoryVerificationException: [repository_ec2]\
    \ path [ec2_base_path] is not accessible on master node\nCaused by: java.io.IOException:\
    \ Unable to upload object [ec2_base_path/tests-X8eE8K6AQYesIdw10Xk6vw/master.dat]\
    \ using a single upload\n\tat org.elasticsearch.repositories.s3.S3BlobContainer.executeSingleUpload(S3BlobContainer.java:466)\n\
    \tat org.elasticsearch.repositories.s3.S3BlobContainer.lambda$writeBlob$1(S3BlobContainer.java:144)\n\
    \tat java.base/java.security.AccessController.doPrivileged(AccessController.java:569)\n\
    \tat org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedIOException(SocketAccess.java:38)\n\
    \tat org.elasticsearch.repositories.s3.S3BlobContainer.writeBlob(S3BlobContainer.java:142)\n\
    \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.blobstore.BlobContainer.writeBlob(BlobContainer.java:124)\n\
    \tat org.elasticsearch.repositories.s3.S3BlobContainer.writeBlobAtomic(S3BlobContainer.java:304)\n\
    \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:2205)\n\
    \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.repositories.RepositoriesService.lambda$validatePutRepositoryRequest$11(RepositoriesService.java:373)\n\
    \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.action.ActionRunnable$1.doRun(ActionRunnable.java:37)\n\
    \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1023)\n\
    \tat org.elasticsearch.server@8.18.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)\n\
    \tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\
    \tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\
    \tat java.base/java.lang.Thread.run(Thread.java:833)\nCaused by: com.amazonaws.SdkClientException:\
    \ Unable to calculate a request signature: Unable to calculate a request signature:\
    \ Key size for HMAC must be at least 112 bits in approved mode: SHA-256/HMAC\n\
    \tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:109)\n\t\
    at com.amazonaws.auth.AWS4Signer.newSigningKey(AWS4Signer.java:639)\n\tat com.amazonaws.auth.AWS4Signer.deriveSigningKey(AWS4Signer.java:404)\n\
    \tat com.amazonaws.auth.AWS4Signer.sign(AWS4Signer.java:253)\n\tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1320)\n\
    \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)\n\
    \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)\n\
    \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)\n\
    \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)\n\
    \tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)\n\
    \tat com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)\n\
    \tat com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)\n\t\
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)\n\t\
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456)\n\
    \tat com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403)\n\
    \tat com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:421)\n\
    \tat com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:6531)\n\
    \tat com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1861)\n\
    \tat com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1821)\n\
    \tat org.elasticsearch.repositories.s3.S3BlobContainer.lambda$executeSingleUpload$16(S3BlobContainer.java:464)\n\
    \tat org.elasticsearch.repositories.s3.SocketAccess.lambda$doPrivilegedVoid$0(SocketAccess.java:47)\n\
    \tat java.base/java.security.AccessController.doPrivileged(AccessController.java:318)\n\
    \tat org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedVoid(SocketAccess.java:46)\n\
    \tat org.elasticsearch.repositories.s3.S3BlobContainer.executeSingleUpload(S3BlobContainer.java:464)\n\
    \t... 14 more\nCaused by: com.amazonaws.SdkClientException: Unable to calculate\
    \ a request signature: Key size for HMAC must be at least 112 bits in approved\
    \ mode: SHA-256/HMAC\n\tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:132)\n\
    \tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:105)\n\t\
    ... 37 more\nCaused by: org.bouncycastle.crypto.IllegalKeyException: Key size\
    \ for HMAC must be at least 112 bits in approved mode: SHA-256/HMAC\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsSHS$MACOperatorFactory.createMAC(Unknown\
    \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsSHS$MACOperatorFactory.createMAC(Unknown\
    \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsMACOperatorFactory.createOutputMACCalculator(Unknown\
    \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.crypto.fips.FipsMACOperatorFactory.createOutputMACCalculator(Unknown\
    \ Source)\n\tat org.bouncycastle.fips.core/org.bouncycastle.jcajce.provider.BaseHMac.engineInit(Unknown\
    \ Source)\n\tat java.base/javax.crypto.Mac.init(Mac.java:432)\n\tat com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:127)\n\
    \t... 38 more\n"
status: 500
]

Issue Reasons:

  • [8.x] 2 consecutive failures in step openjdk17_checkpart1_java-fips-matrix
  • [8.x] 6 failures in class org.elasticsearch.repositories.s3.RepositoryS3ClientYamlTestSuiteIT (1.5% fail rate in 413 executions)
  • [8.x] 6 failures in step openjdk17_checkpart1_java-fips-matrix (75.0% fail rate in 8 executions)
  • [8.x] 6 failures in pipeline elasticsearch-periodic (54.5% fail rate in 11 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

Metadata

Metadata

Assignees

Labels

:Distributed Coordination/Snapshot/RestoreAnything directly related to the `_snapshot/*` APIs>test-failureTriaged test failures from CITeam:Distributed CoordinationMeta label for Distributed Coordination teamlow-riskAn open issue or test failure that is a low risk to future releases

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions