[FLINK-26300][connectors/common] Increased timeout for LocalstackCont…

[CrynetLogistics]

…ainer to start

What is the purpose of the change

Fix the LocalstackContainer startup loop.

Explanation: The localstack container is used to mock aws services so tests can hit a mock endpoint. When we start the container, it occasionally fails with the exception: IOException - closed before protocol could be determined. This is what happens whenever we make a request (e.g. read from mock s3) prior to the container being ready**. A similar issue was resolved in Kinesalite by increasing the timeout from 1s to 10s, but Localstack is a larger box. So here we increase the wait to 30s.

** This is a known limitation, please see issue #1202 in Localstack/localstack or this blog post.

Brief change log

Increased timeout of container startup to 30s from 10s.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): (yes / no)n
• The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)n
• The serializers: (yes / no / don't know)n
• The runtime per-record code paths (performance sensitive): (yes / no / don't know)n
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)n
• The S3 file system connector: (yes / no / don't know)n

Documentation

• Does this pull request introduce a new feature? (yes / no)n
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)n

[CrynetLogistics]

@flinkbot azure run

[CrynetLogistics]

@flinkbot run azure

[flinkbot]

Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.

Automated Checks

Last check on commit b16fd24 (Tue Feb 22 16:57:27 UTC 2022)

Warnings:

• No documentation files were touched! Remember to keep the Flink docs up to date!

_{Mention the bot in a comment to re-run the automated checks.}

Review Progress

• ❓ 1. The [description] looks good.
• ❓ 2. There is [consensus] that the contribution should go into to Flink.
• ❓ 3. Needs [attention] from.
• ❓ 4. The change fits into the overall [architecture].
• ❓ 5. Overall code [quality] is good.

Please see the Pull Request Review Guide for a full explanation of the review process.

Details

The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot approve description to approve one or more aspects (aspects: description, consensus, architecture and quality)
• @flinkbot approve all to approve all aspects
• @flinkbot approve-until architecture to approve everything until architecture
• @flinkbot attention @username1 [@username2 ..] to require somebody's attention
• @flinkbot disapprove architecture to remove an approval you gave earlier

[flinkbot]

CI report:

• Unknown: CANCELED
• 4c10f8e UNKNOWN

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[dannycranmer]

Please justify the reason for this.

Is there any other way we can establish if the test stack is ready?

[CrynetLogistics]

I couldn't see a good way to poll whether the container is ready. On the testcontainers side, I tried tracking the output of isRunning()/isCreated()/isReusable() through successful and unsuccessful runs and none of them are telling. Anything on the localstack api side puts it in a funny state in the same way the s3bucket read strategy does.

[dannycranmer]

There is a solution here: https://ignas.me/tech/waiting-localstack-s3-start/

I’ve noticed that hitting Refresh button in LocalStack Dashboard calls /graph endpoint to know if and how many nodes (mock services) are available, i.e. healthy and ready to accept incoming requests. Bullseye!

[dannycranmer]

Can you create a follow up for this improvement and we can merge this to unblock the testing?

[CrynetLogistics]

Thanks, I have created this: https://issues.apache.org/jira/browse/FLINK-26309.

[CrynetLogistics]

@flinkbot run azure

[CrynetLogistics]

@flinkbot run azure

[CrynetLogistics]

@flinkbot run azure