Adding AWSBatch JobExists and JobExecutionComplete waiter

[hugoprudente]

Adding the JobExists and JobExecutionComplete to manipulate AWS Bach jobs easily using boto3

Usage:

JobExists:

import boto3
client = boto3.client('batch')
waiter = client.get_waiter('job_exists')

print(waiter.wait(jobs=['JOB_ID']))

JobExecutionComplete:

import boto3
client = boto3.client('batch')
waiter = client.get_waiter('job_execution_complete')

print(waiter.wait(jobs=['JOB_ID']))

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (develop@7aacea5). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             develop    #1307   +/-   ##
==========================================
  Coverage           ?   92.54%           
==========================================
  Files              ?       53           
  Lines              ?     9958           
  Branches           ?        0           
==========================================
  Hits               ?     9216           
  Misses             ?      742           
  Partials           ?        0

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7aacea5...55435e3. Read the comment docs.

[hugoprudente]

Interesting.

This file was not changed from the latest commit and I have rebased and there is no difference that explains the above Delta.

[hugoprudente]

Any news on it? it's already 5 months.

[heyalexchoi]

bump

[johnchenghk01]

Any news?

[Raphtor]

Bump

[lohmuller]

+1

[sdanbury]

What do we need to do to get this in please? Looks like checks have passed, etc.

[quazzuk]

This would be really useful, why has nobody merged yet?!

[dazza-codes]

Airflow AWS Batch Operator tries to use this, but it's not available yet, right?

• https://github.com/apache/airflow/blob/master/airflow/contrib/operators/awsbatch_operator.py#L117

The wide use of Airflow means that the AWS Batch API could get hit more frequently than necessary without this feature?

• https://issues.apache.org/jira/browse/AIRFLOW-5218

The retry frequency and the max-attempts should be surfaced as parameters on the methods rather than hard-coded, right?

[hugoprudente]

Ensuring that the CI is still cool to merge so one day this get's to the mainline.

[AlJohri]

bumping this PR. can we get this merged?

[m4ce]

+1

[dazza-codes]

According to https://github.com/boto/botocore/blob/develop/CONTRIBUTING.rst#contributing-code, this PR should be considered and merged. In apache/airflow#6811 I was able to confirm that these waiters can work OK, including moto unit-tests that work. If this is pending because it needs unit tests, please see that Airflow PR for examples.

[cariaso]

+1 many people have asked for the merge, and it seems all checks and concerns have been addressed.

[hugoprudente]

Happy Birthday PR!!! You successfully completed 2 years in the waiting queue and had only 1 Rebase :) So proud of you!

[stobrien89]

Hi all,

Thanks for the request and our apologies for the lack of attention to this. We are now submitting requests for paginators and waiters upstream so that all AWS SDKs benefit from a consistent implementation. I will pass on the request for these waiters to the service team. We will also update the guidance in the contributing guide to better reflect this policy. I've added an issue to track this here:

aws/aws-sdk#84

Thanks again for bringing it to our attention!

[hugoprudente]

That's is really sad in 2 years you guys have changed the policy and found a way to not merge the code that was ready.

Now the new aws-sdk will take more 2 years to have a simple waiter...

[cariaso]

2 years ....