[AIRFLOW-2325] Add cloudwatch task handler (IN PROGRESS)

[fangpenlin]

Make sure you have checked all steps below.

JIRA

• My PR addresses the following Airflow JIRA issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"
  • https://issues.apache.org/jira/browse/AIRFLOW-2325

Description

• Here are some details about my PR, including screenshots of any UI changes:

Tests

• My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

• My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

Documentation

• In case of new functionality, my PR adds documentation that describes how to use it.
  • When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.

Code Quality

• Passes git diff upstream/master -u -- "*.py" | flake8 --diff

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ericabertugli]

Hallo, is it possible to re-open this PR?
I think this code is really useful for everybody working with AWS, and I've been using it for a while without any problem. Thanks!

[potiuk]

@ericabertugli maybe you can take it over (seems it's been abandoned by @fangpenlin ). Id you could rebase and re-submit for review? I am happy to merge it but I think what is missing here are some basic tests.

[mik-laj]

https://airflow.readthedocs.io/en/latest/howto/write-logs.html
There is a lack of documentation in this PR. Can you add it?

[ericabertugli]

@ericabertugli maybe you can take it over (seems it's been abandoned by @fangpenlin ). Id you could rebase and re-submit for review? I am happy to merge it but I think what is missing here are some basic tests.

@potiuk, I'll try to do some tests and re-submit it. I'll send updates in one week.

[fangpenlin]

@ericabertugli @potiuk @mik-laj yeah, sorry I have no time to work on this. Feel free to take over my PR and continue working on it 👍

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[dhegberg]

@ericabertugli Are you still working on this?

I started doing some testing and I'm happy to take over.

I'd write some tests, add to the logging documentation and add an entry in airflow_local_settings.

@potiuk
I was thinking of using a URL preflix like 'cloudwatch://' in the remote_base_log_folder. This seems a bit weird since it's not a folder but it looks like the stackdriver option has already gone this route.

[dhegberg]

Updated PR: #7437