Add loggingconfig rule

[Sangarshanan]

Closes: #11046

Adds LoggingConfigurationRule rule to upgrade/rules as per:

https://github.com/apache/airflow/blob/master/UPDATING.md#logging-configuration-has-been-moved-to-new-section

---

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

[turbaszek]

I would suggest checking if any of this key is used by users (not empty or different than default - this can be tricky) and then print a message for each incompatibility. See #11056 for reference. In this way, the message will be more precise. What do you think?

[Sangarshanan]

Ah I was not thinking of this cause this change is backwards compatible and not something that can break if not moved but you are right, showing only the incompatibility really helps while taking actions, I can do a comparison and return only the actively used keys but what of the default/ empty keys is it right that they still remain in the old section ?

[turbaszek]

what of the default/ empty keys is it right that they still remain in the old section ?

I think it will be ok to hardcode them in rule using default values from airflow.cfg of 1.10.x. WDYT @kaxil ?

[kaxil]

Yea default values can be hardcoded.

And also agree that showing a precise message for the used config will be better

[turbaszek]

Will this work if the logging section is not present? Have you tested it?

[Sangarshanan]

Yep has_option returns false when the section is not present

[turbaszek]

Look good to me 🚀

Logging configuration has been moved to new section
---------------------------------------------------
The logging configurations have been moved from [core] to the new [logging] section.

Problems:
  1.  task_log_prefix_template has been moved from [core] to a the new [logging] section.

[turbaszek]

Please rebase your PR, we added few changes that should fix the CI issues. Please do rebase and try to avoid merge commits, more information:
https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#id9

[turbaszek]

@Sangarshanan I think that the Conditional MySQL Client installation (#11174) and Allow overrides for pod_template_file (#11162) should not be part of your change

[github-actions]

The CI and PROD Docker Images for the build are prepared in a separate "Build Image" workflow,
that you will not see in the list of checks (you will see "Wait for images" jobs instead).

You can checks the status of those images in The workflow run

[turbaszek]

@Sangarshanan it seems that one test is failing:

_______________________________________ TestLoggingConfigurationRule.test_valid_check _______________________________________
/usr/local/lib/python3.7/contextlib.py:73: in inner
    with self._recreate_cm():
/usr/local/lib/python3.7/contextlib.py:112: in __enter__
    return next(self.gen)
tests/test_utils/config.py:45: in conf_vars
    conf.set(section, key, value)
/usr/local/lib/python3.7/site-packages/backports/configparser/__init__.py:1239: in set
    super(ConfigParser, self).set(section, option, value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <airflow.configuration.AirflowConfigParser object at 0x7fa366479bd0>, section = 'logging', option = 'base_log_folder'
value = 'DUMMY'

    def set(self, section, option, value=None):
        """Set an option."""
        if value:
            value = self._interpolation.before_set(self, section, option,
                                                   value)
        if not section or section == self.default_section:
            sectdict = self._defaults
        else:
            try:
                sectdict = self._sections[section]
            except KeyError:
>               raise from_none(NoSectionError(section))
E               backports.configparser.NoSectionError: No section: 'logging'

Personally I would remove the test_valid_check test 👍

[mik-laj]

@turbaszek This is an interesting problem that also occurs in other cases. I think that it is also worth solving separately. If you have a configuration file from Airflow 2 and you run it on Airflow 1.10, then you can't run: ʻairflow config`

[turbaszek]

@turbaszek This is an interesting problem that also occurs in other cases. I think that it is also worth solving separately. If you have a configuration file from Airflow 2 and you run it on Airflow 1.10, then you can't run: ʻairflow config`

I agree, but as you said, this should be done in separate PR

[github-actions]

The CI and PROD Docker Images for the build are prepared in a separate "Build Image" workflow,
that you will not see in the list of checks (you will see "Wait for images" jobs instead).

You can checks the status of those images in The workflow run

[turbaszek]

All upgrade tests passed:

tests/upgrade/test_formattes.py .                                        [ 86%]
706
tests/upgrade/test_problem.py ..                                         [ 86%]
707
tests/upgrade/rules/test_base_rule.py .                                  [ 86%]
708
tests/upgrade/rules/test_conn_id_is_unique.py .                          [ 86%]
709
tests/upgrade/rules/test_conn_type_is_not_nullable.py .                  [ 86%]
710
tests/upgrade/rules/test_fernet_enabled.py ..                            [ 86%]
711
tests/upgrade/rules/test_import_changes.py ...                           [ 86%]
712
tests/upgrade/rules/test_logging_configuration.py .                      [ 86%]
713
tests/upgrade/rules/test_send_grid_moved.py ..                           [ 86%]