Respect deprecated options in default config parser

[jason810496]

• related: Use the regular OTel environment variables for configuring traces and metrics #56150

Why

Previously, the default value parser logic in AirflowConfigParser did not check for version_deprecated or deprecation_reason for default config.yaml

This means that if we really deprecated a [config/key], like the case of #56150 (comment) (not renaming or moving to other section like AF2 -to AF3), the conf.get will still get default value from the config.yaml.

From my perspective, the default value parser should respect deprecated options (the option that provide either version_deprecated or deprecation_reason) and ignore it.

What

• Update the config parser to respect deprecated options for default value parser
• Extract the configure_parser_from_configuration_description logic to shared lib

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Copilot (ChatGPT 4.1), Gemini-CLI (auto model) following the guidelines