Introduce deprecated features management

[dumbbell]

This introduces a way to declare deprecated features in the code, not only in our communication. The new module allows to disallow the use of a deprecated feature and/or warn the user when he relies on such a feature.

Why

Currently, we only tell people about deprecated features through blog posts and the mailing-list. This might be insufficiant for our users that a feature they use will be removed in a future version:

• They may not read our blog or mailing-list
• They may not understand that they use such a deprecated feature
• They might wait for the big removal before they plan testing
• They might not take it seriously enough

The idea behind this patch is to increase the chance that users notice that they are using something which is about to be dropped from RabbitMQ. Anopther benefit is that they should be able to test how RabbitMQ will behave in the future before the actual removal. This should allow them to test and plan changes.

How

When a feature is deprecated in other large projects (such as FreeBSD where I took the idea from), it goes through a lifecycle:

1. The feature is still available, but users get a warning somehow when they use it. They can disable it to test.
2. The feature is still available, but disabled out-of-the-box. Users can re-enable it (and get a warning).
3. The feature is disconnected from the build. Therefore, the code behind it is still there, but users have to recompile the thing to be able to use it.
4. The feature is removed from the source code. Users have to adapt or they can't upgrade anymore.

The solution in this patch offers the same lifecycle. A deprecated feature will be in one of these deprecation phases:

1. permitted_by_default: The feature is available. Users get a warning if they use it. They can disable it from the configuration.
2. denied_by_default: The feature is available but disabled by default. Users get an error if they use it and RabbitMQ behaves like the feature is removed. They can re-enable is from the configuration and get a warning.
3. disconnected: The feature is present in the source code, but is disabled and can't be re-enabled without recompiling RabbitMQ. Users get the same behavior as if the code was removed.
4. removed: The feature's code is gone.

The whole thing is based on the feature flags subsystem, but it has the following differences with other feature flags:

• The semantic is reversed: the feature flag behind a deprecated feature is disabled when the deprecated feature is permitted, or enabled when the deprecated feature is denied.
• The feature flag behind a deprecated feature is enabled out-of-the-box (meaning the deprecated feature is denied):
  • if the deprecation phase is permitted_by_default and the configuration denies the deprecated feature
  • if the deprecation phase is denied_by_default and the configuration doesn't permit the deprecated feature
  • if the deprecation phase is disconnected or removed
• Feature flags behind deprecated feature don't appear in feature flags listings.

Otherwise, deprecated features' feature flags are managed like other feature flags, in particular inside clusters.

To declare a deprecated feature:

-rabbit_deprecated_feature(
   {my_deprecated_feature,
    #{deprecation_phase => permitted_by_default,
      msgs => #{when_permitted => "This feature will be removed in RabbitMQ X.0"}
     }}).

Then, to check the state of a deprecated feature in the code:

case rabbit_deprecated_features:is_permitted(my_deprecated_feature) of
    true ->
        %% The deprecated feature is still permitted.
        ok;
    false ->
        %% The deprecated feature is gone or should be considered
        %% unavailable.
        error
end.

Warnings and errors are logged automatically. A message is generated automatically, but it is possible to define a message in the deprecated feature flag declaration like in the example above.

Here is an example of a logged warning that was generated automatically:

Feature `my_deprecated_feature` is deprecated.
By default, this feature can still be used for now.
Its use will not be permitted by default in a future minor RabbitMQ version and the feature will be removed from a future major RabbitMQ version; actual versions to be determined.
To continue using this feature when it is not permitted by default, set the following parameter in your configuration:
    "deprecated_features.permit.my_deprecated_feature = true"
To test RabbitMQ as if the feature was removed, set this in your configuration:
    "deprecated_features.permit.my_deprecated_feature = false"

To override the default state of permitted_by_default and denied_by_default deprecation phases, users can set the following configuration:

# In rabbitmq.conf:
deprecated_features.permit.my_deprecated_feature = true # Or false

The actual behavior protected by a deprecated feature check is out of scope for this subsystem. It is the repsonsibility of each deprecated feature code to determine what to do when the deprecated feature is denied.

[dumbbell]

This is a work in progress, but here is what it could look like:

First, you declare a deprecated feature:

-rabbit_deprecated_feature(
   {classic_mirrored_queues,
    #{deprecation_phase => optout,
      warning =>
      "Classic mirrored queues are deprecated and will go away in RabbitMQ 4.x",
      doc_url => "https://blog.rabbitmq.com/posts/2021/08/4.0-deprecation-announcements/"
     }}).

The deprecation phases would be:

1. opt-out; the feature is still enabled by default and can be disabled in the configuration
2. opt-in; the feature is disabled by default and can be re-enabled in the configuration
3. disconnected; the feature is disabled and can't be re-enabled, except by recompiling RabbitMQ or the plugin
4. removed; the feature is gone entirely; we need to keep the declaration because for the same purpose we have required feature flags

Then in the code, we can verify if the feature is allowed to be used:

%% Example with `rabbit_mirror_queue_misc:validate_policy/1
validate_policy(KeyList) ->
    FeatureName = classic_mirrored_queues,
    case rabbit_deprecated_features:is_permitted(FeatureName) of
        false ->
            Warning = rabbit_deprecated_features:get_warning(FeatureName),
            {error, "~ts", [Warning]};
        true ->
            Mode = proplists:get_value(<<"ha-mode">>, KeyList, none),
            Params = proplists:get_value(<<"ha-params">>, KeyList, none),
            ...

Calling rabbit_deprecated_features:is_permitted/1 would log a warning on a regular basis, say once a day.

With this example, the warning is also returned as an error when someone tries to add an HA policy. For instance, in the management UI:

[dumbbell]

As an example, here is a branch where classic mirrored queues are deprecated:
https://github.com/rabbitmq/rabbitmq-server/tree/deprecate-classic-mirrored-queues