Enforce deprecation process

[bruceg]

This is a mini-RFC proposal for a new deprecation process.

While Vector has a deprecation policy, we don't have any process for actually ensuring the policy is followed in a timely and visible manner. This has showed up in areas like the Lua transform version 2 mode being introduced in March 2020 with an indication that version 1 would be deprecated, but the actual deprecation not actually being announced for over two years.

• New file, docs/DEPRECATIONS.md, with three sections, each corresponding to one of the stages, consolidating references to all the affected facets in one location:
  1. Facets that are planned to be deprecated,
  2. Facets that are planned to be migrated, and
  3. Facets that are planned to be removed.
• Each facet in the file will have an identifier tag, a version number indicating the Vector version by which we need to take the appropriate action, and a brief one-line description as a reminder (unused). The identifiers will be machine readable to allow for more automated enforcement.
  Examples:
  • lua_v1 v0.52 The lua transform v1 mode is obsolete and needs to be removed
• The release process script will be updated to ensure that:
  1. No facets remain marked as to be deprecated, migrated, or removed in this version, and
  2. All new deprecations, migrations, and removals have a section in the upgrade notes.

Tasks:

• Set up and populate new deprecations list document
• Update release process to check deprecations

---

Future work:

• Source code will be marked with the tag at every point the deprecation is relevant. These will not contain data about when the deprecation should happen, as that will be tracked separately. Descriptive text is fine if there are special instructions or other additional information needed.
  Example:
  // DEPRECATION: lua_v1 Remove everything under this module
• A new CI check script will be added to scan all PRs. It will look for any source code that looks like a deprecation has been added, and ensure that:
  1. There is a deprecation marker near by, and
  2. The deprecation marker is listed in the above document.
     Together this will help ensure that all deprecations are tracked, and that the removal process checks all the locations where a deprecation is noted.

• Create deprecation CI check script

[bruceg]

Example DEPRECATIONS.md file:

# To be deprecated

* lua_v1 v0.52 The lua transform v1 mode is obsolete and needs to be removed
* foobar v0.26 Foobar needs to die a slow and painful death

# To be migrated

* barbaz v0.28 We are moving users to the Babar instead

# To be removed

* thatthing v0.29 That thing is obsolete and has been migrated

[fuchsnj]

I'm a fan of the DEPRECATIONS.md file , but I'm not sure the code comments will be useful, especially the wording around "at every point the deprecation is relevant", which could be very extensive depending on what is deprecated.

Not even sure if the automated script is really needed, I think just having a document with the deprecations and the version numbers will get us 90% of the improvement here

[tobz]

One thing to note is that I had planned, in part, to utilize some of the Configuration Schema work to drive deprecation automation.

To wit, I've been unofficially adding #[configurable(deprecated)] to configuration fields that were already marked/listed as deprecated. This simply adds data to the configuration schema to mark those fields as being deprecated. It doesn't yet power anything in the resulting website documentation, nor anything in CI.

My plan was/is to generate a schema document every time we cut an official, versioned release. These might live in this repository, or in another repository. Every time we cut an official, versioned release from then on, we would check which parts of the schema had changed, and for fields that had been changed/removed, we would check the previous version's schema to see if they were already marked as deprecated... the idea being that then we could detect when we were about to make user-visible changes that were not properly communicated as being deprecated.

I also believe this data, in the configuration schema, could potentially be used to power generating the above DEPRECATIONS file as well, even.

Additionally, we could potentially change the Configurable helper attribute to take more inputs, such that we could use it to drive compile-time-based warnings that code which was previously deprecated, is now still present when it should be removed.

As an example, we might deprecate a field and mark it as deprecated in version 0.25. Once the project bumps to version 0.26 -- even when not released yet -- the compiler could potentially start emitting a non-blocking warning that the code still exists, even though it should theoretically be removed as of the current 0.26 version.