HEP 2: Release and deprecation policies

[maximlt]

Warning: builds on #386

---

This PR adds HEP 2: Release and deprecation policies. It is motivated by a discussion we had in holoviz/panel#5221, and by my interest in ensuring that the HoloViz projects apply consistent policies, both for its users and maintainers. The HEP formalizes practices that have been in place for a long time and adds a few more rules on top of those. For instance, I've tried to come up with a clearer process for maintainers to adopt when they deprecate a feature. I've also suggested a minimum deprecation period of 18 months before removal, I'm sure there will be opinions about this period, and I'm also quite convinced we need to define such a period.

I drew some inspiration from:

• https://docs.djangoproject.com/en/dev/internals/release-process/#release-cadence
• https://numpy.org/neps/nep-0023-backwards-compatibility.html

I haven't touched all things on this topic, either because I didn't want to or didn't know what to suggest, I'd say these can be out of scope for now:

• I've addressed deprecation + removal, the HEP however says nothing about what should be done when changing the behavior of some API without removal (e.g. announcing the change of a default parameter value)
• Can a single maintainer decide to move/remove some API without getting anyone else involved? What if maintainers/contributors/users disagree?

[hoxbro]

Overall, this HEP is great.

However, I'm very much against being unable to put a version of when a deprecation is removed.

Some technical details have also made it into this HEP. An example of this is the release project section. It is not that it is wrong, but if we, at some point, want to change it to no longer be triggered by pushing a tag, we need to update this HEP and get the approval.

[hoxbro]

I don't buy this. Adding a version when the removal is being made is fine for both the users and us as developers.

I also think defining the deprecation period of 18 months is too long for some packages (Panel) because a lot of releases can have happened in that time period, whereas for other packages, it may not be enough (colorcet). In a worst-case scenario, a user could download a package on the last day of the 17th month and start using it, only to see it removed the next day because of a new release.

By keeping a table, updating it, and checking it, we also add extra burden on us as maintainers to keep it up to date. Having the deprecations as close to the code as possible makes it easier to avoid it from getting out of sync.

Again, I'm very much on having at least the option to put the version where the removal is gonna happen in the warning message, like what Matplotlib and xarray does. To be clear, this is not what all packages do. An example of this is pandas, which postpone it to the future, but they seem to be moving towards more major releases with breaking changes.

[maximlt]

Adding a version when the removal is being made is fine for both the users and us as developers.

My main goal here is to define a minimum deprecation period that all HoloViz projects can adhere to. I think it'd have benefits for its users and maintainers. I could see allowing projects to specify the removal version as long as they ensure that it meets the minimum deprecation period. For example, if a project releases 1.0.0 and announces a feature will be removed in 1.3.0, the feature can only be removed X months after 1.0.0, even if 1.3.0 happens before.

I also think defining the deprecation period of 18 months is too long for some packages (Panel) because a lot of releases can have happened in that period, whereas for other packages, it may not be enough (colorcet).

I don't think the number or frequency of releases matters so much for the deprecation period. The deprecation period is mostly a tradeoff between letting users have enough time to adapt (apps and libraries) and reducing the cost of maintaining a project. I want HoloViz as a whole project to agree on a minimum deprecation period. Individual projects are free to adopt longer deprecation periods.

In a worst-case scenario, a user could download a package on the last day of the 17th month and start using it, only to see it removed the next day because of a new release.

This isn't a very good counter-example, as this is all about probability, and in that case well, bad luck! The same could happen with a shorter or longer deprecation period. Just to be clear, I'm not saying you should make a new release right after the deprecation period is over.

By keeping a table, updating it, and checking it, we also add extra burden on us as maintainers to keep it up to date. Having the deprecations as close to the code as possible makes it easier to avoid it from getting out of sync.

I don't think HoloViz projects are used to deprecating and removing features, while I'd argue they should do it more to clean up their ever-growing API that sometimes confuses their users. For instance, I personally rarely check whether some code could be removed when I do a new minor release. I haven't built that muscle working on HoloViz projects. How about other maintainers? So I'm suggesting here a systematic approach to dealing with deprecated features, I don't think it's so much extra burden, and I anyway think it's part of the maintainer's job and more precisely of the release manager.

Again, I'm very much on having at least the option to put the version where the removal is gonna happen in the warning message, like what Matplotlib and xarray does. To be clear, this is not what all packages do. An example of this is pandas, which postpone it to the future, but they seem to be moving towards more major releases with breaking changes.

• Matplotlib has great docs on how to deprecate a feature (https://matplotlib.org/devdocs/devel/api_changes.html#rules), it's a minimum of 2 minor releases later and possibly more. Its 3.x releases were: 3.8 (09/2023), 3.7 (02/2023), 3.6 (06/2022), 3.5 (11/2021), 3.4 (03/2021), 3.3 (07/2020), 3.2 (03/2020), 3.1 (05/2019), 3.0 (09/2018). The time between two minor releases is in months: 18, 14, 12, 16, 15, 15 and 15. It's a pretty stable release cadence so I think it's fair for them to announce they will remove a feature in 2 releases. HoloViz projects could also decide to adopt a regular-ish release cadence, though it might be more extra burden than what I'm suggesting.
• Xarray's contributor docs doesn't say you should say when a feature is removed, maybe their docs and code isn't in sync https://docs.xarray.dev/en/stable/contributing.html#backwards-compatibility.

[hoxbro]

I think the tag message should be removed or at least loosened, as it is, in my opinion, not important.

[maximlt]

I suggested that because I observed that the message was pretty inconsistent across releases in a project, and across projects. I like consistency. But you're also right that it is not the most important part of the HEP.

[hoxbro]

This could also explicitly say that no new features can be added after the first RC, and only regressions and bugs should be handled.

Must is also a strong word. I would say another of the dev releases is also good enough for a patch release.

An expectation of a release candidate for some is that it has a time period where people have a chance to test it. What I see in our cases is that we make a release candidate to test if our CI runs correctly (with downstream test), and when it is finished, we make the official release.

[maximlt]

This could also explicitly say that no new features can be added after the first RC, and only regressions and bugs should be handled.

Yes, I didn't think about that this is interesting.

Must is also a strong word. I would say another of the dev releases is also good enough for a patch release.

I'm suggesting replacing the alpha releases (often just one) made before a patch release with release candidates.

An expectation of a release candidate for some is that it has a time period where people have a chance to test it.

Yes, that's true. The HEP could explain that release candidates are always announced sometime before the final release.

What I see in our cases is that we make a release candidate to test if our CI runs correctly (with downstream test), and when it is finished, we make the official release.

Not always, we have applications that test release candidates before the final release. It's not the most common case though, even if it's difficult to say whether other users rely on dev releases in their internal code bases.

[maximlt]

Thanks for your review @hoxbro !

However, I'm very much against being unable to put a version of when a deprecation is removed.

I've commented on that in #388 (comment).

Some technical details have also made it into this HEP. An example of this is the release project section. It is not that it is wrong, but if we, at some point, want to change it to no longer be triggered by pushing a tag, we need to update this HEP and get the approval.

Yep true that's a bit too technical and I can remove it, it's already documented in the HoloViz Contributing Guide.

But I have to say I'm not sure yet what exactly could and should get into a HEP. For instance, Matplotlib developer docs (https://matplotlib.org/devdocs/devel/) include a section with MEPs and many other sections on e.g. Dependency version policy (the equivalent of our HEP 1) or API Changes (the equivalent of this HEP).

So in which case should we need a new HEP (if we need HEPs at all?)? @droumis pinging you since you told me you were thinking about writing HEP0.

[maximlt]

I just talked with Simon and told him I was open to adopting another way to define the deprecation period, as long as there's some sort of consistency across the projects. I suggested a minimum deprecation period of 18 months, he prefers a minimum number of minor versions (correct me if I'm wrong). I still prefer a minimum deprecation period, but I think that in the end whatever we adopt will be better than the status quo!

[maximlt]

Simon opened holoviz/geoviews#701 to remove some features of GeoViews. I started to have a look at it and felt it was a good use case, and well, it was just in front of my eyes at the right time! This helps me understand the whole deprecation process and will hopefully help improve the HEP. By the way, this is all very factual, I don't intend to blame anyone :)

Shared

• The deprecations were released in GeoViews 1.10.0 on the 25th of May 2023. They were planned to be removed one minor release later in GeoViews 1.11. Version 1.11.0 was released on the 1st of November 2023, 5 months later. Removing the deprecations was not done in this first patch release, it seems this is going to happen in version 1.11.1.
• It looks like GeoViews has some code to error when the removal version matches the current working version (in https://github.com/holoviz/geoviews/blame/main/geoviews/_warnings.py#L63). I guess it didn't catch this because the deprecated features were not exerted in the test suite.
• The warning emitted is a GeoviewsDeprecationWarning, a subclass of DeprecationWarning. Given how users run their code, this type of warning is not always emitted (see the section at the end)

load_tiff

• It's not clear to me why gv.util.load_tiff was deprecated in the first place, one user reported in deprecated synthaxe to open tiff using xarray geoviews#639 that this function was using a deprecated API. Maybe load_tiff could have been adapted instead.
• Reading the PR that deprecated gv.util.load_tiff (Deprecate load_tiff geoviews#640), it isn't clear to me that the class methods Image.load_tiff and RGB.load_tiff were also deprecated.
• The deprecation was included in the release notes (Github and website) of version 1.10.0. The version targeted for removal was not mentioned there (thought it could have been inferred from the indication that the Wikipedia tile source was to be removed in 1.11.0)
• The deprecation was not mentioned in the docstrings. GeoViews has no API docs.
• gv.util.load_tiff isn't used in the documentation, so it didn't have to be updated :)
• The programmatic warning included an alternative with geoviews.util.from_xarray(rioxarray.open_rasterio(filename). The release notes/documentation included an alternative too but not as clear as the programmatic one, with rioxarray.open_rasterio to load GeoTIFFs into a xarray.DataArray.
• load_tiff isn't used much out there (I can't tell if it was more used before 1.10 was released). The first two code bases have no test suite.
  • https://github.com/venuswku/data-visualizer/blob/eebc31e71f499b4f37ba48861c8abe9c62d2bd5f/data_visualizer/components/DataMap.py#L403
  • https://github.com/GlacioHack/geoprofile/blob/0a48380f05bf2f8824d98ece14237f537e345f5c/geoprofile/core/core.py#L357
  • EarthSim:
    • https://github.com/holoviz-topics/EarthSim/blob/5d6bde55feaa6e95e292a90575dfc5179f5b3beb/earthsim/grabcut.py#L384
    • (gv.RGB.load_tiff) https://github.com/holoviz-topics/EarthSim/blob/5d6bde55feaa6e95e292a90575dfc5179f5b3beb/examples/user_guide/WMTS_Example.ipynb#L102

Wikipedia WMTS tile

• The issue (Deprecate Wikipedia tile geoviews#623) and the PR (Deprecate Wikipedia tile and CI maintenance geoviews#630) don't mention why this is deprecated.
• Tile source replacement warnings holoviews#4941 (from May 2021) has more context on this deprecation. I have no time to dig more into it, there's a lot of discussion about how things should be deprecated, with a Parameter controlling some of the warning/error behavior. It seems the way this was deprecated in HoloViews vs GeoViews was quite different. The deprecated API was removed in HoloViews in Remove deprecations holoviews#5654 (March 2023). The default tile was changed from Wikipedia to OSM in hvPlot in Switch default tile source to OSM hvplot#610 (May 2021), the documentation wasn't updated accordingly (I'll open a PR for that).
• The deprecation was included in the release notes (Github and website) of version 1.10.0, stating the version targeted for removal was 1.11.0.
• Wikipedia is an instance of WMTS, there wasn't any docstring to update :)
• The documentation was updated in the PR, replacing Wikipedia with OSM.
• The programmatic warning included an alternative with OSM. The release notes/documentation suggested users to pick OSM too.
• It seems gv.tile_sources.Wikipedia has some more usage, some examples:
  • https://github.com/umr-lops/xsar/blob/cd3bfa72afaae95aa544f2efeb8f5c7f29a3ad79/docs/examples/xsar_tops_slc.ipynb#L50
  • https://github.com/umr-lops/xsarsea/blob/b12c73140137ba95197f8693c648c0e2d52bea67/docs/examples/streaks.ipynb#L34
  • https://github.com/OpenEnergyPlatform/open-MaStR/blob/bfa686f509a19a447610d3073fc40cfe07737406/postprocessing/helpers.py#L30
  • https://github.com/otro-mas1/ZBS_Madrid/blob/1dcee97e045c576188f7f6b5a1bf80c8a6f1c62f/ZBS_Mapa_Interactivo.ipynb#L79
  • https://github.com/fccoelho/epigrass/blob/5a94cbc236dbfc60a82d71e8af4ab367779bf44c/Epigrass/epipanel.py#L11

---

I hope I don't get this wrong as Python warnings confuse me a little, but running the code below with python foo.py doesn't print the warning. It does print it if I disable the default warning filters with python -W default foo.py, showing:

/home/liquetm/dev/bar.py:5: DeprecationWarning: Bottom is deprecated
return bottom()

# foo.py
from bar import top

top()

# bar.py
import warnings

def top():
    return bottom()

def bottom():
    warnings.warn('Bottom is deprecated', DeprecationWarning, stacklevel=2)

[maximlt]

My take away from this as a maintainer:

• I'd like us to reach a consensus on how to deprecate a feature, the difference between how the Wikipedia tile source was deprecated in HoloViews and GeoViews is quite big, this can only lead to endless discussions
• I'd like much better guidelines, the whole deprecation cycle involves many actions and is quite complex to get right

[maximlt]

Made some changes:

• Removed the too detailed section on the release process as suggested by Simon (989e935)
• Clarified definition/expectation of release candidates (1e5a7ad)
• Improved the deprecation guidelines and added an explanation on Python warnings (44f52dd)

I think more work is needed:

• (Re)-exploring the various ways Python warnings are filtered/displayed depending on their type and the context made me realize the guidelines should be more clear about them.
• I suggested maintainers should update a deprecation table somewhere, I want to see how this might be done with Param. Simon on his side has looked into exploring a code base to find its warnings and parse them (with the AST). It'd be great to find a systematic approach that isn't too over-engineered.

[maximlt]

I finally got around to dedicating some time to the HEP and updated it in 43cb239. I probably need to give it a good review with fresh eyes, but the next step for me will be to introduce it to the team in a HoloViz meeting.

[maximlt]

@philippjfr here are some deprecation periods for other libraries:

• Django:
  • Feature releases are made roughly every 8 months. Deprecations need to wait for a minimum of 2 feature releases (carrying over major versions if needed). For example, Django 4.2 deprecated something, next version is Django 5.0, the feature is removed in Django 5.1.
  • So this is a minimum of 16 months
• Numpy:
  • Removal shall be done after at least 2 releases assuming the current 6-monthly release cycle; if that changes, there shall be at least 1 year between deprecation and removal. Removal can be done in any minor, but not bugfix, release.
• Matplotlib:
  • Removal minimum of 2 minor releases later and possibly more.
  • Matplotlib 3.x releases were: 3.8 (09/2023), 3.7 (02/2023), 3.6 (06/2022), 3.5 (11/2021), 3.4 (03/2021), 3.3 (07/2020), 3.2 (03/2020), 3.1 (05/2019), 3.0 (09/2018). The time between two minor releases is in months: 18, 14, 12, 16, 15, 15 and 15.
• Scipy:
  • Wait till at least 6 months after the release date of the release that introduced the DeprecationWarning before removing the functionality

18 months is for sure quite high. Listening to your suggestions on:

• a shorter grace period
• special-casing deprecations made before a major release, perhaps with an even shorter grace period

[maximlt]

@jlstevens suggested it would be great if there was automation to enforce some of the requirements. I think that:

• Ideally, yes, but building and setting up automation over multiple repos and in the correct way would require some effort
• Proper review of PRs deprecating and removing features should already be sufficient to make sure this is done correctly
• The most important part, which is to decide to deprecate something and to implement it, cannot be automated.

[droumis]

Looks good, added a couple suggestions

[droumis]

Let's add a concise 'Resolution' section after 'Summary' for a quick recap of the technical plan of action. Something like:

## Resolution
- Adopt a tiered warning strategy for deprecations:
  - Initially emit a `DeprecationWarning`, ensuring it's set with the appropriate stacklevel for accurate source identification.
  - After a 12-month period, elevate the warning to a `FutureWarning` to capture the attention of all users, including those who might not be configured to see DeprecationWarnings.
- Allow exceptions in rare cases, such as beginning with a `PendingDeprecationWarning` for potentially reversible or highly disruptive changes, or immediately using a `FutureWarning` for urgent cases.

[maximlt]

I don't really want to summarize the plan of action as I think all the steps of the deprecation cycle are important.

[jlstevens]

If automation is too high a bar, then the next best thing is establishing a clear, concrete process that does things correctly. For instance, maybe we should have a standardized approach/template for a 'deprecation PR' that we need to create every release.

That way a maintainer can just look up at a reference deprecation PR (e.g. the previous one), follow the steps and without too much cognitive effort be fairly confident that the recommendations in this HEP have been followed. At a minimum I would want a standardized PR name (e.g. Deprecations for version X.X.X) and the first comment to link to this HEP!

[maximlt]

If automation is too high a bar, then the next best thing is establishing a clear, concrete process that does things correctly. For instance, maybe we should have a standardized approach/template for a 'deprecation PR' that we need to create every release.

That way a maintainer can just look up at a reference deprecation PR (e.g. the previous one), follow the steps and without too much cognitive effort be fairly confident that the recommendations in this HEP have been followed. At a minimum I would want a standardized PR name (e.g. Deprecations for version X.X.X) and the first comment to link to this HEP!

But isn't the clear process the Specification section of the HEP?

[jlstevens]

But isn't the clear process the Specification section of the HEP?

Yes but you also want something concrete as well that integrates well into your release cycle. Checking the Specification section of all HEPs at release for instructions is a rather painful workflow.

I'm trying to suggest an approach to make our lives easier, not harder! :-)

[philippjfr]

I would remove the sentence about "significant new functionality", I think we should be perfectly comfortable making a major release solely to make API changes.

[maximlt]

The sentence says should normally include significant new functionality so it doesn't make it a strong requirement.

By the way, I literally copied these definitions from a comment from @jbednar holoviz/panel#5221 (comment).

[jbednar]

Right after that comment I wrote "I'd be interested in whether other team members think this description is accurate.", but no one engaged with it. Reading it now, I'm happy for @philippjfr to make the call -- would we normally provide some carrot along with the stick, i.e. provide some enticement to make it worth switching? You make more releases than I do, so I leave it to your discretion.

[maximlt]

FWIW I re-used your description as I found it appropriate.

[maximlt]

• b5def36: Enhanced the warnings.warn / @deprecated section by putting @deprecated first and documenting that calling @deprecated(msg, category=None) disables the run-time warning.
• f8e0067: Added docs on the skip_file_prefixes parameter of warnings.warn added in Python 3.12 that I think could be used in place of setting stacklevel manually or with find_stack_level.

[maximlt]

HEP2 is ready for review. I think the first reviews and discussions have helped resolve some points of contention, but there may still be some. According to HEP0, we should aim to reach a consensus. To make sure I capture everyone's perspective, please add a comment. It could be as simple as 👍, if there are parts of the HEP you disagree with please raise them so they can be further discussed.

cc @jbednar @philippjfr @jlstevens @MarcSkovMadsen @hoxbro @droumis @ahuang11

[droumis]

minor comment about docs, but either way 👍

[jbednar]

I've given it a detailed review, and made a bunch of tiny wording fixes and clarifications, none of which I think should be controversial. I have a couple of minor questions below, but overall, I'm happy with it and happy for it to be merged. Good job! (And I vote for a 6 month period; longer than that and we will have no memory of what we were doing!)

[maximlt]

@jbednar I didn't include the main channel originally as that feels like this shouldn't be the responsibility of a HoloViz maintainer, but of an Anaconda employee, and it turns out the intersection of both is close to 100% at the moment (it may not always be the case).

[philippjfr]

Left some minor comments but I'm quite happy with this! Really appreciate the effort you put into this.

[jlstevens]

Looks good to me!

Thanks for driving this forward @maximlt !

[maximlt]

Thanks all for the reviews! With approvals from @droumis, @jbednar, @philippjfr, @hoxbro, @jlstevens, and @ahuang11 (one thumb up somewhere 🙃 ), I think it's fair to consider we've reached a consensus, HEP2 is approved! So next time you intend to deprecate a feature somewhere, please check the HEP :)

(I'll now see with Demetris how to merge it since it depends on #386)

[droumis]

I don't think merging this depends on exposing HEPs on the website. It certainly wasn't a blocker for HEP1. Let's go ahead and merge this and then continue the website work.

[maximlt]

I don't think merging this depends on exposing HEPs on the website. It certainly wasn't a blocker for HEP1. Let's go ahead and merge this and then continue the website work.

This PR was branched off #386. Instead, I rebased it on main, and "pre-accepted" it by changing the status and adding it to the accepted HEPs file.

A HoloViz core developer will then formally review the pull request for format, scope, and evidence of consensus. They may also notify other interested parties to weigh in on the HEP. If consensus has been reached, the HoloViz core developer will update the status of the pull request, assign a HEP number, add a link to the discussion in the HEP table, and merge it. All HEPs will be resolved as either 'rejected', 'accepted', or 'deferred' depending on the consensus of the community.

@droumis, could you have a final look and merge if it's all okay?