Deprecation notice for Python 2.7 (and 3.5) support. Target July 2021

[jmcnamara]

I have just added the following notice to the Changes page of the XlsxWriter docs:

Deprecation Notice: Python 2.7 reached the end of its life on January 1st,
2020 and is no longer being supported in the community. XlsxWriter support for
Python 2.7 will end by mid-year 2021 (probably in July 2021). No new features
or fixes for Python 2.7 will be added to XlsxWriter after that date/release.

If anyone has any concerns about this please raise them now.

[jmcnamara]

Also, by the time this deprecation happens Python 3.5 will also be end of life. Any strong opinions on dropping support for Pythons <= 3.5 in early 2021?

There isn't any code that differentiates between the Python 3 versions so there isn't any urgency to do this but it reduces the test cycle.

If I do this it will happen in a number of steps:

• Last Python 2 release in January 2021
• Strip out Python 2 code and release Python 3 only version
• Optimize for Python 3
• Drop Python 3.4 and 3.5 support

[jkyeung]

Not that I have official data or anything, but I suspect that there are very few folks out there who are stuck on Python 3.4 or 3.5 (unable to upgrade to 3.6). For those who are truly stuck, I think they will be able to suffice on whatever XlsxWriter version is available that still supports their Python.

[jmcnamara]

Not that I have official data or anything, but I suspect that there are very few folks out there who are stuck on Python 3.4 or 3.5 (unable to upgrade to 3.6).

I'm not 100% committed to dropping Python 3.4 and 3.5 support since there isn't anything in the code that depends on different Python3 versions.

For what it is worth here are the download stats for that last 30 days:

$ pypinfo --percent XlsxWriter pyversion
Served from cache: False
Data processed: 114.56 GiB
Data billed: 114.56 GiB
Estimated cost: $0.56

| python_version | percent | download_count |
| -------------- | ------- | -------------- |
| 3.7            |  46.78% |      1,925,733 |
| 3.8            |  19.05% |        784,098 |
| 3.6            |  16.26% |        669,387 |
| 2.7            |  15.04% |        619,205 |
| 3.5            |   2.37% |         97,397 |
| 3.9            |   0.44% |         18,028 |
| 3.4            |   0.07% |          2,922 |
| 3.3            |   0.00% |             55 |
| 2.6            |   0.00% |             44 |
| 3.10           |   0.00% |             22 |
| Total          |         |      4,116,891 |



# Or in version order:

| python_version | percent | download_count |
| -------------- | ------- | -------------- |
| 2.6            |   0.00% |             44 |
| 2.7            |  15.04% |        619,205 |
| 3.3            |   0.00% |             55 |
| 3.4            |   0.07% |          2,922 |
| 3.5            |   2.37% |         97,397 |
| 3.6            |  16.26% |        669,387 |
| 3.7            |  46.78% |      1,925,733 |
| 3.8            |  19.05% |        784,098 |
| 3.9            |   0.44% |         18,028 |
| 3.10           |   0.00% |             22 |

As a general observation, there is still quite a large percentage using Python 2.7.

[jkyeung]

For what it is worth here are the download stats for that last 30 days:

As a general observation, there is still quite a large percentage using Python 2.7.

Interesting. The number of people still using Python 2.7 is probably much, much greater. What is surprising to me is that there seem to be about as many new Python 2.7 users as there are new Python 3.6 users.

I don't know how many of those 2.7 downloaders are confused newbies versus how many are die-hard Python 2 fans who think Python 3 is genuinely worse. Either way, it's fine to cut them off. The newbies will learn soon enough that they should be on Python 3; and the folks who doggedly stick to Python 2 most likely know what they're doing, at least enough to cope with the landscape they're faced with.

[Garoun]

@jkyeung some may also be stuck in a position where they are forced to use py2 while waiting to get approvals and provisioned environments (which can be extremely slow depending on the work sector, can't go into how long I've been waiting). I'd be more than happy to move to py3 on more than a testing basis :). Not saying this is a reason to keep py2 support but wanted to be sure it was on the radar that it's not always by choice; though I guess you could group those folks in the 'cope with' group.

[jkyeung]

@Garoun - Fair enough, and I actually have a significant amount of 2.7 code that I have to keep in production because it was developed for an unusual platform, with a platform-specific implementation of Python. You're right, we are both stuck and just kind of coping. But it's still striking to me that the new download numbers for Python 2.7 are so high.

I guess it's possible that a chunk of those downloads represent legacy hardware reaching end of life, with legacy software that still needs to run on the replacement hardware, and so you need to install legacy Python once again.

[jmcnamara]

For what it is worth I'm not dropping Python 2 support for the sake of it. I maintain the Perl version of this library back to version 5.8 (2002) and the C version as ANSI C (~1990 for Windows MSVC compatibility). Python 2 adds a small amount of additional maintenance burden but not a lot.

The main reason is that a pure Python 3 version should be faster since I could drop the use of codecs and use the native UTF-8 support instead. I need to retest the performance difference but when I went in the other direction the library was about 10-15% slower.

There are also benefits in terms of simplified documentation and examples.

[jkyeung]

This is embarrassing. I totally misunderstood the download counts posted above. I didn't bother to parse this line

$ pypinfo --percent XlsxWriter pyversion

In my head, I thought I was looking at the number of downloads of Python itself. Maybe that's just what I was interested in seeing at the time, and it didn't occur to me where those numbers were coming from or what else they could mean.

[jmcnamara]

For what it is worth here are the current XlsxWriter download figures. Python 2.7 use is still highish:

$ pypinfo --percent XlsxWriter pyversion

| python_version | percent | download_count |
| -------------- | ------- | -------------- |
| 3.7            |  51.81% |      1,351,107 |
| 3.8            |  19.12% |        498,647 |
| 2.7            |  14.79% |        385,623 |
| 3.6            |  10.63% |        277,076 |
| 3.5            |   2.14% |         55,828 |
| 3.9            |   1.44% |         37,497 |
| 3.4            |   0.07% |          1,906 |
| 3.10           |   0.00% |             29 |
| 3.3            |   0.00% |              7 |
| 2.6            |   0.00% |              4 |
| Total          |         |      2,607,724 |

[Garoun]

Appreciate you looking into those numbers again. We're still stuck on py2 for now with plans for py3; unfortunately no set timeline due to factors outside of our control.

[jmcnamara]

Percentage downloads as of May 2021.

$ pypinfo --percent XlsxWriter pyversion

| python_version | percent | download_count |
| -------------- | ------- | -------------- |
| 3.7            |  58.95% |      3,778,280 |
| 3.8            |  16.59% |      1,063,181 |
| 3.6            |  10.93% |        700,372 |
| 2.7            |   9.68% |        620,335 |
| 3.9            |   1.94% |        124,071 |
| 3.5            |   1.86% |        119,078 |
| 3.4            |   0.06% |          3,597 |
| 3.10           |   0.00% |             71 |
| 3.2            |   0.00% |             39 |
| 2.6            |   0.00% |             32 |
| Total          |         |      6,409,056 |

The Python 2 percentage is below 10% but still relatively high. This inclines me to put off the deprecation of Python 2 in XlsxWriter until January 2022.

[jpmckinney]

Using f-strings (Python 3.6+) instead of % in some of the most frequent methods (_xml_inline_string) does yield some small performance improvements.

I also wonder about writing bytes instead of strings (I don't know if this would spare some time on encoding; I also don't know if this would require changing a lot of code or not).

[jmcnamara]

I've created a Python 3 test branch or XlsxWriter (python3). I would be grateful if people could try it out and report any issues (or lack of issues). This branch is about 10-15% faster than main.

@jpmckinney I have also created a python3.6 branch with f-string support in xmlwriter. However, I don't see any performance improvements. In fact it seems to be 1-2% slower in my testing. Update: In another test configuration the f-string version was 2% faster.

[jmcnamara]

And for what it is worth here is the current XlsxWriter download percentages between the different Python versions:

$ pypinfo --percent XlsxWriter pyversion

| python_version | percent | download_count |
| -------------- | ------- | -------------- |
| 3.7            |  51.19% |      2,786,268 |
| 3.8            |  22.46% |      1,222,448 |
| 3.6            |  13.32% |        724,691 |
| 2.7            |   7.86% |        427,518 |
| 3.9            |   2.85% |        155,284 |
| 3.5            |   2.25% |        122,597 |
| 3.4            |   0.06% |          3,476 |
| 3.10           |   0.01% |            298 |
| 3.3            |   0.00% |              5 |
| Total          |         |      5,442,585 |

Python 2.7 usage is < 8% and almost half of what it was last November.

[jpmckinney]

@jmcnamara The branch works for me without issue!

Yeah, I think f-strings have a clearer performance difference the more substitutions there are (and might have a worse performance if there are only 1-2 substitutions). I think the most substitutions in any frequent code path is 3 in _xml_inline_string. So, I don't expect the difference to be major, and I think it's fine to leave f-strings out.

[jmcnamara]

Python2 downloads for XlsxWriter are now below 6%. It has also been over a year since the initial deprecation notice went out and 18 months since the Python community dropped maintenance support for Python 2. I am going to start the transition to Python3 only support in the next 1-2 days.

$ pypinfo --percent XlsxWriter pyversion

| python_version | percent | download_count |
| -------------- | ------- | -------------- |
| 3.7            |  50.64% |      3,041,573 |
| 3.8            |  25.19% |      1,512,816 |
| 3.6            |  13.53% |        812,633 |
| 2.7            |   5.92% |        355,390 |
| 3.9            |   2.78% |        166,859 |
| 3.5            |   1.87% |        112,539 |
| 3.4            |   0.06% |          3,872 |
| 3.10           |   0.00% |            153 |
| 3.2            |   0.00% |              1 |
| 3.3            |   0.00% |              1 |
| Total          |         |      6,005,837 |

[jmcnamara]

Python 2 support has been dropped from main.

[jmcnamara]

Python 2 support has been dropped in the latest version of XlsxWriter (3.0.0). Closing this feature request.