datetime.strftime("%Y") not consistent for years < 1000

[florentx]

BPO	13305
Nosy	@jaraco, @abalkin, @vstinner, @ned-deily, @ezio-melotti, @florentx, @vadmium, @elenaoat, @pganssle, @Fronkan
Superseder	bpo-3173: external strftime for Python?
Files	issue13305_xmlrpc_patch.diff doc.patch issue13305.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2011-10-31.20:12:37.023>
labels = ['easy', 'type-bug', '3.8', '3.9', '3.7', 'library', 'docs']
title = 'datetime.strftime("%Y") not consistent for years < 1000'
updated_at = <Date 2022-02-10.19:47:24.483>
user = 'https://github.com/florentx'

bugs.python.org fields:

activity = <Date 2022-02-10.19:47:24.483>
actor = 'jaraco'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation', 'Library (Lib)']
creation = <Date 2011-10-31.20:12:37.023>
creator = 'flox'
dependencies = []
files = ['23577', '41476', '42136']
hgrepos = []
issue_num = 13305
keywords = ['patch', 'easy']
message_count = 30.0
messages = ['146740', '146743', '146745', '146750', '146751', '146752', '146765', '146767', '146967', '146972', '146976', '146978', '252165', '257271', '257383', '257397', '261577', '261607', '261608', '261609', '261637', '261638', '261639', '261640', '307417', '359735', '359740', '394251', '413015', '413021']
nosy_count = 13.0
nosy_names = ['jaraco', 'belopolsky', 'vstinner', 'ned.deily', 'ezio.melotti', 'flox', 'docs@python', 'python-dev', 'martin.panter', 'Elena.Oat', 'p-ganssle', 'Fronkan', 'cklarson']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = '3173'
type = 'behavior'
url = 'https://bugs.python.org/issue13305'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

[florentx]

See msg146725 on bpo-13291.

on linux
>>> datetime(1,  2, 10, 11, 41, 23).strftime("%Y")
'1'

on osx
>>> datetime(1,  2, 10, 11, 41, 23).strftime("%Y")
'0001'


>>> datetime.strptime('0001', '%Y')
datetime.datetime(1, 1, 1, 0, 0)

>>> datetime.strptime('1', '%Y')
ValueError: time data '1' does not match format '%Y'

[florentx]

FWIW, issue bpo-1777412 added support for years < 1000 to Python 3.3 strftime.

[abalkin]

I am not sure this can be fixed without distributing our own implementation of strftime. See bpo-3173.

[florentx]

There's many discrepancies between OS X and Linux about time formatting...

OS X
>>> from datetime import datetime
>>> datetime(1900, 1, 1).strftime("%6Y")
'6Y'

Linux
>>> from datetime import datetime
>>> datetime(1900, 1, 1).strftime("%6Y")
'001900'

BTW, these discrepancies are already mentioned:
http://docs.python.org/dev/library/datetime.html#strftime-strptime-behavior

“The full set of format codes supported varies across platforms, because Python calls the platform C library’s strftime() function, and platform variations are common.”

We should had an asterisk to the "%Y" saying that the padding is not consistent across platforms.

[florentx]

Proposed patch to fix the issue in xmlrpc.client

[python-dev]

New changeset 3f025427f02b by Florent Xicluna in branch 'default':
Fix regression due to changeset 2096158376e5 (issue bpo-13305).
http://hg.python.org/cpython/rev/3f025427f02b

[vstinner]

Le 01/11/2011 00:07, Roundup Robot a écrit :

Roundup Robot<devnull@psf.upfronthosting.co.za> added the comment:

New changeset 3f025427f02b by Florent Xicluna in branch 'default':
Fix regression due to changeset 2096158376e5 (issue bpo-13305).
http://hg.python.org/cpython/rev/3f025427f02b

I don't like this hack. If there is a bug in time.strftime(), we need to
fix time.strftime(), not xmlrpclib.

Is there a test for the hack?

[python-dev]

New changeset 230f0956aaa3 by Florent Xicluna in branch 'default':
Strengthen the tests for format '%Y', in relation with issue bpo-13305.
http://hg.python.org/cpython/rev/230f0956aaa3

[vstinner]

Did your commit fix the issue or not?

[florentx]

I understand that the issue is because the C standard does not specify the length of the string returned by '%Y'.

The changeset 230f0956aaa3 adds a test to verify that either '%Y' or '%4Y' returns a 4-digits value usable to produce ISO-8601 representations.

Now it is a documentation issue: add a comment to the "%Y" specifier saying that the padding is not applicable to all platforms and in such case the "%4Y" specifier should return the 4-digit value.

[vstinner]

Since the changeset 55a3b563f0dbed04af317f632f7f3c0f6abe175b, test_strptime is failing on "AMD64 Gentoo Wide 3.x" buildbot:

======================================================================
FAIL: test_strptime (test.test_time.TimeTestCase)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.ochtman-gentoo-amd64/build/Lib/test/test_time.py", line 159, in test_strptime
    time.strptime(strf_output, format)
ValueError: time data 'LMT' does not match format '%Z'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.ochtman-gentoo-amd64/build/Lib/test/test_time.py", line 162, in test_strptime
    (format, strf_output))
AssertionError: conversion specifier '%Z' failed with 'LMT' input.

---

http://www.python.org/dev/buildbot/all/builders/AMD64%20Gentoo%20Wide%203.x/builds/2661

[florentx]

other test_time related errors are followed with bpo-13309, bpo-13312 and bpo-13313

[abalkin]

Can someone recap the status of this issue? It is classified as a documentation bug, but I don't see a clear statement of what is wrong with the documentation.

[ezio-melotti]

See msg146972 -- AFAIU the padding of %Y is inconsistent for years <1000 (e.g. 0042 vs 42), and this is not documented in the note (2) of https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior
Also, using %4Y seems to consistently produce zero-padded values across different platforms.

[vadmium]

At the bottom of <https://docs.python.org/dev/library/time.html#time.strftime\> it suggests that this four-digit field width specifier is not portable. Does that also hold for the datetime version? It seems like a bad idea to recommend an unportable workaround for a portability problem.

Also, in the patch it is not clear if you are referring to strptime(), strftime(), or both. On Linux, datetime.strptime(..., "%4Y") fails for me.

[ezio-melotti]

Also, using %4Y seems to consistently produce zero-padded values across
different platforms.

Actually I was wrong. Based on some more testing I did on Linux and Windows 8 and the previous comments in this thread, it seems that, with 1 as year:

• '%4Y' does produces '0001' on Linux but gives a value error on Windows and '4Y' on OS X;
• '%Y' produces '1' on Linux and '0001' on Windows and OS X;

So the problem is only on Linux and only with years <1000, where to have a zero-padded value %4Y can be used instead.
I don't know if these results are true for all versions of Linux/Win/OSX, so the note could just say that:

1. some platforms (e.g. Linux) don't zero-pad years <1000;
2. some platforms (e.g. Linux) support %4Y to add padding, but this doesn't work on other platforms.

[elenaoat]

I submitted a patch for documentation changes related to strftime "%Y" directive inconsistencies. I am not sure that specifying the OS is correct, because I haven't tested the inconsistencies on all Linux PCs, Windows versions or OS X. I still left them there though, so let me know what's your opinion on it.

[python-dev]

New changeset 291d47954618 by Victor Stinner in branch 'default':
Always test datetime.strftime("%4Y")
https://hg.python.org/cpython/rev/291d47954618

[vstinner]

About Elena Oat's patch bpo-13305.diff: I'm not sure that strftime("%4Y") works on all platforms, so I enabled the test on strftime("%4Y"). I will check our buildbots.

[vstinner]

acucci's patch doc.patch suggests to use format "%4Y". Problem: I tried it on Linux, and it looks like it doesn't work.

>>> datetime.datetime.strptime("1980", "%Y")
datetime.datetime(1980, 1, 1, 0, 0)
>>> datetime.datetime.strptime("1980", "%4Y")
Traceback (most recent call last):
  ...
ValueError: '4' is a bad directive in format '%4Y'

Or maybe I misunderstood the doc change. Do you suggest to use %4Y format with strptime(), with strftime() or with both?

[ezio-melotti]

About Elena Oat's patch bpo-13305.diff: I'm not sure that
strftime("%4Y") works on all platforms

It doesn't, that's what the patch is trying to document.
AFAICT it works on my Linux (but not on yours), it gives '4Y' on Mac, and raises an error on Windows. This should depend on the libc used by system, so the same OS can give different results depending on which libc it uses, and the only sure thing is that %4Y is not a portable solution.
FWIW the patch LGTM (except a couple minor nits -- zero-pad should be hyphenated and there should be a double space after the dot), but I'm not sure if it's better mentioning platforms at all or just being vague and say that it works on some platforms but not others and leave up to the user figuring out where.

[vstinner]

> About Elena Oat's patch bpo-13305.diff: I'm not sure that
> strftime("%4Y") works on all platforms

It doesn't, that's what the patch is trying to document.

Oh. I confirm: %4Y gives "4Y" for any year on FreeBSD and Mac OS X, it raises ValueError("Invalid format string") on Windows, etc.

[vstinner]

Elena's bpo-13305.diff:

"Zero padding can be forced on some platforms by using e.g. %4Y"

Sorry, I still doesn't understand the change. %4Y doesn't work with strptime() and it only work with strftime() on very few platforms.

Can you explain when %4Y should be used? strftime() and/or strptime()? Which platform?

To me, it looks wrong to document %4Y since it almost never works...

[python-dev]

New changeset e54224e8d6a9 by Victor Stinner in branch 'default':
Revert change 291d47954618
https://hg.python.org/cpython/rev/e54224e8d6a9

[ned-deily]

(See also msg307413 (#76376 (comment)) in duplicate bpo-32195 for more discussion.)

[jaraco]

In bpo-39103, I filed a bug relating to this issue.

I'd like for Python to provide a portable implementation of strftime instead of just documenting that the version isn't portable.

Given that this ticket assigned to 'docs' suggests that a portable version is out of the question. Can we expand the scope of this issue to actually providing a portable version?

[vstinner]

I'd like for Python to provide a portable implementation of strftime instead of just documenting that the version isn't portable.

If someone wants to do that, I suggest to first start with a project on PyPI. It sounds like a tricky project. Date, time, locales and localization are hard problems!

[Fronkan]

This also causes an issue when using stftime and strptime together on Linux (Ubuntu 20.04.1 LTS).
When encoding a datetime using strftime and decoding with strptime using the same pattern an error is raised if year < 1000.

Example:
>>> pattern = "%Y-%m-%d"
>>> datetime.strptime(
...     datetime(1, 1, 1).strftime(pattern),
...     pattern
... )
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/_strptime.py", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/usr/lib/python3.8/_strptime.py", line 349, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data '1-01-01' does not match format '%Y-%m-%d'

[cklarson]

Has there been any work/progress on this? Alternatively, what suggested work around/mitigations are suggested?

[jaraco]

The tempora library implements a portable strftime.

[codethief]

Given the age of the ticket and the lack of clarity on what to do, I don't think the "easy" label still applies here, does it? :-)

[jaraco]

I'd like for Python to provide a portable implementation of strftime instead of just documenting that the version isn't portable.

If someone wants to do that, I suggest to first start with a project on PyPI. It sounds like a tricky project. Date, time, locales and localization are hard problems!

The wrapper in tempora has worked well for me for a couple of years. It defers most of the processing to the existing C implementation and merely performs the substitution early, allowing the C implementation to handle the rest.

Would it make sense for CPython to adopt a similar technique to address the concern at hand and avoid doing a full re-implementation (with all of the bugs and performance regressions that would likely entail)?

[blhsing]

Given the age of the ticket and the lack of clarity on what to do, I don't think the "easy" label still applies here, does it? :-)

A fix for %Y alone would've been "easy" by PyOS_snprintf-formatting the year with "%04ld", but the issue is made more complex when taking into account the need to support %G (credit to @serhiy-storchaka for pointing this out).

The issue is now resolved with PR #120820 by conditionally delegating the said format specifiers to time.strftime and wrapping the value with PyOS_snprintf.

[serhiy-storchaka]

Closing as a duplicate of #120713.