Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

time: decimal comma not supported in fractional seconds #6189

Open
gopherbot opened this issue Aug 19, 2013 · 21 comments
Open

time: decimal comma not supported in fractional seconds #6189

gopherbot opened this issue Aug 19, 2013 · 21 comments
Milestone

Comments

@gopherbot
Copy link

@gopherbot gopherbot commented Aug 19, 2013

by dmitri.m:

What steps will reproduce the problem?
If possible, include a link to a program on play.golang.org.
The Go time package expects the decimal mark in fractional seconds to always be a dot.
Attempting to parse a time string that uses a decimal comma results in an error:
http://play.golang.org/p/d8qQasN0z1

Decimal commas are standard in many locales, but there is currently no way to parse the
fractional part of the second in a time string if it's separated by a comma.
https://en.wikipedia.org/wiki/Decimal_mark#Countries_using_Arabic_numerals_with_decimal_comma

What is the expected output?
In case the provided layout also uses a decimal comma, the time string should be parsed
without errors.

What do you see instead?
parsing time "2013-08-19 22:56:01,234" as "2006-01-02 15:04:05,000":
cannot parse "234" as ",000"

Which compiler are you using (5g, 6g, 8g, gccgo)?
6g

Which operating system are you using?
OS X 10.8.4

Which version are you using?  (run 'go version')
go version devel +5037426bea2f Mon Aug 19 23:09:24 2013 +0400 darwin/amd64

Please provide any additional information below.
Note that (*time.Time).Format() currently outputs decimal commas just fine since it
doesn't see them as a special character. But making the straightforward change to
support decimal commas in Parse() - by adding them to the stdFracSecond{0,9} classes -
breaks this due to hardcoded '.' in formatNano().
@robpike
Copy link
Contributor

@robpike robpike commented Aug 19, 2013

Comment 1:

Here is what I wrote in mail responding to a CL to fix this:
Thanks for filing the issue but I might prefer to leave it unaddressed
for the moment. Proper locale treatment is coming (I hope soon) and this
seems like the wrong place to start insinuating special locale handling
into the standard library. Strconv can't do it yet, for instance.
I'll mark the bug as to be fixed for Go 1.3.

Labels changed: added go1.3, removed go1.2maybe.

@robpike
Copy link
Contributor

@robpike robpike commented Aug 19, 2013

Comment 2:

Status changed to Accepted.

@robpike
Copy link
Contributor

@robpike robpike commented Aug 19, 2013

Comment 3:

Labels changed: added priority-soon, removed priority-triage.

@robpike
Copy link
Contributor

@robpike robpike commented Aug 20, 2013

Comment 4:

Labels changed: removed go1.3.

@rsc
Copy link
Contributor

@rsc rsc commented Nov 27, 2013

Comment 5:

Labels changed: added go1.3maybe.

@rsc
Copy link
Contributor

@rsc rsc commented Dec 4, 2013

Comment 6:

Labels changed: added release-none, removed go1.3maybe.

@rsc
Copy link
Contributor

@rsc rsc commented Dec 4, 2013

Comment 7:

Labels changed: added repo-main.

@gopherbot
Copy link
Author

@gopherbot gopherbot commented Jan 3, 2014

Comment 8 by phmmnd:

I'm not sure this is just a localization issue.
According to http://en.wikipedia.org/wiki/ISO_8601 the latest version (ISO 8601:2004(E))
suggests that a comma is the preferred separator, which means some software will use a
comma even when not localized.
As a real life use case, I'm currently trying to parse a log file generated by log4j
where the most commonly used date formatters use a comma (eg:
http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/helpers/ISO8601DateFormat.html)
@gopherbot
Copy link
Author

@gopherbot gopherbot commented Jun 8, 2014

Comment 9 by mike.gaffney:

I am running into the same issue as described in comment #8. The default format for
log4j 1.2.x uses a comma. Here is an example copied from one of our log files: 
2014-04-02 12:00:57,376
There's a lot of software using log4j out there.
@rsc rsc added this to the Unplanned milestone Apr 10, 2015
@rsc rsc removed priority-soon labels Apr 10, 2015
@actgardner
Copy link

@actgardner actgardner commented May 11, 2016

Would a PR be welcome to add this functionality? We have a workaround in our codebase but it'd be great to have support in the standard library.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented May 11, 2016

@alanctgardner, if you already have a fix, feel free to send it so we can see how invasive it might be.

@mpvl, this bug was punted in 2013 because proper locale support was coming. Is it still coming?

@bradfitz bradfitz modified the milestones: Go1.8Maybe, Unplanned May 11, 2016
@gopherbot
Copy link
Author

@gopherbot gopherbot commented May 12, 2016

CL https://golang.org/cl/23063 mentions this issue.

@bradfitz bradfitz modified the milestones: Unplanned, Go1.8Maybe May 12, 2016
@mpvl
Copy link
Member

@mpvl mpvl commented May 13, 2016

Date localization is definitely still in the planning, but fairly low on the priority list at the moment. Extraction, segmentation, numbers, among other things are higher on the list right now.

That said, I think it is not unreasonable for the standard lib to support case 2 of the play example:
{"2006-01-02 15:04:05,000", "2013-08-19 22:56:01,234"}, // decimal comma
as it is unrelated to localization, I would say, and a localization that wants to build upon the existing time package would need such support.

The third case:
{"2006-01-02 15:04:05.000", "2013-08-19 22:56:01,234"},
seems to be more a localization feature.

@actgardner
Copy link

@actgardner actgardner commented May 13, 2016

That makes sense to me - the patch I uploaded doesn't support using a comma where a period is specified in the format string (and vice versa). That does seem like localization.

The patch does support the case where there's no fractional seconds part in the format but there are fractional seconds with a comma in the string, like:

{"2006-01-02 15:04:05", "2013-08-19 22:56:01,234"}

but I would be willing to remove that if it's too localization-related. The real crux of the issue is that there's no way to parse a date that uses a comma as a delimiter using any format string.

@cprior
Copy link

@cprior cprior commented May 25, 2017

I am completely new to Go, thanks for this Issue which saved me a ton of time.

I ended up strings-Replace'ing the log4j-comma.
Ugly, but in this code of mine are uglier things for sure! ;)

@AlekSi
Copy link
Contributor

@AlekSi AlekSi commented Aug 9, 2019

In https://golang.org/cl/23063 @robpike said:

While I understand the desire, this is not the right fix. As listed in the issue comments, the right answer is proper localization support and that should come soon. I hope.

But hacking it up like this in various places is not the answer. It sets a bad precedent.

But the next day @mpvl said there:

That said, I think it is not unreasonable for the standard lib to support case 2 of the play example:
{"2006-01-02 15:04:05,000", "2013-08-19 22:56:01,234"}, // decimal comma
as it is unrelated to localization, I would say, and a localization that wants to build upon the existing time package would need such support.

That sends a mixed message. Can you please decide about if you want to support 2006-01-02 15:04:05,000 as layout?

My use-case: I parse supervisord log, and there is no way to configure timestamps format. strings.Replace works, but ugly.

sunshine69 pushed a commit to sunshine69/nsre that referenced this issue Nov 18, 2019
golang does not support fraction of sec time layout using coma (,) - golang/go#6189. This is a work around.
@raneq
Copy link

@raneq raneq commented Jan 14, 2020

I am using @grafana 's promtail to parse logs from java, python and go applications. Not all of them use the same format that go assumes as "neutral" and "unlocalized".

@AlekSi and @actgardner have strong points

I could expect apps not being flexible enough to configure log formatting, but not the programming language behind any of them.

IMO, little bugs like this make golang much less attractive for developers looking for a versatile language.

Regarding localization, claiming that using a english standard like decimal dot is "unlocalized" sounds pretty colonialist, and the fact that "proper localization" is still coming, tells to me that diversity for different languages and cultures is totally not a priority for @golang

As constructive data, I analyzed a bit the linux /usr/share/i18n/locales/ dir and found that the most used separators are:

    208 "."
    134 ","
      7 "<U002E>" -> same as regular ascii dot
      3 "<U066B>" -> arabic decimal separator ٫

So adding latin and arabic comma would be a great step.

@djaglowski
Copy link

@djaglowski djaglowski commented Jun 12, 2020

I do not agree that this is a localization issue.

I need my application to be able to parse both period-milliseconds and comma-milliseconds.

It's discouraging to see such a simple limitation still in place after 7 years.

@RobbanHoglund
Copy link

@RobbanHoglund RobbanHoglund commented Jun 18, 2020

I have the same problem with colon delimiter before fraction of a second.
2020-06-12_11:48:20:466

cannot parse "466" as ".000""

@ChangSurrey
Copy link

@ChangSurrey ChangSurrey commented Jul 2, 2020

I find it hard to understand why this is a localization issue.

Using comma as the millisecond delimiter is the default format used by log4j2.

https://logging.apache.org/log4j/2.x/manual/garbagefree.html#Layouts

@jvsteiner
Copy link

@jvsteiner jvsteiner commented Aug 6, 2020

This is absolutely not a localization issue. It's a developer usability issue. let's say I have a timestamps like the following:

2020/6/17 06:31:59.573
2020/6/17 06:31:59,573
2020/6/17 06:31:59:573
2020/6/17 06:31:59|573

The first one, I can parse, the rest I cannot. It doesn't matter at all that some other maniac developer is feeding me a timestamp format that makes no sense at all, and is non standard as hell. The pattern matching technique in go is just bad, and inflexible. Microsoft Excel has a better mechanism, see: here.

Get it together people.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.