Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

d3.time.format directive for negative year #1782

Closed
esjewett opened this issue Mar 17, 2014 · 12 comments

Comments

@esjewett
Copy link

commented Mar 17, 2014

Our project is used by historians who will often have data sets including years before the year 0. These years will generally be represented by a negative number. The %Y directive doesn't parse these years properly. Would it be possible to define a new directive that parses negative and positive years, allowing at least anything in the range [-9999,9999]?

@mbostock

This comment has been minimized.

Copy link
Member

commented Mar 17, 2014

I wonder if this would interact poorly with time formats that have dashes in them, like the common ISO 8601 format. I think the only risk that you would get a false match with a format like "%d%m%Y" against a string like "17-03-2014" where it would think the date would be -3 and the year -2014.

How is this handled in other languages with strptime-like functionality?

@esjewett

This comment has been minimized.

Copy link
Author

commented Mar 17, 2014

Agreed on the dashes being a potential problem. That's why I think it must be an entirely new directive as I don't see a way to be fully backwards compatible with %Y. It would probably also have to be documented that it shouldn't be used with dash-delimited formats, though I guess if the new directive is captured last, it should properly parse out "17-03-2014" vs "17-03--2014".

I'm kind of shocked by this, but I've done a little research and can't find a single strptime-like (or other) implementation that supports negative years, so this may be somewhat uncharted territory.

@esjewett

This comment has been minimized.

Copy link
Author

commented Mar 17, 2014

Oh, also, ISO 8601 does address this problem, but it's fairly unhelpful. It says that the sender and receiver must agree on a specific number of year digits and if negative years will be supported, then all years must be in the format +YYYY or -YYYY (I suppose they are maintaining constant character lengths). I suppose this (an explicit "+" or "-" on all years) could be workable for our project.

@mbostock

This comment has been minimized.

Copy link
Member

commented Mar 17, 2014

Yeah, having an explicit + for positive numbers would definitely resolve the ambiguity, so I’d rather do that if it works for you.

@esjewett

This comment has been minimized.

Copy link
Author

commented Mar 17, 2014

That would be fine. We're not going to require our users to provide positive years with a + in front of them, but since we know which values are supposed to be dates and all of our dates start with a year, we can just prepend a + if we don't see a - at the beginning of them. Simple enough.

@mbostock mbostock added this to the 3.5 milestone Mar 18, 2014

@esjewett esjewett closed this Mar 26, 2014

@esjewett esjewett reopened this Mar 26, 2014

esjewett added a commit to esjewett/d3 that referenced this issue Mar 26, 2014
%V directive and tests for issue d3#1782
Provides a time.format directive for a signed 4-digit year
@mbostock

This comment has been minimized.

Copy link
Member

commented Mar 26, 2014

I have a related question here: are you using full dates (meaning month, date, hour, etc.) for negative years, or are you just using years?

I ask because the Gregorian calendar used by JavaScript’s Date class has only existed since 1582, and the Julian calendar since 46 BC. The Roman calendar dates back to ~750 BC and is similar, and I suppose you could use Gregorian dates to represent Roman dates, but this approach would fall apart if you tried to do any calendar math (including d3.time.scale).

Of course we can retroactively apply the Gregorian calendar to dates before it existed, but I wonder if it would be simpler to just use years? (And thus d3.scale.linear instead of d3.time.scale, and d3.format instead of d3.time.format.) Can you elaborate a bit on how you’ll use this functionality?

@esjewett

This comment has been minimized.

Copy link
Author

commented Mar 26, 2014

It's a good question. We're developing a platform for humanities-oriented data called Palladio. You can see a demo at http://palladio.designhumanities.org Right now we have a timeline visualization type and we support date dimensions in %Y and %Y-%m-%d formats, with more to come. On the backend at the moment we convert %Y formats into the %Y-%m-%d, but longer term we will use a different internal format (still based on a %Y-%m-%d concept) that allows for timespans with start/end dates as well as accuracy estimates. Eventually we also hope to be able to convert other types of time-specifications into our internal format (years and seasons are one example we would like to support). That's probably neither here nor there.

We have one specific use-case for negative years at the moment that I'm aware of and my understanding is that they are using a retroactively applied Gregorian calendar on years. But we expect we will have other researchers using retroactively applied Gregorian calendars, potentially at higher levels of detail and/or researchers with data sets including both ancient events and more modern events (at a higher level of detail). I'll check into it more, but my understanding is that these researchers are generally aware of the issues around converting into Gregorian dates.

Regardless, the basic issue is that since it's a general purpose platform, we want to be able to put dates on the same timeline regardless of how ancient those dates happen to be. Longer-term, we may need to develop a whole new date processing library to support this efficiently, but as a start, it seems like it would be helpful to historians to support the concept of a negative Gregorian date.

@esjewett

This comment has been minimized.

Copy link
Author

commented Mar 29, 2014

I was able to check with our coordinator and apparently we actually have 3 or 4 researchers who all have negative Gregorian dates in their data sets. I guess it's a fairly common use-case in the historical research community.

@mbostock

This comment has been minimized.

Copy link
Member

commented Mar 30, 2014

Thanks for the additional info. I’m onboard with supporting negative years for dates.

@esjewett

This comment has been minimized.

Copy link
Author

commented Mar 30, 2014

Thanks - that's great to hear. Is the pull request helpful, or are there changes you'd like to see or another better way I can help contribute towards this support? Otherwise, I can sit back and wait for 3.5. Thanks again!

@esjewett

This comment has been minimized.

Copy link
Author

commented May 1, 2014

Update: For unrelated performance reasons we have switched to using a custom date parser. Switching over to code that can make more assumptions about our internal format results in a big speed-up for us, so I took to opportunity to roll in handling negative date handling in our custom parser as well.

So, we will probably no longer use this directly in our current project if it is added to D3. I think it would still be useful to some historians and researchers.

@mbostock mbostock modified the milestones: Icebox, 3.5 May 30, 2014

@eric-poitras

This comment has been minimized.

Copy link

commented Dec 9, 2014

Hello !
I indented to use the %V for 1-based week of year as we needs it in our project. Can't you use another directive ?

It is defined in the strftime API: http://pubs.opengroup.org/onlinepubs/007908799/xsh/strftime.html

%V
is replaced by the week number of the year (Monday as the first day of the week) as a decimal number [01,53]. If the week containing 1 January has four or more days in the new year, then it is considered week 1. Otherwise, it is the last week of the previous year, and the next week is week 1.

@mbostock mbostock referenced this issue Nov 12, 2015
1 of 5 tasks complete

@mbostock mbostock closed this Jan 6, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
You can’t perform that action at this time.