New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NSDate being truncated to the second #875

Closed
dginsburg opened this Issue Sep 5, 2014 · 22 comments

Comments

Projects
None yet
@dginsburg

dginsburg commented Sep 5, 2014

NSDates are being truncated to a whole second when being stored and then retrieved from the realm. Looking at the date setter/getter, the NSTimeInterval is being cast to a std::time_t. NSTimeInterval is a double with sub second precision. My C++ is rusty but I believe time_t is a integer value of whole seconds. Could the underlying storage of NSDate be changed to a double?

@jpsim

This comment has been minimized.

Show comment
Hide comment
@jpsim

jpsim Sep 8, 2014

Contributor

Nice find! Indeed, Realm discards sub-second information from NSDates. This is because Realm's underlying data type is based on the unix standard std::time_t and most realistic dates are representable in 4 bytes.

If you need to store more precise time information in Realm, you can create NSTimeInterval properties on your models, which will have no loss of precision from NSDate's or other NSTimeIntervals.

We'll update our docs to clarify this point.

Contributor

jpsim commented Sep 8, 2014

Nice find! Indeed, Realm discards sub-second information from NSDates. This is because Realm's underlying data type is based on the unix standard std::time_t and most realistic dates are representable in 4 bytes.

If you need to store more precise time information in Realm, you can create NSTimeInterval properties on your models, which will have no loss of precision from NSDate's or other NSTimeIntervals.

We'll update our docs to clarify this point.

@jpsim jpsim closed this Sep 8, 2014

@jpsim jpsim reopened this Sep 8, 2014

@jpsim

This comment has been minimized.

Show comment
Hide comment
@jpsim

jpsim Sep 8, 2014

Contributor

I just reopened the ticket because we'll be looking into alternatives to store NSDate's in a lossless way.

Contributor

jpsim commented Sep 8, 2014

I just reopened the ticket because we'll be looking into alternatives to store NSDate's in a lossless way.

@dginsburg

This comment has been minimized.

Show comment
Hide comment
@dginsburg

dginsburg Sep 8, 2014

I think that's a smart choice. It would be better to not support NSDate than to lose precision. Even if you documented the discrepancy, it would be an unending source of confusion and bugs because the expectation is that when you put a value in, you get the same value out.

dginsburg commented Sep 8, 2014

I think that's a smart choice. It would be better to not support NSDate than to lose precision. Even if you documented the discrepancy, it would be an unending source of confusion and bugs because the expectation is that when you put a value in, you get the same value out.

@youens

This comment has been minimized.

Show comment
Hide comment
@youens

youens Oct 13, 2014

1+ for this. I ran into this my first day of testing out Realm. In my case, I was just using the iOS Master-Detail Xcode template and quickly hitting + while recording the date. Using date to sort in that case led to weirdness as I added a few within the same second.

Obviously can work around it, but for clarity if nothing else, seems like it would be a good default behavior. 👍

youens commented Oct 13, 2014

1+ for this. I ran into this my first day of testing out Realm. In my case, I was just using the iOS Master-Detail Xcode template and quickly hitting + while recording the date. Using date to sort in that case led to weirdness as I added a few within the same second.

Obviously can work around it, but for clarity if nothing else, seems like it would be a good default behavior. 👍

@jpsim

This comment has been minimized.

Show comment
Hide comment
@jpsim

jpsim Nov 12, 2014

Contributor

Just a small update on this, we're still investigating changes to Realm's date support, our main concern being maintaining accuracy and interoperability with common date formats across platforms.

In the meantime, if you need sub-second accurate dates, please use NSTimeInterval or any other supported Realm type with suitable accuracy.

Contributor

jpsim commented Nov 12, 2014

Just a small update on this, we're still investigating changes to Realm's date support, our main concern being maintaining accuracy and interoperability with common date formats across platforms.

In the meantime, if you need sub-second accurate dates, please use NSTimeInterval or any other supported Realm type with suitable accuracy.

@pyrtsa

This comment has been minimized.

Show comment
Hide comment
@pyrtsa

pyrtsa Dec 10, 2014

Another find: persisting NSDate.distantFuture() doesn't seem to be allowed (and probably NSDate.distantPast() too).

pyrtsa commented Dec 10, 2014

Another find: persisting NSDate.distantFuture() doesn't seem to be allowed (and probably NSDate.distantPast() too).

@yasirmturk

This comment has been minimized.

Show comment
Hide comment
@yasirmturk

yasirmturk Feb 15, 2015

how to convert a string date from json to NSDate or NSTimeInterval i am getting this:
*Invalid value '2014-12-13T12:00:00+01:00' for property

yasirmturk commented Feb 15, 2015

how to convert a string date from json to NSDate or NSTimeInterval i am getting this:
*Invalid value '2014-12-13T12:00:00+01:00' for property

@pizthewiz

This comment has been minimized.

Show comment
Hide comment
@pizthewiz

pizthewiz Mar 10, 2015

Contributor

I have to agree with @dginsburg that having NSDate in the list of supported property types sends a mixed message. Until a resolution is found, perhaps the docs could link to the limitation detail directly in the property types list to be more explicit?


Realm supports the following property types: BOOL, bool, int, NSInteger, long, long long, float, double, CGFloat, NSString, NSDate (truncated to the second), and NSData.

Contributor

pizthewiz commented Mar 10, 2015

I have to agree with @dginsburg that having NSDate in the list of supported property types sends a mixed message. Until a resolution is found, perhaps the docs could link to the limitation detail directly in the property types list to be more explicit?


Realm supports the following property types: BOOL, bool, int, NSInteger, long, long long, float, double, CGFloat, NSString, NSDate (truncated to the second), and NSData.

@jpsim

This comment has been minimized.

Show comment
Hide comment
@jpsim

jpsim Mar 10, 2015

Contributor

@pizthewiz thanks for the suggestion, @yoshyosh just made the change on the website: http://realm.io/docs/cocoa/0.91.0/#property-types

Contributor

jpsim commented Mar 10, 2015

@pizthewiz thanks for the suggestion, @yoshyosh just made the change on the website: http://realm.io/docs/cocoa/0.91.0/#property-types

@ulfie22

This comment has been minimized.

Show comment
Hide comment
@ulfie22

ulfie22 Mar 13, 2015

Is it correct that the Realm database is storing the seconds but your Realm Browser app does not display them? Is there a way to see the seconds in the Realm Browser?

ulfie22 commented Mar 13, 2015

Is it correct that the Realm database is storing the seconds but your Realm Browser app does not display them? Is there a way to see the seconds in the Realm Browser?

@jpsim

This comment has been minimized.

Show comment
Hide comment
@jpsim

jpsim Mar 13, 2015

Contributor

Is it correct that the Realm database is storing the seconds but your Realm Browser app does not display them?

Yes, seconds are being persisted, but the browser doesn't display seconds.

To see seconds in the browser, you could build it from source and change this line to use NSDateFormatterMediumStyle.

Contributor

jpsim commented Mar 13, 2015

Is it correct that the Realm database is storing the seconds but your Realm Browser app does not display them?

Yes, seconds are being persisted, but the browser doesn't display seconds.

To see seconds in the browser, you could build it from source and change this line to use NSDateFormatterMediumStyle.

@marcoscurvello

This comment has been minimized.

Show comment
Hide comment
@marcoscurvello

marcoscurvello Jun 18, 2015

What's the status on NSDate property type being lossless? @jpsim

marcoscurvello commented Jun 18, 2015

What's the status on NSDate property type being lossless? @jpsim

@segiddins

This comment has been minimized.

Show comment
Hide comment
@segiddins

segiddins Jun 18, 2015

Contributor

@marcoscurvello it's something that's still being discussed, since it would require changes at the core database level as to how we store dates. JP's earlier suggestion as to how to workaround the issue still stands:

In the meantime, if you need sub-second accurate dates, please use NSTimeInterval or any other supported Realm type with suitable accuracy.

Contributor

segiddins commented Jun 18, 2015

@marcoscurvello it's something that's still being discussed, since it would require changes at the core database level as to how we store dates. JP's earlier suggestion as to how to workaround the issue still stands:

In the meantime, if you need sub-second accurate dates, please use NSTimeInterval or any other supported Realm type with suitable accuracy.

@marcoscurvello

This comment has been minimized.

Show comment
Hide comment
@marcoscurvello

marcoscurvello Jun 18, 2015

@segiddins Okay, thanks. That's what I am currently doing. 😏

marcoscurvello commented Jun 18, 2015

@segiddins Okay, thanks. That's what I am currently doing. 😏

@bmunkholm bmunkholm added T:Enhancement and removed T:Bug labels Sep 1, 2015

@WolfTrinity

This comment has been minimized.

Show comment
Hide comment
@WolfTrinity

WolfTrinity Nov 30, 2015

A question about NSTimeInterval: when I declare a property of type NSTimeInterval I can't use the dynamic keyword because its type cannot be represented in Objc. Is that ok (meaning that we won't have notifications about this property) or is there a way to workaround this?

WolfTrinity commented Nov 30, 2015

A question about NSTimeInterval: when I declare a property of type NSTimeInterval I can't use the dynamic keyword because its type cannot be represented in Objc. Is that ok (meaning that we won't have notifications about this property) or is there a way to workaround this?

@jpsim

This comment has been minimized.

Show comment
Hide comment
@jpsim

jpsim Nov 30, 2015

Contributor

@WolfTrinity are you sure you're not declaring that property as a NSTimeInterval? (Optional<NSTimeInterval>)? Objective-C primitives can't be represented as dynamic and Optional due to limitations in the Swift runtime.

If you need this property to be optional, you can mark it as RealmOptional<NSTimeInterval>.

Contributor

jpsim commented Nov 30, 2015

@WolfTrinity are you sure you're not declaring that property as a NSTimeInterval? (Optional<NSTimeInterval>)? Objective-C primitives can't be represented as dynamic and Optional due to limitations in the Swift runtime.

If you need this property to be optional, you can mark it as RealmOptional<NSTimeInterval>.

@WolfTrinity

This comment has been minimized.

Show comment
Hide comment
@WolfTrinity

WolfTrinity Nov 30, 2015

@jpsim Oh sorry, my fault: I've just substituted NSDate? with NSTimeInterval?, but I forgot that not all the optional types are supported in the standard swift way, but need to use RealmOptional.
Anyway, this still prevents me to use dynamic (I can't declare dynamic let myDate = RealmOptional<NSTimeInterval>()) so I can't use KVO on it.

Thanks for the hint!

WolfTrinity commented Nov 30, 2015

@jpsim Oh sorry, my fault: I've just substituted NSDate? with NSTimeInterval?, but I forgot that not all the optional types are supported in the standard swift way, but need to use RealmOptional.
Anyway, this still prevents me to use dynamic (I can't declare dynamic let myDate = RealmOptional<NSTimeInterval>()) so I can't use KVO on it.

Thanks for the hint!

@jpsim

This comment has been minimized.

Show comment
Hide comment
@jpsim

jpsim Nov 30, 2015

Contributor

You can't specify a RealmOptional property as dynamic:

There are two exceptions to this: List and RealmOptional properties cannot be declared as dynamic because generic properties cannot be represented in the Objective-C runtime, which is used for dynamic dispatch of dynamic properties, and should always be declared with let.

Source: https://realm.io/docs/swift/latest/#property-attributes

But they're still KVO-compliant.

Contributor

jpsim commented Nov 30, 2015

You can't specify a RealmOptional property as dynamic:

There are two exceptions to this: List and RealmOptional properties cannot be declared as dynamic because generic properties cannot be represented in the Objective-C runtime, which is used for dynamic dispatch of dynamic properties, and should always be declared with let.

Source: https://realm.io/docs/swift/latest/#property-attributes

But they're still KVO-compliant.

@jpsim jpsim modified the milestone: 1.0 Feb 12, 2016

@jpsim jpsim modified the milestones: 1.0, 0.100 Apr 19, 2016

@marchy

This comment has been minimized.

Show comment
Hide comment
@marchy

marchy Apr 22, 2016

Has this been assigned to a milestone / sprint? Seems JSON parsing is blowing up due to this

marchy commented Apr 22, 2016

Has this been assigned to a milestone / sprint? Seems JSON parsing is blowing up due to this

@bdash

This comment has been minimized.

Show comment
Hide comment
@bdash

bdash Apr 22, 2016

Contributor

This is being actively worked on. It's hard to see how it would break JSON parsing though. Feel free to file a new issue with more details if you'd like some help understanding whatever bad behavior you're seeing.

Contributor

bdash commented Apr 22, 2016

This is being actively worked on. It's hard to see how it would break JSON parsing though. Feel free to file a new issue with more details if you'd like some help understanding whatever bad behavior you're seeing.

@marchy

This comment has been minimized.

Show comment
Hide comment
@marchy

marchy Apr 22, 2016

That's great to hear it. Can't wait. Replacing the model with yet more DB shims to store NSTimeInterval but expose NSDates in the code is making our models look crazier than they already are.

Yes it's a place you would least expect it. But unfort when the truncation happens NSDate serialization returns without the decimal and the JSON parsing fails since it expects "eee dd-MMM-yyyy GG HH:mm:ss.SSS ZZZ" but gets "d MMM yyyy HH:mm:ss ZZZ". Maybe we're just using a crappy library (SwiftDate) but the downstream results happen none the less.

marchy commented Apr 22, 2016

That's great to hear it. Can't wait. Replacing the model with yet more DB shims to store NSTimeInterval but expose NSDates in the code is making our models look crazier than they already are.

Yes it's a place you would least expect it. But unfort when the truncation happens NSDate serialization returns without the decimal and the JSON parsing fails since it expects "eee dd-MMM-yyyy GG HH:mm:ss.SSS ZZZ" but gets "d MMM yyyy HH:mm:ss ZZZ". Maybe we're just using a crappy library (SwiftDate) but the downstream results happen none the less.

@bdash

This comment has been minimized.

Show comment
Hide comment
@bdash

bdash Apr 22, 2016

Contributor

That sounds like a bug that should be fixed in your JSON handling either way. It's completely possible for a date to legitimately have no fractional seconds.

Contributor

bdash commented Apr 22, 2016

That sounds like a bug that should be fixed in your JSON handling either way. It's completely possible for a date to legitimately have no fractional seconds.

@jpsim jpsim modified the milestones: 0.100, 0.101 Apr 26, 2016

@bdash bdash added S:In Progress and removed S:P1 Backlog labels May 3, 2016

@tgoyne tgoyne closed this in #3503 May 4, 2016

@tgoyne tgoyne removed the S:In Progress label May 4, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment