Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add precision option for datetime string dumping #370

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

add precision option for datetime string dumping #370

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 2 additions & 1 deletion dynamoid.gemspec
View file
Open in desktop
Expand Up @@ -23,7 +23,8 @@ Gem::Specification.new do |spec|
'Pascal Corpet',
'Brian Glusman',
'Peter Boling',
'Andrew Konchin'
'Andrew Konchin',
'Ryan Foster'
]
spec.email = ['peter.boling@gmail.com', 'brian@stellaservice.com']

Expand Down
6 changes: 5 additions & 1 deletion lib/dynamoid/dumping.rb
View file
Open in desktop
Expand Up @@ -222,7 +222,11 @@ def format_datetime(value, options)

if use_string_format
value_in_time_zone = Dynamoid::DynamodbTimeZone.in_time_zone(value)
value_in_time_zone.iso8601
if options[:iso_precision].nil?
value_in_time_zone.iso8601
else
value_in_time_zone.iso8601(options[:iso_precision].to_i)
end
else
unless value.respond_to?(:to_i) && value.respond_to?(:nsec)
value = value.to_time
Expand Down
14 changes: 11 additions & 3 deletions lib/dynamoid/type_casting.rb
View file
Open in desktop
Expand Up @@ -227,14 +227,22 @@ def process(value)
nil
elsif value.is_a?(String)
dt = begin
# parse first for stricter constraint failure before attempting to parse iso for precision support
# e.g. DateTime.iso8601("2018-12") is valid
DateTime.parse(value)
DateTime.iso8601(value)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need these changes in type casting. Anything that is like date/time could be casted to DateTime. Type casting isn't connected with writing/reading attributes to/from DynamoDB.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andrykonchin Per the comment, the existing expected behaviour is to break for poorly parsed strings. https://github.com/Dynamoid/dynamoid/blob/master/spec/dynamoid/type_casting_spec.rb#L143

I wrote this work around to support precision, and also keep the same erroneous expectations. Parse is expected to throw an exception, where as constructing with iso8601 has less strict input expectations.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Well.

                 DateTime.parse(value)
                 DateTime.iso8601(value)

Here. DateTime.parse supports precision, doesn't it?

irb(main):003:0> s = DateTime.now.iso8601(2)
=> "2019-07-29T22:06:24.62+03:00"
irb(main):004:0> DateTime.parse(s).usec
=> 620000

Here

            if dt_has_precision
              dt.utc

completely ignores ApplicationTimeZone.utc_offset what's wrong. If passed string doesn't have timezone - ApplicationTimeZone.utc_offset should be used as default timezone.

I thinks precision support could be added directly in this line:

DateTime.new(dt.year, dt.mon, dt.mday, dt.hour, dt.min, dt.sec, offset)

seconds argument could have fraction so this line could be changed in this way:

DateTime.new(dt.year, dt.mon, dt.mday, dt.hour, dt.min, dt.sec + dt.sec_fraction, offset)

Look at the example:

irb(main):015:0> s = DateTime.now.iso8601(4)
=> "2019-07-29T22:24:20.8163+03:00"
irb(main):016:0> DateTime.parse(s).sec
=> 20
irb(main):018:0> DateTime.parse(s).sec_fraction
=> (8163/10000)

rescue StandardError
nil
end
if dt
seconds = string_utc_offset(value) || ApplicationTimeZone.utc_offset
offset = seconds_to_offset(seconds)
DateTime.new(dt.year, dt.mon, dt.mday, dt.hour, dt.min, dt.sec, offset)
dt_has_precision = dt.to_f % 1 != 0
if dt_has_precision
dt.utc
else
seconds = string_utc_offset(value) || ApplicationTimeZone.utc_offset
offset = seconds_to_offset(seconds)
DateTime.new(dt.year, dt.mon, dt.mday, dt.hour, dt.min, dt.sec, offset)
end
end
else
value.to_datetime
Expand Down
2 changes: 1 addition & 1 deletion lib/dynamoid/undumping.rb
View file
Open in desktop
Expand Up @@ -202,7 +202,7 @@ def process(value)
else
@options[:store_as_string]
end
value = DateTime.iso8601(value).to_time.to_i if use_string_format
value = DateTime.iso8601(value).to_time if use_string_format
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it should be tested as well. It's a new feature - storing milliseconds when use string format for datetime.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, I think this could be a clearer abstraction. FWIW, This was amended to make the current and additional spec coverage pass

ApplicationTimeZone.at(value)
end
end
Expand Down
2 changes: 1 addition & 1 deletion lib/dynamoid/version.rb
View file
Open in desktop
@@ -1,5 +1,5 @@
# frozen_string_literal: true

module Dynamoid
VERSION = '3.2.0'
VERSION = '3.2.1'
end
14 changes: 14 additions & 0 deletions spec/dynamoid/dumping_spec.rb
View file
Open in desktop
Expand Up @@ -239,6 +239,8 @@
Dynamoid.config.store_datetime_as_string = store_datetime_as_string
end



Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excessive new lines detected

it 'prioritize field option over global one' do
store_datetime_as_string = Dynamoid.config.store_datetime_as_string
Dynamoid.config.store_datetime_as_string = false
Expand Down Expand Up @@ -366,6 +368,18 @@
expect(raw_attributes(obj)[:signed_up_on]).to eql('2017-09-25')
end

it 'stores in string format with precision when :iso_precision is present' do
klass = new_class do
field :signed_up_on, :datetime, store_as_string: true, iso_precision: 6
end

signed_up_on = DateTime.now.utc.iso8601(6)

obj = klass.create(signed_up_on: signed_up_on)

expect(reload(obj).signed_up_on.to_f).to eql(DateTime.iso8601(signed_up_on).to_f)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think i't completely OK to use to_f here but it's a bad practice to check equality of floating point numbers. Comparing milliseconds or string representation would be more correct.

The precision is 6 in the test so it's OK, but with precision 9 (with nanoseconds) this test could become flaky because MRI isn't accurate in preserving nanoseconds for Time/DateTime.

end

it 'stores in string format when global option :store_date_as_string is true' do
klass = new_class do
field :signed_up_on, :date
Expand Down