Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Methods for ActiveSupport::Duration parsing from ISO 8601 and output to it #16917

Closed
wants to merge 21 commits into from

Conversation

@Envek
Copy link
Contributor

@Envek Envek commented Sep 14, 2014

This PR add methods to ActiveSupport::Duration to allow present it in ISO 8601 Duration format and instantiate from it.

ISO 8601 Duration format is standartized way to represent duration for interchange, it's already recognized by some database engines (e.g. PostgreSQL) and client side libraries (e.g. plugin for Moment.js durations). So, I think it should be part of ActiveSupport::Duration.

Some parts of code and tests are taken from ISO8601 gem by Arnau Siches (@arnau) and contributors. Many thanks to them.

This PR is required for PostgreSQL interval datatype support (for converting it from and to ActiveSupport::Duration) as I've proposed in Google Group here. See #16919. Because of that my parsing allows individual datetime parts to be negative (see PostgreSQL interval output).

There is no backward incompatible changes as I want it to be included in upcoming 4.2 release, if it still possible (pleeeaasee!).

end
end

def test_iso8601_parsing_valid_patterns

This comment has been minimized.

@egilburg

egilburg Sep 14, 2014
Contributor

does this test just repeat part of what's already tested in test_iso8601_output_and_reparsing?

This comment has been minimized.

@Envek

Envek Sep 14, 2014
Author Contributor

Yes, it repeat. This test only checks that validations doesn't complain. test_iso8601_output_and_reparsing tries to test some logic (as possible with weird ActiveSupport::Duration).
Should I remove this test case?

(?<weeks>-?\d+(?:[.,]\d+)?W)
) # Duration
$/x) || raise(ISO8601ParsingError.new("Invalid ISO 8601 duration: #{iso8601duration}"))
sign = (match[:sign].nil? || match[:sign] == '+') ? 1 : -1

This comment has been minimized.

@egilburg

egilburg Sep 14, 2014
Contributor

simpler to check for sign = match[:sign] == '-' ? -1 : 1

This comment has been minimized.

@Envek

Envek Sep 14, 2014
Author Contributor

Fixed

) # Duration
$/x) || raise(ISO8601ParsingError.new("Invalid ISO 8601 duration: #{iso8601duration}"))
sign = (match[:sign].nil? || match[:sign] == '+') ? 1 : -1
parts = match.names.zip(match.captures).select{|_k,v| v.present? }.map do |k, v|

This comment has been minimized.

@egilburg

egilburg Sep 14, 2014
Contributor

Do you expect non-nil blanks here? If not, you can use compact on hash.

This comment has been minimized.

@Envek

Envek Sep 14, 2014
Author Contributor

No, I don't expect non-nil blanks here, but I want to avoid constructing Hash here as it will be transformed again into Array on the next map call on the same line. I can replace it with .reject{|_k,v| v.nil? }.

Compare:
::Hash[match.names.zip(match.captures)].compact
match.names.zip(match.captures).reject{|_k,v| v.nil? }

arnau added a commit to arnau/ISO8601 that referenced this pull request Sep 14, 2014
Thanks to @egilburg comment in rails/rails#16917 pull
request for catching this.
arnau added a commit to arnau/ISO8601 that referenced this pull request Sep 14, 2014
Thanks to @Envek for catching it in rails/rails#16917
# Initialize new duration
time = ::Time.now
new(time.advance(parts) - time, parts)
end

This comment has been minimized.

@jeremy

jeremy Sep 14, 2014
Member

There's enough going on here to extract a separate ISO8601DurationParser

This comment has been minimized.

@Envek

Envek Sep 15, 2014
Author Contributor

Where it should be placed? Same file (inside Duration) or separate file? Its location?

def iso8601(precision=nil)
output = 'P'
# First, trying to summarize duration parts (they can be repetitive)
parts = self.parts.inject(::Hash.new(0)) {|p,(k,v)| p[k] += v; p }

This comment has been minimized.

@jeremy

jeremy Sep 14, 2014
Member

Normalizing parts may be extracted to a separate method

parts = parts.inject(::Hash.new(0)) {|p,(k,v)| p[k] = -v; p }
end
# Building output string
output << "#{parts[:years]}Y" if parts[:years].nonzero?

This comment has been minimized.

@jeremy

jeremy Sep 14, 2014
Member

Would expect normalized parts to not include a component if it's zero, so these could be nil checks

This comment has been minimized.

@Envek

Envek Sep 15, 2014
Author Contributor

There is ::Hash.new(0) used in normalization above, so there is no nils, there is zeroes: Hash.new(0)[:a].nil? # => false. I can change default value back to nil. for example.

@@ -1,3 +1,14 @@
* ActiveSupport::Duration can be parsed from and outputted to ISO 8601 duration format.
Parts of code and tests are taken from ISO8601 gem by Arnau Siches (@arnau).

This comment has been minimized.

@jeremy

jeremy Sep 14, 2014
Member

Is its license compatible? Where is its copyright notice?

This comment has been minimized.

@Envek

Envek Sep 15, 2014
Author Contributor

https://github.com/arnau/ISO8601/blob/master/LICENSE — MIT License (as in rails). Should be compatible AFAIK. Should be license information added here?

@@ -80,7 +80,7 @@ def ago(time = ::Time.current)
def inspect #:nodoc:
parts.
reduce(::Hash.new(0)) { |h,(l,r)| h[l] += r; h }.
sort_by {|unit, _ | [:years, :months, :days, :minutes, :seconds].index(unit)}.
sort_by {|unit, _ | [:years, :months, :weeks, :days, :hours, :minutes, :seconds].index(unit)}.

This comment has been minimized.

@jeremy

jeremy Sep 14, 2014
Member

Weeks and hours weren't separate parts before. Is this the only place they need to be supported?

This comment has been minimized.

@Envek

Envek Sep 15, 2014
Author Contributor

They were'nt separate parts when ActiveSupport::Duration was only constructed via methods on Numerics, like this:

dur1 = 1.hour
# => 3600 seconds # Hey, where is my hour?
dur2 = ActiveSupport::Duration.parse('PT1H')
# => 1 hour

IMO all parts should be saved whenever possible:

dur1.iso8601 # => "PT3600S"
dur2.iso8601 # => "PT1H"

This is because I've added them there. I've found no other place where all parts were listed.

This comment has been minimized.

@pixeltrix

pixeltrix Nov 24, 2015
Member

My only concern is that we're going to get different output depending on how the duration is constructed, e.g:

>> ActiveSupport::Duration.parse('P1W')
=> 1 week
>> 1.week
=> 7 days

Pondering whether we should not decompose values in Rails 5.

This comment has been minimized.

@Envek

Envek Nov 24, 2015
Author Contributor

I can try to fix that in some another pull request in a few days.

@Envek Envek force-pushed the Envek:iso8601_duration branch from b4c3c4f to 6cdfea9 Sep 15, 2014
@Envek
Copy link
Contributor Author

@Envek Envek commented Sep 15, 2014

Fixed some issues:

  • extracted parsing in separate ActiveSupport::Duration::ISO8601DurationParser class in separate file
  • extracted parts normalization into separate method
  • fixed parts normalization so returned hash default value is nil (as usual)

Method parse! throws exception ActiveSupport::Duration::ISO8601DurationParser::ParsingError on invalid input while parse returns nil.

Rebased branch on top of current master due to changes in ActiveSupport::Duration in recently merged #16574 .

@Envek
Copy link
Contributor Author

@Envek Envek commented Sep 16, 2014

Fixed some more remarks.

I've noticed that now there is next error occurs when I run activerecord's tests for #16919 on top of this branch:

/Users/anovikov/rails/activesupport/lib/active_support/duration/iso8601_duration_parser.rb:2: warning: loading in progress, circular require considered harmful - /Users/anovikov/rails/activesupport/lib/active_support/duration.rb
    from /Users/anovikov/.rvm/gems/ruby-2.1.2-gost/gems/rake-10.3.2/lib/rake/rake_test_loader.rb:4:in  `<main>'
    from /Users/anovikov/.rvm/gems/ruby-2.1.2-gost/gems/rake-10.3.2/lib/rake/rake_test_loader.rb:4:in  `select'
    from /Users/anovikov/.rvm/gems/ruby-2.1.2-gost/gems/rake-10.3.2/lib/rake/rake_test_loader.rb:15:in  `block in <main>'
    from /Users/anovikov/.rvm/gems/ruby-2.1.2-gost/gems/rake-10.3.2/lib/rake/rake_test_loader.rb:15:in  `require'
    from /Users/anovikov/rails/activerecord/test/cases/adapter_test.rb:1:in  `<top (required)>'
    from /Users/anovikov/rails/activerecord/test/cases/adapter_test.rb:1:in  `require'
    from /Users/anovikov/rails/activerecord/test/cases/helper.rb:15:in  `<top (required)>'
    from /Users/anovikov/rails/activesupport/lib/active_support/dependencies.rb:248:in  `require'
...
    from /Users/anovikov/rails/activesupport/lib/active_support/dependencies.rb:248:in  `block in require'
    from /Users/anovikov/rails/activesupport/lib/active_support/dependencies.rb:248:in  `require'
    from /Users/anovikov/rails/activesupport/lib/active_support/duration/iso8601_duration_parser.rb:1:in  `<top (required)>'
    from /Users/anovikov/rails/activesupport/lib/active_support/duration/iso8601_duration_parser.rb:2:in  `<module:ActiveSupport>'

How can I fix it? The reason seems to be in that I'm explicitly requires file with parser in duration.rb

@Envek Envek force-pushed the Envek:iso8601_duration branch 2 times, most recently from 929ed6f to fa2672d Feb 8, 2015
@Envek
Copy link
Contributor Author

@Envek Envek commented Feb 8, 2015

To avoid circular require warnings (absolutely don't understand why are they appearing such a lot) included ISO8601 parser into file activesupport/lib/active_support/duration.rb. Squashed and rebased on top of current master. Please review one more time.

@kaspth
Copy link
Member

@kaspth kaspth commented Feb 8, 2015

Did you require duration in the parser and vice versa? That sounds like it could cause the circular requires.

@Envek
Copy link
Contributor Author

@Envek Envek commented Feb 8, 2015

No, I had required parser only from duration (no requires from file with parser at all): commit that caused a lot of warnings.

@kaspth
Copy link
Member

@kaspth kaspth commented Feb 8, 2015

Duration was being autoloaded. It hit class Duration in the parser which would trigger another autoload and thus the files would keep trying to load each other.

raise ParsingError.new("Invalid ISO 8601 duration: #{iso8601duration} (only last part can be fractional)")
end
end

This comment has been minimized.

@kaspth

kaspth Feb 8, 2015
Member

✂️ this line

end

end

This comment has been minimized.

@kaspth

kaspth Feb 8, 2015
Member

✂️ this too

# See http://en.wikipedia.org/wiki/ISO_8601#Durations
# Parts of code are taken from ISO8601 gem by Arnau Siches (@arnau).
# This parser isn't so strict and allows negative parts to be present in pattern.
class ISO8601DurationParser

This comment has been minimized.

@kaspth

kaspth Feb 8, 2015
Member

The indentation level should match #method_missing above.

) # Duration
$/x) || raise(ParsingError.new("Invalid ISO 8601 duration: #{iso8601duration}"))
sign = match[:sign] == '-' ? -1 : 1
@parts = match.names.zip(match.captures).reject{|_k,v| v.nil? }.map do |k, v|

This comment has been minimized.

@kaspth

kaspth Feb 8, 2015
Member

I'd extract this to a private method. Also we prefer spaces in blocks reject { |_, v| v.nil? }.

This comment has been minimized.

@Envek

Envek Feb 8, 2015
Author Contributor

What exactly should go into private method? Only @parts construction from regexp match? Or validations too?

This comment has been minimized.

@kaspth

kaspth Feb 8, 2015
Member

I'd extract all of those into separate private methods.

end
@parts = ::Hash[parts].slice(:years, :months, :weeks, :days, :hours, :minutes, :seconds)
# Validate that is not empty duration or time part is empty if 'T' marker present
if parts.empty? || (match[:time].present? && match[:time][1..-1].empty?)

This comment has been minimized.

@kaspth

kaspth Feb 8, 2015
Member

You switch back and forth from using parts the instance variable and the reader method. Ditch the attr_reader and stick to @parts.

# Parts of code are taken from ISO8601 gem by Arnau Siches (@arnau).
# This parser isn't so strict and allows negative parts to be present in pattern.
class ISO8601DurationParser
attr_reader :parts

This comment has been minimized.

@kaspth

kaspth Feb 8, 2015
Member

After sticking to @parts just delete this.

This comment has been minimized.

@Envek

Envek Feb 8, 2015
Author Contributor

It's a kind of this parser's API. That parts are being fed to duration in ActiveSupport::Duration.parse! method (see line 111)

This comment has been minimized.

@kaspth

kaspth Feb 8, 2015
Member

Ah, didn't catch that 👍

(3.years + 3.days).iso8601
# => "P3Y3D"

*Andrey Novikov*

This comment has been minimized.

@gsamokovarov

gsamokovarov Feb 8, 2015
Contributor

I think it'd be fair to put Arnau Siches name here as well. This can also act as a reminder for us to special case it in the contributors app.

This comment has been minimized.

@kaspth

kaspth Feb 8, 2015
Member

@Envek you can also add a reference in the commit using [] around your comma separated names then I believe GitHub will attribute you both.

I don't know if the contributors app recognizes this though (cc @fxn).

This comment has been minimized.

@Envek

Envek Feb 8, 2015
Author Contributor

I've tried to feed in git commit's --author option various combinations like Andrey Novikov <envek@envek.name>, Arnau Siches <asiches@gmail.com> or [Andrey Novikov <envek@envek.name> Arnau Siches <asiches@gmail.com>], but no luck: git always strips everything after first email. Git just doesn't support multiple authors.
Pushed commit with author Andrey Novikov, Arnau Siches <envek@envek.name>

@Envek Envek force-pushed the Envek:iso8601_duration branch 2 times, most recently from a4013e1 to 621fdff Feb 8, 2015
@jeremy
Copy link
Member

@jeremy jeremy commented Apr 17, 2016

If we're pulling in code under MIT license, we need to include the license and copyright statement.

Also, if we're pulling in code, perhaps we should do a gem dep instead?

Nice work on this, and patience 😁

@Envek
Copy link
Contributor Author

@Envek Envek commented Apr 17, 2016

@jeremy there is not a much code left from the original code from ISO8601 gem as I've rewritten parser from regexp to string scanner by @pixeltrix suggestion. As far as I can see for now only duration examples strings inside tests are same with ISO8601. See the original code and tests.

Anyway where I should place license and copyright statement if I should? @arnau what do you think?

Gem dependency means that we should throw away whole ActiveSupport::Duration and replace it with ISO8601::Duration. Are you serious with that? 😄

@jeremy please take a look at #22806 while I'm fixing your notes— it's extracted from this pull request.

Envek added 2 commits Apr 17, 2016
This reverts commit c130b5c.
@Envek
Copy link
Contributor Author

@Envek Envek commented Apr 17, 2016

@jeremy fixed your notes

@jeremy
Copy link
Member

@jeremy jeremy commented Apr 17, 2016

@Envek We can't replace AS::Duration with ISO8601::Duration, but perhaps we could delegate implementation to it. You tell me :)

If you've rewritten the code, then you're no longer using the original code and aren't redistributing it, so you needn't pull in its license.

If you are reusing code, include the license+copyright along with the code, in a Ruby comment.

@arnau
Copy link

@arnau arnau commented Apr 18, 2016

@Envek do whatever makes more sense to you :)

Envek added 2 commits Apr 18, 2016
…ect current status
@Envek
Copy link
Contributor Author

@Envek Envek commented Apr 18, 2016

@jeremy, @arnau, I changed notices about licensing, take a look at 3a92929 . Is it enough?

If yes I would be glad to rebase and squash.

@@ -224,6 +224,9 @@ def test_comparable
assert_equal(1, (61 <=> 1.minute))
end

# ISO8601 string examples are taken from ISO8601 gem at https://github.com/arnau/ISO8601/blob/b93d466840/spec/iso8601/duration_spec.rb
# published under the conditions of MIT license: https://github.com/arnau/ISO8601/blob/b93d466840/LICENSE

This comment has been minimized.

@jeremy

jeremy Apr 18, 2016
Member

The license is pretty clear about what we need to include here:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

So we need to include the license text in a comment here.

This comment has been minimized.

@Envek

Envek Apr 18, 2016
Author Contributor

Included the license text in a comment

@Envek Envek force-pushed the Envek:iso8601_duration branch to dbb6a1d Apr 18, 2016
@@ -130,6 +133,23 @@ def respond_to_missing?(method, include_private=false) #:nodoc:
@value.respond_to?(method, include_private)
end

# Creates a new Duration from string formatted according to ISO 8601 Duration.
#
# See http://en.wikipedia.org/wiki/ISO_8601#Durations

This comment has been minimized.

@vipulnsward

vipulnsward Apr 18, 2016
Member

See http://en.wikipedia.org/wiki/ISO_8601#Durations .

class Duration
# Parses a string formatted according to ISO 8601 Duration into the hash .
#
# See http://en.wikipedia.org/wiki/ISO_8601#Durations

This comment has been minimized.

@vipulnsward

vipulnsward Apr 18, 2016
Member

http://en.wikipedia.org/wiki/ISO_8601#Durations.

This comment has been minimized.

@Envek

Envek Apr 18, 2016
Author Contributor

Placing dot in end of URL will make it broken.

This comment has been minimized.

@vipulnsward

vipulnsward Apr 18, 2016
Member

Actually this should be wrapped in as a markdown link.
See [ISO 8601](http://en.wikipedia.org/wiki/ISO_8601#Durations) for more information.

This comment has been minimized.

@Envek

Envek Apr 18, 2016
Author Contributor

Links have been markdownified.


private

# Return pair of duration's parts and whole duration sign

This comment has been minimized.

@vipulnsward

vipulnsward Apr 18, 2016
Member

missing .. There are a couple of places its missing, @Envek can you take a look?

This comment has been minimized.

@Envek

Envek Apr 18, 2016
Author Contributor

Fixed!


# Return pair of duration's parts and whole duration sign
# Parts are summarized (as they can become repetitive due to addition, etc)
# zero parts are removed as not significant.

This comment has been minimized.

Envek added 2 commits Apr 18, 2016
class Duration
# Parses a string formatted according to ISO 8601 Duration into the hash.
#
# See [ISO 8601](http://en.wikipedia.org/wiki/ISO_8601#Durations) for more information.

This comment has been minimized.

@vipulnsward

vipulnsward Apr 18, 2016
Member

😢 Sorry this is rdoc, my bad. Let me see.

This comment has been minimized.

@vipulnsward

vipulnsward Apr 18, 2016
Member

So rdoc makes sure the link is proper even if it's followed by a '.' Anyway, lets change to -

See {ISO 8601}[http://en.wikipedia.org/wiki/ISO_8601#Durations] for more information.

@@ -130,6 +133,23 @@ def respond_to_missing?(method, include_private=false) #:nodoc:
@value.respond_to?(method, include_private)
end

# Creates a new Duration from string formatted according to ISO 8601 Duration.
#
# See [ISO 8601](http://en.wikipedia.org/wiki/ISO_8601#Durations) for more information.

This comment has been minimized.

@vipulnsward

vipulnsward Apr 18, 2016
Member

See {ISO 8601}[http://en.wikipedia.org/wiki/ISO_8601#Durations] for more information.

This comment has been minimized.

@Envek

Envek Apr 18, 2016
Author Contributor

RDocified links, thanks.

@jeremy
Copy link
Member

@jeremy jeremy commented Apr 18, 2016

Merged! 04c512d

@jeremy jeremy closed this Apr 18, 2016
vipulnsward added a commit to vipulnsward/rails that referenced this pull request Apr 18, 2016
spastorino added a commit that referenced this pull request Apr 19, 2016
Add #16917 to release notes
@Envek
Copy link
Contributor Author

@Envek Envek commented Apr 19, 2016

@jeremy thank you for merging! You have just unlocked a special #16919!

P.S> I have a one more small ActiveSupport goodness: #20625, you can review it too if you have free time and wish :-)

@Envek Envek deleted the Envek:iso8601_duration branch Apr 19, 2016
chancancode added a commit that referenced this pull request Jun 27, 2016
@github-staff github-staff deleted a comment Jul 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

You can’t perform that action at this time.