Methods for ActiveSupport::Duration parsing from ISO 8601 and output to it #16917

Envek · 2014-09-14T13:54:05Z

This PR add methods to ActiveSupport::Duration to allow present it in ISO 8601 Duration format and instantiate from it.

ISO 8601 Duration format is standartized way to represent duration for interchange, it's already recognized by some database engines (e.g. PostgreSQL) and client side libraries (e.g. plugin for Moment.js durations). So, I think it should be part of ActiveSupport::Duration.

Some parts of code and tests are taken from ISO8601 gem by Arnau Siches (@arnau) and contributors. Many thanks to them.

This PR is required for PostgreSQL interval datatype support (for converting it from and to ActiveSupport::Duration) as I've proposed in Google Group here. See #16919. Because of that my parsing allows individual datetime parts to be negative (see PostgreSQL interval output).

There is no backward incompatible changes as I want it to be included in upcoming 4.2 release, if it still possible (pleeeaasee!).

egilburg · 2014-09-14T17:16:17Z

activesupport/test/core_ext/duration_test.rb

+    end
+  end
+
+  def test_iso8601_parsing_valid_patterns


does this test just repeat part of what's already tested in test_iso8601_output_and_reparsing?

Yes, it repeat. This test only checks that validations doesn't complain. test_iso8601_output_and_reparsing tries to test some logic (as possible with weird ActiveSupport::Duration).
Should I remove this test case?

@egilburg

Thanks to @egilburg comment in rails/rails#16917 pull request for catching this.

@Envek

Thanks to @Envek for catching it in rails/rails#16917

jeremy · 2014-09-14T22:03:09Z

activesupport/lib/active_support/duration.rb

+      # Initialize new duration
+      time  = ::Time.now
+      new(time.advance(parts) - time, parts)
+    end


There's enough going on here to extract a separate ISO8601DurationParser

Where it should be placed? Same file (inside Duration) or separate file? Its location?

Envek · 2014-09-15T08:19:49Z

Fixed some issues:

extracted parsing in separate ActiveSupport::Duration::ISO8601DurationParser class in separate file
extracted parts normalization into separate method
fixed parts normalization so returned hash default value is nil (as usual)

Method parse! throws exception ActiveSupport::Duration::ISO8601DurationParser::ParsingError on invalid input while parse returns nil.

Rebased branch on top of current master due to changes in ActiveSupport::Duration in recently merged #16574 .

Envek · 2014-09-16T08:09:22Z

Fixed some more remarks.

I've noticed that now there is next error occurs when I run activerecord's tests for #16919 on top of this branch:

/Users/anovikov/rails/activesupport/lib/active_support/duration/iso8601_duration_parser.rb:2: warning: loading in progress, circular require considered harmful - /Users/anovikov/rails/activesupport/lib/active_support/duration.rb
    from /Users/anovikov/.rvm/gems/ruby-2.1.2-gost/gems/rake-10.3.2/lib/rake/rake_test_loader.rb:4:in  `<main>'
    from /Users/anovikov/.rvm/gems/ruby-2.1.2-gost/gems/rake-10.3.2/lib/rake/rake_test_loader.rb:4:in  `select'
    from /Users/anovikov/.rvm/gems/ruby-2.1.2-gost/gems/rake-10.3.2/lib/rake/rake_test_loader.rb:15:in  `block in <main>'
    from /Users/anovikov/.rvm/gems/ruby-2.1.2-gost/gems/rake-10.3.2/lib/rake/rake_test_loader.rb:15:in  `require'
    from /Users/anovikov/rails/activerecord/test/cases/adapter_test.rb:1:in  `<top (required)>'
    from /Users/anovikov/rails/activerecord/test/cases/adapter_test.rb:1:in  `require'
    from /Users/anovikov/rails/activerecord/test/cases/helper.rb:15:in  `<top (required)>'
    from /Users/anovikov/rails/activesupport/lib/active_support/dependencies.rb:248:in  `require'
...
    from /Users/anovikov/rails/activesupport/lib/active_support/dependencies.rb:248:in  `block in require'
    from /Users/anovikov/rails/activesupport/lib/active_support/dependencies.rb:248:in  `require'
    from /Users/anovikov/rails/activesupport/lib/active_support/duration/iso8601_duration_parser.rb:1:in  `<top (required)>'
    from /Users/anovikov/rails/activesupport/lib/active_support/duration/iso8601_duration_parser.rb:2:in  `<module:ActiveSupport>'

How can I fix it? The reason seems to be in that I'm explicitly requires file with parser in duration.rb

Envek · 2015-02-08T17:36:01Z

To avoid circular require warnings (absolutely don't understand why are they appearing such a lot) included ISO8601 parser into file activesupport/lib/active_support/duration.rb. Squashed and rebased on top of current master. Please review one more time.

kaspth · 2015-02-08T17:40:43Z

Did you require duration in the parser and vice versa? That sounds like it could cause the circular requires.

Envek · 2015-02-08T17:45:21Z

No, I had required parser only from duration (no requires from file with parser at all): commit that caused a lot of warnings.

kaspth · 2015-02-08T18:03:33Z

Duration was being autoloaded. It hit class Duration in the parser which would trigger another autoload and thus the files would keep trying to load each other.

kaspth · 2015-02-08T18:05:40Z

activesupport/lib/active_support/duration.rb

+          raise ParsingError.new("Invalid ISO 8601 duration: #{iso8601duration} (only last part can be fractional)")
+        end
+      end
+


✂️ this line

fxn · 2015-02-08T19:09:44Z

@kaspth the contrib app certainly understands a trailing [...] with authors separated by commas and other separators. See for example 6f8d9bd.

I didn't know GitHub had that feature... you sure it does?

Envek · 2015-02-08T19:13:29Z

I've accounted all the notes and force pushed changes in the same commit. I've tried split initializer on separate methods, but dislike current implementations (they are linked too tight with each other).

P.S> in 6f8d9bd only one author specified. Git doesn't support multiple authors (at least with each one's email)

kaspth · 2015-02-08T19:18:12Z

@fxn I did remember seeing commits like that. I assumed it was for GitHub and not the contributors app, but it was the other way. Sorry about that!

Envek · 2015-02-08T19:18:24Z

Ah, sorry, misunderstood you. Added both authors in [] in end of commit message.

fxn · 2015-02-08T19:18:51Z

@Envek If the body of the commit message ends with

[Andrey Novikov, Arnau Siches]

at the very bottom, last line, the contrib app will pick it up automatically and give both of you credit.

See for example 84c0f73. That one uses "&" but a comma also works.

jeremy · 2016-04-16T23:33:23Z

activesupport/lib/active_support/duration.rb

+      parse!(iso8601duration)
+    rescue ISO8601Parser::ParsingError
+      nil
+    end


IMO we don't need a method that swallows the parse error.

jeremy · 2016-04-17T00:25:04Z

If we're pulling in code under MIT license, we need to include the license and copyright statement.

Also, if we're pulling in code, perhaps we should do a gem dep instead?

Nice work on this, and patience 😁

Envek · 2016-04-17T15:59:44Z

@jeremy there is not a much code left from the original code from ISO8601 gem as I've rewritten parser from regexp to string scanner by @pixeltrix suggestion. As far as I can see for now only duration examples strings inside tests are same with ISO8601. See the original code and tests.

Anyway where I should place license and copyright statement if I should? @arnau what do you think?

Gem dependency means that we should throw away whole ActiveSupport::Duration and replace it with ISO8601::Duration. Are you serious with that? 😄

@jeremy please take a look at #22806 while I'm fixing your notes— it's extracted from this pull request.

This reverts commit c130b5c.

Envek · 2016-04-17T16:22:17Z

@jeremy fixed your notes

jeremy · 2016-04-17T21:56:38Z

@Envek We can't replace AS::Duration with ISO8601::Duration, but perhaps we could delegate implementation to it. You tell me :)

If you've rewritten the code, then you're no longer using the original code and aren't redistributing it, so you needn't pull in its license.

If you are reusing code, include the license+copyright along with the code, in a Ruby comment.

arnau · 2016-04-18T07:50:03Z

@Envek do whatever makes more sense to you :)

…ect current status

Envek · 2016-04-18T08:36:08Z

@jeremy, @arnau, I changed notices about licensing, take a look at 3a92929 . Is it enough?

If yes I would be glad to rebase and squash.

jeremy · 2016-04-18T15:59:44Z

activesupport/test/core_ext/duration_test.rb

@@ -224,6 +224,9 @@ def test_comparable
    assert_equal(1, (61 <=> 1.minute))
  end

+  # ISO8601 string examples are taken from ISO8601 gem at https://github.com/arnau/ISO8601/blob/b93d466840/spec/iso8601/duration_spec.rb
+  # published under the conditions of MIT license: https://github.com/arnau/ISO8601/blob/b93d466840/LICENSE
+


The license is pretty clear about what we need to include here:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

So we need to include the license text in a comment here.

Included the license text in a comment

vipulnsward · 2016-04-18T20:27:07Z

activesupport/lib/active_support/duration.rb

@@ -130,6 +133,23 @@ def respond_to_missing?(method, include_private=false) #:nodoc:
      @value.respond_to?(method, include_private)
    end

+    # Creates a new Duration from string formatted according to ISO 8601 Duration.
+    #
+    # See http://en.wikipedia.org/wiki/ISO_8601#Durations


See http://en.wikipedia.org/wiki/ISO_8601#Durations .

vipulnsward · 2016-04-18T20:49:10Z

activesupport/lib/active_support/duration/iso8601_parser.rb

+  class Duration
+    # Parses a string formatted according to ISO 8601 Duration into the hash.
+    #
+    # See [ISO 8601](http://en.wikipedia.org/wiki/ISO_8601#Durations) for more information.


😢 Sorry this is rdoc, my bad. Let me see.

So rdoc makes sure the link is proper even if it's followed by a '.' Anyway, lets change to -

See {ISO 8601}[http://en.wikipedia.org/wiki/ISO_8601#Durations] for more information.

jeremy · 2016-04-18T23:31:08Z

Merged! 04c512d

[ci skip]

Add #16917 to release notes

Envek · 2016-04-19T18:55:13Z

@jeremy thank you for merging! You have just unlocked a special #16919!

P.S> I have a one more small ActiveSupport goodness: #20625, you can review it too if you have free time and wish :-)

egilburg reviewed Sep 14, 2014
View reviewed changes

arnau added a commit to arnau/ISO8601 that referenced this pull request Sep 14, 2014

Change sign checking for Durations.

2880c74

Thanks to @egilburg comment in rails/rails#16917 pull request for catching this.

arnau added a commit to arnau/ISO8601 that referenced this pull request Sep 14, 2014

Fix atomizing durations with ,

8535b7e

Thanks to @Envek for catching it in rails/rails#16917

Envek mentioned this pull request Sep 14, 2014

PostgreSQL interval datatype support #16919

Closed

jeremy reviewed Sep 14, 2014
View reviewed changes

seuros added the activesupport label Sep 16, 2014

kaspth reviewed Feb 8, 2015
View reviewed changes

jeremy reviewed Apr 16, 2016
View reviewed changes

Envek added 2 commits April 17, 2016 19:03

Revert "Changed duration output as json to ISO 8601 duration format"

00b8a51

This reverts commit c130b5c.

Removed error-swallowing parsing method from duration. parse! → parse

d22524e

Envek added 2 commits April 18, 2016 11:27

Merge branch 'master' into iso8601_duration

79fc484

Change notices about licensing status and code and data usage to refl…

3a92929

…ect current status

jeremy reviewed Apr 18, 2016
View reviewed changes

Include the whole MIT license to comments for the duration tests.

dbb6a1d

vipulnsward reviewed Apr 18, 2016
View reviewed changes

Envek added 2 commits April 18, 2016 23:42

Dots. Dots are everywhere.

c0f3c72

Markdonw links in comments to Wikipedia article

498df33

vipulnsward reviewed Apr 18, 2016
View reviewed changes

Markdown links → RDoc links

6776417

jeremy closed this Apr 18, 2016

vipulnsward added a commit to vipulnsward/rails that referenced this pull request Apr 18, 2016

Add rails#16917 to release notes

f2ac3ea

[ci skip]

spastorino added a commit that referenced this pull request Apr 19, 2016

Merge pull request #24615 from vipulnsward/16917-release-notes

324cd2d

Add #16917 to release notes

Envek deleted the iso8601_duration branch April 19, 2016 18:55

chancancode added a commit that referenced this pull request Jun 27, 2016

Add #16917 to 5.0.0 release notes

bdf8036

rails deleted a comment Jul 16, 2019

Methods for ActiveSupport::Duration parsing from ISO 8601 and output to it #16917

Methods for ActiveSupport::Duration parsing from ISO 8601 and output to it #16917

Envek commented Sep 14, 2014

egilburg Sep 14, 2014

Envek Sep 14, 2014

jeremy Sep 14, 2014

Envek Sep 15, 2014

Envek commented Sep 15, 2014

Envek commented Sep 16, 2014

Envek commented Feb 8, 2015

kaspth commented Feb 8, 2015

Envek commented Feb 8, 2015

kaspth commented Feb 8, 2015

kaspth Feb 8, 2015

fxn commented Feb 8, 2015

Envek commented Feb 8, 2015

kaspth commented Feb 8, 2015

Envek commented Feb 8, 2015

fxn commented Feb 8, 2015

jeremy Apr 16, 2016

jeremy commented Apr 17, 2016

Envek commented Apr 17, 2016

Envek commented Apr 17, 2016

jeremy commented Apr 17, 2016

arnau commented Apr 18, 2016

Envek commented Apr 18, 2016

jeremy Apr 18, 2016

Envek Apr 18, 2016

vipulnsward Apr 18, 2016

vipulnsward Apr 18, 2016

vipulnsward Apr 18, 2016

jeremy commented Apr 18, 2016

Envek commented Apr 19, 2016

Methods for ActiveSupport::Duration parsing from ISO 8601 and output to it #16917

Methods for ActiveSupport::Duration parsing from ISO 8601 and output to it #16917

Conversation

Envek commented Sep 14, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Envek commented Sep 15, 2014

Envek commented Sep 16, 2014

Envek commented Feb 8, 2015

kaspth commented Feb 8, 2015

Envek commented Feb 8, 2015

kaspth commented Feb 8, 2015

Choose a reason for hiding this comment

fxn commented Feb 8, 2015

Envek commented Feb 8, 2015

kaspth commented Feb 8, 2015

Envek commented Feb 8, 2015

fxn commented Feb 8, 2015

Choose a reason for hiding this comment

jeremy commented Apr 17, 2016

Envek commented Apr 17, 2016

Envek commented Apr 17, 2016

jeremy commented Apr 17, 2016

arnau commented Apr 18, 2016

Envek commented Apr 18, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jeremy commented Apr 18, 2016

Envek commented Apr 19, 2016