Adds group_by_exp filter #5513

thiagoarrais · 2016-10-25T20:39:24Z

Tasklist:

{% data | group_by_exp: "item", "item" %}
{% data | group_by_exp: "item", "item.version" %}
{% data | group_by_exp: "item", "item.version | split: '.' | first" %}
update docs

pathawks

literally nothing

😆

thiagoarrais · 2016-10-26T12:38:17Z

I'm thinking I need a parse_expression method here. But I'm not sure. Maybe modifying the parse_condition method will do?

thiagoarrais · 2016-10-26T12:42:15Z

thiagoarrais · 2016-10-31T16:39:53Z

Current version actually works and groups stuff. I need to learn one or thing or two about Liquid parsing to implement @caspervonb's example fully, though. I'm working on that.

Rubocop is also complaning about the size of lib/jekyll/filters.rb. We'll need to get that fixed before merging the PR, right? Any suggestions?

thiagoarrais · 2016-11-01T14:33:45Z

@caspervonb: can you take a look if this is what you had in mind?

caspervonb · 2016-11-01T15:51:13Z

itens... OCD 😭

The result looks correct but we should probaly yield the same object type as group_by? A group has a name (the value it was grouped by), an array of items and a size (group.items.size == group.size)

Relevant group_by tests
https://github.com/thiagoarrais/jekyll/blob/5415-group_by_exp-filter/test/test_filters.rb#L595

should "allow filters in expression" do
  items = [
    { "version"=>"1.0", "result"=>"slow" },
    { "version"=>"1.1.5", "result"=>"medium" },
    { "version"=>"2.7.3", "result"=>"fast" }
  ]

  result = @filter.group_by_exp(items, "item", "item.version | split: '.' | first")
  expected = [
    {
      "name" => "1",
      "items" => [
        { "version"=>"1.0", "result"=>"slow" },  
        { "version"=>"1.1.5", "result"=>"medium" },
      ]
    },
    {
      "name" => "2",
      "items" => { "version"=>"2.7.3", "result"=>"fast" }
    },
  ]

  assert_equal expected, result
end

But yeah its grouping by the expression, filters and all so 👍

thiagoarrais · 2016-11-01T16:59:20Z

itens... OCD 😭

😆

thiagoarrais · 2016-11-01T17:11:53Z

Awesome, @caspervonb! Will take a look at that next. See 679a0c1#diff-77a9bfa5e00815266a775b7f497e3595

caspervonb · 2016-11-02T06:56:38Z

These tests should cover everything I think, tho may want to add some more coverage for more complex expressions

context "group_by_exp filter" do
  should "successfully group array of Jekyll::Page's" do
    @filter.site.process
    grouping = @filter.group_by_exp(@filter.site.pages, "page", "page.layout | upcase")
    grouping.each do |g|
      assert(
        ["DEFAULT", "NIL", ""].include?(g["name"]),
        "#{g["name"]} isn't a valid grouping."
      )
      case g["name"]
      when "DEFAULT"
        assert(
          g["items"].is_a?(Array),
          "The list of grouped items for 'default' is not an Array."
        )
        assert_equal 5, g["items"].size
      when "nil"
        assert(
          g["items"].is_a?(Array),
          "The list of grouped items for 'nil' is not an Array."
        )
        assert_equal 2, g["items"].size
      when ""
        assert(
          g["items"].is_a?(Array),
          "The list of grouped items for '' is not an Array."
        )
        assert_equal 15, g["items"].size
      end
    end

  should "include the size of each grouping" do
    grouping = @filter.group_by_exp(@filter.site.pages, "page", "page.layout")
    grouping.each do |g|
      p g
      assert_equal(
        g["items"].size,
        g["size"],
        "The size property for '#{g["name"]}' doesn't match the size of the Array."
      )
    end
  end

  should "be equivalent of group_by" do
    grouping = @filter.group_by_exp(@filter.site.pages, "page", "page.layout")
    expected = @filter.group_by(@filter.site.pages, "layout")

    assert_equal grouping, expected
  end
end

thiagoarrais · 2016-11-04T12:20:31Z

lib/jekyll/filters.rb

-    #   "items" => [...] } # all the items where `property` == "larry"
-    def group_by(input, property)
-      if groupable?(input)
-        groups = input.group_by { |item| item_property(item, property).to_s }


I did this mostly because Rubocop was complaining about module length. But I'm torn about this refactoring mostly because of this call to item_property. This line got moved to the new module, but keeps calling code on the original one. In other words, the new module is bound to be included by the old one.

Should we keep this as per this commit? Should we extract this common code to a third module? Should we break the original Filters module in a completely different way?

thiagoarrais · 2016-11-04T12:26:34Z

I'm mostly happy about this PR now. But some feedback on the way I split the Filters module would be awesome. See 7f18ac8#r86635564.

thiagoarrais · 2016-11-04T21:07:43Z

This is done at 7f18ac8. Let me know if you need any help merging this.

caspervonb · 2016-11-04T21:17:42Z

I'm mostly happy about this PR now. But some feedback on the way I split the Filters module would be awesome. See 379f917#r86533966.

Think you merged that link/commitish into non existence ;)

thiagoarrais · 2016-11-04T21:18:11Z

Yeah. Will rebuild it.

caspervonb · 2016-11-04T21:26:41Z

Might as-well do RDoc comments too!

# Group an array of items by an expression
#
# input - the object array
# variable - the variable to assign each item to in the expression 
# expression -a Liquid comparison expression passed in as a string
#
# Returns the filtered array of objects

thiagoarrais · 2016-11-04T21:27:04Z

lib/jekyll/filters/grouping_filters.rb

+      #   "items" => [...] } # all the items where `property` == "larry"
+      def group_by(input, property)
+        if groupable?(input)
+          groups = input.group_by { |item| item_property(item, property).to_s }


I did this to make Rubocop happy. But I'm torn about this refactoring and the main reason is this call to item_property. It binds the new module to being included to the old one.

Should we keep this as is? Should we split item_property to its own module? Should we split Filters in a totally different way?

thiagoarrais · 2016-11-04T21:28:09Z

Yeah, sure!

thiagoarrais · 2016-11-04T21:29:46Z

@caspervonb: I fixed the link and edited the comment. Sorry.

caspervonb

LGTM

parkr

This LGTM once CI passes. 👌

parkr · 2016-11-29T05:33:17Z

lib/jekyll/filters/grouping_filters.rb

+      end
+
+      private
+      def make_grouped_array(groups)


I don't like the name of this method – something about Ruby conventions which I always loved is that the method could be considered like a variable, so you name things as though they aren't dynamic. In this case, you'd remove make_ and just have grouped_array(groups) or group_metadata_array(groups). Would either of those suit you?

something about Ruby conventions which I always loved is that the method could be considered like a variable

Totally agree! That is something I appreciate in Ruby too.

I guess I've been dayjobbing on Javaland for too long, though. ;-)

Would either of those suit you?

Sure. I chose grouped_array(groups). See 91f0b91

parkr · 2016-11-29T05:34:27Z

test/test_filters.rb

+      should "include the size of each grouping" do
+        groups = @filter.group_by_exp(@filter.site.pages, "page", "page.layout")
+        groups.each do |g|
+          p g


4ed4155

;-)

thiagoarrais · 2016-12-09T16:38:51Z

@parkr I've finally got around to fixing this. CI is now passing. Let me know if you want this to be squashed/rebased.

parkr · 2016-12-10T00:01:21Z

Looks great to me, thank you!

@jekyllbot: merge +minor

caspervonb · 2016-12-10T13:22:15Z

Great, thanks for the work @thiagoarrais 👍

thiagoarrais · 2016-12-10T15:55:04Z

You're welcome, @caspervonb. Let me know if you choose to use this in jsperf.

* master: (40 commits) Update history to reflect merge of #5658 [ci skip] Fix a couple of typos in the docs Update history to reflect merge of #5657 [ci skip] Update variables.md Update history to reflect merge of #5653 [ci skip] Improve Permalinks documentation. Update history to reflect merge of #5652 [ci skip] Use `assert_nil` instead of `assert_equal nil` Update history to reflect merge of #5513 [ci skip] Ignore symlinked file in windows Update history to reflect merge of #5612 [ci skip] Update history to reflect merge of #5643 [ci skip] Update Core team list in README Update history to reflect merge of #5641 [ci skip] use backticks for Gemfile for consistency since in the next sentence _config.yml file has backtick add a set of steps in site_configuration.feature update documentation for Windows narrow it down to only Windows revert and adjust site_configuration.feature add 'tzinfo-data' gem to generated Gemfile ...

thiagoarrais changed the title ~~Adds group_by_exp filter~~ WIP: Adds group_by_exp filter Oct 25, 2016

thiagoarrais mentioned this pull request Oct 25, 2016

Add group_by_exp filter #5415

Closed

pathawks reviewed Oct 25, 2016

View reviewed changes

pathawks added this to the flexible milestone Oct 25, 2016

pathawks added the feature label Oct 25, 2016

DirtyF added the needs-work label Oct 25, 2016

thiagoarrais mentioned this pull request Oct 26, 2016

Missing data point? hasharray/jsperf#62

Closed

jekyllbot removed the needs-work label Oct 26, 2016

thiagoarrais force-pushed the 5415-group_by_exp-filter branch from 5c8de61 to 1d163de Compare October 26, 2016 18:52

thiagoarrais force-pushed the 5415-group_by_exp-filter branch from fe4e672 to 8a96f6f Compare November 1, 2016 13:59

thiagoarrais commented Nov 4, 2016

View reviewed changes

thiagoarrais changed the title ~~WIP: Adds group_by_exp filter~~ Adds group_by_exp filter Nov 4, 2016

thiagoarrais force-pushed the 5415-group_by_exp-filter branch from a2c0be1 to 04837ac Compare November 4, 2016 21:11

Group using arbitraty Liquid expressions

7f18ac8

thiagoarrais force-pushed the 5415-group_by_exp-filter branch from 04837ac to 7f18ac8 Compare November 4, 2016 21:17

thiagoarrais commented Nov 4, 2016

View reviewed changes

caspervonb approved these changes Nov 4, 2016

View reviewed changes

RDoc for group_by_exp

7ac9653

parkr approved these changes Nov 29, 2016

View reviewed changes

thiagoarrais added 3 commits November 30, 2016 17:54

Whoops!

4ed4155

Rename for more idiomatic Ruby

91f0b91

Ignore symlinked file in windows

d4c8d7f

jekyllbot merged commit 8ed3240 into jekyll:master Dec 10, 2016

jekyllbot added the enhancement label Dec 10, 2016

jekyllbot added a commit that referenced this pull request Dec 10, 2016

Update history to reflect merge of #5513 [ci skip]

b02f306

thiagoarrais deleted the 5415-group_by_exp-filter branch December 10, 2016 02:39

dylanahsmith mentioned this pull request Dec 16, 2016

Upgrade to Liquid v4 #4362

Merged

parkr pushed a commit that referenced this pull request Dec 16, 2016

Update history to reflect merge of #5513 [ci skip]

8d10e9a

DirtyF removed this from the flexible milestone Nov 10, 2017

jekyll locked and limited conversation to collaborators Jul 11, 2019

jekyllbot added the frozen-due-to-age label Jul 11, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds group_by_exp filter #5513

Adds group_by_exp filter #5513

thiagoarrais commented Oct 25, 2016 •

edited

Loading

pathawks left a comment

thiagoarrais commented Oct 26, 2016

thiagoarrais commented Oct 26, 2016 •

edited

Loading

thiagoarrais commented Oct 31, 2016 •

edited

Loading

thiagoarrais commented Nov 1, 2016

caspervonb commented Nov 1, 2016

thiagoarrais commented Nov 1, 2016

thiagoarrais commented Nov 1, 2016 •

edited

Loading

caspervonb commented Nov 2, 2016 •

edited

Loading

thiagoarrais Nov 4, 2016

thiagoarrais commented Nov 4, 2016 •

edited

Loading

thiagoarrais commented Nov 4, 2016 •

edited

Loading

caspervonb commented Nov 4, 2016 •

edited

Loading

thiagoarrais commented Nov 4, 2016

caspervonb commented Nov 4, 2016

thiagoarrais Nov 4, 2016 •

edited

Loading

thiagoarrais commented Nov 4, 2016

thiagoarrais commented Nov 4, 2016 •

edited

Loading

caspervonb left a comment

parkr left a comment

parkr Nov 29, 2016

thiagoarrais Nov 30, 2016

parkr Nov 29, 2016

thiagoarrais Nov 30, 2016

thiagoarrais commented Dec 9, 2016

parkr commented Dec 10, 2016

caspervonb commented Dec 10, 2016

thiagoarrais commented Dec 10, 2016

Adds group_by_exp filter #5513

Adds group_by_exp filter #5513

Conversation

thiagoarrais commented Oct 25, 2016 • edited Loading

pathawks left a comment

Choose a reason for hiding this comment

thiagoarrais commented Oct 26, 2016

thiagoarrais commented Oct 26, 2016 • edited Loading

thiagoarrais commented Oct 31, 2016 • edited Loading

thiagoarrais commented Nov 1, 2016

caspervonb commented Nov 1, 2016

thiagoarrais commented Nov 1, 2016

thiagoarrais commented Nov 1, 2016 • edited Loading

caspervonb commented Nov 2, 2016 • edited Loading

thiagoarrais Nov 4, 2016

Choose a reason for hiding this comment

thiagoarrais commented Nov 4, 2016 • edited Loading

thiagoarrais commented Nov 4, 2016 • edited Loading

caspervonb commented Nov 4, 2016 • edited Loading

thiagoarrais commented Nov 4, 2016

caspervonb commented Nov 4, 2016

thiagoarrais Nov 4, 2016 • edited Loading

Choose a reason for hiding this comment

thiagoarrais commented Nov 4, 2016

thiagoarrais commented Nov 4, 2016 • edited Loading

caspervonb left a comment

Choose a reason for hiding this comment

parkr left a comment

Choose a reason for hiding this comment

parkr Nov 29, 2016

Choose a reason for hiding this comment

thiagoarrais Nov 30, 2016

Choose a reason for hiding this comment

parkr Nov 29, 2016

Choose a reason for hiding this comment

thiagoarrais Nov 30, 2016

Choose a reason for hiding this comment

thiagoarrais commented Dec 9, 2016

parkr commented Dec 10, 2016

caspervonb commented Dec 10, 2016

thiagoarrais commented Dec 10, 2016

thiagoarrais commented Oct 25, 2016 •

edited

Loading

thiagoarrais commented Oct 26, 2016 •

edited

Loading

thiagoarrais commented Oct 31, 2016 •

edited

Loading

thiagoarrais commented Nov 1, 2016 •

edited

Loading

caspervonb commented Nov 2, 2016 •

edited

Loading

thiagoarrais commented Nov 4, 2016 •

edited

Loading

thiagoarrais commented Nov 4, 2016 •

edited

Loading

caspervonb commented Nov 4, 2016 •

edited

Loading

thiagoarrais Nov 4, 2016 •

edited

Loading

thiagoarrais commented Nov 4, 2016 •

edited

Loading