Add save hooks on MiqReportResult to remove groupings #17598

NickLaMuro · 2018-06-15T21:53:20Z

The report.extras[:groupings] on MiqReportResult were being serialized to the report column, and on certain chargeback reports, can get quite large. This data is not needed, so we remove it prior to saving.

The after_commit exists in case there is a use for the saved data after it has been saved and still used within the original instance of the MiqReportResult for building the result. Most likely this is overkill, but for now it is in place as a safety measure.

Links

https://bugzilla.redhat.com/show_bug.cgi?id=1590908

NickLaMuro · 2018-06-15T21:55:37Z

app/models/miq_report_result.rb

@@ -35,6 +35,9 @@ class MiqReportResult < ApplicationRecord
    end
  end

+  before_save  { @_extra_groupings = report.extras.delete(:grouping) }
+  after_commit { report.extras[:grouping] = @_extra_groupings }


Worth noting: This has to be an after_commit instead of a after_save, otherwise it puts the ActiveRecord object into a dirty state, and causes an infinite loop. If this gets changed to the latter, the test should catch it.

What is this witchcraft... So, are we removing it before we commit it to the DB but put it back into the in-memory object?

Should we clear @_extra_groupings after?

So, are we removing it before we commit it to the DB but put it back into the in-memory object?

Yup, basically. Trying to play it safe just incase I missed something where this is used after it is saved.

Should we clear @_extra_groupings after?

I had that in there previously, but nothing is using that var, so I am tempted to just not bother. We aren't .duping it, so just passing a reference of it around. Up to you though if you feel strongly one way or the other.

jrafanie · 2018-06-15T21:59:23Z

can get quiet large

quite

NickLaMuro · 2018-06-15T22:02:08Z

can get quiet large

quite

@jrafanie quiet you...

NickLaMuro · 2018-06-15T22:36:07Z

Test failures are from this PR. Getting them fixed.

jrafanie · 2018-06-18T16:04:20Z

app/models/miq_report_result.rb

@@ -35,6 +35,9 @@ class MiqReportResult < ApplicationRecord
    end
  end

+  before_save  { @_extra_groupings = report.extras.delete(:grouping) if report.kind_of?(MiqReport) }


I feel like this needs a short comment about why we're doing this and a link to the BZ.

jrafanie · 2018-06-18T16:04:33Z

spec/models/miq_report_result_spec.rb

@@ -105,6 +105,23 @@
      expect(report_result.report_results.table.data).not_to be_nil
    end

+    it "should not include `extras[:grouing]` in the report column" do


typo: grouping

Butts... will fix

jrafanie · 2018-06-18T16:05:05Z

LGTM with the minor nitpicks I suggested.

JPrause · 2018-06-18T17:55:18Z

@miq-bot add_label blocker

JPrause · 2018-06-18T17:56:40Z

@NickLaMuro if this can be backported, please add the label gaprindashvili/yes

jrafanie · 2018-06-18T18:39:55Z

app/models/miq_report_result.rb

@@ -35,6 +35,17 @@ class MiqReportResult < ApplicationRecord
    end
  end

+  # These two hooks of a fix for the following BZ:


Maybe something like this: These two hooks prevent temporary chargeback aggregation data to be retained in each miq_report_results row found in the following BZ:

jrafanie · 2018-06-18T18:40:54Z

app/models/miq_report_result.rb

+  #
+  #   https://bugzilla.redhat.com/show_bug.cgi?id=1590908
+  #
+  # The remove extra data that is not necessary to be saved to the DB, but


Is The remove a typo? Is it supposed to be These remove

jrafanie · 2018-06-18T18:41:24Z

app/models/miq_report_result.rb

+  #   https://bugzilla.redhat.com/show_bug.cgi?id=1590908
+  #
+  # The remove extra data that is not necessary to be saved to the DB, but
+  # allow it to be retained for the remainder of the instanation incase the


instantiation
in case

jrafanie · 2018-06-18T18:43:19Z

app/models/miq_report_result.rb

+  # The remove extra data that is not necessary to be saved to the DB, but
+  # allow it to be retained for the remainder of the instanation incase the
+  # data is used to finish building the report results (the `after_commit` hook
+  # specifically, most likely this is overkill.


the after_commit hook specifically, is probably overkill but allows us to not break existing code that expects the temporary chargeback data in the instantiated object

^ Maybe?

https://bugzilla.redhat.com/show_bug.cgi?id=1590908 The `report.extras[:groupings]` on MiqReportResult were being serialized to the `report` column, and on certain chargeback reports, can get quite large. This data is not needed, so we remove it prior to saving. The `after_commit` exists in case there is a use for the saved data after it has been saved and still used within the original instance of the MiqReportResult for building the result. Most likely this is overkill, but for now it is in place as a safety measure. Also, `after_commit` is used because a `after_save` will cause an "stack level too deep" error since editing the report value causes the record to be put into a `dirty` state, and the ActiveRecord internals don't like that.

jrafanie · 2018-06-18T19:01:32Z

Line 50, Col 26 - Style/AsciiComments - Use only ascii symbols in comments.

Wow, bot is harsh

miq-bot · 2018-06-18T19:07:54Z

Checked commit NickLaMuro@253cf7a with ruby 2.3.3, rubocop 0.52.1, haml-lint 0.20.0, and yamllint 1.10.0
2 files checked, 0 offenses detected
Everything looks fine. 🏆

NickLaMuro · 2018-06-18T19:09:17Z

@miq-bot add_label reporting, enhancement, gaprindashvili/yes

jrafanie · 2018-06-19T14:34:25Z

@miq-bot add_label fine/yes

…g_saved_on_report_results Add save hooks on MiqReportResult to remove groupings (cherry picked from commit 28dc337) https://bugzilla.redhat.com/show_bug.cgi?id=1594387

simaishi · 2018-06-22T19:39:12Z

Fine backport details:

$ git log -1
commit 49df80e32e4ebf2646b5cd759430e5db3df29d7e
Author: Nick Carboni <ncarboni@redhat.com>
Date:   Mon Jun 18 18:03:39 2018 -0400

    Merge pull request #17598 from NickLaMuro/prevent_groupings_from_being_saved_on_report_results
    
    Add save hooks on MiqReportResult to remove groupings
    (cherry picked from commit 28dc33781dbe888dd8b0aaff4ada7af070891f7e)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1594387

…g_saved_on_report_results Add save hooks on MiqReportResult to remove groupings (cherry picked from commit 28dc337) https://bugzilla.redhat.com/show_bug.cgi?id=1594386

simaishi · 2018-06-22T19:45:56Z

Gaprindashvili backport details:

$ git log -1
commit d80c21a173aea3c0249dbea0ba0039428d44290d
Author: Nick Carboni <ncarboni@redhat.com>
Date:   Mon Jun 18 18:03:39 2018 -0400

    Merge pull request #17598 from NickLaMuro/prevent_groupings_from_being_saved_on_report_results
    
    Add save hooks on MiqReportResult to remove groupings
    (cherry picked from commit 28dc33781dbe888dd8b0aaff4ada7af070891f7e)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1594386

NickLaMuro commented Jun 15, 2018

View reviewed changes

NickLaMuro force-pushed the prevent_groupings_from_being_saved_on_report_results branch from 9f21198 to 071fd83 Compare June 15, 2018 22:01

NickLaMuro force-pushed the prevent_groupings_from_being_saved_on_report_results branch 2 times, most recently from 4cd1f28 to af169b6 Compare June 16, 2018 02:58

jrafanie reviewed Jun 18, 2018

View reviewed changes

NickLaMuro force-pushed the prevent_groupings_from_being_saved_on_report_results branch from af169b6 to e534eb2 Compare June 18, 2018 16:09

NickLaMuro mentioned this pull request Jun 18, 2018

Add tool to remove grouping from report results #17589

Merged

miq-bot added the blocker label Jun 18, 2018

jrafanie reviewed Jun 18, 2018

View reviewed changes

NickLaMuro force-pushed the prevent_groupings_from_being_saved_on_report_results branch 3 times, most recently from 0bfcd41 to 0addd38 Compare June 18, 2018 18:48

NickLaMuro force-pushed the prevent_groupings_from_being_saved_on_report_results branch from 0addd38 to 253cf7a Compare June 18, 2018 18:57

jrafanie approved these changes Jun 18, 2018

View reviewed changes

miq-bot added enhancement gaprindashvili/yes labels Jun 18, 2018

miq-bot added the core/reporting label Jun 18, 2018

carbonin approved these changes Jun 18, 2018

View reviewed changes

carbonin self-assigned this Jun 18, 2018

carbonin merged commit 28dc337 into ManageIQ:master Jun 18, 2018

carbonin added this to the Sprint 88 Ending Jun 18, 2018 milestone Jun 18, 2018

NickLaMuro mentioned this pull request Jun 19, 2018

Add nil check for report.extras on save hooks #17605

Merged

miq-bot added the fine/yes label Jun 19, 2018

simaishi added fine/backported and removed fine/yes labels Jun 22, 2018

simaishi added gaprindashvili/backported and removed gaprindashvili/yes labels Jun 22, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add save hooks on MiqReportResult to remove groupings #17598

Add save hooks on MiqReportResult to remove groupings #17598

NickLaMuro commented Jun 15, 2018 •

edited

NickLaMuro Jun 15, 2018

jrafanie Jun 15, 2018

jrafanie Jun 15, 2018

NickLaMuro Jun 15, 2018

jrafanie commented Jun 15, 2018

NickLaMuro commented Jun 15, 2018

NickLaMuro commented Jun 15, 2018

jrafanie Jun 18, 2018

jrafanie Jun 18, 2018

NickLaMuro Jun 18, 2018

jrafanie commented Jun 18, 2018

JPrause commented Jun 18, 2018

JPrause commented Jun 18, 2018

jrafanie Jun 18, 2018

jrafanie Jun 18, 2018 •

edited

jrafanie Jun 18, 2018

jrafanie Jun 18, 2018

jrafanie commented Jun 18, 2018

miq-bot commented Jun 18, 2018

NickLaMuro commented Jun 18, 2018

jrafanie commented Jun 19, 2018

simaishi commented Jun 22, 2018

simaishi commented Jun 22, 2018

Add save hooks on MiqReportResult to remove groupings #17598

Add save hooks on MiqReportResult to remove groupings #17598

Conversation

NickLaMuro commented Jun 15, 2018 • edited

Links

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrafanie commented Jun 15, 2018

NickLaMuro commented Jun 15, 2018

NickLaMuro commented Jun 15, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrafanie commented Jun 18, 2018

JPrause commented Jun 18, 2018

JPrause commented Jun 18, 2018

Choose a reason for hiding this comment

jrafanie Jun 18, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrafanie commented Jun 18, 2018

miq-bot commented Jun 18, 2018

NickLaMuro commented Jun 18, 2018

jrafanie commented Jun 19, 2018

simaishi commented Jun 22, 2018

simaishi commented Jun 22, 2018

NickLaMuro commented Jun 15, 2018 •

edited

jrafanie Jun 18, 2018 •

edited