Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exclude Service::AGGREGATE_ALL_VM_ATTRS from MiqExp.to_sql #16915

Conversation

NickLaMuro
Copy link
Member

@NickLaMuro NickLaMuro commented Jan 31, 2018

These calls, while they are Arel and could be converted, would most likely be used in "filter" type scenarios, and would not be good candidates for that since they would perform horribly when used on more than one record at a time (which filtering would do that).

Instead of preventing these records from showing up as a filter option, just make sure they can't be used with SQL when they are. That said, they will cause N+1 hell, so they really shouldn't be filtered on... but...

¯\(°_o)/¯

Alternative solution explanation

This is one possible solution for this BZ. The other would be wrapping the arel_attribute values in the MiqExpression#to_arel method with a Arel::Nodes::Grouping call, which would allow eq calls to be done on these, and the other arel-based MiqExpression operators. This solution was proposed first because of the concerns raised in #11502 which add the Service::AGGREGATE_ALL_VM_ATTRS virtual_attributes:

These are DEFINITELY not queries that you want to run in a index type fashion, as they will not scale. This is because the nested selects will individually run for each service record returned. In this case, it will end up in a tying up a process, so these methods need to be used with care.

As mentioned above, this means if a user does decide to filter based on these columns, it will most likely be slow, but it will be a bunch of short queries to the DB instead of a single massive and crippling one. Ideally these filters just shouldn't be used for filtering Services.

Links

Steps for Testing/QA

I will add some tests in another commit/rebased commit, but for now, you can test with that the following doesn't raise an error in a bin/rails c:

irb> MiqExpression.new("=" => {"field" => "Service-aggregate_all_vm_cpus", "value" => "454"}).to_sql
#=> [nil, nil, {:supported_by_sql=>false}]

Instead of passing just the column name into
.excluded_col_for_expression, pass the entire field object into the
method so that the model can also be referenced if needed (will be used
in a future commit).

This method is only used where it has been edited here in our entire
codebase, so this should be a relatively safe change.  I also don't see
this as something that would called in an automate script.
These calls, while they are Arel and could be converted, would most
likely be used in "filter" type scenarios, and would not be good
candidates for that since they would perform horribly when used on more
than one record at a time (which filtering would do that).

Instead of preventing these records from showing up as a filter option,
just make sure they can't be used with SQL when they are.  That said,
they will cause N+1 hell, so they really shouldn't be filtered on...
but...

¯\(°_o)/¯
@NickLaMuro
Copy link
Member Author

cc/ @imtayadeway @kbrock

@miq-bot
Copy link
Member

miq-bot commented Jan 31, 2018

Checked commits NickLaMuro/manageiq@e7955bf~...0d32b87 with ruby 2.3.3, rubocop 0.52.0, haml-lint 0.20.0, and yamllint 1.10.0
1 file checked, 0 offenses detected
Everything looks fine. ⭐

@gtanzillo gtanzillo self-assigned this Jan 31, 2018
@gtanzillo gtanzillo added this to the Sprint 79 Ending Feb 12, 2018 milestone Jan 31, 2018
@gtanzillo gtanzillo merged commit 5709538 into ManageIQ:master Jan 31, 2018
@NickLaMuro
Copy link
Member Author

@gtanzillo Seeing as this was originally targeted for a 5.9 BZ, should we be backporting this to Gaprindashvili? Also, I have confirmed that this causes issues as far back as Darga as well (though, I think we don't support that version any more), should we backport to other branches as well?

simaishi pushed a commit that referenced this pull request Jan 31, 2018
…service-agg-col-with-miq-exp

Exclude Service::AGGREGATE_ALL_VM_ATTRS from MiqExp.to_sql
(cherry picked from commit 5709538)

https://bugzilla.redhat.com/show_bug.cgi?id=1540698
@simaishi
Copy link
Contributor

Gaprindashvili backport details:

$ git log -1
commit 01a01787c8c75d76ebf2d6f84d0b6ffdb9678021
Author: Gregg Tanzillo <gtanzill@redhat.com>
Date:   Wed Jan 31 10:24:37 2018 -0500

    Merge pull request #16915 from NickLaMuro/bz-1539710-quick-fix-avoid-service-agg-col-with-miq-exp
    
    Exclude Service::AGGREGATE_ALL_VM_ATTRS from MiqExp.to_sql
    (cherry picked from commit 57095386dfb0fe1d799a1ad7bfc6fef6278dac98)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1540698

simaishi pushed a commit that referenced this pull request Feb 1, 2018
…service-agg-col-with-miq-exp

Exclude Service::AGGREGATE_ALL_VM_ATTRS from MiqExp.to_sql
(cherry picked from commit 5709538)

https://bugzilla.redhat.com/show_bug.cgi?id=1540699
@simaishi
Copy link
Contributor

simaishi commented Feb 1, 2018

Fine backport details:

$ git log -1
commit a6d06865680a9741aa22ee35d21227d86c01077e
Author: Gregg Tanzillo <gtanzill@redhat.com>
Date:   Wed Jan 31 10:24:37 2018 -0500

    Merge pull request #16915 from NickLaMuro/bz-1539710-quick-fix-avoid-service-agg-col-with-miq-exp
    
    Exclude Service::AGGREGATE_ALL_VM_ATTRS from MiqExp.to_sql
    (cherry picked from commit 57095386dfb0fe1d799a1ad7bfc6fef6278dac98)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1540699

d-m-u pushed a commit to d-m-u/manageiq that referenced this pull request Jun 6, 2018
…x-avoid-service-agg-col-with-miq-exp

Exclude Service::AGGREGATE_ALL_VM_ATTRS from MiqExp.to_sql
(cherry picked from commit 5709538)

https://bugzilla.redhat.com/show_bug.cgi?id=1540699
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants