fixes #16725 - add plugin extension point for smart proxies #3904

timogoebel · 2016-09-29T12:47:00Z

This commit adds the ability to easily add smart proxies to models from plugins. This also includes tiny dsl to specify the proxies in a model directly.
The proxies automatically show up in the model's form and api.

If this gets merged, I'll send a pr to foreman_discovery to make use of the feature.

domcleal · 2016-09-30T13:01:30Z

app/services/foreman/plugin.rb

+    def smart_proxy_for(klass, name, options)
+      @smart_proxies[klass.name] ||= {}
+      @smart_proxies[klass.name][name] = options
+      klass.register_smart_proxy(name, options)


This probably won't work well with class autoloading, as if klass gets reloaded at any point the belongs_to association (etc) will be lost. You could either:

Add an included block to the concern so after it's loaded, it queries Foreman::Plugin to reload any smart proxies on the model.

Remove the @smart_proxies stuff (though it's my preferred method) and use to_prepare instead, which would be called on every development request (in the dev environment where reloading occurs).

domcleal · 2016-09-30T13:06:23Z

app/views/api/v2/subnets/main.json.rabl

-
-child :dns => :dns do
-  extends "api/v2/smart_proxies/base"
+Subnet.smart_proxies.keys.each do |proxy|


It has to be Subnet, because @object would be Subnet::Ipv4 for example. The smart proxies are registered only on `Subnet´.

class_attribute may help here, as its value is inherited by subclasses if a more specific subclass value isn't set. See domcleal@4d05ce2 for example.

Note that's specifically fetching, merging, then setting the value of smart_proxies_from_model rather than modifying the same hash, which should allow for subclasses to vary their registered smart proxies from the parent (they'll receive a copy of the parent + their own alterations).

domcleal · 2016-09-30T13:08:18Z

app/helpers/subnets_helper.rb

@@ -1,4 +1,6 @@
 module SubnetsHelper
+  include SharedSmartProxiesHelper


Shouldn't be needed, all helpers get loaded usually. Ditto in others.

domcleal · 2016-09-30T13:08:37Z

app/helpers/realms_helper.rb

@@ -0,0 +1,3 @@
+module RealmsHelper


Not required? All helpers should be loaded.

domcleal · 2016-09-30T13:10:43Z

app/validators/proxy_features_validator.rb

@@ -6,7 +6,7 @@ def initialize(args)

  def validate_each(record, attribute, value)
    if value && !value.has_feature?(@options[:feature])
-      record.errors["#{attribute}_id"] << _(@options[:message])
+      record.errors["#{attribute}_id"] << _('does not have the %s feature') % @options[:feature]


Best to prefer @options[:message] here if it's set, else (||) use the built in one - it's more like Rails ones then, and it's possible a plugin is using this validator.

domcleal · 2016-09-30T13:12:06Z

app/views/api/v2/domains/main.json.rabl

@@ -3,3 +3,9 @@ object @domain
 extends "api/v2/domains/base"

 attributes :fullname, :dns_id, :created_at, :updated_at
+
+@object.smart_proxies.keys.each do |proxy|


I'd suggest putting this into a partial like api/v2/taxonomies/children_nodes, it's identical in every location.

domcleal · 2016-09-30T13:12:37Z

app/views/hosts/_form.html.erb

@@ -73,7 +73,10 @@

        <%= host_puppet_environment_field f %>

-        <%= puppet_master_fields f, true, @host.hostgroup && @host.hostgroup_id_was.nil? %>
+        <%= smart_proxy_fields f,
+          :can_override => true,


Nitpick: indentation should be more than the smart_proxy_fields word itself.

timogoebel · 2016-09-30T15:13:49Z

@domcleal : Updated. Thanks for your comments and domcleal/foreman@4d05ce2! I have also added api doc handling. It has some redundancy but it am unsure how to write an extension point for this without having to monkey-patch apipie-rails. So I haven't really looked into that.

domcleal · 2016-10-03T08:04:50Z

app/views/api/v2/realms/main.json.rabl

@@ -1,3 +1,5 @@
 object @realm
 extends "api/v2/realms/base"
+extends "api/v2/smart_proxies/children_nodes"
+
 attributes :name, :realm_proxy_id, :realm_type, :created_at, :updated_at


realm_proxy_id and _name should be replaced by the smart_proxy/children_nodes too I think, it can be generated easily.

Ditto in other views, e.g. host has puppet_proxy_id/puppet_ca_proxy_name which can be replaced.

@domcleal : I saw the params, however we break compatibility if we just remove these. What is the normal procedure for this?

Oh sorry - I hadn't meant to remove them. I meant to replace the explicit listings of attributes :realm_proxy_id by code that automatically calls attributes with _id and _name for each registered smart proxy association. IDs and names for new associations from plugins will automatically appear in the output too then, not just through full child nodes.

timogoebel · 2016-10-03T12:47:47Z

@domcleal : Added the requested change and parameter_filter handling. Hope, we have all covered now. The smart proxy params for the host model were defined in host_base, I have moved them to host_common. Was there any reason they were in host_base?

timogoebel · 2016-10-04T08:28:18Z

I just tested this a little more. Somehow the inheritance thing still does not work. I need to take a closer look.

timogoebel · 2016-10-04T09:02:22Z

@domcleal : I reworked the registration from plugins a bit. They are also inherited now.

domcleal

Thanks, other than one small inline comment the registration system does seem to be more reliable now and it works well.

It would be reassuring if you could add some tests (either to the concern tests or subnet model tests) to check that the inheritance is indeed working between Subnet, Subnet::Ipv4/6 as I can't see any coverage of this right now.

domcleal · 2016-10-03T13:11:44Z

app/validators/proxy_features_validator.rb

+      else
+        message = _(@options[:message])
+      end
+      record.errors["#{attribute}_id"] << message


Use record.errors.add("#{attribute}_id", message), it's the preferred API. (Deprecated later.)

domcleal · 2016-10-04T14:52:49Z

The smart proxy params for the host model were defined in host_base, I have moved them to host_common. Was there any reason they were in host_base?

No, probably a mistake from when protected attributes were initially added.

@domcleal : I reworked the registration from plugins a bit. They are also inherited now.

Per the review comment, it'd be great if there was some test coverage of this, I don't see it.

timogoebel · 2016-10-05T13:00:39Z

The tests run fine on my laptop. I have rebased the code. Let's see if they run through now.

domcleal · 2016-10-05T14:53:50Z

It's an interference between some of the unit tests that permanently change the behaviour of models via the plugin interface, then the controller tests fail later. You can reproduce it with:

be ruby -Itest:. -e "require 'test/unit/plugin_test.rb'; require 'test/controllers/api/v1/subnets_controller_test.rb'"

You may want to stub some of the class modifications (e.g. adding validators) in the plugin unit tests rather than letting them modify the class (and attempting to revert changes).

timogoebel · 2016-10-05T17:25:53Z

Thanks a lot, @domcleal . You are awesome. Saved me the time of looking into this. That is why the test now runs against the "Awesome" module defined in the test class. Well, and it should fix the tests. I also renamed smart_proxies to ´registered_smart_proxies´ as apparently the former term is used quite a lot throughout the code and I didn't want to shadow it.

timogoebel · 2016-10-05T19:52:25Z

There is still one failing test. I'll investigate tomorrow.

timogoebel · 2016-10-06T08:09:48Z

@domcleal : That was it. Tests are finally green.

domcleal · 2016-10-07T11:09:00Z

Merged, thanks @timogoebel!

Please could you add a section to http://projects.theforeman.org/projects/foreman/wiki/How_to_Create_a_Plugin, noting that it requires Foreman 1.14 or higher.

timogoebel · 2016-10-07T11:28:48Z

I added the section to the docs.

lzap · 2016-10-11T12:47:13Z

Nice patch!

theforeman-bot added Needs testing Not yet reviewed labels Sep 29, 2016

timogoebel mentioned this pull request Sep 29, 2016

fixes #14036,#16722 - use plugin hook for subnet smart proxy theforeman/foreman_discovery#303

Merged

domcleal suggested changes Sep 30, 2016

View reviewed changes

domcleal added Waiting on contributor and removed Not yet reviewed labels Sep 30, 2016

timogoebel force-pushed the 16725-smart-proxy-hooks branch from f63d202 to bf610a4 Compare September 30, 2016 15:11

theforeman-bot added Needs re-review Needs testing and removed Waiting on contributor labels Sep 30, 2016

domcleal suggested changes Oct 3, 2016

View reviewed changes

domcleal added Waiting on contributor and removed Needs re-review labels Oct 3, 2016

timogoebel force-pushed the 16725-smart-proxy-hooks branch from bf610a4 to 8e0bb69 Compare October 3, 2016 12:45

theforeman-bot added Needs re-review Needs testing and removed Waiting on contributor labels Oct 3, 2016

timogoebel added Waiting on contributor and removed Needs re-review labels Oct 4, 2016

timogoebel force-pushed the 16725-smart-proxy-hooks branch from 8e0bb69 to 01185e9 Compare October 4, 2016 08:57

theforeman-bot added Needs re-review Needs testing and removed Waiting on contributor labels Oct 4, 2016

domcleal suggested changes Oct 4, 2016

View reviewed changes

domcleal added the Waiting on contributor label Oct 4, 2016

domcleal added Waiting on contributor and removed Needs re-review labels Oct 5, 2016

timogoebel force-pushed the 16725-smart-proxy-hooks branch from 799eb83 to 1843c17 Compare October 5, 2016 12:50

theforeman-bot added Needs re-review Needs testing and removed Waiting on contributor labels Oct 5, 2016

domcleal added Waiting on contributor and removed Needs re-review labels Oct 5, 2016

timogoebel force-pushed the 16725-smart-proxy-hooks branch from 1843c17 to acc6b80 Compare October 5, 2016 17:23

theforeman-bot added Needs re-review Needs testing and removed Waiting on contributor labels Oct 5, 2016

timogoebel force-pushed the 16725-smart-proxy-hooks branch from acc6b80 to 0e1a508 Compare October 5, 2016 19:08

timogoebel added the Waiting on contributor label Oct 5, 2016

domcleal removed the Needs re-review label Oct 6, 2016

fixes #16725 - add plugin extension point for smart proxies

9560791

timogoebel force-pushed the 16725-smart-proxy-hooks branch from 0e1a508 to 9560791 Compare October 6, 2016 07:27

theforeman-bot added Needs re-review Needs testing and removed Waiting on contributor labels Oct 6, 2016

domcleal approved these changes Oct 7, 2016

View reviewed changes

domcleal merged commit c676093 into theforeman:develop Oct 7, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fixes #16725 - add plugin extension point for smart proxies #3904

fixes #16725 - add plugin extension point for smart proxies #3904

timogoebel commented Sep 29, 2016

domcleal Sep 30, 2016

domcleal Sep 30, 2016

timogoebel Sep 30, 2016

domcleal Sep 30, 2016 •

edited

Loading

domcleal Sep 30, 2016 •

edited

Loading

domcleal Sep 30, 2016

domcleal Sep 30, 2016

domcleal Sep 30, 2016

domcleal Sep 30, 2016

timogoebel commented Sep 30, 2016

domcleal Oct 3, 2016 •

edited

Loading

timogoebel Oct 3, 2016

domcleal Oct 3, 2016 •

edited

Loading

timogoebel commented Oct 3, 2016

timogoebel commented Oct 4, 2016

timogoebel commented Oct 4, 2016

domcleal left a comment

domcleal Oct 3, 2016 •

edited

Loading

timogoebel Oct 4, 2016

domcleal commented Oct 4, 2016

timogoebel commented Oct 5, 2016

domcleal commented Oct 5, 2016

timogoebel commented Oct 5, 2016 •

edited

Loading

timogoebel commented Oct 5, 2016

timogoebel commented Oct 6, 2016

domcleal commented Oct 7, 2016

timogoebel commented Oct 7, 2016

lzap commented Oct 11, 2016

		@@ -1,4 +1,6 @@
		module SubnetsHelper
		include SharedSmartProxiesHelper

fixes #16725 - add plugin extension point for smart proxies #3904

fixes #16725 - add plugin extension point for smart proxies #3904

Conversation

timogoebel commented Sep 29, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

domcleal Sep 30, 2016 • edited Loading

Choose a reason for hiding this comment

domcleal Sep 30, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timogoebel commented Sep 30, 2016

domcleal Oct 3, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

domcleal Oct 3, 2016 • edited Loading

Choose a reason for hiding this comment

timogoebel commented Oct 3, 2016

timogoebel commented Oct 4, 2016

timogoebel commented Oct 4, 2016

domcleal left a comment

Choose a reason for hiding this comment

domcleal Oct 3, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

domcleal commented Oct 4, 2016

timogoebel commented Oct 5, 2016

domcleal commented Oct 5, 2016

timogoebel commented Oct 5, 2016 • edited Loading

timogoebel commented Oct 5, 2016

timogoebel commented Oct 6, 2016

domcleal commented Oct 7, 2016

timogoebel commented Oct 7, 2016

lzap commented Oct 11, 2016

domcleal Sep 30, 2016 •

edited

Loading

domcleal Sep 30, 2016 •

edited

Loading

domcleal Oct 3, 2016 •

edited

Loading

domcleal Oct 3, 2016 •

edited

Loading

domcleal Oct 3, 2016 •

edited

Loading

timogoebel commented Oct 5, 2016 •

edited

Loading