Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
[dev.icinga.com #12093] Data Field - Dictionary type/nested variable currently not possible #337
This issue has been migrated from Redmine: https://dev.icinga.com/issues/12093
Created by lebvanih on 2016-07-04 11:05:38 +00:00
We are currently trying to import our running configuration in Icinga Director (1.1.0). Our configuration rely on dictionaries and "nested variables" in our hosts. These are used in our "apply for" rules.
Our configuration is mostly based on the following example from the official documentation (see [[http://docs.icinga.org/icinga2/latest/doc/module/icinga2/toc\#!/icinga2/latest/doc/module/icinga2/chapter/monitoring-basics#using-apply-for-custom-attribute-override\]\]):
We also use this kind of structure:
The structure above was used for three reasons in our configuration:
I tried to workaround with a data field named like this: "an_ncpa.token", but it get renamed to "an_npcatoken" as soon as the configuration is rendered.
If this kind of data structure is not "valid" could you please guide us on what we should use instead?
Updated by msmol on 2016-08-03 19:50:09 +00:00
In the aforementioned example :
The "an_ncpa" form definition could be represented by a DataObject such as:
Where "notification_threshold" is another DataObject that can be used generically for both 'load' and 'cpuload' etc.
That being said, the "disk" structure above would be difficult to implement using this approach. A possible workaround would be to use a structure like this instead:
This would be generated by a DataObject like so:
Would this be an acceptable solution for you? Open to suggestions / alternatives / reasons why this will fail miserably.
Updated by tgelf on 2016-08-03 22:05:23 +00:00
thanks a lot for your input, this absolutely makes sense to me. What about calling it "Dictionary" instead of "Data Object" to follow Icinga 2 naming conventions? Should the value really be part of the type? I'd consider leaving it away. Right now, for all data fields default values are specified when assigning a field to a real object. Then, a DataType currently consists of a class name stored in an entry in the datafield table. So why not basing the dictionary on this, creating a list of datafield_ids and related arity?
Let me show you the DB tables that could reflect this:
We could combine is_required and allow_multiple into a single "arity" property or use min/max-count, but as is_required is already used in the *_field table it seemed to fit better. Please note that this schema would also allow one to create two different types for the same key, something that could also be desired. Your initial disk example should perfectly be possible with this, your workaround (Array of disks) isn't. But that's what we should then add as a new DataType "Array of SomeType" while renaming the current Array to "Array of Strings".
Updated by tgelf on 2016-08-04 15:02:40 +00:00
Fantastic! Can't wait to see the result ;-) This is gonna be a pretty cool feature I guess.
Updated by msmol on 2016-08-08 19:57:47 +00:00
We started working on this feature and have a a few things done, and a few things left to do.
So far, we've added 2 new tabs in datalist section, for creating Dictionaries and DictionaryFields.
What we're missing is the ability to set a custom attribute with type Dictionary, as we're not really sure how to show this kind of nested data in a reasonably nice way. cardeois tells me that he thinks you're already working on something like this. Can you confirm whether or not you are, and if so if you've made much progress?
You can see our internal PR inside our fork here: internap#1
Updated by tgelf on 2016-08-09 15:20:08 +00:00
Perfect. I had a quick look at it, looks good to me. What was the reason for the "Rename dictionary_field to dictionaryfield" commit? All the other implemented fields use "_field" as a postfix, did you run in any problems with this?
One missing part is the data type itself, please have a look into library/Director/DataType for examples and register your new type in configuration.php. Registration might seem useless, but it is pretty cool as it allows other Icinga Web 2 modules to provide their very own data types in an upgrade-safe way.
The DataType must provide a form field, and this is where it starts to become tricky. Such a form field is basically a ZF1 form element. You'll need to create one for Dictionaries in library/Director/Web/Form/Element. That element can specify a custom view helper, you'll also need one, should be placed into application/views/helpers/.
The web form is the trickiest part of your feature request and the main reason why I avoided to start working on it so far ;-) You could have a look at the ExtensibleSet for an example, but that one isn't isolated very good.
What I'm currently working on (I think that's what cardeois was referring to) is a form element for "Filters", allowing one to create deep nested filter definitions while allowing the developer to handle it like a single-element form field. That component got larger than expected, as I want to use it for nested apply rules and added support for data types. Please have a look at the attached screenshot:
Look & feel might be improved. However, it is pretty cool. Every icon is a form button, you can nest it as far as your screen allows. And still, in your code you just write
...and that's is. $element->getValue() gives you a full Filter object that can be serialized and stored to DB. I wanted to have this feature in v1.1.1, even if I haven't been forced to do so. By the end, it became the main reason why I didn't release v1.1.1 yet :p I'm currently on vacation, but I'll continue to work on this these days. Still a little bit unhappy with how I have to deal with data types, but I'll find a solution.
As you can see, the problems I have to tackle for this feature are very similar to yours. So it could absolutely make sense for you to wait for this, but you could also start with this in parallel. Regardless of this I'd suggest to go on with the provided DataType itself. Then make your first steps with a rough custom FormElement. Do not spend to much time there, eventually cheat by using a simple text field asking for JSON in the meantime. Try to figure out whether you can use CustomVariableDictionary for validation/serialization, or what other components might be missing.
I hope this helps, please do not hesitate letting me know when I just confused you instead ;-)
Updated by msmol on 2016-08-10 19:27:13 +00:00
Yes, I don't remember exactly what the issue was, but between whatever the issue was (been a few days now), and seeing how "DirectorDatafield" uses "Datafield" and not "DataField", it seemed like a simple solution that solved the issue at the time.
Yes, already started work on that, but isn't committed yet. Basically for the reasons you listed, i.e. wasn't sure what to do for the web form, and seeing as cardeois mentioned you were maybe already working on something for this, wrote the comment above :-)
I think I'll take your advice and cheat for now with a simple text field.
Thanks for the help, and enjoy your vacation
I'am currently writing my bachelor thesis about the Icinga Director. One point of this is to analyze how to migrate an existing configuration to the Director. There I ran into the issue with the dictionary-support. So are there any news about this? Is there a workaround or something?
Propose to add an ability to insert user-defined functions also. Icinga2 core supports complex data types since 2015 (new-features-in-icinga-2-3) and the following vars definition is perfectly valid in Icinga2 world:
Director object inspection already represents functions as:
Therefore comment sound reasonable to me.
We need them too.
Currently we have an service set with the defaults and then create a new service set for each exception
I wanted to migrate too and stuck now again, after I remembered, why I did not move month ago to the Director module: I need the dictonary support. I have a bigger Ceph storage with many disks (and types) and a lot of switches. For example on some I have SSDs, HDDs, Raid and virtual disks which which can be present as /dev/sda ... I have no idea, how can I solve it without the dictionary type.
Is there workaround for that problem, without moving the whole node to icinga2 conf.d/ again?
2 similar comments
Had related discussions multiple times, trying to sum it up, just for the records:
Said all this, sooner or later we might implement this. It has low priority, as we see no gain in it. It will have the form of custom nested data types, allow to combine different data fields with related rules like explained in this and other related issues. We are currently preparing the next generation of our base web form libraries. Ones we have them, weird nested form elements will become easier to implement. We have no plans to allow for free-form Dictionaries, and in our believes not doing so is the right way.
As always, there might be different opinions and use-cases we didn't think about. I'd love to learn more about them!
Nexted vars helps alot when you work with other Systems that provide Data for the Monitoring.
To manage our systems we use Jira. We have workflows in Jira for order a host. Also this workflow "lives" as long the Server is in use.
Cause we have all the data of the Servers in Jira we created the subtask Monitoring. In this we have a field where our stuff can write JSON like:
URL Checks in the notation:
"URL_" : "",""
We are applying some rules inside the Director DB to make a loop over all this entry which creates a check like this:
The problem is that it will publish all check relevant Data to the public (all users that are able to see the service).
But now we also want SNMP Checks to be applyed that way.
I dont see how can i accomplish this without nested Vars. But maybe i am doing it wrong as you mentioned
Lukas Matecki from Cologne
p.s. I still own you some Gin for the next OSMC. Have found some special here in Cologne. Will bring it with me for this year osmc ;-)
I don't see any way to deploy Service Apply Rules with individual parameters without nested vars or dictionaries.
There are some many usecases that can't be deployed without dictionaries.
Even if you use dictionaries, you have to have some way of determining what partitions need to be monitored on each system. If you know this ahead of time, then you can just apply one service for each partition to a host template and you're done. If it needs to be dynamic, as in my case, you need to do something else to determine what partitions each system has.
ok. I understand. what the director cannot do your third-party tool can.
You can still use dictionary variables so long as you define the service and apply rules in Icinga2, outside of Director. It just means that you won't be able to manage that particular service and associated thresholds in Director which is not ideal.