Facts not being reported by the Microkernel to Hanlon on some hardware #29
Comments
|
I think I have a related issue. Received a number of Super Micro machines and only getting partial tags/facts being reported. Note the lack of some of the microkernel facts not defined so the system tags do not appear. Specifically mk_hw_cpu_count, mk_hw_mem_size and mk_hw_nic_count. Probably related is the exception that is occurring in hnl_mk_hardware_facter.rb. |
|
@hickey I created this doc a while back, maybe if we increase the logging we can figure out where the problem is. |
|
Yes, I was starting to look at that this morning.... I am also starting to cut a new microkernel image that print out values throughout the routine to determine what things look like as the routine is executing. Not sure how the debug level gets transferred to the microkernel (figure it has to be statically written into the docker image when image add executes), but the lack of controls when starting up the docker containers is getting frustrating.... I have already started to extend the hanlon_docker.sh script to be more 12 factorish. I guess I have another setting to add. |
|
@hickey see https://github.com/csc/Hanlon-Microkernel/blob/master/hnl_mk_web_server.rb#L57 Most of the configurations of Hanlon don't need to be changed which is the reason why we did not add those options when starting container. Its easy enough to enter the container modify the config temporarily for testing/debugging and restart puma. |
|
I have grabbed a copy of the log for analysis. Here is the gist: https://gist.github.com/hickey/6207183c78ea0903cea1 |
|
My guess (from looking at line 79 of the This is the first time I’ve seen this sort of error before…can you run a Cheers, Tom
|
|
Here is the value of hash_map (prettied up to make it readable) just before the exception: |
|
So clearly in my output it is memory_array rather than bank_array. OK that is an easy fix. I walked through the rest of the sections and executed the commands to look at the output. Everything else looks like it will parse correctly. Well everything up to the point of trying to gather BMC/IPMI information. I have already created an issue (#31) for this. It will be fairly critical to gather this information also, so hopefully this will be able to be overcome. |
|
A though I was just having (more of a question) is why was the hardware gathering done this way instead of using the regular Facter interface? If all these gathering processes were written as either Facter modules or even using the facter-dot-d interface, then any one of them blowing up would not disturb the other bits of code gathering information. At lease then in my case only the memory information would be missing. It would also be easier to add new code to test and generate facts. The other advantage is that running facter on the command line would yield all the regular facts along with the the ones being created by the hanlon code. Would not be that much trouble to break it apart and make it Facter modules. Although the more I think about it, the more I like adding it as facts-dot-d scripts. One of the principle reasons being that it would make it pretty easy to interface to an external PAAS system through a hook (yes the microkernel needs to support an easy way for someone to drop scripts in a directory and have them executed prior to and after registration--maybe even at each checkin) to retrieve information and drop a JSON/YAML/text file in the facter-dot-d directory to add PAAS values as facts. This would allow Hanlon tags to be created from exposed PAAS information. |
|
I an report back that the initial changes I have made to support memory_array and bank_array are working. I did not get any real memory information back, so I will look to solve this before I submit a PR. But I am seeing the facts to generate number of CPUs and NICs. So improvements :-) |
|
While I still have the patch for this issue in my local repo, I would suggest that the PR I just created for issue #32 be used instead. That code base also solves some (if not all) of the issues on this thread. |
A user recently reported that when he used the Microkernel to discover HP BL460cG8 blade servers that included with 10Gb Flex Fabric NICs, there were no system tags assigned to the nodes after they registered with the Hanlon. This seems to be associated with facts not being returned to Hanlon from the Microkernel in the node registration process. This issue needs to be explored further to determine the root cause (whether it's a Hanlon issue, a Hanlon-Microkernel issue, or both).
On the plus side, the node seems to be checking in successfully, so this hints at an issue with discovering (and parsing?) the underlying facts from the Hanlon-Microkernel side. This issue is associated with Issue #422 in the Hanlon project
The text was updated successfully, but these errors were encountered: