Skip to content

Update monasca-agent cookbook and barclamp#2

Merged
artur-ba merged 64 commits intoFujitsuEnablingSoftwareTechnologyGmbH:monascafrom
matrixik:monasca
Mar 9, 2017
Merged

Update monasca-agent cookbook and barclamp#2
artur-ba merged 64 commits intoFujitsuEnablingSoftwareTechnologyGmbH:monascafrom
matrixik:monasca

Conversation

@matrixik
Copy link
Member

@matrixik matrixik commented Jan 9, 2017

No description provided.

end

node.save

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style/TrailingBlankLines: 1 trailing blank lines detected. (https://github.com/bbatsov/ruby-style-guide#newline-eof)

ha_enabled = node[:monasca][:ha][:enabled]

my_admin_host = CrowbarHelper.get_host_for_admin_url(node, ha_enabled)
my_public_host = CrowbarHelper.get_host_for_public_url(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint/UselessAssignment: Useless assignment to variable - my_public_host. (https://github.com/bbatsov/ruby-style-guide#underscore-unused-vars)

monasca_protocol = ssl_enabled ? "https" : "http"
ha_enabled = node[:monasca][:ha][:enabled]

my_admin_host = CrowbarHelper.get_host_for_admin_url(node, ha_enabled)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint/UselessAssignment: Useless assignment to variable - my_admin_host. (https://github.com/bbatsov/ruby-style-guide#underscore-unused-vars)


monasca_port = node[:monasca][:api][:bind_port]
ssl_enabled = node[:monasca][:api][:ssl]
monasca_protocol = ssl_enabled ? "https" : "http"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint/UselessAssignment: Useless assignment to variable - monasca_protocol. Did you mean monasca_port? (https://github.com/bbatsov/ruby-style-guide#underscore-unused-vars)


keystone_settings = KeystoneHelper.keystone_settings(node, @cookbook_name)

monasca_port = node[:monasca][:api][:bind_port]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint/UselessAssignment: Useless assignment to variable - monasca_port. Did you mean monasca_protocol? (https://github.com/bbatsov/ruby-style-guide#underscore-unused-vars)

@matrixik matrixik changed the title Add monasca-reconfigure and sudoers file Update monasca-agent cookbook and barclamp Jan 16, 2017
action :add_service
end

keystone_register "register monasca-agent endpoint" do

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that needs to go api.rb, you install monasca-api you register an endpoint for it.


# agent_nodes.each do |agent_node|
# base["attributes"][@bc_name]["keystone_agents_credentials"]["node_name"] = "#{agent_node.name}"
# base["attributes"][@bc_name]["keystone_agents_credentials"]["username"] = "mon-agent-#{agent_node.name}"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metrics/LineLength: Line is too long. [112/100] (https://github.com/SUSE/style-guides/blob/master/Ruby.md#metricslinelength)


# agent_nodes.each do |agent_node|
# base["attributes"][@bc_name]["keystone_agents_credentials"]["node_name"] = "#{agent_node.name}"
# base["attributes"][@bc_name]["keystone_agents_credentials"]["username"] = "mon-agent-#{agent_node.name}"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metrics/LineLength: Line is too long. [112/100] (https://github.com/SUSE/style-guides/blob/master/Ruby.md#metricslinelength)

base["attributes"][@bc_name][:db][:password] = random_password

# agent_nodes.each do |agent_node|
# base["attributes"][@bc_name]["keystone_agents_credentials"]["node_name"] = "#{agent_node.name}"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metrics/LineLength: Line is too long. [103/100] (https://github.com/SUSE/style-guides/blob/master/Ruby.md#metricslinelength)

base["attributes"][@bc_name][:db][:password] = random_password

# agent_nodes.each do |agent_node|
# base["attributes"][@bc_name]["keystone_agents_credentials"]["node_name"] = "#{agent_node.name}"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metrics/LineLength: Line is too long. [103/100] (https://github.com/SUSE/style-guides/blob/master/Ruby.md#metricslinelength)

mode 0o640
variables agent_settings
variables (
agent_settings: agent_settings,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unexpected token tLABEL

mode 0o640
variables agent_settings
variables (
agent_settings: agent_settings,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unexpected token tLABEL


agent_settings[:monasca_url] = "#{monasca_protocol}://#{monasca_public_host}/v2.0"

agent_settings[:dimensions] = { "service": "monitoring" }

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unexpected token tCOLON

monasca_protocol = ssl_enabled ? "https" : "http"
ha_enabled = monasca_server[:monasca][:ha][:enabled]

monasca_public_host = CrowbarHelper.get_host_for_public_url(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint/UselessAssignment: Useless assignment to variable - monasca_public_host. Did you mean monasca_protocol? (https://github.com/bbatsov/ruby-style-guide#underscore-unused-vars)

@kornicameister kornicameister force-pushed the monasca branch 2 times, most recently from d78d8db to 9208ba5 Compare January 25, 2017 06:32
answer
end

def create_proposal

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metrics/AbcSize: Assignment Branch Condition size for create_proposal is too high. [31.08/30] (https://github.com/SUSE/style-guides/blob/master/Ruby.md#metricsabcsize, http://c2.com/cgi/wiki?AbcMetric)

"suse" => "< 12.1",
"windows" => "/.*/"
}
},

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style/TrailingCommaInLiteral: Avoid comma after the last item of a hash. (https://github.com/bbatsov/ruby-style-guide#no-trailing-array-commas)

@@ -0,0 +1,5 @@
name "monasca-server"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style/FileName: The name of this source file (monasca-server.rb) should use snake_case. (https://github.com/bbatsov/ruby-style-guide#snake-case-files)

@@ -0,0 +1,5 @@
name "monasca-agent"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style/FileName: The name of this source file (monasca-agent.rb) should use snake_case. (https://github.com/bbatsov/ruby-style-guide#snake-case-files)


if CrowbarRoleRecipe.node_state_valid_for_role?(node, "monasca", "monasca-server")
include_recipe "monasca::server"
# include_recipe "monasca::api"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style/CommentIndentation: Incorrect indentation detected (column 0 instead of 2).

mode 0o0640
variables(
database_connection: database_connection,
keystone_settings: KeystoneHelper.keystone_settings(node, @cookbook_name),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style/TrailingCommaInArguments: Avoid comma after the last parameter of a method call. (https://github.com/bbatsov/ruby-style-guide#no-trailing-params-comma)

db_settings = fetch_database_settings
db_conn_scheme = db_settings[:url_scheme]

if db_settings[:backend_name] == "mysql"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style/IfUnlessModifier: Favor modifier if usage when having a single-line body. Another good alternative is the usage of control flow &&/||. (https://github.com/bbatsov/ruby-style-guide#if-as-a-modifier)

agent_settings = node[:monasca][:agent][:config]
keystone_settings = KeystoneHelper.keystone_settings(node, @cookbook_name)

monasca_port = node[:monasca][:api][:bind_port]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint/UselessAssignment: Useless assignment to variable - monasca_port. Did you mean monasca_protocol? (https://github.com/bbatsov/ruby-style-guide#underscore-unused-vars)

@matrixik
Copy link
Member Author

matrixik commented Jan 31, 2017

I have problems with the GUI part:

======== Migrating barbican, ceilometer, ceph, cinder, cisco_ucs, crowbar, database, deployer, dns, glance, heat, horizon, hyperv, ipmi, keystone, logging, magnum, manila, network, neutron, nfs_client, nova, ntp, openstack, pacemaker, provisioner, rabbitmq, sahara, suse_manager_client, swift, tempest, trove, updater barclamps -- Tue Jan 31 09:32:42 2017 ========
WARNING: Nokogiri was built against LibXML version 2.9.1, but has dynamically loaded 2.9.4
active
rake aborted!
can not load translations from /opt/dell/crowbar_framework/config/locales/monasca/en.yml: #<Psych::SyntaxError: (/opt/dell/crowbar_framework/config/locales/monasca/en.yml): did not find expected key while parsing a block mapping at line 31 column 11>
/opt/dell/crowbar_framework/app/models/proposal.rb:13:in `<class:Proposal>'
/opt/dell/crowbar_framework/app/models/proposal.rb:1:in `<top (required)>'
/opt/dell/crowbar_framework/lib/schema_migration.rb:104:in `get_template_for_barclamp'
/opt/dell/crowbar_framework/lib/schema_migration.rb:34:in `run_for_bc'
/opt/dell/crowbar_framework/lib/tasks/crowbar.rake:30:in `block (3 levels) in <top (required)>'
/opt/dell/crowbar_framework/lib/tasks/crowbar.rake:29:in `each'
/opt/dell/crowbar_framework/lib/tasks/crowbar.rake:29:in `block (2 levels) in <top (required)>'
Tasks: TOP => crowbar:schema_migrate

Edit:
Fixed it, I had to many ' in one line

Copy link

@kornicameister kornicameister left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Part that I don't like is the one where you try to copy custom templates. That has little chance of working reliably.

And rename all variables to reflect this change.
@@ -0,0 +1,5 @@
name "monasca-metric-agent"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style/FileName: The name of this source file (monasca-metric-agent.rb) should use snake_case. (https://github.com/bbatsov/ruby-style-guide#snake-case-files)

@@ -1,4 +1,4 @@
# # Copyright 2017 FUJITSU LIMITED
# Copyright 2017 FUJITSU LIMITED

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style/FileName: The name of this source file (monasca-log-agent.rb) should use snake_case. (https://github.com/bbatsov/ruby-style-guide#snake-case-files)

monasca_url = "#{monasca_protocol}://#{monasca_public_host}/v3"
elasticsearch_url = "http://localhost:9200/"
kibana_url = "http://#{monasca_public_host}:5601"
log_api_url = "http://#{monasca_public_host}:#{node[:monasca][:log_api][:bind_port]}"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint/UselessAssignment: Useless assignment to variable - log_api_url. (https://github.com/bbatsov/ruby-style-guide#underscore-unused-vars)

service = "monitoring"
monasca_url = "#{monasca_protocol}://#{monasca_public_host}/v3"
elasticsearch_url = "http://localhost:9200/"
kibana_url = "http://#{monasca_public_host}:5601"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint/UselessAssignment: Useless assignment to variable - kibana_url. (https://github.com/bbatsov/ruby-style-guide#underscore-unused-vars)

agent_dimensions = { service: "monitoring" }
service = "monitoring"
monasca_url = "#{monasca_protocol}://#{monasca_public_host}/v3"
elasticsearch_url = "http://localhost:9200/"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint/UselessAssignment: Useless assignment to variable - elasticsearch_url. (https://github.com/bbatsov/ruby-style-guide#underscore-unused-vars)

packages: [],
services: [monasca_log_agent_service_name]
services: [
monasca_agent_service_name,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the variable name is monasca_metric_agent_service_name


agent_dimensions = { service: "monitoring" }
service = "monitoring"
monasca_url = "#{monasca_protocol}://#{monasca_public_host}/v3"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about port?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

service = "monitoring"
monasca_url = "#{monasca_protocol}://#{monasca_public_host}/v3"

template "/etc/sudoers.d/monasca-agent_sudoers" do

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IHMO the _sudoers suffix is redundant, I'd remove it

@@ -0,0 +1,32 @@
#
# Copyright 2011-2013, Dell

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it should be Copyright 2015 FUJITSU LIMITED?

service = "monitoring"
monasca_url = "#{monasca_protocol}://#{monasca_public_host}/v3"

template "/etc/sudoers.d/monasca-agent" do

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This belongs into packaging not in the barclamp

Copy link
Member Author

@matrixik matrixik Mar 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I did not found any sudo in rpm-packaging however I found it in crowbar-openstack...:
https://github.com/crowbar/crowbar-openstack/blob/master/chef/cookbooks/nova/recipes/monitor.rb#L50-L55
But we will move it if you want.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added sudoers configuration here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code for sudoers removed from barclamp

# metric-agent default service settings
default[:monasca][:metric_agent][:service_name] = monasca_metric_agent_service_name
default[:monasca][:metric_agent][:user] = "monasca-agent"
default[:monasca][:metric_agent][:group] = "monasca-agent"
Copy link

@artur-ba artur-ba Mar 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Group should be monasca. As we discussed with other guys from community about rpm packaging.

@matrixik
Copy link
Member Author

matrixik commented Mar 7, 2017

Added SLES to supported dists to monasca-agent:
https://review.openstack.org/#/c/442381/

Otherwise barclamp is erroring with:

[2017-03-07T11:30:16+01:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: execute[monasca-setup detect services] (monasca::metric_agent line 56) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /usr/local/bin/monasca-reconfigure ----
STDOUT:
STDERR: WARNING: SUSE Linux Enterprise Server is not a support Linux distribution
Traceback (most recent call last):
File "/usr/bin/monasca-setup", line 10, in
sys.exit(main())
File "/usr/lib/python2.7/site-packages/monasca_setup/main.py", line 50, in main
agent_service.enable()
File "/usr/lib/python2.7/site-packages/monasca_setup/service/linux.py", line 62, in enable
LinuxInit.enable(self)
File "/usr/lib/python2.7/site-packages/monasca_setup/service/linux.py", line 37, in enable
os.chown(self.log_dir, user.pw_uid, user.pw_gid)
OSError: [Errno 1] Operation not permitted: '/var/log/monasca/agent'
---- End output of /usr/local/bin/monasca-reconfigure ----
Ran /usr/local/bin/monasca-reconfigure returned 1 

@artur-ba artur-ba merged commit 126cdaa into FujitsuEnablingSoftwareTechnologyGmbH:monasca Mar 9, 2017
@matrixik matrixik deleted the monasca branch March 9, 2017 10:09
samirjorina pushed a commit that referenced this pull request Mar 28, 2017
* Add monasca-reconfigure and sudoers file

* Fix monasca-setup bin path

* Fix template variables

* Keystone settings for agent

* Keystone settings for agent

* Keystone settings for agent

* Add log

* More readable config templates

* Remove log

* Remove per-node credentials todos

* Fix json indentations

* Rename agent dimensions variable name

* Run monasca-reconfigure every time with chef client

* Add firts GUI elements

* Add GUI elements

* Update for new code arrangement for keystone

* Fix yaml for translations

* Fix monasca helper module name

* Use root user for agent temporally

* Update monasca agent GUI parts

* WIP: Add custom checks for monasca-agent

* Extract variables for templates

* Fix testing filepath

* Fix template for monasca agent checks

* Fix template paths again

* Address Hound warning

* Add more monasca agent checkers

* Fix some variables

* Add log api port to data_bag

* Build Kibana and log-api urls

* Add more info to GUI

* Add user_managed: false

* Fix yaml indentation

* Add some logs

* Fix monasca-reconfigure

* Fix ruby boolean name

* Uncomment agent.yaml template code

* Fix providing service variable to template

* Fix insecure args for monasca-setup

* Fix monasca-reconfigure

* Configure more variables with monasca-reconfigure

* Add more config to UI

* Disable generating agent.yaml

It's regenerated every time chef client is run, it's running
monasca-reconfigure.

* Change default overwrite config to false

* Add monasca-agent to sudoers

* Run monasca-setup as monasca-agent user

* Remove logs from monasca-reconfigure

* Rename monasca-agent to monasca-metric-agent

And rename all variables to reflect this change.

* Fix hound warnings

* Address comments

* Fix copyright

* Use same order in data_bag schema like in json

* Restore user mapping in log_agent schema

* Change monasca-master unique to false

* Fix monasca-metric-agents names in json

* Use keystone settings from data_bag for metric agent

* Replace `owner` with `user` in execute command

* Replace group `monasca-agent` with `monasca`

* Remove adding monasca-agent to sudoers

Will be added from rpm.

* Add port to monasca_api_url for monasca-agent

* Use helper for getting api url

* Fix bad default config name
samirjorina pushed a commit to samirjorina/crowbar-openstack that referenced this pull request Mar 30, 2017
…chnologyGmbH#2)

* Add monasca-reconfigure and sudoers file

* Fix monasca-setup bin path

* Fix template variables

* Keystone settings for agent

* Keystone settings for agent

* Keystone settings for agent

* Add log

* More readable config templates

* Remove log

* Remove per-node credentials todos

* Fix json indentations

* Rename agent dimensions variable name

* Run monasca-reconfigure every time with chef client

* Add firts GUI elements

* Add GUI elements

* Update for new code arrangement for keystone

* Fix yaml for translations

* Fix monasca helper module name

* Use root user for agent temporally

* Update monasca agent GUI parts

* WIP: Add custom checks for monasca-agent

* Extract variables for templates

* Fix testing filepath

* Fix template for monasca agent checks

* Fix template paths again

* Address Hound warning

* Add more monasca agent checkers

* Fix some variables

* Add log api port to data_bag

* Build Kibana and log-api urls

* Add more info to GUI

* Add user_managed: false

* Fix yaml indentation

* Add some logs

* Fix monasca-reconfigure

* Fix ruby boolean name

* Uncomment agent.yaml template code

* Fix providing service variable to template

* Fix insecure args for monasca-setup

* Fix monasca-reconfigure

* Configure more variables with monasca-reconfigure

* Add more config to UI

* Disable generating agent.yaml

It's regenerated every time chef client is run, it's running
monasca-reconfigure.

* Change default overwrite config to false

* Add monasca-agent to sudoers

* Run monasca-setup as monasca-agent user

* Remove logs from monasca-reconfigure

* Rename monasca-agent to monasca-metric-agent

And rename all variables to reflect this change.

* Fix hound warnings

* Address comments

* Fix copyright

* Use same order in data_bag schema like in json

* Restore user mapping in log_agent schema

* Change monasca-master unique to false

* Fix monasca-metric-agents names in json

* Use keystone settings from data_bag for metric agent

* Replace `owner` with `user` in execute command

* Replace group `monasca-agent` with `monasca`

* Remove adding monasca-agent to sudoers

Will be added from rpm.

* Add port to monasca_api_url for monasca-agent

* Use helper for getting api url

* Fix bad default config name
samirjorina pushed a commit that referenced this pull request Mar 30, 2017
* Add monasca-reconfigure and sudoers file

* Fix monasca-setup bin path

* Fix template variables

* Keystone settings for agent

* Keystone settings for agent

* Keystone settings for agent

* Add log

* More readable config templates

* Remove log

* Remove per-node credentials todos

* Fix json indentations

* Rename agent dimensions variable name

* Run monasca-reconfigure every time with chef client

* Add firts GUI elements

* Add GUI elements

* Update for new code arrangement for keystone

* Fix yaml for translations

* Fix monasca helper module name

* Use root user for agent temporally

* Update monasca agent GUI parts

* WIP: Add custom checks for monasca-agent

* Extract variables for templates

* Fix testing filepath

* Fix template for monasca agent checks

* Fix template paths again

* Address Hound warning

* Add more monasca agent checkers

* Fix some variables

* Add log api port to data_bag

* Build Kibana and log-api urls

* Add more info to GUI

* Add user_managed: false

* Fix yaml indentation

* Add some logs

* Fix monasca-reconfigure

* Fix ruby boolean name

* Uncomment agent.yaml template code

* Fix providing service variable to template

* Fix insecure args for monasca-setup

* Fix monasca-reconfigure

* Configure more variables with monasca-reconfigure

* Add more config to UI

* Disable generating agent.yaml

It's regenerated every time chef client is run, it's running
monasca-reconfigure.

* Change default overwrite config to false

* Add monasca-agent to sudoers

* Run monasca-setup as monasca-agent user

* Remove logs from monasca-reconfigure

* Rename monasca-agent to monasca-metric-agent

And rename all variables to reflect this change.

* Fix hound warnings

* Address comments

* Fix copyright

* Use same order in data_bag schema like in json

* Restore user mapping in log_agent schema

* Change monasca-master unique to false

* Fix monasca-metric-agents names in json

* Use keystone settings from data_bag for metric agent

* Replace `owner` with `user` in execute command

* Replace group `monasca-agent` with `monasca`

* Remove adding monasca-agent to sudoers

Will be added from rpm.

* Add port to monasca_api_url for monasca-agent

* Use helper for getting api url

* Fix bad default config name
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants