Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decrease amount of node saves #80

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

vuntz
Copy link
Member

@vuntz vuntz commented Dec 19, 2016

Saving is not cheap, and most of the time, it's not needed (as no attribute changed, or because we will already save later).

(Here's it's not critical as the saves only occur on initial setup)

dirty = true
end
# if journal_device is nil, this will still work as expected
if node["ceph"]["osd_devices"][index]["journal"] != journal_device

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metrics/BlockNesting: Avoid more than 3 levels of block nesting. (https://github.com/bbatsov/ruby-style-guide#three-is-the-number-thou-shalt-count)

@@ -198,8 +208,15 @@
end
end
end
node.set["ceph"]["osd_devices"][index]["status"] = "deployed"
node.set["ceph"]["osd_devices"][index]["journal"] = journal_device unless journal_device.nil?
if node["ceph"]["osd_devices"][index]["status"] != "deployed"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metrics/BlockNesting: Avoid more than 3 levels of block nesting. (https://github.com/bbatsov/ruby-style-guide#three-is-the-number-thou-shalt-count)

@@ -88,7 +94,11 @@
has_ssds = unclaimed_disks.any? { |d| node[:block_device][d.name.gsub("/dev/", "")]["rotational"] == "0" }
has_hdds = unclaimed_disks.any? { |d| node[:block_device][d.name.gsub("/dev/", "")]["rotational"] == "1" }

node.set["ceph"]["osd"]["use_ssd_for_journal"] = false unless has_ssds && has_hdds
use_ssd_for_journal = has_ssds && has_hdds
if node["ceph"]["osd"]["use_ssd_for_journal"] != use_ssd_for_journal

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metrics/BlockNesting: Avoid more than 3 levels of block nesting. (https://github.com/bbatsov/ruby-style-guide#three-is-the-number-thou-shalt-count)

min_size_blocks = node["ceph"]["osd"]["min_size_gb"] * 1024 * 1024 * 2
unclaimed_disks = BarclampLibrary::Barclamp::Inventory::Disk.unclaimed(node).sort.select { |d| d.size >= min_size_blocks }

# if devices for journal are explicitely listed, do not use automatic journal assigning to SSD
if !node["ceph"]["osd"]["journal_devices"].empty?
node.set["ceph"]["osd"]["use_ssd_for_journal"] = false
# explicit comparison because we don't want a condition that uses nil
if node["ceph"]["osd"]["use_ssd_for_journal"] != false

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metrics/BlockNesting: Avoid more than 3 levels of block nesting. (https://github.com/bbatsov/ruby-style-guide#three-is-the-number-thou-shalt-count)

@@ -69,13 +69,19 @@
end

if is_crowbar?
node.set["ceph"]["osd_devices"] = [] if node["ceph"]["osd_devices"].nil?
dirty = false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would call it changed ;)

Copy link

@matelakat matelakat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vuntz I see a pattern evolving here (and looking at the other changes as well) Can we make some sort of accessor/proxy/decorator that wraps the node, set attributes on the decorator, and it knows only to save the decorated object if any value changed?

@Itxaka
Copy link
Member

Itxaka commented Dec 20, 2016

Agreed with @matelakat here, I was thinking the same thing. Maybe cache the current attributes on the Nodeobject and override the save node to check if the current attributes have changed compared to the cached ones and call the parent save in that case?

Not a ruby expert so this may not be possible or may incur a performance hit to have a cache of attributes?

@vuntz
Copy link
Member Author

vuntz commented Dec 20, 2016

@matelakat @Itxaka let me think about it over lunch and see if I can come with something.

@vuntz vuntz added the wip label Dec 20, 2016
@vuntz
Copy link
Member Author

vuntz commented Dec 20, 2016

So I'm torn. I can find ways to do something like suggested, but nothing that will end up looking like:

node.set[:a][:b][:c] = foobar
node.save

The most simple approach (from the caller perspective) would be something like:

node.set_value(:a, :b, :c, foobar)
node.save_if_changed

But I kind of dislike the fact that we move away from what looks like standard chef code.

I don't see a good way to simply override set in a compatible way (and I know we don't want to override save as sometimes we want to really just save as usual, so we may as well get a new method). Of course, I'm not really the best ruby experts out there :-)

Anyone having feelings on this?

@aspiers
Copy link
Member

aspiers commented Jan 4, 2017

It's evil, but it might be possible to monkey-patch Chef::Node#set (which is an alias for #normal) so that it always checks to see whether the value being set is different to what it was before, and if so, sets a dirty attribute on the node object. Then it would be trivial to implement Chef::Node#save_if_dirty via monkey-patching.

@aspiers
Copy link
Member

aspiers commented Jan 4, 2017

And actually, if we did it right, probably we could just monkey-patch #save too, since presumably there's no good reason to save a pre-existing node if there are no changes.

@dirkmueller dirkmueller added this to the Cloud 7 Update2 milestone Jan 18, 2017
scottwulf added a commit to scottwulf/crowbar-ceph that referenced this pull request Aug 2, 2017
Rebase of PR crowbar#80
Only save the node when attribute(s) have changed
Do not fetch monitor secret if we're the master
scottwulf added a commit to scottwulf/crowbar-ceph that referenced this pull request Aug 2, 2017
Rebase of PR crowbar#80
Only save the node when attribute(s) have changed
Do not fetch monitor secret if we're the master
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8 participants