Make it possible to assign externally managed Arakoon clusters to dedicated Backend

[kvanhijf]

From @kvanhijf on August 3, 2017 12:33

Our NSM checkup currently claims any Arakoon cluster which has the 'in_use' flag set to False when a Backend needs more NSM clusters.
This behavior is not optimal and should be improved

Possible solutions:

1. We could add a flag in the Arakoon cluster's metadata called: 'backend_name'. This flag could then be used to check for which backend the cluster was created. Our NSM checkup should then check on the name flag and act accordingly.
   This could prevent that certain unused arakoon clusters, which have been deployed on 5 nodes, are registered to an ABM cluster only deployed on 3 nodes.
2. Do nothing at all when NSM checkup notices all NSM clusters being overloaded, just log a warning for monitoring to react upon.

Downsides for 1st option:

• What to do when no 'name' flag is provided? Always claim or log warning?
• What to do when unknown backend name is provided?

Downsides for 2nd option:

• We need for every NSM cluster a service in our model. This would have to become a scheduled task to keep these services in sync.

Copied from original issue: openvstorage/framework#1711

[kvanhijf]

From @JeffreyDevloo on August 3, 2017 12:34

Shouldn't we just start moving away from internal and external arakoons and just leave it up to customer where he wants his arakoons?

[kvanhijf]

Yes we definitely should, but that would be a whole lot more work
This is an enhancement which shouldn't take too much time and allows our OPS boys to have more control

PS: created a ticket (FR) for it: see openvstorage/framework#1712

[kvanhijf]

From @wimpers on August 24, 2017 9:23

• Implement option 2.
• extend Allow to add nsm arakoon to a backend (api) #284 so it also registers the services in the model.

[kvanhijf]

openvstorage/framework#1797 --> openvstorage-2.11.0-dev.1506520895.af0293a
#491 --> openvstorage-backend-1.11.0-dev.1506522259.f180102

[kvanhijf]

Solution implemented is the following:
Whenever externally managed arakoon NSM clusters are overloaded, a CRITICAL log message is logged. Operations then needs to react upon this to add more arakoon clusters, by executing the 'expand_nsm_clusters' API on 'alba/backends'
For internally managed Arakoon clusters, nothing's changed. Whenever overloaded clusters are detected there, we automatically add new NSMs