Provide a mechanism to run only one tezos-node pod per k8s node.

[elric1]

We also turn this functionality on by default in pulumi if we are configured to have 1 node per vm.

[harryttd]

I don't think we should be customizing such a specific requested feature like this directly in the nodes.yaml. We should templatize the affinity spec of the statefulsets however. A user should be able to specify any affinity spec they want for every node class. In oxheadinfra for example, we are going to want to provide different affinities for different sts's.

I think we should instead simply templatize the affinity spec so that a user can customize it however they want for each sts they spin up. In addition, we write a helper function in _helpers.yaml that contains the new added affinity yaml here. So that a user can still set the flag. But the main node.yaml file should not begin to accumulate many different specific user requested features.

And then obviously user provided affinities should be merged with globally set ones.

[orcutt989]

What would be the use case for that? Isn't that sort of undoing kubernetes?

[elric1]

Another thing that we could do is provide a way of setting the other resources beyond storage like CPU or RAM. That way if you want 1 tezos node per EC2 VM, say, you could request exclusive access to enough CPUs to make that a reality based on the instance type that you have selected.

[nicolasochem]

Closing as this is probably not the right way to do it (per our weekly discussion).