clean up default spawner_ui_config.yaml

[thesuperzapper]

This PR significantly cleans up (but does not change) the default spawner_ui_config.yaml to make it much easier to understand and customize.

---

The only change to default values is that I have removed the default gpus.value.vendors list, and left the previous values as a commented example, as most clusters don't have GPUs, so there is no reason to include them by default.

I have also updated the examples for affinityConfig and tolerationGroup to demonstrate how a user can cause Notebook to be scheduled on exclusive Nodes (no other Notebooks), using the new namespaceSelector for pod-anti-affinity added in K8S 1.22.

[thesuperzapper]

I think this is a big quality-of-life improvement so would appreciate a review!

/assign @kimwnasptd

[thesuperzapper]

@kimwnasptd can you take a quick look at this, its a formatting change but makes the config much easier to use.

[thesuperzapper]

@kimwnasptd can we try and get this merged soon, as I think its a significant improvement in readability for the spawner_ui_config.yaml?

[thesuperzapper]

@orfeas-k @tasos-ale @elenzio9, as you are quite active now in the web apps, you also might want to review.

[kimwnasptd]

@thesuperzapper I'm not really fond of putting all this documentation in the ConfigMap itself. IMO this makes it more difficult to read both the docs bits (i.e. no table of contents) and the YAML itself.

I'd propose to have a separation of concerns. A different place to document the API and a different place for configuring it (yaml). We've also seen multiple requests for more documentation and this can be a good start #6914

So I'd propose to instead:

1. Create a PR in the docs and fully expose there the different parts of this ConfigMap and provide examples
2. Keep these YAMLs as minimal as possible

[thesuperzapper]

@kimwnasptd I agree that we can provide improved API docs, especially on the notebook section of the website (and am happy to help with this) but I think the changes proposed in this PR are still worth having.

The spawner_ui_config.yaml file, already has a lot of docstrings & examples, but they lack clarity on what each section actually does, and the file is ordered in a very hard-to-understand way.

This PR simply makes the existing docstrings/examples more clear and re-orders the file.

If you think this PR adds too much detail, can you please comment on the lines which you think are not needed?

[thesuperzapper]

@kimwnasptd I really think we should try and get this cleanup merged as it will make the spawner_ui_config.yaml better.

Do you think you will have time this week to make a GitHub review so we can get this merged?

You mentioned that you thought some of the docstrings were a bit long, can you point them out.

[StefanoFioravanzo]

@thesuperzapper I tend to agree with this

1. Create a PR in the docs and fully expose there the different parts of this ConfigMap and provide examples
2. Keep these YAMLs as minimal as possible

I suggest adding a new page under the API Reference. In general, it's dangerous to over-document source files, as it results in not exposing that information in the official docs. Anyway, since you've done a lot of great work here, a possible path forward could be:

1. Merge this PR
2. Commit to moving most of the comments to the docs

[thesuperzapper]

I suggest adding a new page under the API Reference. In general, it's dangerous to over-document source files, as it results in not exposing that information in the official docs. Anyway, since you've done a lot of great work here, a possible path forward could be:

1. Merge this PR
2. Commit to moving most of the comments to the docs

@StefanoFioravanzo I am happy with that plan, if you are too, can you give it an /approve so we can move it forward?

Note, this won't make it to Kubeflow 1.7 either way, so it won't affect anyone until Kubeflow 1.8.

Once we have merged, I will look at updating the kubeflow.org docs with similar notes as I have added in this PR, probably improving on what the person did in kubeflow/website#3414

[StefanoFioravanzo]

@thesuperzapper I don't know the details of how the spawner config works. I'd love for @kimwnasptd to approve the technical review as well

/lgtm

[thesuperzapper]

@StefanoFioravanzo do you mind giving this an /approve as well? If @kimwnasptd doesn'tt have time to review now.

[StefanoFioravanzo]

Sure @thesuperzapper , sorry for the delay, I caught up with my GitHub emails just now. Sure let's go ahead and merge this. Can we also start a discussion to bring this to docs? If you ping me in a PR or issue I am happy to help you out or review

/approve

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: StefanoFioravanzo, Talador12

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• components/crud-web-apps/OWNERS [StefanoFioravanzo]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment