Get kubernetes versions for all cloud providers

[iameskild]

Closes #854

[costrouc]

Could you add some information in https://docs.qhub.dev/en/stable/source/installation/configuration.html?highlight=kuberentes%20version#providers about how the Kubernetes versions for all providers update frequently and if nothing is explicitly specified qhub init will get the latest Kubernetes version.

And remove this text from the docs in favor of the more general message.

Note: DigitalOcean regularly updates Kubernetes versions hence, the field kubernetes_version will most likely have to be changed. See available instance types for DigitalOcean. If you used qhub init this version will automatically be compute for you Do not copy the version you see bellow

[costrouc]

Otherwise this PR looks good to me 😄

[iameskild]

Thanks for taking a look @costrouc, I'll make sure to update the docs and fix these tests. I was hoping to add a Kubernetes check for GCP as well.

[costrouc]

I was hoping to add a Kubernetes check for GCP as well.

Absolutely. If you feel it would be best to include that work in this PR go for it.

Like I mentioned in the meeting long term goal wold be that for QHub we are able to get (for each provider):

• list all regions that are available
• list all available Kubernetes versions within region
• list all available instance types within region
• get the memory/cpu for a given instance type

By no means something that needs to be done in a single PR.

[iameskild]

I apologize, the scope of this PR grew larger than I intended...
Besides adding/updating kubernetes_versions functions for all the cloud providers, I have made the following motivated changes:

• Standardized how all of the kubernetes_versions functions behave. For all cloud providers, they return a list of available kubernetes versions from oldest to latest.
• If the user doesn't specify a kubernetes version during qhub init, the latest available version will be used.
• If the user tries to specify an "unsupported" version, it will cause a ValueError
  • Now each render_config will check the kubernetes version, it causes the "Test QHub Provider" tests to fail since we do not have the appropriate credentials
    • My proposed remedy is to include service account credentials as GH secrets but also include a pytest fixture that will remove them when running unit tests.
• Updates to test suite include a setup_fixture which can be reused for any test that calls render_config, reducing the amount of duplicated code.
• Updates to the docs based on the Documentation linter - it included many docs which I did not touch...

[costrouc]

Thanks @iameskild this is a great contribution and glad the scope grew! This will make us handle all cloud providers more consistently. And no problem with the linting issues on documentation. It is going to throw a lot of errors for awhile.

[iameskild]

@costrouc This PR is ready for another look whenever you get a chance. The biggest change is that Test QHub Provider workflow now requires cloud credentials/secrets to check the kubernetes versions for each cloud provider; service accounts or equivalent given read-only access where possible.
The Test QHub Provider is now it's own workflow and the reason for the failures is because of a GH Actions precaution (I presume), noted here. When testing in my fork, I was able to get these tests to pass. I'm still unsure why the Kubernetes Test are failing , but I can take a closer look.

[iameskild]

Many of the CI workflows are failing because this PR is coming from my fork and thus it doesn't have access to the Secrets. I will reopen this PR and make sure all the tests pass before requesting another review 👍

[iameskild]

Replaced by #927