provide the ability to add imagePullSecrets to different ServiceAccounts so that private images can be fetched

[kkasravi]

ServiceAccounts should have an imagePullSecrets that references a docker-registry name when the image is coming from a different registry or different gcr.io account.

For example for jupyterhub - adding a parameter jupyterHubImageSecret which is then added to the jupyterhub ServiceAccount as

imagePullSecrets: 
- name: gcr-accountfoo

would reference this secret. The secret itself could be added to deploy.sh so that it does something like

kubectl create secret docker-registry --namespace=${KUBEFLOW_NAMESPACE} gcr-accountfoo --docker-server=https://gcr.io --docker-username=_json_key --docker-email=${SA_EMAIL} --docker-password="$(cat ./${SA_EMAIL}.json)"

[kkasravi]

/cc @jlewi

[jlewi]

What would be a good pattern?

Option 1

Expose parameter for imagePullSecrets

Option 2

Users does something like

parts.jupyterhub +  {
      spec: +{
        template: +{
          spec: +{
             imagePullSecrets: [
                  "mySecret"
              ]
          }
     }

Option 3

Admission controller.

Option 1 isn't nearly as flexible as Option 2 or Option 3.

Regardless of whether we do 1, I think it would make sense to refactor our ksonnet to support Option 2 better.

1. parts in libsonnet should contain a map so we can get a particular part by name
2. Our prototype should work by getting all values in the dictionary containing parts

[kkasravi]

@jlewi i like the idea of refactoring the ksonnet libsonnet.
I did earlier refactoring on making them all consistent .all(params) as well as the tests.

[kkasravi]

@jlewi admission controller could be implemented using metacontroller which in turn uses ksonnet.
See article. I think the logic of the controller could be embedded in existing libsonnets.

[kkasravi]

@jlewi I'm currently working on this

[jlewi]

@kkasravi That's great. Is your solution only going to handle ImagePullSecrets? Or will you address the more general case of #56?

/assign @kkasravi

[kkasravi]

i'm providing a parts mapping as you've suggested so i think i could pick up #56 after this.

[jlewi]

@kkasravi How's this going?

[kkasravi]

@jlewi: Update:
I have the following working:
Given something like:

local jupyterhub = map(import 'kubeflow/core/jupyterhub.libsonnet');

the 'map' function maps all resources under jupyterhub as keys of "Kind/metadata.name".
For example - given the above line you could now do

jupyterhub['StatefulSet/tf-hub']
jupyterhub['ConfigMap/jupyterhub-config']
...

and each key would give you that Resource.

I also have functions where you can add, modify or delete anything under the resource
for example add(jupyterhub['StatefulSet/tf-hub'], {
spec+: {
replicas: 2,
},
},
)

Right now I'm creating a fix for this ticket but I could use your guidance on how you would like
add, modify, delete to be applied. Is it done in kfctl? In which case would it be done prior to apply, or maybe after apply using kubecfg?

[jlewi]

Why do you need to do the map command?

I thought jupyterhub.libsonnet might just look like the following

{
   tfhub: {
    ...
  } ,
  configMap: {
   ... 
   },
}

So then you can just do

local parts = import 'kubeflow/core/jupyterhub.libsonnet';

local tfhub = parts.tfhub + {
  ...
};

Right now I'm creating a fix for this ticket but I could use your guidance on how you would like
add, modify, delete to be applied. Is it done in kfctl? In which case would it be done prior to apply, or maybe after apply using kubecfg?

I don't think modifications should be done in kfctl.sh. I'm trying to prevent scope creep in kfctl.sh.

My thinking was that if you wanted to modify the ConfigMap you would do the following

kfctl.sh generate k8s

Now edit components/jupyterhub.jsonnet and change it to something like.

local tfhub = parts.tfhub + {
  ...
};

[jlewi]

@kkasravi any thoughts?

[kkasravi]

copying from the PR

This now works

local jupyterhub = import "kubeflow/core/jupyterhub.libsonnet";
local updated = jupyterhub + {
  StatefulSet+: {
     spec+: {
       template+: {
         spec+: {
          imagePullSecrets: [
            {name: "mySecret",}
          ],
        },
      },
    },
  },
};
updated.all(updatedParams).list