Can't install kubeprod

[meodemsao]

after create cluster with eksctl + create cognito pool + create route53 DNS record

eksctl create cluster --name=xxx --node=2 --version=1.15

i run install kubeprod

kubeprod install eks \
  --email "xxxx@gmail.com" \
  --dns-zone "xxx.xxx" \
  --user-pool-id "xxxxxxxx"

this output logs

Unknown options: eks,get-token
INFO  Validating clusterrole cert-manager-issuers

Unknown options: eks,get-token
ERROR Error in clusterrole cert-manager-issuers: Unable to fetch schema: Get "https://CC5AA6253875D0F46C3BFFD4D5926565.gr7.ap-southeast-1.eks.amazonaws.com/openapi/v2": getting token: exec: exit status 252

Unknown options: eks,get-token
INFO  Validating serviceaccount kubeprod.cert-manager

Unknown options: eks,get-token
ERROR Error in serviceaccount kubeprod.cert-manager: Unable to fetch schema: Get "https://CC5AA6253875D0F46C3BFFD4D5926565.gr7.ap-southeast-1.eks.amazonaws.com/openapi/v2": getting token: exec: exit status 252

Unknown options: eks,get-token
INFO  Validating customresourcedefinition certificaterequests.cert-manager.io

Unknown options: eks,get-token
ERROR Error in customresourcedefinition certificaterequests.cert-manager.io: Unable to fetch schema: Get "https://CC5AA6253875D0F46C3BFFD4D5926565.gr7.ap-southeast-1.eks.amazonaws.com/openapi/v2": getting token: exec: exit status 252

Unknown options: eks,get-token
INFO  Validating customresourcedefinition certificates.cert-manager.io

Unknown options: eks,get-token
ERROR Error in customresourcedefinition certificates.cert-manager.io: Unable to fetch schema: Get "https://CC5AA6253875D0F46C3BFFD4D5926565.gr7.ap-southeast-1.eks.amazonaws.com/openapi/v2": getting token: exec: exit status 252

[meodemsao]

I found relate with this issues

#293

I don't know but when i using

kubecfg update --gc-tag=kube_prod_runtime --ignore-unknown kubeprod-manifest.jsonnet

every thing install perfect

[javsalgar]

Hi,

Thanks for letting us know. We will try to reproduce the issue. However, please note that version 1.15 is not still in our compatibility matrix. We are working on bumping the supported Kubernetes version. Did you try a lower version of K8s?

[abubakar-malik]

I am having exactly same issue when deploying BRK using below;
kubeprod install eks \ --email ${AWS_EKS_USER} \ --dns-zone "${BKPR_DNS_ZONE}" \ --user-pool-id "${AWS_COGNITO_USER_POOL_ID}"

Error like:
Unknown options: eks,get-token
ERROR Error in statefulset kubeprod.alertmanager: Unable to fetch schema: Get https://XXXX.eks.amazonaws.com/openapi/v2: getting token: exec: exit status 255

I am using;
aws cli version: 1.18.52
OS: Both Ubuntu and MacOS
Kubernetes version: 1.14
kubeprod version: 1.5.1

Thanks to @meodemsao above kubecfg worked. But it needs to be fixed and the documentation should be adjusted too
https://github.com/bitnami/kube-prod-runtime/blob/master/docs/quickstart-eks.md

Even simple version output command returns same error message
$ kubeprod version

Unknown options: eks,get-token
Installer version: v1.5.1
Server manifests version: unknown

[javsalgar]

Hi,

This is strange, I don't see this message when downloading bkpr. Do you need to install it in eks in order to get that message?

❯ ./kubeprod version
Installer version: v1.5.1
Server manifests version: not installed

[abubakar-malik]

No. I didn't install it in eks. Only on my local machines with Ubunut and Mac OS.

Does it matter which aws cli and k8s version you are using?

I believe i had it working with an older version of aws cli

[javsalgar]

In my case this is the aws version that I have in my machine and didn't find any issue

❯ aws --version
aws-cli/2.0.0 Python/3.8.1 Darwin/19.4.0 botocore/2.0.0dev4

[abubakar-malik]

That's strange. I have exactly same major versions as yours.

aws --version
aws-cli/2.0.10 Python/3.8.2 Darwin/19.4.0 botocore/2.0.0dev14
./kubeprod version
Unknown options: eks,get-token
Installer version: v1.5.1
Server manifests version: unknown

anything else you suggest i should try? any logging maybe?

[abubakar-malik]

Any other suggestions please? still getting the error.
Below workaround is working;
kubecfg update --gc-tag=kube_prod_runtime --ignore-unknown kubeprod-manifest.jsonnet

[javsalgar]

Hi,

I tried updating to the exact same aws client and did not find that issue. I will try installing it to an eks cluster to see if the same issue happens.

[abubakar-malik]

You got time to check this?

[meodemsao]

Hi all,
I using

• macos catalia
• aws kubernetes 1.14
• eksctl 0.16.0
• aws-cli 2.0.7
• kubeprod 1.14 ( i try using 1.15 before )

[jjo]

hi @meodemsao

kubeprod 1.14 ( i try using 1.15 before )

Note there's no such release but rather v1.4 I'd guess.
Could you please give v1.6.0-rc2 a try (many small fixes) ?
I just cut it, should have binaries ready in a few.

[vsimon]

Just hit this problem with similar output as the reporter.

aws kubernetes 1.15
aws-cli 2.0.6

$ kubeprod version
Unknown options: eks,get-token
Installer version: v1.5.1
Server manifests version: unknown

Should version be reporting "Unknown options: ..."?

Is kubeprod utilizing kubeconfig (~/.kube/config) in any way, in kubeconfig, under users, there are args "eks", "get-token" like so:

users:
- name: arn:aws:eks:us-east-1:XXXXXXXXXXXX:cluster/eks
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      args:
      - --region
      - us-east-1
      - eks
      - get-token
      - --cluster-name
      - eks
      command: aws

@jjo Does your kubeconfig look similar?

[vsimon]

bkpr-v1.6.0-rc4 worked better for me, getting past the "Unable to fetch schema" errors.

[javsalgar]

Hi,

After going a bit deeper, I believe that the unknown options issue has to do with the version of the Kubernetes go libraries we are using. I've seen other projects experiencing the same issue and solving it by updating them. Could you confirm if the issue appears in BKPR 1.6?

[vsimon]

Hi,

After going a bit deeper, I believe that the unknows options issue has to do with the version of the Kubernetes go libraries we are using. I've seen other projects experiencing the same issue and solving it by updating them. Could you confirm if the issue appears in BKPR 1.6?

thanks, the unknown options issue goes away in 1.6.

[javsalgar]

Thank you very much for letting us know! Did you experience any other issue when installing?

[vsimon]

Thank you very much for letting us know! Did you experience any other issue when installing?

hey @javsalgar kubeprod install completed successfully but the main issue I ran into was I wasn't able to see any DNS records created in the new hosted zone in AWS Route53. I believe the other errors I saw cascaded from that (like certificates from cert-manager needed the records to be created). I didn't have time to go deeper and uninstalled (running into issue #739)

[javsalgar]

Thank you very much for the input. We don't see this issue in our testing but we will update the ticket if we become able to reproduce it.