[FEATURE]: csm replication - repctl cluster inject --use-sa not working when CA certificate is not present in kubeconfig

[anurajpd]

Bug Description

csm replication - repctl cluster inject --use-sa not working

Logs

[anuraj@adm01 ocp04]$ repctl cluster inject --use-sa
[2024-05-17 13:45:14] INFO Generating config maps from existing service accounts
[2024-05-17 13:45:15] INFO Generating config for service account in ocp04 cluster
Compiling kubeconfig for "dell-replication-controller-sa"[ OK ]
Namespace: dell-replication-controller, cmd:
Extracting secret [ OK ]
Extracting token [ OK ]
Extracting CA [ FAIL ]
[2024-05-17 13:45:15] INFO
[2024-05-17 13:45:15] INFO Generating config for service account in ocp05 cluster
Compiling kubeconfig for "dell-replication-controller-sa"[ OK ]
Namespace: dell-replication-controller, cmd:
Extracting secret [ OK ]
Extracting token [ OK ]
Extracting CA [ FAIL ]
[2024-05-17 13:45:15] INFO
[2024-05-17 13:45:15] INFO Custom configs provided, injecting them into clusters
[2024-05-17 13:45:15] FATAL cluster inject: error checking cfg /tmp/repctl/ocp04: stat /tmp/repctl/ocp04: no such file or directory

Screenshots

No response

Additional Environment Information

No response

Steps to Reproduce

repctl cluster inject --use-sa

Expected Behavior

this to work - repctl cluster inject --use-sa

CSM Driver(s)

PowerFlex

Installation Type

Operator v1.5.0

Container Storage Modules Enabled

CSM Replication

Container Orchestrator

Openshift 4.14.23

Operating System

CoreOS

[csmbot]

@anurajpd: Thank you for submitting this issue!

The issue is currently awaiting triage. Please make sure you have given us as much context as possible.

If the maintainers determine this is a relevant issue, they will remove the needs-triage label and respond appropriately.

---

We want your feedback! If you have any questions or suggestions regarding our contributing process/workflow, please reach out to us at container.storage.modules@dell.com.

[shanmydell]

@gallacher can we sync this issue

[gallacher]

/sync

[csmbot]

link: 24798

[boyamurthy]

Hi @anurajpd , could you try the same using the latest csm-replication release "https://github.com/dell/csm-replication/releases/tag/v1.8.1" .

we tried the same on local lab and see it is working fine.

[ repctl]# ./repctl cluster inject --use-sa
[2024-06-10 09:54:25] INFO Generating config maps from existing service accounts
[2024-06-10 09:54:25] INFO Generating config for service account in cluster210 cluster
Compiling kubeconfig for "dell-replication-controller-sa"[ OK ]
Namespace: dell-replication-controller, cmd:
Extracting secret [ OK ]
Extracting token [ OK ]
Extracting CA [ OK ]
Extracting server URL [ OK ]
DONE. Kubeconfig location: /tmp/repctl/cluster210
[2024-06-10 09:54:28] INFO
[2024-06-10 09:54:28] INFO Generating config for service account in cluster234 cluster
Compiling kubeconfig for "dell-replication-controller-sa"[ OK ]
Namespace: dell-replication-controller, cmd:
Extracting secret [ OK ]
Extracting token [ OK ]
Extracting CA [ OK ]
Extracting server URL [ OK ]
DONE. Kubeconfig location: /tmp/repctl/cluster234
[2024-06-10 09:54:29] INFO
[2024-06-10 09:54:29] INFO Custom configs provided, injecting them into clusters
W0610 09:54:30.060642 408759 warnings.go:70] Use tokens from the TokenRequest API or manually created secret-based tokens instead of auto-generated secret-based tokens.
W0610 09:54:30.144091 408759 warnings.go:70] Use tokens from the TokenRequest API or manually created secret-based tokens instead of auto-generated secret-based tokens.
[2024-06-10 09:54:30] INFO Creating/Updating cluster234 secret in cluster210 cluster
[2024-06-10 09:54:30] INFO secret cluster234 created
[2024-06-10 09:54:30] INFO Creating/Updating replication config map in cluster210 cluster
[2024-06-10 09:54:30] INFO config map dell-replication-controller-config created
[2024-06-10 09:54:30] INFO Creating/Updating cluster210 secret in cluster234 cluster
[2024-06-10 09:54:30] INFO secret cluster210 created
[2024-06-10 09:54:30] INFO Creating/Updating replication config map in cluster234 cluster
[2024-06-10 09:54:31] INFO config map dell-replication-controller-config created

[anurajpd]

I am getting below error on RHEL8.10 with the new version of repctl 1.8.1
[anuraj@adm01 ps01]$ repctl -h
repctl: /lib64/libc.so.6: version GLIBC_2.34' not found (required by repctl) repctl: /lib64/libc.so.6: version GLIBC_2.32' not found (required by repctl)

[adarsh-dell]

Hi @anurajpd ,

Regarding the issue repctl: /lib64/libc.so.6: version GLIBC_2.34 not found (required by repctl) repctl: /lib64/libc.so.6: version GLIBC_2.32 not found (required by repctl), we believe that PR #137 will resolve this problem. We are currently verifying this in our lab.

Addressing your initial query about repctl cluster inject --use-sa not working: We discovered that the tool is failing with Extracting CA [FAIL]. The kubeconfig file you provided does not contain the certificate-authority-data field, which is required. Instead, it has insecure-skip-tls-verify: true.

The absence of a CA certificate and the use of insecure-skip-tls-verify: true in your kubeconfig means that TLS verification is bypassed. This configuration is not recommended for production environments due to security risks. The tool repctl requires the CA certificate for its operation as designed.

To address this, please ensure the CA certificate is included in the kubeconfig. Here are the steps to add the CA certificate:

1. Obtain the CA Certificate:

   • Locate the CA certificate file on your OpenShift or Kubernetes cluster (typically found at /etc/kubernetes/pki/ca.crt).

2. Encode the CA Certificate:

   • Use the following command to base64 encode the certificate:

     base64 -w 0 /etc/kubernetes/pki/ca.crt

   • Copy the output of this command.

3. Update the kubeconfig File:

   • Open your kubeconfig file in a text editor.
   • Replace the line insecure-skip-tls-verify: true with:

     certificate-authority-data: <base64-encoded-ca-cert>

   • Save the file.

Including the CA certificate ensures secure communication with your cluster by verifying the server's certificate. Once updated, please try using the repctl tool again.

Best regards,
Adarsh

[adarsh-dell]

If anyone encounter an issues with a static build while creating the binary, please refer to dell/csm-replication#143 for a solution.

[WilsonRadadia20]

I have discussed with @anuraj_pd and here is the update:

• Use this command to get the configmaps: oc get configmap
  

• After this, describe the configmap: oc describe configmap kube-root-ca.crt
  

• This will give the certificate and then add it using the comment that Adarsh has added.

I have updated the previous comment with correct commands.

Best regards,
Wilson

[anurajpd]

@WilsonRadadia20 I am not clear of what you are asking me to do. Could you please give the name of the configmap and the namespace.

[WilsonRadadia20]

I have modified the commands in previous comment with the configmap name: kube-root-ca.crt and the namespace: default.

[adarsh-dell]

Hi @anurajpd ,

We have consulted with our SMEs and concluded that the CA certificate must be present in the kubeconfig. This is essential for ensuring that when the --use-sa flag (which uses an existing service account) is used, the server's identity can be validated by checking the certificate in the response from the API server.

Key Points:

1. Importance of CA Certificate in kubeconfig:

   • Security: The CA certificate ensures secure communication by validating the server's identity.
   • Functionality: Without the CA certificate, tools like repctl and functionalities using the -use-sa flag may not work correctly.

2. Steps to Add CA Certificate:

   • After an OCP cluster is created, you can add the CA certificate to the kubeconfig. Here’s a helpful link: How to add certificates to kubeconfig file.
   • Steps Overview:
     1. Locate the CA certificate file on your cluster.
     2. Update the kubeconfig file to include the path to this CA certificate.

3. Recommendations:

   • Contact RH/OCP Team: For specific guidance on adding the CA certificate to the kubeconfig after the cluster is created, it is advisable to reach out to the RH/OCP team.
   • Production Environment: It is highly recommended to include the CA certificate in the kubeconfig for production environments to ensure secure and reliable operations.

Conclusion:

As per our design, the presence of the CA certificate in the kubeconfig is mandatory when using the -use-sa flag. Therefore, we are closing this issue.

Thank you for your understanding and cooperation.

Best Regards,
Adarsh

[adarsh-dell]

We had a call with @anurajpd , who requested that the repctl should function even if the user does not configure the CA certificate in the kubeconfig.

To us, this appears to be more of an enhancement, so we are converting it to a feature request. This will require further analysis and design changes. For as of now, We will document in the csm-docs that if the user wants to use the --use-sa flag, then the certificate-authority field must be present in the kubeconfig for repctl.

@anurajpd Can you please provide more details to us so we will get more insight and will not miss that while incorporating it for upcoming releases.

Thanks