Seeing re-conclier error in the controller manager logs.

[majidma-ag]

I am using the operator-sdk (native ansible operator for k8s) to onboard clients in keycloak,

1. I did the necessary pre-requisites to initialise the operator and creating the api.
2. In the roles/tasks.yml of the operator directory i have defined the standard playbook for keycloak client creation, i am using the following:

• name: Create or update Keycloak client
  local_action:
  module: keycloak_client
  auth_client_id: admin-cli
  auth_keycloak_url: https://auth.example.com/auth
  auth_realm: master
  auth_username: USERNAME
  auth_password: PASSWORD
  client_id: "{{ client_id }}"
  state: present

3. make docker-build docker-push IMG=repname/keycloak_test

4. make install

5. export IMG=repname/keycloak_test

6. make deploy

7. I ran the kubectl command (kubectl apply -f my_crd.yaml) and the yaml looks like the following.

apiVersion: sso.xxx.com/v1alpha1
kind: Ar-sso-fd
metadata:
name: ar-sso-abc-224
spec:
client_id: test224

Type of question

• General operator-related help

Question

What did you do?

Ran the command: kubectl apply -f my_crd.yml

What did you expect to see?

The CRD (my_crd.yml) applied above should have triggered the client onboarding in keycloak

What did you see instead? Under which circumstances?

The controller logs show this event in the logs "error":"did not receive playbook_on_stats event",

Environment

MacOS

Operator type:

/language ansible

Kubernetes cluster type:

$ operator-sdk version

operator-sdk version: "v1.1.0", commit: "9d27e224efac78fcc9354ece4e43a50eb30ea968", kubernetes version: "v1.18.2", go version: "go1.15.2 darwin/amd64", GOOS: "darwin", GOARCH: "amd64"

$ go version (if language is Go)

$ kubectl version

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.8", GitCommit:"9f2892aab98fe339f3bd70e3c470144299398ace", GitTreeState:"clean", BuildDate:"2020-08-13T16:12:48Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.2", GitCommit:"f5743093fd1c663cb0cbc89748f730662345d44d", GitTreeState:"clean", BuildDate:"2020-09-16T13:32:58Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}

The following are the manager logs, it shows my ansible playbook has action which is conflicting but i don't see any conflicting actions.

t']\r\n ansible python module location = /usr/local/lib/python3.6/site-packxxes/ansible\r\n executable location = /usr/local/bin/ansible-playbook\r\n python version = 3.6.8 (default, Dec 5 2019, 15:45:45) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]\r\nUsing /etc/ansible/ansible.cfg as config file\r\nERROR! conflicting action statements: keycloak_client, state\r\n\r\nThe error appears to be in '/opt/ansible/roles/xx-sso-yyy/tasks/main.yml': line 3, column 3, but may\r\nbe elsewhere in the file depending on the exact syntax problem.\r\n\r\nThe offending line appears to be:\r\n\r\n# tasks file for xx-sso-yyy\r\n- name: hhhhhhssoyyy-client\r\n ^ here\r\n","job":"4548432111829895923","name":"xx-sso-yyy-224","namespace":"xx-sso-yyy-system","error":"did not receive playbook_on_stats event","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tpkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128\ngithub.com/operator-framework/operator-sdk/internal/ansible/controller.(*AnsibleOperatorReconciler).Reconcile\n\tsrc/github.com/operator-framework/operator-sdk/internal/ansible/controller/reconcile.go:229\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tpkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/internal/controller/controller.go:235\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tpkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/internal/controller/controller.go:209\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\tpkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/internal/controller/controller.go:188\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\tpkg/mod/k8s.io/apimachinery@v0.18.8/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\tpkg/mod/k8s.io/apimachinery@v0.18.8/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tpkg/mod/k8s.io/apimachinery@v0.18.8/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.Until\n\tpkg/mod/k8s.io/apimachinery@v0.18.8/pkg/util/wait/wait.go:90"}
{"level":"error","ts":1604585003.780023,"logger":"controller","msg":"Reconciler error","controller":"xx-sso-yyy-controller","name":"xx-sso-yyy-224","namespace":"xx-sso-yyy-system","error":"did not receive playbook_on_stats event","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tpkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tpkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/internal/controller/controller.go:237\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tpkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/internal/controller/controller.go:209\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\tpkg/mod/sigs.k8s.io/controller-runtime@v0.6.2/pkg/internal/controller/controller.go:188\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\tpkg/mod/k8s.io/apimachinery@v0.18.8/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\tpkg/mod/k8s.io/apimachinery@v0.18.8/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tpkg/mod/k8s.io/apimachinery@v0.18.8/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.Until\n\tpkg/mod/k8s.io/apimachinery@v0.18.8/pkg/util/wait/wait.go:90"}

[camilamacedo86]

HI @majidma-ag,

Shows that your role has a syntax error. See the logs:

ERROR! conflicting action statements: keycloak_client, state\r\n\r\nThe error appears to be in '/opt/ansible/roles/xx-sso-yyy/tasks/main.yml': line 3, column 3, but may\r\nbe elsewhere in the file depending on the exact syntax problem.\r\n\r\nThe offending line appears to be:\r\n\r\n# tasks file for xx-sso-yyy\r\n- name: hhhhhhssoyyy-client\r\n ^ here\r\n","job":"4548432111829895923",

[majidma-ag]

Hi @camilamacedo86 ,
I saw the error, but the play looks fine.

[majidma-ag]

@camilamacedo86,
As i mentioned in the previous comment the playbook works if you execute it using regular ansible-playbook command, not sure why it is complaining that playbook has conflicting actions.
And when i do the playbook changes and apply the config to trigger the re-concilation, it takes the values what i have specified in the config as i see in the logs along with the old error, it is not taking new changes of the playbook what i did.

[camilamacedo86]

Hi @majidma-ag,

We are unable to reproduce this issue with the sample in the /testdata/ansible or by following the tutorials. So, we would need to have more information. It really shows some role and/or playbook with invalid syntax. Could not be that your not might be executing the same playbook/role that the project is complaining about. So, could you share a link for your project for we are able to reproduce and check it to see if we can help you to sort it out?

[majidma-ag]

Hi @camilamacedo86,

This does not looks like a issue, you may go ahead and close the issue.
Need an advice from you.
The controller logs will show the details of all the previous runs, i wanted to see the logs of only the current run, what settings should i tweak it. ? Please let me know.

I see we need to tweak the config/manager/manager.yaml and config/default/manager_auth_proxy_patch.yaml to have enable debugging or disabling the debugging or add annotations in the CR "ansible.sdk.operatorframework.io/verbosity": "4"

Is there a way to show only the logs of the current run ?

[camilamacedo86]

Hi @majidma-ag,

Thank you for your reply. In the controller-manager, you will find all logs since it is initialized. You can have one terminal open to tail the logs and in another one, you do the operations such as apply the CR to troubleshooting.

What you can do is enable/disable the ANSIBLE LOGS. More info: https://github.com/operator-framework/operator-sdk/blob/9adf15540db110eb46ea247f8e86bb8068c1a498/website/content/en/docs/building-operators/ansible/development-tips.md#viewing-the-ansible-logs.

I hope that it is what you are looking for. for. I am closing this one, however, feel free to raise new issues if you need.