Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failed to deploy applicationconfiguration service-tracker with two warnings #1578

Closed
zzxwill opened this issue May 26, 2020 · 6 comments
Closed

Failed to deploy applicationconfiguration service-tracker with two warnings #1578

zzxwill opened this issue May 26, 2020 · 6 comments
Labels
bug Something isn't working

Comments

@zzxwill
Copy link
Member

zzxwill commented May 26, 2020

What happened?

Followed Run application to run the application by kubectl apply -f https://raw.githubusercontent.com/crossplane/crossplane/release-0.11/docs/snippets/run/appconfig.yaml and it hit two issues.

➜  /Users/zhouzhengxi/Downloads/crossplane k describe applicationconfiguration service-tracker
Name:         service-tracker
Namespace:    default
Labels:       <none>
Annotations:  API Version:  core.oam.dev/v1alpha2
Kind:         ApplicationConfiguration
Metadata:
  Creation Timestamp:  2020-05-25T09:21:33Z
  Generation:          1
  Resource Version:    12517799
  Self Link:           /apis/core.oam.dev/v1alpha2/namespaces/default/applicationconfigurations/service-tracker
  UID:                 ef5992d7-8d9c-40e1-9edd-3ca64ecf0e52
Spec:
  Components:
    Component Name:  data-api-database
    Parameter Values:
      Name:          database-secret
      Value:         tracker-database-secret
    Component Name:  data-api
    Parameter Values:
      Name:          database-secret
      Value:         tracker-database-secret
    Component Name:  flights-api
    Parameter Values:
      Name:   data-uri
      Value:  http://data-api.default.svc.cluster.local:3009/
    Traits:
      Trait:
        API Version:  core.oam.dev/v1alpha2
        Kind:         ManualScalerTrait
        Metadata:
          Name:  flights-api
        Spec:
          Replica Count:  2
    Component Name:       quakes-api
    Parameter Values:
      Name:   data-uri
      Value:  http://data-api.default.svc.cluster.local:3009/
    Traits:
      Trait:
        API Version:  core.oam.dev/v1alpha2
        Kind:         ManualScalerTrait
        Metadata:
          Name:  quakes-api
        Spec:
          Replica Count:  2
    Component Name:       weather-api
    Parameter Values:
      Name:   data-uri
      Value:  http://data-api.default.svc.cluster.local:3009/
    Traits:
      Trait:
        API Version:  core.oam.dev/v1alpha2
        Kind:         ManualScalerTrait
        Metadata:
          Name:  weather-api
        Spec:
          Replica Count:  2
    Component Name:       service-tracker-ui
    Parameter Values:
      Name:   flights-uri
      Value:  http://flights-api.default.svc.cluster.local:3003/
      Name:   weather-uri
      Value:  http://weather-api.default.svc.cluster.local:3015/
      Name:   quakes-uri
      Value:  http://quakes-api.default.svc.cluster.local:3012/
Status:
  Conditions:
    Last Transition Time:  2020-05-25T09:57:13Z
    Message:               cannot apply components: cannot apply workload "app-postgresql": cannot get object: postgresqlinstancerequirements.database.example.org "app-postgresql" is forbidden: User "system:serviceaccount:oam-system:crossplane-oam" cannot get resource "postgresqlinstancerequirements" in API group "database.example.org" in the namespace "default"
    Reason:                Encountered an error during resource reconciliation
    Status:                False
    Type:                  Synced
Events:
  Type     Reason                 Age                    From                                       Message
  ----     ------                 ----                   ----                                       -------
  Warning  CannotApplyComponents  36m (x12 over 36m)     oam/applicationconfiguration.core.oam.dev  cannot apply workload "app-postgresql": cannot get object: postgresqlinstancerequirements.database.example.org "app-postgresql" is forbidden: User "system:serviceaccount:oam-system:crossplane-oam" cannot get resource "postgresqlinstancerequirements" in API group "database.example.org" in the namespace "default"
  Warning  CannotApplyComponents  16m (x2157 over 36m)   oam/applicationconfiguration.core.oam.dev  cannot apply trait "flights-api": cannot create object: Internal error occurred: failed calling webhook "manualscalertrait.mutate.core.oam.dev": Post https://crossplane-oam-crossplane.oam-system.svc:443/mutate-core-oam-dev-v1alpha2-manualscalertrait?timeout=30s: no endpoints available for service "crossplane-oam-crossplane"
  Normal   RenderedComponents     102s (x3777 over 36m)  oam/applicationconfiguration.core.oam.dev  Successfully rendered components
  Normal   RenderedComponents     102s (x3896 over 36m)  oam/applicationconfiguration.core.oam.dev  Successfully rendered components

How can we reproduce it?

Followed Run application

What environment did it happen in?

Crossplane version:

v 0.11.0

  • Cloud provider or hardware configuration

  • Kubernetes version (use kubectl version)

➜  /Users/zhouzhengxi/Downloads kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:56:40Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.6-aliyun.1", GitCommit:"27fbcf7", GitTreeState:"", BuildDate:"2020-03-11T10:33:57Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/amd64"}
  • Kubernetes distribution (e.g. Tectonic, GKE, OpenShift)
Alibaba ACS
  • OS (e.g. from /etc/os-release)
    System Version: macOS 10.15.3 (19D76)
  • Kernel (e.g. uname -a)
Darwin zhouzhengxi.local 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64
@zzxwill zzxwill added the bug Something isn't working label May 26, 2020
@zzxwill
Copy link
Member Author

zzxwill commented May 26, 2020

For No. 1 warning, it could be fixed by k create clusterrolebinding system_serviceaccount_oam-system_crossplane-oam --clusterrole=postgresqlinstances.database.example.org --user=system:serviceaccount:oam-system:crossplane-oam --group=database.example.org.

Status:
  Conditions:
    Last Transition Time:  2020-05-26T02:46:25Z
    Message:               cannot apply components: cannot apply trait "flights-api": cannot create object: Internal error occurred: failed calling webhook "manualscalertrait.mutate.core.oam.dev": Post https://crossplane-oam-crossplane.oam-system.svc:443/mutate-core-oam-dev-v1alpha2-manualscalertrait?timeout=30s: no endpoints available for service "crossplane-oam-crossplane"
    Reason:                Encountered an error during resource reconciliation
    Status:                False
    Type:                  Synced
Events:
  Type    Reason              Age                       From                                       Message
  ----    ------              ----                      ----                                       -------
  Normal  RenderedComponents  3m (x112692 over 17h)     oam/applicationconfiguration.core.oam.dev  Successfully rendered components
  Normal  RenderedComponents  2m58s (x116084 over 17h)  oam/applicationconfiguration.core.oam.dev  Successfully rendered components

For No. 2 warning, I have no idea.

@hasheddan
Copy link
Member

@zzxwill did you have any errors when installing the OAM helm chart?

@zzxwill
Copy link
Member Author

zzxwill commented May 26, 2020

@zzxwill did you have any errors when installing the OAM helm chart?

Thank you for the reminder, I noticed that pod crossplane-oam-localstack failed.

➜  /Users/zhouzhengxi/Downloads/crossplane k describe svc crossplane-oam-crossplane -n oam-system
Name:              crossplane-oam-crossplane
Namespace:         oam-system
Labels:            app.kubernetes.io/instance=crossplane
                   app.kubernetes.io/managed-by=Helm
                   app.kubernetes.io/name=crossplane-oam
                   app.kubernetes.io/version=0.10.0-rc.24.g61146c2
                   helm.sh/chart=crossplane-oam-1.0.1
Annotations:       meta.helm.sh/release-name: crossplane
                   meta.helm.sh/release-namespace: oam-system
Selector:          app.kubernetes.io/instance=crossplane,app.kubernetes.io/name=crossplane-oam
Type:              ClusterIP
IP:                10.0.0.248
Port:              http  443/TCP
TargetPort:        9443/TCP
Endpoints:
Session Affinity:  None
Events:            <none>
➜  /Users/zhouzhengxi/Downloads/crossplane k get pods -n oam-system
NAME                                           READY   STATUS             RESTARTS   AGE
crossplane-oam-core-6fbb6f558b-fcvsv           1/1     Running            31         19d
crossplane-oam-localstack-c66449f44-mgl2j      1/2     ImagePullBackOff   1          19d
crossplane-oam-stack-manager-77d7d6bd4-8zw9k   1/1     Running            0          19d

By fixing the image pulling issue, the service endpoint shows up.

➜  /Users/zhouzhengxi/Downloads/crossplane k describe svc crossplane-oam-crossplane -n oam-system
Name:              crossplane-oam-crossplane
Namespace:         oam-system
Labels:            app.kubernetes.io/instance=crossplane
                   app.kubernetes.io/managed-by=Helm
                   app.kubernetes.io/name=crossplane-oam
                   app.kubernetes.io/version=0.10.0-rc.24.g61146c2
                   helm.sh/chart=crossplane-oam-1.0.1
Annotations:       meta.helm.sh/release-name: crossplane
                   meta.helm.sh/release-namespace: oam-system
Selector:          app.kubernetes.io/instance=crossplane,app.kubernetes.io/name=crossplane-oam
Type:              ClusterIP
IP:                10.0.0.248
Port:              http  443/TCP
TargetPort:        9443/TCP
Endpoints:         192.168.0.201:9443
Session Affinity:  None
Events:            <none>

And the No. 2 warning disappears.

➜  /Users/zhouzhengxi/Downloads/crossplane k describe applicationconfiguration service-tracker
Name:         service-tracker
Namespace:    default
Labels:       <none>
Annotations:  API Version:  core.oam.dev/v1alpha2
Kind:         ApplicationConfiguration
Metadata:
  Creation Timestamp:  2020-05-26T03:34:30Z
  Generation:          1
  Resource Version:    13458382
  Self Link:           /apis/core.oam.dev/v1alpha2/namespaces/default/applicationconfigurations/service-tracker
  UID:                 9f0ac698-935f-406a-b8c8-a4f039c4fdda
Spec:
  Components:
    Component Name:  data-api-database
    Parameter Values:
      Name:          database-secret
      Value:         tracker-database-secret
    Component Name:  data-api
    Parameter Values:
      Name:          database-secret
      Value:         tracker-database-secret
    Component Name:  flights-api
    Parameter Values:
      Name:   data-uri
      Value:  http://data-api.default.svc.cluster.local:3009/
    Traits:
      Trait:
        API Version:  core.oam.dev/v1alpha2
        Kind:         ManualScalerTrait
        Metadata:
          Name:  flights-api
        Spec:
          Replica Count:  2
    Component Name:       quakes-api
    Parameter Values:
      Name:   data-uri
      Value:  http://data-api.default.svc.cluster.local:3009/
    Traits:
      Trait:
        API Version:  core.oam.dev/v1alpha2
        Kind:         ManualScalerTrait
        Metadata:
          Name:  quakes-api
        Spec:
          Replica Count:  2
    Component Name:       weather-api
    Parameter Values:
      Name:   data-uri
      Value:  http://data-api.default.svc.cluster.local:3009/
    Traits:
      Trait:
        API Version:  core.oam.dev/v1alpha2
        Kind:         ManualScalerTrait
        Metadata:
          Name:  weather-api
        Spec:
          Replica Count:  2
    Component Name:       service-tracker-ui
    Parameter Values:
      Name:   flights-uri
      Value:  http://flights-api.default.svc.cluster.local:3003/
      Name:   weather-uri
      Value:  http://weather-api.default.svc.cluster.local:3015/
      Name:   quakes-uri
      Value:  http://quakes-api.default.svc.cluster.local:3012/
Status:
  Conditions:
    Last Transition Time:  2020-05-26T06:18:34Z
    Reason:                Successfully reconciled resource
    Status:                True
    Type:                  Synced
  Workloads:
    Component Name:  data-api-database
    Workload Ref:
      API Version:   database.example.org/v1alpha1
      Kind:          PostgreSQLInstanceRequirement
      Name:          app-postgresql
    Component Name:  data-api
    Workload Ref:
      API Version:   core.oam.dev/v1alpha2
      Kind:          ContainerizedWorkload
      Name:          data-api
    Component Name:  flights-api
    Traits:
      Trait Ref:
        API Version:  core.oam.dev/v1alpha2
        Kind:         ManualScalerTrait
        Name:         flights-api
    Workload Ref:
      API Version:   core.oam.dev/v1alpha2
      Kind:          ContainerizedWorkload
      Name:          flights-api
    Component Name:  quakes-api
    Traits:
      Trait Ref:
        API Version:  core.oam.dev/v1alpha2
        Kind:         ManualScalerTrait
        Name:         quakes-api
    Workload Ref:
      API Version:   core.oam.dev/v1alpha2
      Kind:          ContainerizedWorkload
      Name:          quakes-api
    Component Name:  weather-api
    Traits:
      Trait Ref:
        API Version:  core.oam.dev/v1alpha2
        Kind:         ManualScalerTrait
        Name:         weather-api
    Workload Ref:
      API Version:   core.oam.dev/v1alpha2
      Kind:          ContainerizedWorkload
      Name:          weather-api
    Component Name:  service-tracker-ui
    Workload Ref:
      API Version:  core.oam.dev/v1alpha2
      Kind:         ContainerizedWorkload
      Name:         web-ui
Events:
  Type    Reason              Age                     From                                       Message
  ----    ------              ----                    ----                                       -------
  Normal  RenderedComponents  9m55s (x315 over 165m)  oam/applicationconfiguration.core.oam.dev  Successfully rendered components
  Normal  RenderedComponents  5m11s (x367 over 165m)  oam/applicationconfiguration.core.oam.dev  Successfully rendered components
  Normal  AppliedComponents   5m11s (x30 over 5m59s)  oam/applicationconfiguration.core.oam.dev  Successfully applied components

@zzxwill
Copy link
Member Author

zzxwill commented May 26, 2020

Sorry, please ignore this bug.

I deployed a Kubernetes cluster from scratch and did another PoC. No need to apply additional clusterrolebinding for warning No. 1 and No. 2 didn't ever show up. And I could see the application UI.
image

@zzxwill
Copy link
Member Author

zzxwill commented May 26, 2020

I think this issue could be closed :)

@hasheddan
Copy link
Member

Sounds good! Thanks for following up :)

@zzxwill zzxwill mentioned this issue Jul 20, 2021
Closed
6 tasks
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@zzxwill @hasheddan