Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky tests for --now flag #2642

Closed
kadel opened this issue Feb 26, 2020 · 14 comments · Fixed by #2870
Closed

Flaky tests for --now flag #2642

kadel opened this issue Feb 26, 2020 · 14 comments · Fixed by #2870
Assignees
@mik-dass
Labels
area/testing Issues or PRs related to testing, Quality Assurance or Quality Engineering flake Categorizes issue or PR as related to a flaky test. priority/Critical We need to stop doing everything and fix this ASAP. It is making the project unusable. triage/needs-information Indicates an issue needs more information in order to work on it.
Projects
Sprint 182

Comments

@kadel
Copy link
Member

kadel commented Feb 26, 2020 

c] 'Running odo with args [odo config unset --now --env hello --context /tmp/363231666]
[odo]  •  Checking component  ...
[odo]  ✓  Checking component [66ms]
[odo]  ✓  Environment variables were successfully updated
[odo] 
[odo] Configuration changes
[odo]  •  Retrieving component data  ...
[odo]  ✓  Retrieving component data [88ms]
[odo]  •  Applying configuration  ...
[odo]  ✗  Applying configuration [86ms]
[odo]  ✗  Failed To Update Config To Component Deployed.
[odo] Error: unable to update DeploymentConfig for nodejs component: unable to update the current DeploymentConfig nodejs-app: unable to update DeploymentConfig nodejs-app: Operation cannot be fulfilled on deploymentconfigs.apps.openshift.io "nodejs-app": the object has been modified; please apply your changes to the latest version and try again
Deleting dir: /tmp/363231666
Deleting project: usaszqseld
Running odo with args [odo project delete usaszqseld -f]
[odo] This project contains the following applications, which will be deleted
[odo] Application app
[odo] This application has following components that will be deleted
[odo] component named nodejs
[odo] No services / could not get services
[odo]  •  Deleting project usaszqseld  ...
[odo]  ✓  Deleting project usaszqseld [6s]
[odo]  ✓  Deleted project : usaszqseld
• Failure [9.242 seconds]
odo preference and config command tests
/go/src/github.com/openshift/odo/tests/integration/cmd_pref_config_test.go:20
  when using --now with config command
  /go/src/github.com/openshift/odo/tests/integration/cmd_pref_config_test.go:310
    should successfully set and unset variables [It]
    /go/src/github.com/openshift/odo/tests/integration/cmd_pref_config_test.go:322
    No future change is possible.  Bailing out early after 0.427s.

https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_odo/2637/pull-ci-openshift-odo-master-v4.3-integration-e2e-benchmark/728#1:build-log.txt%3A395

/priority high
/kind flake
/assign @mohammedzee1000
/area testing
@openshift-ci-robot openshift-ci-robot added priority/High Important issue; should be worked on before any other issues (except priority/Critical issue(s)). flake Categorizes issue or PR as related to a flaky test. area/testing Issues or PRs related to testing, Quality Assurance or Quality Engineering labels Feb 26, 2020
@openshift-ci-robot
Copy link
Collaborator

@kadel: The label(s) area/ cannot be applied, because the repository doesn't have them

In response to this:

c] 'Running odo with args [odo config unset --now --env hello --context /tmp/363231666]
[odo]  •  Checking component  ...
[odo]  ✓  Checking component [66ms]
[odo]  ✓  Environment variables were successfully updated
[odo] 
[odo] Configuration changes
[odo]  •  Retrieving component data  ...
[odo]  ✓  Retrieving component data [88ms]
[odo]  •  Applying configuration  ...
[odo]  ✗  Applying configuration [86ms]
[odo]  ✗  Failed To Update Config To Component Deployed.
[odo] Error: unable to update DeploymentConfig for nodejs component: unable to update the current DeploymentConfig nodejs-app: unable to update DeploymentConfig nodejs-app: Operation cannot be fulfilled on deploymentconfigs.apps.openshift.io "nodejs-app": the object has been modified; please apply your changes to the latest version and try again
Deleting dir: /tmp/363231666
Deleting project: usaszqseld
Running odo with args [odo project delete usaszqseld -f]
[odo] This project contains the following applications, which will be deleted
[odo] Application app
[odo] This application has following components that will be deleted
[odo] component named nodejs
[odo] No services / could not get services
[odo]  •  Deleting project usaszqseld  ...
[odo]  ✓  Deleting project usaszqseld [6s]
[odo]  ✓  Deleted project : usaszqseld
• Failure [9.242 seconds]
odo preference and config command tests
/go/src/github.com/openshift/odo/tests/integration/cmd_pref_config_test.go:20
 when using --now with config command
 /go/src/github.com/openshift/odo/tests/integration/cmd_pref_config_test.go:310
   should successfully set and unset variables [It]
   /go/src/github.com/openshift/odo/tests/integration/cmd_pref_config_test.go:322
   No future change is possible.  Bailing out early after 0.427s.

https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_odo/2637/pull-ci-openshift-odo-master-v4.3-integration-e2e-benchmark/728#1:build-log.txt%3A395

/priority high
/kind flake
/assign @mohammedzee1000
/area testing

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@amitkrout
Copy link
Contributor 

[odo]  ✗  Applying configuration [86ms]
[odo]  ✗  Failed To Update Config To Component Deployed.
[odo] Error: unable to update DeploymentConfig for nodejs component: unable to update the current DeploymentConfig nodejs-app: unable to update DeploymentConfig nodejs-app: Operation cannot be fulfilled on deploymentconfigs.apps.openshift.io "nodejs-app": the object has been modified; please apply your changes to the latest version and try again

This is where the issue is hit. I think @mik-dass is looking into very similar issue.

@mik-dass
Copy link
Contributor

This is where the issue is hit. I think @mik-dass is looking into very similar issue.

I was looking into a similar but different issue earlier: #1981. This issue is arlready closed.

@kadel
Copy link
Member Author

kadel commented Mar 2, 2020 •
edited

another occurrence https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_odo/2656/pull-ci-openshift-odo-master-v4.1-integration-e2e-benchmark/777

@amitkrout
Copy link
Contributor

same failure can be seen against 4.4 cluster aswell - https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_release/6892/rehearse-6892-pull-ci-openshift-odo-master-v4.4-integration-e2e-benchmark/3#1:build-log.txt%3A1186

@johnmcollier
Copy link
Member

Seen again on 4.1: https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_odo/2656/pull-ci-openshift-odo-master-v4.1-integration-e2e-benchmark/816

@kadel kadel added this to For consideration in Sprint 181 via automation Mar 3, 2020
@kadel
Copy link
Member Author

kadel commented Mar 3, 2020

this is getting more serious, bumping priority again
/remove-priority high
/priority critical

@openshift-ci-robot openshift-ci-robot added priority/Critical We need to stop doing everything and fix this ASAP. It is making the project unusable. and removed priority/High Important issue; should be worked on before any other issues (except priority/Critical issue(s)). labels Mar 3, 2020
@johnmcollier johnmcollier mentioned this issue Mar 10, 2020
Merged
@mohammedzee1000
Copy link
Contributor

mohammedzee1000 commented Mar 11, 2020 •
edited

Unable to apply deployment config interesting 🤔
This also seems to be very common with nodejs apps, I could move to test all now flag with python apps but that will only mitigate the problem, not fix it

Also considering other possibilities

@mohammedzee1000
Copy link
Contributor

@kadel ^

@girishramnani girishramnani added the triage/needs-information Indicates an issue needs more information in order to work on it. label Mar 11, 2020
@girishramnani girishramnani moved this from For consideration to To do in Sprint 181 Mar 11, 2020
@girishramnani
Copy link
Contributor

any updates on this @mohammedzee1000 ?

@amitkrout
Copy link
Contributor

Another occurrence - https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_odo/2739/pull-ci-openshift-odo-master-v4.3-integration-e2e-benchmark/1289#1:build-log.txt%3A375

@mohammedzee1000 Can you please take this as high priority as CI pain is increasing day by day for such flakes.

@dharmit
Copy link
Member

dharmit commented Mar 24, 2020

Faced this couple of times on #2739 😢

@amitkrout amitkrout mentioned this issue Mar 24, 2020
Merged
@dharmit
Copy link
Member

dharmit commented Mar 25, 2020

Hit this again today on same PR 2739: https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_odo/2739/pull-ci-openshift-odo-master-v4.2-integration-e2e-benchmark/1294. But on 4.2 instead of 4.3 earlier.

@dharmit dharmit mentioned this issue Mar 25, 2020
Merged
This was referenced Mar 26, 2020
Merged
Merged
@girishramnani girishramnani added this to For consideration in Sprint 182 via automation Mar 30, 2020
@girishramnani girishramnani removed this from To do in Sprint 181 Mar 30, 2020
@girishramnani girishramnani moved this from For consideration to To do in Sprint 182 Mar 30, 2020
@dharmit dharmit mentioned this issue Apr 9, 2020
Merged
@dharmit
Copy link
Member

dharmit commented Apr 9, 2020

@mohammedzee1000 #2841 (comment)

@dharmit dharmit mentioned this issue Apr 9, 2020
Merged
@amitkrout amitkrout mentioned this issue Apr 11, 2020
Merged
@mik-dass mik-dass mentioned this issue Apr 13, 2020
Merged
@mik-dass mik-dass assigned mik-dass and unassigned mohammedzee1000 Apr 13, 2020
@mik-dass mik-dass moved this from To do to For review in Sprint 182 Apr 13, 2020
Sprint 182 automation moved this from For review to Done Apr 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mik-dass mik-dass

Labels
area/testing Issues or PRs related to testing, Quality Assurance or Quality Engineering flake Categorizes issue or PR as related to a flaky test. priority/Critical We need to stop doing everything and fix this ASAP. It is making the project unusable. triage/needs-information Indicates an issue needs more information in order to work on it.
Projects
No open projects
Sprint 182
  
Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fixes now flag push. mik-dass/odo
8 participants
@kadel @dharmit @girishramnani @johnmcollier @mohammedzee1000 @mik-dass @amitkrout @openshift-ci-robot