Fix flaky chart tests #1820
Fix flaky chart tests #1820
Conversation
| - -c | ||
| - curl -o /tmp/output $ASSETSVC_HOST:$ASSETSVC_PORT/v1/ns/{{ .Release.Namespace }}/charts && cat /tmp/output && cat /tmp/output | grep wordpress | ||
| - | |
There was a problem hiding this comment.
Always worth a comment so in 3 months we know why this was added (and whether it can be removed etc.)
| if curl -o /tmp/output $ASSETSVC_HOST:$ASSETSVC_PORT/v1/ns/{{ .Release.Namespace }}/charts && cat /tmp/output && cat /tmp/output | grep wordpress; then | ||
| break | ||
| fi | ||
| sleep 10 |
There was a problem hiding this comment.
You're missing an increment ((n+=1)) here, so this would just keep going until timing out?
| @@ -288,8 +288,12 @@ else | |||
| fi | |||
|
|
|||
| # Wait for Kubeapps Jobs | |||
| k8s_wait_for_job_completed kubeapps apprepositories.kubeapps.com/repo-name=stable | |||
| info "Job apprepositories.kubeapps.com/repo-name=stable ready" | |||
There was a problem hiding this comment.
You said that
After a bit of investigation I noticed that since we are running the install plus the upgrade in a short time, the
upgradehook may finish after the repos are populated. This causes that the catalog doesn't include the chart required for the tests.
Rather than making CI do something different to what people do in real life, could we not instead:
- Install (and upgrade) with only the bitnami repository
- Update the above wait which you've deleted to wait for the bitnami repository sync to be completed
Wouldn't that effectively do what you've done below, ensuring that the upgrade hook cannot begin until the (single) repo is initially populated? Then, the upgrade happens, which may re-populate, but we'll still wait for the job to complete. Not sure if I'm missing something.
+1'ing but see what you think.
There was a problem hiding this comment.
Wouldn't that effectively do what you've done below, ensuring that the upgrade hook cannot begin until the (single) repo is initially populated? Then, the upgrade happens, which may re-populate, but we'll still wait for the job to complete. Not sure if I'm missing something.
Nope, the wait just ensures that any sync job for the given repo is finished, that's why I am removing all the jobs beforehand. Also, the problem is that the flow is the following:
- Application install, a sync job is started.
- Application upgrade, a second sync job and the upgrade hook are started.
- The two sync jobs finish.
- The upgrade hook finishes, deleting the database. This is sometimes slower than the sync jobs because (I assume) it needs to pull the images.
That way, no matter how much we wait, the repo won't be populated, that's why I am refreshing the repo again.
Description of the change
After adding the
upgradetest, the scenario in which that test is run (helm3+pg) fails many times. After a bit of investigation I noticed that since we are running the install plus the upgrade in a short time, theupgradehook may finish after the repos are populated. This causes that the catalog doesn't include the chart required for the tests.This PR, adds:
bitnamicharts rather thanstable(which is being deprecated).