[7833] Flaky HA unit test + update cypress config#7834
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7834 +/- ##
=======================================
Coverage 75.09% 75.10%
=======================================
Files 430 430
Lines 22838 22846 +8
Branches 6052 6055 +3
=======================================
+ Hits 17151 17159 +8
Misses 5687 5687
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Steve-Mcl
left a comment
Steve-Mcl
left a comment
There was a problem hiding this comment.
Approved with nit - you decide 😸
| cy.get('[data-el="tokens-table"] tbody').find('tr').last().should('contain', '31') | ||
| cy.get('[data-el="tokens-table"] tbody').find('tr').last().should('contain', '12') | ||
| cy.get('[data-el="tokens-table"] tbody').find('tr').last().should('contain', '2050') |
There was a problem hiding this comment.
nit - are these exact presence checks handled in other test? If not, I would be tempted to include (a version of) them in the should => callback on line 113?
not a blocker.
There was a problem hiding this comment.
This is just checking that the correct expiry date was set on the token, I'd say it's good enough
cstns
left a comment
cstns
left a comment
There was a problem hiding this comment.
I'll push a fix for the flaky admin test in order to get this in sooner
| module.exports = { | ||
| viewportWidth: 1024, | ||
| viewportHeight: 768, | ||
| retries: { |
There was a problem hiding this comment.
I was certain we had this enabled for this runner.. I must have been thinking of the dashboard repo
| cy.get('[data-el="tokens-table"] tbody').find('tr').last().should('contain', '31') | ||
| cy.get('[data-el="tokens-table"] tbody').find('tr').last().should('contain', '12') | ||
| cy.get('[data-el="tokens-table"] tbody').find('tr').last().should('contain', '2050') |
There was a problem hiding this comment.
This is just checking that the correct expiry date was set on the token, I'd say it's good enough
|
|
||
| // retryable read - a synchronous Cypress.$ count can beat the table render on slow CI, undercounting the baseline | ||
| let rows | ||
| cy.get('[data-el="tokens-table"] tbody').find('tr.ff-data-table--row').then($rows => { rows = $rows.length }) |
There was a problem hiding this comment.
counting the number of rows in a table was never a good idea to begin with. If there's a next time, we should rethink how we validate entries exist in the table
|
|
||
| cy.url().should('match', /\/team\/[^/]+/) | ||
| // wait for the team redirect to settle on /overview, else the Applications click below can race the pending redirect | ||
| cy.url().should('match', /\/team\/[^/]+\/overview/) |
There was a problem hiding this comment.
I think this was just a gross oversight.. the url should never have matched team after navigating away from the admin teams list. Running the suites locally were just fast enough to catch the old href before navigation.
We should only be checking if the page matches overview after nav
3 participants
Description
Several unrelated CI flakes, green locally, red on runners. Different causes, not one bug.
ha/index_spec.jsstate === 'suspended'check that raced the restart's re-write; kept the reliableinflightcheck.InstanceStatusPolling.vuetokens.spec.jscy.get()(not syncCypress.$pre-render); expiry assertion made timezone-safe.admin.spec.js/overviewbefore clicking the Applications nav. The click was racing the pending redirect and getting bounced back.cypress-shared.config.jsretries: { runMode: 2 }absorbs CI variance. Retried passes still logAttempt 2 of 3. See below.Retries: on CI a failing test is re-run up to 2 more times (3 attempts total); it only counts as failed if all 3 fail, so a one-off hiccup no longer reds the build. Local
cypress openkeeps 0 retries for instant feedback.Downside: a genuinely flaky test can now pass silently, but will torture us less. Check run logs for
Attempt 2 of 3when that matters.Related Issue(s)
Resolves #7833
Checklist
flowforge.yml?FlowFuse/helmto update ConfigMap TemplateFlowFuse/CloudProjectto update values for Staging/ProductionLabels
area:migrationlabel