Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Add CI Troubleshooting doc #6988

Merged
merged 8 commits into from Nov 9, 2023
Merged

[CI] Add CI Troubleshooting doc #6988

merged 8 commits into from Nov 9, 2023

Conversation

unlikelyzero
Copy link
Collaborator

@unlikelyzero unlikelyzero commented Aug 24, 2023

Closes #4576

Describe your changes:

  • Documents how to bust cache
  • Update flakiness docs
  • How to run CI locally
  • Update code coverage document

All Submissions:

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Is this change backwards compatible? For example, developers won't need to change how they are calling the API or how they've extended core plugins such as Tables or Plots.

Author Checklist

  • Changes address original issue?
  • Tests included and/or updated with changes?
  • Command line build passes?
  • Has this been smoke tested?
  • Testing instructions included in associated issue OR is this a dependency/testcase change?

Reviewer Checklist

  • Changes appear to address issue?
  • Reviewer has tested changes by following the provided instructions?
  • Changes appear not to be breaking changes?
  • Appropriate automated tests included?
  • Code style and in-line documentation are appropriate?
  • Has associated issue been labelled unverified? (only applicable if this PR closes the issue)
  • Has associated issue been labelled bug? (only applicable if this PR is for a bug fix)

@unlikelyzero unlikelyzero added this to the Target:3.0.1 milestone Aug 24, 2023
@unlikelyzero unlikelyzero changed the title document how to bust cache [CI] document how to bust cache Aug 24, 2023
@unlikelyzero unlikelyzero mentioned this pull request Aug 24, 2023
15 tasks
@deploysentinel
Copy link

deploysentinel bot commented Aug 24, 2023

Current Playwright Test Results Summary

✅ 153 Passing - ⚠️ 3 Flaky

Run may still be in progress, this comment will be updated as current testing workflow or job completes...

(Last updated on 11/09/2023 04:15:43pm UTC)

Run Details

Running Job e2e-stable on CircleCI

Commit: 8967b6a

Started: 11/09/2023 04:06:33pm UTC

⚠️ Flakes

📄   functional/plugins/plot/logPlot.e2e.spec.js • 1 Flake

Test Case Results

Test Case Last 7 days Failures Last 7 days Flakes
Log plot tests Log Plot ticks are functionally correct in regular and log mode and after refresh
Retry 2Retry 1Initial Attempt
1.27% (2) 2 / 158 runs
failed over last 7 days
32.28% (51) 51 / 158 runs
flaked over last 7 days

📄   functional/plugins/timer/timer.e2e.spec.js • 1 Flake

Test Case Results

Test Case Last 7 days Failures Last 7 days Flakes
Timer Can perform actions on the Timer
Retry 1Initial Attempt
0% (0) 0 / 157 runs
failed over last 7 days
11.46% (18) 18 / 157 runs
flaked over last 7 days

📄   functional/plugins/notebook/notebookSnapshots.e2e.spec.js • 1 Flake

Test Case Results

Test Case Last 7 days Failures Last 7 days Flakes
Snapshot image tests Can drop an image onto a notebook and create a new entry
Retry 1Initial Attempt
0% (0) 0 / 161 runs
failed over last 7 days
68.32% (110) 110 / 161 runs
flaked over last 7 days

View Detailed Build Results


@ozyx ozyx self-requested a review August 24, 2023 06:08
TESTING.md Show resolved Hide resolved
TESTING.md Outdated Show resolved Hide resolved
@ozyx ozyx modified the milestones: Target:3.0.1, Target:3.1.0 Aug 24, 2023
@depperm depperm mentioned this pull request Aug 24, 2023
15 tasks
Co-authored-by: David 'Epper' Marshall <epper.marshall@gmail.com>
@codecov
Copy link

codecov bot commented Aug 24, 2023

Codecov Report

Merging #6988 (f9c6e82) into master (3c933ea) will decrease coverage by 3.35%.
Report is 1 commits behind head on master.
The diff coverage is n/a.

❗ Current head f9c6e82 differs from pull request most recent head 8967b6a. Consider uploading reports for the commit 8967b6a to get more accurate results

@@            Coverage Diff             @@
##           master    #6988      +/-   ##
==========================================
- Coverage   56.08%   52.74%   -3.35%     
==========================================
  Files         652      652              
  Lines       26178    26253      +75     
  Branches     2525     2525              
==========================================
- Hits        14683    13846     -837     
- Misses      10796    11710     +914     
+ Partials      699      697       -2     
Flag Coverage Δ *Carryforward flag
e2e-full 41.92% <ø> (+0.12%) ⬆️ Carriedforward from 21d8482
e2e-stable 39.52% <ø> (-18.51%) ⬇️
unit 49.00% <ø> (-0.49%) ⬇️

*This pull request uses carry forward flags. Click here to find out more.

see 128 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3c933ea...8967b6a. Read the comment docs.

@unlikelyzero unlikelyzero changed the title [CI] document how to bust cache [CI] Add CI Troubleshooting doc Sep 4, 2023
@akhenry akhenry removed this from the Target:3.1.0 milestone Oct 2, 2023
@unlikelyzero
Copy link
Collaborator Author

@ozyx please review

@unlikelyzero unlikelyzero added this to the Target:3.2.0 milestone Oct 29, 2023
Copy link
Member

@ozyx ozyx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some typos and grammatical stuff but this is looking great overall. Really nice to have the codecov process documented!

TESTING.md Outdated Show resolved Hide resolved
TESTING.md Outdated Show resolved Hide resolved
TESTING.md Outdated Show resolved Hide resolved
TESTING.md Outdated Show resolved Hide resolved
TESTING.md Outdated Show resolved Hide resolved
TESTING.md Show resolved Hide resolved
@unlikelyzero unlikelyzero merged commit 2cc0027 into master Nov 9, 2023
2 of 8 checks passed
@unlikelyzero unlikelyzero deleted the cache-bust branch November 9, 2023 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CI] Document how to troubleshoot CI issues
5 participants