Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(browser-tests): flakiness improvements #407

Merged
merged 28 commits into from
Mar 27, 2025
Merged
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
a173ed5
resolve attempt for editor.spec flakiness
emrberk Mar 19, 2025
75e365f
drag test update
emrberk Mar 19, 2025
3e0ea67
bad integer error - remove screenshot matching
emrberk Mar 19, 2025
298dcc4
remove redundant assertion
emrberk Mar 19, 2025
a9f1477
remvoe warnings test flakiness
emrberk Mar 24, 2025
e9ef0dd
only run warnings
emrberk Mar 24, 2025
b5129ea
open all tests - remove snapshot match
emrberk Mar 24, 2025
ae9ac3f
pass failonsnapshotdiff to config
emrberk Mar 24, 2025
71b9788
introduce built-in type command for warnings test
emrberk Mar 24, 2025
49175cb
disable image snapshot matching temp
emrberk Mar 24, 2025
e437475
cleanup image snapshot package
emrberk Mar 25, 2025
cb0f5d4
add new publish to npm action
emrberk Mar 25, 2025
c9ac9d4
update for typequery
emrberk Mar 26, 2025
72158d7
clear editor programmatically
emrberk Mar 26, 2025
beef5b0
disable retries, try with shorter timeout, run only editor
emrberk Mar 26, 2025
d9d3d74
wait for editor settings
emrberk Mar 26, 2025
a9f339e
reintroduce flakiness check action
emrberk Mar 26, 2025
e231d51
pass env variable to enable matviews
emrberk Mar 26, 2025
9544d0f
take cy screenshots as artifacts on test failure
emrberk Mar 26, 2025
ef151c4
wait for tab availability, don't rely on expansion state
emrberk Mar 26, 2025
f964078
change focusing
emrberk Mar 26, 2025
6b891e8
prevent databaseclosederror in warnings
emrberk Mar 26, 2025
3fc6fab
cleanup test case effects
emrberk Mar 26, 2025
bdfc16e
assert new tab created state - prevent failures
emrberk Mar 26, 2025
939d552
add explicit wait for history persistence
emrberk Mar 26, 2025
1d77d5b
wait for individual tab closes
emrberk Mar 26, 2025
1c82009
retry opening closing archived tabs until command timeout is reached
emrberk Mar 26, 2025
fae7c29
reintroduce old ci action & add screenshot uploads
emrberk Mar 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -81,6 +81,14 @@ jobs:
name: questdb-log
path: tmp/dbroot/log/*

- name: Upload Cypress Screenshots
if: success() || failure()
uses: actions/upload-artifact@v4
with:
name: cypress-screenshots
path: packages/browser-tests/cypress/screenshots
if-no-files-found: ignore

- name: Stop QuestDB
if: success() || failure()
run: ./tmp/questdb-*-rt-linux-x86-64/bin/questdb.sh stop
44 changes: 44 additions & 0 deletions .github/workflows/publish_to_npm.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Publish Packages to NPM

on:
workflow_dispatch:

jobs:
publish:
name: Publish Packages to NPM
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
with:
submodules: "recursive"

- uses: actions/setup-node@v4
with:
node-version: "20"
cache: "yarn"

- name: Install dependencies
run: yarn install --immutable --immutable-cache

- name: Build @questdb/react-components
run: yarn workspace @questdb/react-components run build

- name: Build @questdb/web-console
run: yarn workspace @questdb/web-console run build

- name: Publish @questdb/react-components to npm
uses: JS-DevTools/npm-publish@v1
with:
token: ${{ secrets.NPM_TOKEN }}
access: public
check-version: true
package: ./packages/react-components/package.json

- name: Publish @questdb/web-console to npm
uses: JS-DevTools/npm-publish@v1
with:
token: ${{ secrets.NPM_TOKEN }}
access: public
check-version: true
package: ./packages/web-console/package.json
148 changes: 0 additions & 148 deletions .pnp.cjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file removed .yarn/cache/glur-npm-1.1.2-72a04dedc5-3d991f38cb.zip
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
4 changes: 0 additions & 4 deletions packages/browser-tests/.gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
node_modules
cypress/videos
cypress/screenshots

# folder is used to store screenshots and diffs created by `cy.matchImageSnapshot()`
# ignored now until the snapshots will be stored somewhere else
cypress/snapshots
12 changes: 0 additions & 12 deletions packages/browser-tests/README.md
Original file line number Diff line number Diff line change
@@ -19,15 +19,3 @@ If you want to interact with cypress, you can start it like so:
```
yarn workspace browser-tests run cypress open
```

## Screenshot matching

This package uses [simonsmith/cypress-image-snapshot](https://github.com/simonsmith/cypress-image-snapshot). It adds a `cy.matchImageSnapshot()` command to Cypress.

To take a screenshot while running tests, use `cy.matchImageSnapshot()`.
This will take a screenshot and save it in relevant location (depending
on test name) in `browser-tests/cypress/snapshots`.

On the next run, `cy.matchImageSnapshot()` will compare the screenshot with the one saved in `snapshots` and fail the test if they don't match.

If you want to update the screenshot, run `yarn workspace browser-tests test:update`
9 changes: 2 additions & 7 deletions packages/browser-tests/cypress.config.js
Original file line number Diff line number Diff line change
@@ -1,16 +1,13 @@
const { defineConfig } = require("cypress");
const {
addMatchImageSnapshotPlugin,
} = require("@simonsmith/cypress-image-snapshot/plugin");

const contextPath = process.env.QDB_HTTP_CONTEXT_WEB_CONSOLE || ""
const contextPath = process.env.QDB_HTTP_CONTEXT_WEB_CONSOLE || "";
const baseUrl = `http://localhost:9999${contextPath}`;

module.exports = defineConfig({
defaultCommandTimeout: 10000,
e2e: {
defaultCommandTimeout: 30000,
screenshotOnRunFailure: false,
screenshotOnRunFailure: true,
video: false,
baseUrl: baseUrl,
chromeWebSecurity: false, //if it is true, cypress does not allow redirects
@@ -19,8 +16,6 @@ module.exports = defineConfig({
specPattern: "cypress/integration/**/*.spec.js",
supportFile: "cypress/commands.js",
setupNodeEvents(on) {
addMatchImageSnapshotPlugin(on);

on("before:browser:launch", (browser = {}, launchOptions) => {
if (browser.family === "chromium" && browser.name !== "electron") {
launchOptions.args.push(
Loading
Oops, something went wrong.