Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test PR for testing new Dockerized testing #233

Open
wants to merge 52 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
83e104e
Updated tests:e2e script for compatibility with new dockerized testin…
Jurredr Jan 27, 2025
4268066
Modified e2e docker script and added local e2e testing script
Jurredr Jan 27, 2025
9be18d0
Added dockerfile for E2E testing
Jurredr Jan 27, 2025
a8bd82d
Added trace npm scripts for Playwright tracing
Jurredr Jan 27, 2025
e8382d9
Renamed docker image and container names
Jurredr Jan 28, 2025
6738907
Updated node version in Dockerfile
Jurredr Jan 28, 2025
f104dc8
Added separate npm script for building docker image
Jurredr Jan 28, 2025
3a63184
Changed trace mode to upload traces for each failed test
Jurredr Jan 28, 2025
c4aebd8
Added JUnit xml reporter to Playwright
Jurredr Jan 28, 2025
0aae515
Updated Docker run command
Jurredr Jan 28, 2025
8a2b3f1
Added command to inspect Docker E2E test results
Jurredr Jan 28, 2025
1a9bf77
Changed docker inspect target name to "flarechatui-e2e-container"
Jurredr Jan 28, 2025
f04f641
Added docker clean script / step to container runner
Jurredr Jan 28, 2025
a21663d
Removed test results / last run file from default ui-tests dir
Jurredr Jan 28, 2025
0ae6c09
Added JUnit reporter for Playwright
Jurredr Jan 29, 2025
bffafdc
Migrated jest-playwright to playwright/test
Jurredr Jan 29, 2025
438608b
Added test reports to git ignore
Jurredr Jan 30, 2025
b5f193f
Generate all test results in one separate root directory
Jurredr Jan 30, 2025
9ee2bda
Ignore Dockerfile from eslint
Jurredr Jan 30, 2025
ba9a8aa
Merge branch 'main' into jurredr/dockerized-testing
Jurredr Jan 30, 2025
23b144a
Use npm install instead of ci in docker image
Jurredr Jan 30, 2025
a5acfea
Test fixes for playwright library migration
Jurredr Jan 30, 2025
119bb9b
Removed all existing image snapshots
Jurredr Feb 3, 2025
1775fbb
Updated dockerfile to only include relevant files and folders in dock…
Jurredr Feb 3, 2025
0633e6b
Implemented new GitHub CI script for E2E testing through Docker
Jurredr Feb 3, 2025
623fbb4
Ignore e2e-results folder from eslint
Jurredr Feb 3, 2025
af44e63
ci: only send slack message on non-draft prs
Jurredr Feb 4, 2025
5e3675d
Fixed GitHub new_pr workflow, only runs for non-draft PRs properly now
Jurredr Feb 4, 2025
cb221ed
Fixed non-draft PR check in GitHub workflow
Jurredr Feb 4, 2025
be5bae0
Merge branch 'main' into jurredr/dockerized-testing
Jurredr Feb 4, 2025
c5913b9
Merge branch 'main' into jurredr/dockerized-testing
Jurredr Feb 4, 2025
e4d226c
Made all individual E2E tests run on their own
Jurredr Feb 5, 2025
1bca731
Added and tested form tests to work individually with new E2E flow
Jurredr Feb 5, 2025
dcb6ddf
Fully run all tests in parallel
Jurredr Feb 5, 2025
3fad9f3
Fixed all existing test flows to work with new parallel runner
Jurredr Feb 5, 2025
2c5583e
Remove failure check on uploading of artifacts in new_pr test workflow
Jurredr Feb 11, 2025
9eec00e
Extracted e2e testing to separate workflow, added dockerized E2E step…
Jurredr Feb 11, 2025
67bea9b
Temporarily disabled slack messager for new prs while testing e2e wor…
Jurredr Feb 11, 2025
ef60fd0
Attempt at fixing linting errors
Jurredr Feb 11, 2025
069a00b
Added build step before linting
Jurredr Feb 11, 2025
87151df
New lint fix attempt
Jurredr Feb 11, 2025
0e6f7b2
Fixed incorrect npm install command in e2e build step
Jurredr Feb 11, 2025
ad6321a
e2e build fix
Jurredr Feb 11, 2025
361080a
Force Playwright workers to be 6
Jurredr Feb 11, 2025
6b448da
continue e2e workflow on test suite error
Jurredr Feb 11, 2025
26e985c
Merged main into dockerized testing branch
Jurredr Mar 4, 2025
41aa55e
Updated test result reporter type to jest-junit
Jurredr Mar 4, 2025
36e37a0
Removed commented code
Jurredr Mar 4, 2025
e4e600a
Removed console log from e2e test
Jurredr Mar 4, 2025
59373dd
Updated incorrect ui-test result paths
Jurredr Mar 4, 2025
785be97
Specify branch for snapshot replacement commit, change test reporter …
Jurredr Mar 4, 2025
fc46244
Ignore zip and xml files from eslint
Jurredr Mar 4, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Test fixes for playwright library migration
  • Loading branch information
Jurredr committed Jan 30, 2025
commit a5acfea2e559ee82b6394e5a1bbdc64390456d86
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -10,4 +10,5 @@ node_modules
package-lock.json
.gitcommit
.vscode
__reports__/
__reports__/
e2e-results/
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Page } from 'playwright/test';
import { expect, Page } from 'playwright/test';
import { getSelector, waitForAnimationEnd } from '../../helpers';
import testIds from '../../../../src/helper/test-ids';

@@ -9,6 +9,6 @@ export const cancelFeedbackForm = async (page: Page, skipScreenshots?: boolean):
await waitForAnimationEnd(page);

if (skipScreenshots !== true) {
expect(await page.screenshot()).toMatchImageSnapshot();
expect(await page.screenshot()).toMatchSnapshot();
}
};
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import { Page } from 'playwright/test';
import { expect, Page } from 'playwright/test';
import { getSelector, waitForAnimationEnd } from '../../helpers';
import testIds from '../../../../src/helper/test-ids';

export const renderDownvoteResult = async (page: Page, skipScreenshots?: boolean): Promise<void> => {
await page.evaluate((body) => {
await page.evaluate(() => {
const selectedTabId = window.mynahUI.getSelectedTabId();
if (selectedTabId != null) {
window.mynahUI.updateStore(selectedTabId, {
@@ -26,6 +26,6 @@ export const renderDownvoteResult = async (page: Page, skipScreenshots?: boolean
await waitForAnimationEnd(page);

if (skipScreenshots !== true) {
expect(await page.screenshot()).toMatchImageSnapshot();
expect(await page.screenshot()).toMatchSnapshot();
}
};
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Page } from 'playwright/test';
import { expect, Page } from 'playwright/test';
import { getSelector, waitForAnimationEnd } from '../../helpers';
import testIds from '../../../../src/helper/test-ids';

@@ -41,6 +41,6 @@ export const renderFeedbackForm = async (page: Page, skipScreenshots?: boolean):
expect(cancelButton).toBeDefined();

if (skipScreenshots !== true) {
expect(await page.screenshot()).toMatchImageSnapshot();
expect(await page.screenshot()).toMatchSnapshot();
}
};
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import { Page } from 'playwright/test';
import { expect, Page } from 'playwright/test';
import { getSelector, waitForAnimationEnd } from '../../helpers';
import testIds from '../../../../src/helper/test-ids';

export const renderUpvoteResult = async (page: Page, skipScreenshots?: boolean): Promise<void> => {
await page.evaluate((body) => {
await page.evaluate(() => {
const selectedTabId = window.mynahUI.getSelectedTabId();
if (selectedTabId != null) {
window.mynahUI.updateStore(selectedTabId, {
@@ -26,6 +26,6 @@ export const renderUpvoteResult = async (page: Page, skipScreenshots?: boolean):
await waitForAnimationEnd(page);

if (skipScreenshots !== true) {
expect(await page.screenshot()).toMatchImageSnapshot();
expect(await page.screenshot()).toMatchSnapshot();
}
};
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Page } from 'playwright/test';
import { expect, Page } from 'playwright/test';
import { waitForAnimationEnd } from '../../helpers';
import { openNewTab } from '../open-new-tab';
import { closeTab } from '../close-tab';
@@ -7,7 +7,7 @@ export const renderVoteButtons = async (page: Page, skipScreenshots?: boolean):
await closeTab(page, false, true);
await openNewTab(page, false, true);

await page.evaluate((body) => {
await page.evaluate(() => {
const selectedTabId = window.mynahUI.getSelectedTabId();
if (selectedTabId != null) {
window.mynahUI.updateStore(selectedTabId, {
@@ -25,6 +25,6 @@ export const renderVoteButtons = async (page: Page, skipScreenshots?: boolean):
await waitForAnimationEnd(page);

if (skipScreenshots !== true) {
expect(await page.screenshot()).toMatchImageSnapshot();
expect(await page.screenshot()).toMatchSnapshot();
}
};
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Page } from 'playwright/test';
import { expect, Page } from 'playwright/test';
import { getSelector, waitForAnimationEnd } from '../../helpers';
import testIds from '../../../../src/helper/test-ids';

@@ -35,6 +35,6 @@ export const submitFeedbackForm = async (page: Page, skipScreenshots?: boolean):
await waitForAnimationEnd(page);

if (skipScreenshots !== true) {
expect(await page.screenshot()).toMatchImageSnapshot();
expect(await page.screenshot()).toMatchSnapshot();
}
};
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Page } from 'playwright/test';
import { expect, Page } from 'playwright/test';
import { getSelector, waitForAnimationEnd } from '../../helpers';
import testIds from '../../../../src/helper/test-ids';
import { closeTab } from '../close-tab';
@@ -35,6 +35,6 @@ export const navigateBackToCurrentPromptWithCodeAttachment = async (page: Page,
expect(codeAttachmentContent).toBe('This is an unsent code attachment');

if (skipScreenshots !== true) {
expect(await page.screenshot()).toMatchImageSnapshot();
expect(await page.screenshot()).toMatchSnapshot();
}
};
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Page } from 'playwright/test';
import { expect, Page } from 'playwright/test';
import { getSelector, waitForAnimationEnd } from '../../helpers';
import testIds from '../../../../src/helper/test-ids';
import { closeTab } from '../close-tab';
@@ -27,6 +27,6 @@ export const navigateBackToCurrentPrompt = async (page: Page, skipScreenshots?:
expect(await promptInput.inputValue()).toBe('This is the second unsent user prompt');

if (skipScreenshots !== true) {
expect(await page.screenshot()).toMatchImageSnapshot();
expect(await page.screenshot()).toMatchSnapshot();
}
};
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Page } from 'playwright/test';
import { expect, Page } from 'playwright/test';
import { getSelector, waitForAnimationEnd } from '../../helpers';
import testIds from '../../../../src/helper/test-ids';
import { closeTab } from '../close-tab';
@@ -29,6 +29,6 @@ export const navigatePromptsDown = async (page: Page, skipScreenshots?: boolean)
expect(await promptInput.inputValue()).toBe('This is the second user prompt');

if (skipScreenshots !== true) {
expect(await page.screenshot()).toMatchImageSnapshot();
expect(await page.screenshot()).toMatchSnapshot();
}
};
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Page } from 'playwright/test';
import { expect, Page } from 'playwright/test';
import { getSelector, waitForAnimationEnd } from '../../helpers';
import testIds from '../../../../src/helper/test-ids';
import { closeTab } from '../close-tab';
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Page } from 'playwright/test';
import { expect, Page } from 'playwright/test';
import { getSelector, waitForAnimationEnd } from '../../helpers';
import testIds from '../../../../src/helper/test-ids';
import { closeTab } from '../close-tab';
@@ -22,6 +22,6 @@ export const navigatePromptsToEmpty = async (page: Page, skipScreenshots?: boole
expect(await promptInput.inputValue()).toBe('');

if (skipScreenshots !== true) {
expect(await page.screenshot()).toMatchImageSnapshot();
expect(await page.screenshot()).toMatchSnapshot();
}
};
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// navigatePromptsUp.ts
import { Page } from 'playwright/test';
import { expect, Page } from 'playwright/test';
import { getSelector, waitForAnimationEnd } from '../../helpers';
import testIds from '../../../../src/helper/test-ids';
import { closeTab } from '../close-tab';
@@ -20,6 +20,6 @@ export const navigatePromptsUp = async (page: Page, skipScreenshots?: boolean):
expect(await promptInput.inputValue()).toBe('This is the first user prompt');

if (skipScreenshots !== true) {
expect(await page.screenshot()).toMatchImageSnapshot();
expect(await page.screenshot()).toMatchSnapshot();
}
};
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Page } from 'playwright/test';
import { expect, Page } from 'playwright/test';
import { getSelector, waitForAnimationEnd } from '../../helpers';
import testIds from '../../../../src/helper/test-ids';
import { closeTab } from '../close-tab';
@@ -22,6 +22,6 @@ export const stayOnCurrentPrompt = async (page: Page, skipScreenshots?: boolean)
expect(await promptInput.inputValue()).toBe('This is the first unsent user prompt');

if (skipScreenshots !== true) {
expect(await page.screenshot()).toMatchImageSnapshot();
expect(await page.screenshot()).toMatchSnapshot();
}
};
22 changes: 11 additions & 11 deletions ui-tests/__test__/main.spec.ts
Original file line number Diff line number Diff line change
@@ -211,40 +211,40 @@ test.describe('Open MynahUI', () => {
test('should navigate down to current empty prompt', async ({ page }) => {
await navigatePromptsToEmpty(page);
});
it('should navigate up/down only if on first/last line', async () => {
test('should navigate up/down only if on first/last line', async ({ page }) => {
await navigatePromptsFirstLastLineCheck(page);
});

it('should stay on current prompt', async () => {
test('should stay on current prompt', async ({ page }) => {
await stayOnCurrentPrompt(page);
});

it('should navigate back to current prompt', async () => {
test('should navigate back to current prompt', async ({ page }) => {
await navigateBackToCurrentPrompt(page);
});

it('should navigate back to current prompt with code attachment', async () => {
test('should navigate back to current prompt with code attachment', async ({ page }) => {
await navigateBackToCurrentPromptWithCodeAttachment(page);
});
});

describe('Feedback form', () => {
it('should render vote buttons', async () => {
test.describe('Feedback form', () => {
test('should render vote buttons', async ({ page }) => {
await renderVoteButtons(page);
});
it('should render upvote results', async () => {
test('should render upvote results', async ({ page }) => {
await renderUpvoteResult(page);
});
it('should render downvote results', async () => {
test('should render downvote results', async ({ page }) => {
await renderDownvoteResult(page);
});
it('should render feedback form', async () => {
test('should render feedback form', async ({ page }) => {
await renderFeedbackForm(page);
});
it('should cancel feedback form', async () => {
test('should cancel feedback form', async ({ page }) => {
await cancelFeedbackForm(page);
});
it('should submit feedback form', async () => {
test('should submit feedback form', async ({ page }) => {
await submitFeedbackForm(page);
});
});