Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BCPT Test Report #619

Open
wants to merge 130 commits into
base: main
Choose a base branch
from
Open

BCPT Test Report #619

wants to merge 130 commits into from

Conversation

freddydk
Copy link
Collaborator

@freddydk freddydk commented Jul 24, 2023

This PR will add a BCPT Test Report to the build summary, looking like this:
image

You can also add a bcptBaseLine.json to the project in order to establish a baseline for the performance tests.
It looks like this:
image

TODOs:

  • Add tests
  • Add thresholds to project settings
  • Determine how thresholds should work? threshold on very small items like (enter account no.) doesn't make much sense. It absolutely makes sense to have threshold on scenarios.
  • Determine sorting of test results? (codeunitID, codeunitName or ???)
  • Issue GitHub warnings and errors when thresholds are exceeded
  • Is durationMin in milliseconds, seconds or what? how many decimal digits should be displayed?
  • Get BCPT Backend Team signoff that BCPT Test Results are correctly understood and compared
  • Add scenario documentation

Example of bcpt tests with failures and warnings:
image

@freddydk freddydk changed the title BCPT Test Report [DRAFT] BCPT Test Report Jul 24, 2023
@freddydk freddydk marked this pull request as ready for review May 16, 2024 10:52
@freddydk freddydk requested a review from a team as a code owner May 16, 2024 10:52
@freddydk freddydk changed the title [DRAFT] BCPT Test Report BCPT Test Report May 16, 2024
Copy link
Contributor

@aholstrup1 aholstrup1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good I think :) It's a nice improvement

Actions/AnalyzeTests/AnalyzeTests.ps1 Show resolved Hide resolved
Actions/AnalyzeTests/AnalyzeTests.ps1 Outdated Show resolved Hide resolved
Actions/AnalyzeTests/TestResultAnalyzer.ps1 Outdated Show resolved Hide resolved
Actions/AnalyzeTests/TestResultAnalyzer.ps1 Outdated Show resolved Hide resolved
Actions/AnalyzeTests/TestResultAnalyzer.ps1 Outdated Show resolved Hide resolved
Actions/AnalyzeTests/TestResultAnalyzer.ps1 Outdated Show resolved Hide resolved
Actions/AnalyzeTests/TestResultAnalyzer.ps1 Show resolved Hide resolved
RELEASENOTES.md Show resolved Hide resolved
Scenarios/AddAPerformanceTestApp.md Outdated Show resolved Hide resolved
Scenarios/AddAPerformanceTestApp.md Show resolved Hide resolved
freddydk and others added 2 commits May 16, 2024 16:13
Co-authored-by: Alexander Holstrup <117829001+aholstrup1@users.noreply.github.com>
@freddydk freddydk requested a review from aholstrup1 May 16, 2024 14:19
mazhelez
mazhelez previously approved these changes Jun 3, 2024
Copy link
Collaborator

@mazhelez mazhelez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks okay to me.

Should there be any part of the E2E tests that check any of the new functionality?

Scenarios/AddAPerformanceTestApp.md Outdated Show resolved Hide resolved
Actions/AnalyzeTests/TestResultAnalyzer.ps1 Show resolved Hide resolved
Co-authored-by: Maria Zhelezova <43066499+mazhelez@users.noreply.github.com>
@freddydk freddydk requested a review from mazhelez June 7, 2024 08:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BCPT Test Results Viewer
3 participants