Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Define metrics for Fn test and capture duration #174

Closed
gabog opened this issue Nov 13, 2020 · 1 comment
Closed

Define metrics for Fn test and capture duration #174

gabog opened this issue Nov 13, 2020 · 1 comment
Assignees
@denscollo
Labels
Area: Functional tests The issue is related to end to end tests feature-request A request for new functionality or an enhancement to an existing one. P1 Painful if we don't fix, won't block releasing Size: M The issue is not very complex and it is well understood, it will take 1 to 3 days to complete
Projects
Bot Framework SDK - Functional Tests
Milestone
R13

Comments

@gabog
Copy link
Contributor

gabog commented Nov 13, 2020 •
edited

Some things we need to know:
How long does it take to deploy?
How long does it take to build?
How many tests were run?
How long each test takes?
How long it takes to reconfigure?
Which ones are the slowest tests?
What combination of tests failed? (good error reporting)

Notes:
number pipeline banes in azure devops so they sort better
why can't a see stack traces and failure messages in failure reports?
where do I got to see pipeline failures?
@gabog gabog added feature-request A request for new functionality or an enhancement to an existing one. P0 Must Fix. Release-blocker Area: Functional tests The issue is related to end to end tests Size: M The issue is not very complex and it is well understood, it will take 1 to 3 days to complete labels Nov 13, 2020
@gabog gabog added this to To Be Groomed in Bot Framework SDK - Functional Tests via automation Nov 13, 2020
@gabog gabog added this to the R12 milestone Nov 13, 2020
@JuanAr JuanAr assigned denscollo and unassigned JuanAr Nov 13, 2020
@gabog gabog moved this from To Be Groomed to R12 Issues in Bot Framework SDK - Functional Tests Nov 19, 2020
@gabog gabog added P1 Painful if we don't fix, won't block releasing and removed P0 Must Fix. Release-blocker labels Dec 4, 2020
@gabog gabog modified the milestones: R12, R13 Feb 3, 2021
@gabog
Copy link
Contributor Author

gabog commented Feb 25, 2021

@JuanAr, @denscollo and @mrivera-ms, after reviewing the Azure DevOps in more detail after our call today, I think we have the information to answer the questions stated in this issue, I was even able to see detailed failure messages in the tests drilldown.

I am closing this one then and will open separate ones if we need additional reporting once we have more tests.

@gabog gabog closed this as completed Feb 25, 2021
Bot Framework SDK - Functional Tests automation moved this from R13 Issues to Closed Feb 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@denscollo denscollo

Labels
Area: Functional tests The issue is related to end to end tests feature-request A request for new functionality or an enhancement to an existing one. P1 Painful if we don't fix, won't block releasing Size: M The issue is not very complex and it is well understood, it will take 1 to 3 days to complete
Projects
Bot Framework SDK - Functional Tests
Closed
Milestone
R13
Development

No branches or pull requests
3 participants
@JuanAr @gabog @denscollo