New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seamlessly store test results from the build execution #47

Closed
bartoszmajsak opened this Issue Jun 30, 2017 · 6 comments

Comments

Projects
None yet
4 participants
@bartoszmajsak
Member

bartoszmajsak commented Jun 30, 2017

failed strategy relies on historical build execution. At this point we only support local experience. It has an assumption that reports from previous test run are stored in the .report folder. We don't have however any automated way of getting those reports there other than user doing it manually.

This task should automate this process in the surefire-provider. Every test result file should be moved to this folder. We should also purge the folder after we gather test results data from the previous build, as we will write new ones. I'm open here to other ideas though, as this is suboptimal. Maybe subfolder per build would be better?

We should also implement it such that plugging in publisher of test reports to the "hub" would be a matter of adding it as a dependency. Good candidate for SPI?

@bartoszmajsak

This comment has been minimized.

Show comment
Hide comment
@bartoszmajsak

bartoszmajsak Jun 30, 2017

Member

Should we make the location folder configurable as part of this story or a follow-up one?

Member

bartoszmajsak commented Jun 30, 2017

Should we make the location folder configurable as part of this story or a follow-up one?

@bartoszmajsak bartoszmajsak added this to the 0.0.1 milestone Jun 30, 2017

@lordofthejars

This comment has been minimized.

Show comment
Hide comment
@lordofthejars

lordofthejars Jun 30, 2017

Member

We can do it in this issue. Also when you say purge you mean based on created time, make a configurable strategy, ....

Member

lordofthejars commented Jun 30, 2017

We can do it in this issue. Also when you say purge you mean based on created time, make a configurable strategy, ....

@bartoszmajsak

This comment has been minimized.

Show comment
Hide comment
@bartoszmajsak

bartoszmajsak Jun 30, 2017

Member

based on created time, make a configurable strategy

I think we are going to far. Let's have one good solution for the local scenario first and then think if we really need to open it up for configurable strategies.

Member

bartoszmajsak commented Jun 30, 2017

based on created time, make a configurable strategy

I think we are going to far. Let's have one good solution for the local scenario first and then think if we really need to open it up for configurable strategies.

@bartoszmajsak

This comment has been minimized.

Show comment
Hide comment
@bartoszmajsak

bartoszmajsak Jun 30, 2017

Member

Another question here would be how do we want to store this data in the long run. Is the junit-xml format sufficient enough? Can we just generate stubs based on the junit-xsd and use it as a part of our solution? Especially considering transferring it over the wire? Or do we want to have our own structure tailored to our needs? What that would mean in terms of supporting stuff beyond surefire?

Member

bartoszmajsak commented Jun 30, 2017

Another question here would be how do we want to store this data in the long run. Is the junit-xml format sufficient enough? Can we just generate stubs based on the junit-xsd and use it as a part of our solution? Especially considering transferring it over the wire? Or do we want to have our own structure tailored to our needs? What that would mean in terms of supporting stuff beyond surefire?

@MatousJobanek

This comment has been minimized.

Show comment
Hide comment
@MatousJobanek

MatousJobanek Jun 30, 2017

Contributor

Another area to investigate is getting the information (to be stored) using Surefire/JUnit/TestNg API - using RunResult object that we get in the provider:
and/or JUnit/Surefire listeners

As for local storing - I would do it in similar way as Surefire does for its ordering - have one file that changes during the builds

Contributor

MatousJobanek commented Jun 30, 2017

Another area to investigate is getting the information (to be stored) using Surefire/JUnit/TestNg API - using RunResult object that we get in the provider:
and/or JUnit/Surefire listeners

As for local storing - I would do it in similar way as Surefire does for its ordering - have one file that changes during the builds

@dipak-pawar

This comment has been minimized.

Show comment
Hide comment
@dipak-pawar

dipak-pawar Sep 4, 2017

Contributor

Few options to think:

  • Store only failed build results or all?
  • Store only last build execution result or n subsequent build execution result.
Contributor

dipak-pawar commented Sep 4, 2017

Few options to think:

  • Store only failed build results or all?
  • Store only last build execution result or n subsequent build execution result.

@MatousJobanek MatousJobanek self-assigned this Sep 8, 2017

bartoszmajsak added a commit that referenced this issue Sep 13, 2017

feat(#47): surefire reports are automatically copied in our maven ext…
…ension (#134)

The surefire reports are automatically copied to the directory `.reports` in our maven extension. These directories are removed when the build is finished
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment