New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the output directory configurable #45

Open
marktigno opened this Issue May 25, 2017 · 17 comments

Comments

5 participants
@marktigno

marktigno commented May 25, 2017

Hello Jan,

Is there a way to modify the report folder that can be defined using config file or a command-line argument? As of now after I execute the tests the results are placed on the default folder mentioned. I'm planning to execute a parallel test on different URLs (example website A and B) and create different types of report per URL. By the way serenity-js is a great product. Thanks! ^_^V

@marktigno

This comment has been minimized.

Show comment
Hide comment
@marktigno

marktigno May 26, 2017

It seems I've found it where the default location for the report is being set, is there a way this line here could be set using a config property or from a command line argument? Right now I'll try to find a way how to play with it.

image

marktigno commented May 26, 2017

It seems I've found it where the default location for the report is being set, is there a way this line here could be set using a config property or from a command line argument? Right now I'll try to find a way how to play with it.

image

@jan-molak

This comment has been minimized.

Show comment
Hide comment
@jan-molak

jan-molak May 26, 2017

Owner

Hey @marktigno! Yes, I wanted to make that configurable as soon as someone needs it :-)

By the way, are you running the exact same test against those two different URLs? If the test has the exact same title and steps, how do you know which website has a problem should that test fail?

Owner

jan-molak commented May 26, 2017

Hey @marktigno! Yes, I wanted to make that configurable as soon as someone needs it :-)

By the way, are you running the exact same test against those two different URLs? If the test has the exact same title and steps, how do you know which website has a problem should that test fail?

@nbarrett

This comment has been minimized.

Show comment
Hide comment
@nbarrett

nbarrett May 26, 2017

Contributor
Contributor

nbarrett commented May 26, 2017

@jan-molak

This comment has been minimized.

Show comment
Hide comment
@jan-molak

jan-molak May 27, 2017

Owner

Hey Nick! The identity of a scenario is determined based on the name of the feature, the title of the scenario and its position in the feature file.

Cucumber, which generates the SceneStarts event, has no knowledge of the Protractor browser nor its capabilities, so we'd need to find a better place to obtain and include that information.

Owner

jan-molak commented May 27, 2017

Hey Nick! The identity of a scenario is determined based on the name of the feature, the title of the scenario and its position in the feature file.

Cucumber, which generates the SceneStarts event, has no knowledge of the Protractor browser nor its capabilities, so we'd need to find a better place to obtain and include that information.

@marktigno

This comment has been minimized.

Show comment
Hide comment
@marktigno

marktigno May 30, 2017

Hello Jan! There will be two answers for that:
1.) Yes running the same set of tests on different URLs like UAT environment then prelive environment.
2.) Same environment but different test scenarios by executing using any of the following commands:
a.) npm run protractor -- --baseUrl="http://www.someurl.com/" --suite="smoketest"
b.) npm run protractor -- --baseUrl="http://www.someurl.com/" --cucumberOpts.tags="@regressiontest"

In that way serenity will generate results/reports based on the arguments set. As of now it seems serenity is pre-configured to store results on the "target/site/serenity" when the protractor is invoked during execution. It will be better if that will be configurable before protractor executes the tests and that would be very awesome especially running tests in parallel execution.

Reference: https://github.com/angular/protractor/blob/master/lib/config.ts

marktigno commented May 30, 2017

Hello Jan! There will be two answers for that:
1.) Yes running the same set of tests on different URLs like UAT environment then prelive environment.
2.) Same environment but different test scenarios by executing using any of the following commands:
a.) npm run protractor -- --baseUrl="http://www.someurl.com/" --suite="smoketest"
b.) npm run protractor -- --baseUrl="http://www.someurl.com/" --cucumberOpts.tags="@regressiontest"

In that way serenity will generate results/reports based on the arguments set. As of now it seems serenity is pre-configured to store results on the "target/site/serenity" when the protractor is invoked during execution. It will be better if that will be configurable before protractor executes the tests and that would be very awesome especially running tests in parallel execution.

Reference: https://github.com/angular/protractor/blob/master/lib/config.ts

@jan-molak

This comment has been minimized.

Show comment
Hide comment
@jan-molak

jan-molak May 31, 2017

Owner

Gotcha, I guess two things would be useful:

  • to configure the output directory
  • to include the information about the execution context when generating the unique hash of a given scenario (so that the url is included, for instance) - Addressed in @serenity-js/core 1.4.0
Owner

jan-molak commented May 31, 2017

Gotcha, I guess two things would be useful:

  • to configure the output directory
  • to include the information about the execution context when generating the unique hash of a given scenario (so that the url is included, for instance) - Addressed in @serenity-js/core 1.4.0

@jan-molak jan-molak changed the title from Is the target/site/serenity folder can be modified? to Make the output directory configurable May 31, 2017

@marktigno

This comment has been minimized.

Show comment
Hide comment
@marktigno

marktigno Sep 8, 2017

Addition feature also would be added if the report has archival like capabilities like report created on a specific time and date or some flexibility like:

target_folder = "someDir/location/" + reportDate.madeOn();

marktigno commented Sep 8, 2017

Addition feature also would be added if the report has archival like capabilities like report created on a specific time and date or some flexibility like:

target_folder = "someDir/location/" + reportDate.madeOn();

@jan-molak jan-molak added the ready label Sep 9, 2017

@jan-molak jan-molak closed this in 03b2842 Sep 26, 2017

@jan-molak jan-molak removed the ready label Sep 26, 2017

@jan-molak

This comment has been minimized.

Show comment
Hide comment
@jan-molak

jan-molak Sep 26, 2017

Owner

Hey @marktigno - please update to the latest version and let me know if it's all working for you.

Best,
Jan

Owner

jan-molak commented Sep 26, 2017

Hey @marktigno - please update to the latest version and let me know if it's all working for you.

Best,
Jan

@marktigno

This comment has been minimized.

Show comment
Hide comment
@marktigno

marktigno Oct 2, 2017

Hello @jan-molak!

I've updated to this version of SerenityJS and it seems the screenshots are still generated to the default directory target/site/serenity though I've followed what is written on your site on the configs but the report generated is blank.
image

Also I've find out on the PhotographySchedule class, the outputDirectory property does not exists as shown on this screenshot:
image

Thanks,
Mark

marktigno commented Oct 2, 2017

Hello @jan-molak!

I've updated to this version of SerenityJS and it seems the screenshots are still generated to the default directory target/site/serenity though I've followed what is written on your site on the configs but the report generated is blank.
image

Also I've find out on the PhotographySchedule class, the outputDirectory property does not exists as shown on this screenshot:
image

Thanks,
Mark

@marktigno

This comment has been minimized.

Show comment
Hide comment
@marktigno

marktigno Oct 2, 2017

Disregard the second screenshot on the comment above, I just rebuild the forked copy and it recognized the property now. I'll try some test to find out what's missing on my end. Thanks!

marktigno commented Oct 2, 2017

Disregard the second screenshot on the comment above, I just rebuild the forked copy and it recognized the property now. I'll try some test to find out what's missing on my end. Thanks!

@marktigno

This comment has been minimized.

Show comment
Hide comment
@marktigno

marktigno Oct 2, 2017

Hello @jan-molak,

It is confirmed on my end that the screenshots are still generated to the target/site/serenity directory including the generated .json result file.

Thanks!

marktigno commented Oct 2, 2017

Hello @jan-molak,

It is confirmed on my end that the screenshots are still generated to the target/site/serenity directory including the generated .json result file.

Thanks!

@jan-molak

This comment has been minimized.

Show comment
Hide comment
@jan-molak

jan-molak Oct 2, 2017

Owner

Hey @marktigno - I'm slightly confused, so is that the behaviour you expected, or is there a defect and we should re-open this ticket?

Owner

jan-molak commented Oct 2, 2017

Hey @marktigno - I'm slightly confused, so is that the behaviour you expected, or is there a defect and we should re-open this ticket?

@marktigno

This comment has been minimized.

Show comment
Hide comment
@marktigno

marktigno Oct 2, 2017

Hello @jan-molak,

Yup it's a defect. When I set my outputDirectory property to any specific directory (ex. ${process.cwd()}/some/output/path) I was expecting to generate the result there (screenshots and .json reports) but nothing is generated. It is still uses the default 'target/site/serenity' directory output.

marktigno commented Oct 2, 2017

Hello @jan-molak,

Yup it's a defect. When I set my outputDirectory property to any specific directory (ex. ${process.cwd()}/some/output/path) I was expecting to generate the result there (screenshots and .json reports) but nothing is generated. It is still uses the default 'target/site/serenity' directory output.

@marktigno

This comment has been minimized.

Show comment
Hide comment
@marktigno

marktigno Oct 11, 2017

By the way, I'm using Typescript 2.5.3 on my machine.

marktigno commented Oct 11, 2017

By the way, I'm using Typescript 2.5.3 on my machine.

@jan-molak jan-molak reopened this Oct 29, 2017

@jan-molak jan-molak added the ready label Oct 29, 2017

@Thopthes

This comment has been minimized.

Show comment
Hide comment
@Thopthes

Thopthes Nov 14, 2017

Got the same problem here. After some analysis the reason is the following:
Serenity class (serenity.ts) get config is called before configure is called in serenity_framework_config.ts
So the default value is used before the configured value overrides it.

Thopthes commented Nov 14, 2017

Got the same problem here. After some analysis the reason is the following:
Serenity class (serenity.ts) get config is called before configure is called in serenity_framework_config.ts
So the default value is used before the configured value overrides it.

@jan-molak

This comment has been minimized.

Show comment
Hide comment
@jan-molak

jan-molak Nov 15, 2017

Owner

Thanks for looking into that, @Thopthes! Let me know if you'd like to raise a PR for this - I'd be happy to help out.

Owner

jan-molak commented Nov 15, 2017

Thanks for looking into that, @Thopthes! Let me know if you'd like to raise a PR for this - I'd be happy to help out.

@webit4me

This comment has been minimized.

Show comment
Hide comment
@webit4me

webit4me Jan 26, 2018

Hi @jan-molak, @Thopthes, is there any progress on this?
Having this config available allow us to enable the reporting on our pipelines and I'm keen to contribute too, but unfortunately I even don't know where to begin

webit4me commented Jan 26, 2018

Hi @jan-molak, @Thopthes, is there any progress on this?
Having this config available allow us to enable the reporting on our pipelines and I'm keen to contribute too, but unfortunately I even don't know where to begin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment