New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Join forces #5

Open
christian-bromann opened this Issue Jun 6, 2016 · 6 comments

Comments

Projects
None yet
3 participants
@christian-bromann

Hi,

I stumbled upon this great project and wondering if you would like to join forces and merge this into the webdrivercss project?

What do you think about that?

Cheers

@zinserjan

This comment has been minimized.

Show comment
Hide comment
@zinserjan

zinserjan Jun 6, 2016

Owner

I have to think about to merge everything into webdrivercss. Currently, I think it's better to seperate webdrivercss into multiple packages as suggested in webdriverio-boneyard/webdrivercss#113.

Webdrivercss does at the moment many different things:

  • captures screenshots
  • compares screenshots locally or remotly
  • syncs screenshots

I tried this transformation with wdio-screenshot and wdio-visual-regression-service.
wdio-screenshot is just for taking screenshots, wdio-visual-regression-service is for testing of all kinds of elements in different resolutions with any kind of comparison method. At the moment there's only a local compare algorithm but it's easy to build a custom compare method. For an example have a look at the tests.

Anyway, I'm with you that the projects will definitely benefit from each other.

What do you think about the separation into multiple packages? Are there any drawbacks?

Owner

zinserjan commented Jun 6, 2016

I have to think about to merge everything into webdrivercss. Currently, I think it's better to seperate webdrivercss into multiple packages as suggested in webdriverio-boneyard/webdrivercss#113.

Webdrivercss does at the moment many different things:

  • captures screenshots
  • compares screenshots locally or remotly
  • syncs screenshots

I tried this transformation with wdio-screenshot and wdio-visual-regression-service.
wdio-screenshot is just for taking screenshots, wdio-visual-regression-service is for testing of all kinds of elements in different resolutions with any kind of comparison method. At the moment there's only a local compare algorithm but it's easy to build a custom compare method. For an example have a look at the tests.

Anyway, I'm with you that the projects will definitely benefit from each other.

What do you think about the separation into multiple packages? Are there any drawbacks?

@meleyal meleyal referenced this issue Jun 16, 2016

Closed

4.0 support? #156

@christian-bromann

This comment has been minimized.

Show comment
Hide comment
@christian-bromann

christian-bromann Sep 14, 2016

Sorry for getting back to you so late.

What do you think about the separation into multiple packages?

Yeah this is the plan. Ultimately it would be great to see similar modular structure like in WebdriverIO where you can apply reporters and service as you want. For example let's have data store addons to save images on the file system, on dropbox or any other arbitrary server. A different type of addon could be the comparison mechanism. Options are pixel by pixel, gm and applitools. Each one of these plugins could live in its own repository.

Here is a great graphic @TheSavior did once: http://slides.com/eliwhite/visualtesting#/14

Question is: do we want to have another organization for webdrivercss or is it ok to put it there? Also is wdio-screenshot only the screenshot taker and webdrivercss should manage this whole plugin system? What do you think about this plugin system?

Sorry for getting back to you so late.

What do you think about the separation into multiple packages?

Yeah this is the plan. Ultimately it would be great to see similar modular structure like in WebdriverIO where you can apply reporters and service as you want. For example let's have data store addons to save images on the file system, on dropbox or any other arbitrary server. A different type of addon could be the comparison mechanism. Options are pixel by pixel, gm and applitools. Each one of these plugins could live in its own repository.

Here is a great graphic @TheSavior did once: http://slides.com/eliwhite/visualtesting#/14

Question is: do we want to have another organization for webdrivercss or is it ok to put it there? Also is wdio-screenshot only the screenshot taker and webdrivercss should manage this whole plugin system? What do you think about this plugin system?

@TheSavior

This comment has been minimized.

Show comment
Hide comment
@TheSavior

TheSavior Sep 15, 2016

Wow, cool to see people are still talking about this.

I actually built the project that follows the flow chart that @christian-bromann linked to. All the pieces live here: https://github.com/thea-diffing

It would be great to figure out how to work together to build the platform we all want to see.

Wow, cool to see people are still talking about this.

I actually built the project that follows the flow chart that @christian-bromann linked to. All the pieces live here: https://github.com/thea-diffing

It would be great to figure out how to work together to build the platform we all want to see.

@zinserjan

This comment has been minimized.

Show comment
Hide comment
@zinserjan

zinserjan Sep 23, 2016

Owner

Yeah this is the plan. Ultimately it would be great to see similar modular structure like in WebdriverIO where you can apply reporters and service as you want. For example let's have data store addons to save images on the file system, on dropbox or any other arbitrary server. A different type of addon could be the comparison mechanism. Options are pixel by pixel, gm and applitools. Each one of these plugins could live in its own repository.

That's what I already tried with wdio-visual-regression-service. It's responsible for taking screenshots with wdio-screenshot in different resolutions and provides a mechanism which I called compare methods to control the storage and diffing of screenshots.

Such a method is just a simple implementation of 4 different hooks:

  • before
  • beforeScreenshot
  • afterScreenshot
  • after

At the moment I just implemented a LocalCompare method which just stores screenshots locally and compares them against some reference shots.

Question is: do we want to have another organization for webdrivercss or is it ok to put it there?

What do mean with "put it there"? WebdriverIO?

Also is wdio-screenshot only the screenshot taker and webdrivercss should manage this whole plugin system?

That was my plan. My personal opinion is that the screenshot taking functionality is so complex that it should exist on it's own. This makes it also easier to use it for other projects, too.

Owner

zinserjan commented Sep 23, 2016

Yeah this is the plan. Ultimately it would be great to see similar modular structure like in WebdriverIO where you can apply reporters and service as you want. For example let's have data store addons to save images on the file system, on dropbox or any other arbitrary server. A different type of addon could be the comparison mechanism. Options are pixel by pixel, gm and applitools. Each one of these plugins could live in its own repository.

That's what I already tried with wdio-visual-regression-service. It's responsible for taking screenshots with wdio-screenshot in different resolutions and provides a mechanism which I called compare methods to control the storage and diffing of screenshots.

Such a method is just a simple implementation of 4 different hooks:

  • before
  • beforeScreenshot
  • afterScreenshot
  • after

At the moment I just implemented a LocalCompare method which just stores screenshots locally and compares them against some reference shots.

Question is: do we want to have another organization for webdrivercss or is it ok to put it there?

What do mean with "put it there"? WebdriverIO?

Also is wdio-screenshot only the screenshot taker and webdrivercss should manage this whole plugin system?

That was my plan. My personal opinion is that the screenshot taking functionality is so complex that it should exist on it's own. This makes it also easier to use it for other projects, too.

@christian-bromann

This comment has been minimized.

Show comment
Hide comment
@christian-bromann

christian-bromann Sep 23, 2016

That's what I already tried with wdio-visual-regression-service.

This service only works if you run it with the wdio testrunner. Something that would be great is that we provide VRT functionality for standalone as well as for testrunner mode. This would allow 3rd party libraries like Spectron or Concept to leverage from it. I would almost say that the service could be the next version of WebdriverCSS if we change it to work as a plugin not as a service. However I don't want to do that if you don't want that. I just think that more people know WebdriverCSS as that VRT tool for WebdriverIO.

What do mean with "put it there"? WebdriverIO?

I believe we get more user traktion if we have a GitHub organization for all plugins and tools around VRT with WebdriverIO. Maybe we can just keep it under the WebdriverIO org hat. But like I said, if you would rather want continue this under your account that is fine too.

the screenshot taking functionality is so complex

I absolutely agree. One of the reason why I stand off of it. It took too much time for me to fix all the issues especially on mobile.

That's what I already tried with wdio-visual-regression-service.

This service only works if you run it with the wdio testrunner. Something that would be great is that we provide VRT functionality for standalone as well as for testrunner mode. This would allow 3rd party libraries like Spectron or Concept to leverage from it. I would almost say that the service could be the next version of WebdriverCSS if we change it to work as a plugin not as a service. However I don't want to do that if you don't want that. I just think that more people know WebdriverCSS as that VRT tool for WebdriverIO.

What do mean with "put it there"? WebdriverIO?

I believe we get more user traktion if we have a GitHub organization for all plugins and tools around VRT with WebdriverIO. Maybe we can just keep it under the WebdriverIO org hat. But like I said, if you would rather want continue this under your account that is fine too.

the screenshot taking functionality is so complex

I absolutely agree. One of the reason why I stand off of it. It took too much time for me to fix all the issues especially on mobile.

@zinserjan

This comment has been minimized.

Show comment
Hide comment
@zinserjan

zinserjan Sep 23, 2016

Owner

Something that would be great is that we provide VRT functionality for standalone as well as for testrunner mode. ... I would almost say that the service could be the next version of WebdriverCSS if we change it to work as a plugin not as a service.

Sounds good to me.

I believe we get more user traktion if we have a GitHub organization for all plugins and tools around VRT with WebdriverIO.

Makes sense. I'm definitely open for this.

Question is: Does it make sense to create another organization for this? Another organization just for regression testing gives us probably a better overview. But this dependents on the planned features/plugins. I think we can keep everything as it is until the roadmap is clear.

As next step we should start collecting some ideas. Don't know what's the best way to do this.

Owner

zinserjan commented Sep 23, 2016

Something that would be great is that we provide VRT functionality for standalone as well as for testrunner mode. ... I would almost say that the service could be the next version of WebdriverCSS if we change it to work as a plugin not as a service.

Sounds good to me.

I believe we get more user traktion if we have a GitHub organization for all plugins and tools around VRT with WebdriverIO.

Makes sense. I'm definitely open for this.

Question is: Does it make sense to create another organization for this? Another organization just for regression testing gives us probably a better overview. But this dependents on the planned features/plugins. I think we can keep everything as it is until the roadmap is clear.

As next step we should start collecting some ideas. Don't know what's the best way to do this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment