-
Notifications
You must be signed in to change notification settings - Fork 603
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BACKSTOP CHROME-HEADLESS BETA VERSION IS HERE! #479
Comments
Hi Garris, So happy to see Chromy support! Thanks for making this happen 🙌🏻 I've just grabbed
Any specific config that would be slowing this down? |
That's really great news. But about onReady.js and onBefore.js files I'm not sure if it's possible to convert those or how :) BTW, I noticed beta branch on github doesn't have chromy updates. |
@daveaspinall I am surprised to hear that -- in my environment, running the backstop-feature-tests it is the opposite... could you open a ticket and include console output for both the chrome and the phantom versions? Thanks! |
@garris sure no problem 🙂 The page it's testing does have quite a few selectors (~52 elements), but there must be something with my setup as it took over 20mins and still hadn't finished 🙈 |
@mirzazeyrek -- yes, as I mentioned -- the scripts are totally different and I don't know yet if there is an easy way to automatically convert or polyfill. That said, please check out this test repo https://github.com/garris/backstop-feature-tests/ for examples of setting cookies and manipulating DOM. We have already converted some complex scripts at work and did not find it to be too difficult. @dotneet has been very helpful with questions and feature tweaks as well. I am working on documentation now and will probably help with chromy documentation where I can. |
@daveaspinall I see. please post your example -- I think there is a selector expansion feature optimization I can add which would significantly improve performance for that many selectors in one scenario. |
@garris this is amazing news! We'll give it a try tomorrow! |
Sharp dependency issue for BackstopJS. I've raised this issue with Sharp lovell/sharp#879 but mentioning here too. Unable to install sharp on Amazon Linux AMI (4.9.17-8.31.amzn1.x86_64) related release notes: Steps to reproduce:
More info: |
@kiran-redhat I also experienced that from time to time on mac. The problem here is from time to time - so sometimes after rerunning it it went through. Very weird behaviour. |
Looks like Amazon Linux AMI doesn't have gcc-c++, sharp install fine after 'sudo yum install gcc-c++'. |
Has anybody noticed that backstop using chromy is taking about 40% longer to execute than the exact same config using phantomjs? Also passing of flags to chromy does not seem to work properly - or i'm doing something wrong 😄 btw @garris thank's a lot for pushing the chrome-headless integration 👍 |
@laubsauger thanks for testing this version! Yes. Totally noticed. At least some of it is related to #480. Working on it. Also, please file a ticket for the flags -- probably a bug in there. Cheers! |
Just to be clear -- it looks like the slowdown is 1) chrome launch time, and 2) per selector capturing. The chrome option is faster if you have a test with many scenarios each having one selector. Because, even though boot time is slower, they happen in parallel. But, obviously, I am hoping we can do much better. |
@laubsauger #480 has been fixed and @daveaspinall has reported a very significant speed increase. Please retest at your earliest convenience -- would like to know if your issue is addressed also. Thanks! |
Note: please install the latest backstopjs 3.0.6+ and the latest chrome canary 60+. The |
AS OF 3.0.7 ALL FEATURES ARE SUPPORTED WHEN USING CHROMY OPTION. |
Next steps:
HELP WANTED! ^^^ |
Just tried this out and can report everything is working as expected. My config is very simple but the visual output is better (Phantom seemed to struggled with some bits). Anecdotally I would say it runs faster too. Thanks for the effort, you managed to turn this around very quickly! |
@garris Nice.
edit: opened Issue with Chromy regarding debugWindow OnetapInc/chromy#53 |
Hmm, I have a set of scenarios that pass individually, but when I run them together (even just 9 scenarios), backstop hangs (two chrome instances never close when I run all tests). So tests/comparison etc is never run. I haven't been able to pin this down to anything specific. (There's some Chromy selector errors, but it still takes shots and these scenarios pass when run on their own...). Anyone else come up against this? Im on macosx, node 8.1.4, chrome 60, backstopjs 3.0.7. Happy to debug if anyone has any pointers as to where to start. |
Okay, left it running even longer:
Setting chromy to be viewable I can see the last windows don't progress to a url at all... (digging) |
@josh-wilson-sociomantic @laubsauger @djskinner Also, please download and run the backstop test here... https://github.com/garris/backstop-feature-tests -- please include the summary at the end. (with the perf times) e.g. ...
|
|
@josh-wilson-sociomantic the file where the magic happens is |
Doesn't seem t be any specific test, sadly. Just 10-12+ tests running and it happens.
@garris yeh,that's what I was thinking. Digging in I started digging up in createBitmaps, and the only thing there could be p-map passing them on before they're ready, though I doubt that. I wont be back on this machine for a few days, but I'll be digging in more when I can. Thanks, @garris 👍 |
Super I realy need to migrate to Chrome my tests in the near future. I finally managed to install version 3.0.7 but I hit some problems during the process. The Sharp module does not want to install properly. The first error was related to SDK windows version. So I installed the required version 8.1 . https://developer.microsoft.com/en-us/windows/downloads/windows-8-1-sdk After that, I still had an issue with the same module Sharp : I found this issue (lovell/sharp#341) and follow the solution "…manually installed Visual C++. https://msdn.microsoft.com/en-IN/library/60k1461a.aspx" I post it if it could help someone. |
@josh-wilson-sociomantic thanks for taking a look at it. Just for a sanity check -- if you get a chance -- please download and run the test in this project. Please let me know if it runs as expected. And also -- if you're able to share your scenarios and project I would take a look at that. Please open a new ticket for this if you find more info. |
@patriciamarchand |
On ubuntu 16.04 with node 7.2 latest released chrome default tests are working fine. But my previous tests have some issues. |
@mirzazeyrek thank you for testing! please feel free to open a new ticket for your previous tests. I would like to understand where we need improvements. Cheers. |
I've got an error running tests, I've added here , although maybe it should have been in this repo ? OnetapInc/chromy#45 |
@garris I'm happy to say, I found a Chrome instance that was lurking and seemed to be blocking a port. Once I I've no idea what caused it to block/hang or if it was my normal chrome having done something there... For now seems good, and I'm keeping an eye for the problem, if it happens again I should be able to glean something more useful. 👍 (but i hope it doesn't :P ) |
Hey great -- thanks for following up and letting me know! cheers, GS |
@garris , @dotneet and guys, just dropping by to say a huge CONGRATS on bringing headless chrome to production! I think this is the first project I see moving from older frameworks to headless chrome, and successfully 😄 I don't use BackstopJS but is an active user of CasperJS for couple of years. Had previously spoken to Chromy creator Shinji and I really second what Garris said. I especially like the clean and rapidly extending API and his fast rate of iteration putting features into prod so fast. Sorry for posting and not really adding any contribution to this thread. Just can't contain my excitement that Garris and Shinji pulled this off so fast. |
Thanks @kensoh! |
Looks like attribute selectors no longer work? Not sure if this belongs here or in chromy though. works: doesn't work: |
@lancetharper Thanks for testing! So, attribute selectors still work using the |
@garris correct. chrome runs MUCH faster though so I'm very excited to see this version released! edit: |
@lancetharper Great to hear!!! Yes, some of my tests are running at double speed... So, please file this over on the Chromy repo. @dotneet is really amazing with fixes like these. Cheers! |
CLOSED BY 0535303 |
Chrome headless support is now on master. THANK YOU EVERYONE FOR YOUR SUPPORT! |
HI guys, I was wondering if someone could help me with the error I am facing below. I've tried updating my Backstop JS to Chrome headless but can't seem to install using the command below, is this correct? D:\AEM_BACKSTOPJS_SMOKE_TESTS>npm install -g backstopjs@beta npm ERR! getaddrinfo EAI_AGAIN registry.npmjs.org:443 npm ERR! Please include the following file with any support request: D:\AEM_BACKSTOPJS_SMOKE_TESTS> |
According to http://www.codingdefined.com/2015/06/nodejs-error-errno-eaiagain.html this is a problem with your internet connection or proxy setting. I am using the Microsoft Ubuntu Shell on Windows and not node on Windows directly. Does it work when you install other nom modules or is it really only that single one? |
Thanks for replying so quickly @Primajin I have installed npm modules before which has worked okay. In this case, I also logged into cmd using an admin user but to no avail. I'll check out on the proxy settings though and rule out the fact that the proxy is an issue :) |
Hi @Primajin I spoke to our Network team who said there may not be able to enable/modify our proxy settings in order to run the test below. I tried running the tests from home (using home Wifi) instead of work and came up with a separate error. Any ideas what might be causing this? Many thanks, D:\AEM_BACKSTOPJS_SMOKE_TESTS\backstopjs-smoke-tests>npm install -g backstopjs 404 - Request is serviced by NPM metadata service, remote access not needed from DefaultNpmProxyRepository(id=npmjs)npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Request is serviced by NPM metadata service, remote access not needed from DefaultNpmProxyRepository(id=npmjs) npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: @types/mkdirp@^0.3.29 (node_modules\backstopjs\node_modules\chrome-launcher\node_modules@types\mkdirp): npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Unexpected token < in JSON at position 0 npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: <title>404 - Request is serviced by NPM metadata service, remote access not needed from DefaultNpmProxyRepository(id=npmjs)</title> npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: 404 - Request is serviced by NPM metadata service, remote access not needed from DefaultNpmProxyRepository(id=npmjs)npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Request is serviced by NPM metadata service, remote access not needed from DefaultNpmProxyRepository(id=npmjs) npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: npm ERR! Windows_NT 6.1.7601 npm ERR! argv "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" "install" "-g" "backstopjs" npm ERR! node v6.9.5 npm ERR! npm v3.10.10 npm ERR! Unexpected token < in JSON at position 0 404 - Request is serviced by NPM metadata service, remote access not needed from DefaultNpmProxyRepository(id=npmjs)npm ERR! Request is serviced by NPM metadata service, remote access not needed from DefaultNpmProxyRepository(id=npmjs) npm ERR! npm ERR! npm ERR! npm ERR! npm ERR! If you need help, you may report this error at: npm ERR! https://github.com/npm/npm/issues npm ERR! Please include the following file with any support request: D:\AEM_BACKSTOPJS_SMOKE_TESTS\backstopjs-smoke-tests> |
ATTENTION BACKSTOPPERS!
📣 BackstopJS now supports all existing features using chrome-headless.
I have been working with chrome-headless extensively for the past 6 weeks now and I am happy to report to you that it is WICKED STABLE.
BackstopJS been major-version-bumped to 3.x.x. (
legacy
mode is deprecated)Check it out here!
Documentation is in progress now. Examples and feature tests are here for the moment...
All beta users: Please keep in mind that the chrome-headless feature is built on top of chromy.js https://github.com/OnetapInc/chromy. Any
onBefore
andonReady
scripts will need to use chromy.js syntax and methods in order to work when using the chrome option. I have personally been in touch with @dotneet of chromy.js -- he is very dedicated to the project and has been a huge help supporting this integration!The
chrome-headless
feature is able to capture screen shots in parallel -- This is already 3x faster than the previous version. There are other additions listed in the brandh readme file.LET THE BUG BASHING BEGIN!
closes #454
The text was updated successfully, but these errors were encountered: