A few days ago Lauri Apple and Raffaele Di Fazio approached me with a few interesting questions about docker-selenium that we decided to share here.
The project aims to provide Selenium inside a docker container.
The project started while I was in charge of building and maintaining the test automation suite of an AngularJS project at AppNexus. I was also doing DevOps tasks related to the test infrastructure.
The objective was to run the tests headless, different solutions existed for that and there is PhantomJS for example but we needed real browsers like Chrome or Firefox to run our tests on, one reason was to get better test confidence and the other was that Protractor doesn't play nice with PhantomJS. Recently Wallaby.js announced they will support Electron as an alternative to PhantomJS because it allows to use the latest Chromium/V8 which might be equivalent to running in Chrome however it sill needs a display so xvfb is needed.
With Selenium, you can always run your tests locally but as soon as your tests runs the browser popping up in your main display can be annoying, you could configure your windows manager to move it automatically to another workspace for example and similar solutions exists but why going into all those troubles if you can just docker run selenium.
You can also configure a headless Xvfb selenium as it is a common use case in Jenkins CI but, again, why going into the trouble now docker exists.
So back in 2014 the first thing I did was googling "selenium in docker", looked around 2 to 3 projects that were floating around but not well maintained so decided to create my own.
It started mid 2014 so almost 2 years, it wasn't maintained consistently all these time but yes lately thanks to my role here at Zalando and the fact that some teams here are using it.
It has tests that run seamlessly locally or in Travis plus deploy automation using TravisCI docker infrastructure
The reason for pushing the new releases (docker images) from Travis instead of using docker automated builds is that a CI tool allows running arbitrary scripting like tests before pushing a broken image like happened for example at issue 208
There was no promotion in the beginning, I suppose the success of it is due to the fact that is an obvious use case for any developer that needs Selenium and knows how handy docker technology is regarding disposable infrastructure.
Automation testers probably Google 2 words: selenium and docker then the top results are the official project and secondly this one or sometimes the other way around depending on Google mood (i.e. Google algorithm in relation to the browser user stats).
When I started at Zalando in March 2015 we didn't have Sauce Labs, only a centralized Selenium Grid in our data center which acted as a kind of SPOF (Single Point of Failure) for our Zalando teams let alone the fact you couldn't see the tests running nor have recorded video results which are 2 features of this project.
My team, in charge of the Test Infrastructure, later on managed to acquire Sauce Labs enterprise which has all these features and support hundreds of browsers combinations so is the natural preferred way of getting a front-end testing infrastructure these days but docker-selenium still keeps 3 advantages:
- cost: is free.
- speed: runs around 2x faster than a paid cloud based Selenium solution.
- security: no need to tunnel your local app to a third party cloud solution.
Has there been any noticeable spike in followers/stars/etc that you've noticed during the time it's been public?
There was a slightly spike after one of contributors, @rubytester, made a presentation back in September 2015.
Sometimes when someone with many followers tweets about it but other than that the popularity growth has being consistent. Unfortunately I can't analyze this, back in time, out of github analytics because there is no way to customize the dates, yet.
I suppose most users come from Google search results as shown in the traffic stats.
Some of these users generously asked me to add them in the "Who is using this" list like in the case of Nvidia for example when I was contacted directly by Tony Carrillo via his corporate email @nvidia.com.
In the case of Algolia I asked @vvo after he pointed it out in a github issue.
From now on, when I remember it, I try to ask the users that open issues or send PRs to add their companies to this list as it feels nice to see the list growing.
Good extensive documentation helps gain users for sure but should always include how to get started quickly for people that want to use it ASAP without needing to acquire a deep knowledge of the tool.
A couple of users wrote me they preferred my project because it was well maintained, up to date and issues were addressed quickly (no more than 3 days). I can understand that in a huge widely used project like AngularJS you get to see hundreds of open issues without comments nor assignee nor labels or 36 issues unattended like I see today in docker but some projects that have that huge amount of incoming opened issues and yet you still see them not even labeled, like the author not caring enough.
When Matthew Smith aka @mtscout6 jumped into the project, 4 months after it started, great things happened. He made a few interesting improvements but moreover we started conversations about moving docker-selenium to the official SeleniumHQ organization, Matt pushed this, contacted Mozilla guys and what not so it happened and for a while we were maintaining only that one but I decided to continue maintaining mine with differentiated features and that's why 2 project for the same purpose exist today.
Main technical debt now is issue 31 "Comply with docker official-images requisites" and maybe also issue 40 to "Split browser capabilities into separate images". Probably also to improve the bash scripting by using functions with proper unit tests.
The official one at SeleniumHQ is using CircleCI while elgalu/selenium uses TravisCI so tests basically have a couple of scenarios, running front-end tests on Chrome, on Firefox and restarting the container to check it still works after that. We should potentially add more tests for each bug that comes up to increase coverage in the future.
Right now people tend to build long running selenium grids by using the stock docker selenium images or this one however when you see Sauce Labs or BrowserStack approach you realize the way to go is isolation, i.e. one machine or VM or docker container for each selenium session so those are the next steps for this project and perhaps a Jenkins plugin. Also automate new versions detection e.g. new version of Chrome/Firefox/Selenium might be one of the chores to work on.