Add Makefile support to stay DRY

Author: elgalu

.env

@@ -2,14 +2,45 @@
 # file named docker-compose.yml in the current directory and then each
 # parent directory in succession until a file by that name is found.
 #  https://docs.docker.com/compose/reference/envvars/#compose-file
-COMPOSE_FILE=docker-compose.yml
+export COMPOSE_FILE ?= docker-compose.yml
+
+# We prefer Real VNC client because it allows the
+# -Scaling % option
+export VNC_CLIENT_NAME ?= RealVNC
+export VNC_DOWNLOAD_BASE_URL = https://www.realvnc.com/download/file/vnc.files
+export VNC_CLIENT_VERSION = 5.3.2
+export VNC_CLIENT ?= vncviewer
+export VNC_CLIENT_ERROR_MSG="Required VNC viewer '${VNC_CLIENT_NAME}' is not installed"
+export VNC_FILE_OSX = VNC-${VNC_CLIENT_VERSION}-MacOSX-x86_64.pkg
+export VNC_FILE_LINUX = VNC-${VNC_CLIENT_VERSION}-Linux-x64-DEB.tar.gz
+export VNC_MD5SUM_OSX = a9b2ba07286f04dadc63db252a95b639
+export VNC_MD5SUM_LINUX = 5b0da4c2d3a8f1e3db5982b9e6be2cb6
+# The command we will use to check if proper VNC is installed
+export VNC_CHECK_CMD = ${VNC_CLIENT} -h | grep -i '${VNC_CLIENT_NAME}' >/dev/null
+
+# Timeouts
+export WAIT_ALL_DONE ?= 40s
+
+# Number of nodes
+export chrome ?= 1
+export firefox ?= 1
+
+# To target a specific node number
+# for example to VNC into it via
+#  make see browser=chrome node=1
+export browser ?= chrome
+export node ?= 1
 
 # If you project name is selenium and it includes two services
 # `hub` and `node` then compose starts containers named
-# selenium_hub_1 and selenium_node_1 respectively.
+# grid_hub_1 and grid_node_1 respectively.
 # It will also
 #  https://docs.docker.com/compose/reference/envvars/#compose-project-name
-COMPOSE_PROJECT_NAME=selenium
+export COMPOSE_PROJ_NAME ?= grid
+
+# App name is the service name for the web app under test
+export APP_NAME ?= mock
 
 # We need a fixed port number to know where to connect the selenium tests
-SELENIUM_HUB_PORT=4444
+export HUB_PORT ?= 4444
+export SELENIUM_HUB_PORT ?= ${HUB_PORT}

.gitignore

@@ -4,10 +4,17 @@ binaries/
 READMEBS.md
 READMEZala.sh
 .python-version
-videos/
+videos
 android/
 binaries
 scm-source.json
 TODO.md
 docker_push.log
 bats/
+compose_down.log
+docker-pull.log
+VNC-*-Linux-x64-DEB.tar.gz
+VNC-Viewer-*-Linux-x64.deb
+VNC-*-MacOSX-x86_64.pkg
+vnc_linux.md5
+vnc_osx.md5

.travis.yml

@@ -77,7 +77,7 @@ install:
 
 script:
   - ./test/script_start
-  - travis_retry ./test/script_with_retry
+  - travis_retry ./test/script_scenario_compose_N_N
   - ./test/script_end
 
 # cleanup though doesn't seem to be necessary in Travis infra

CHANGELOG.md

@@ -10,6 +10,8 @@ Note image ids also change after scm-source.json has being updated which trigger
 ## TBD_DOCKER_TAG
  + Date: TBD_DATE
  + Upgrade dockertoolbox cask to 1.12.0-rc4
+ + Add Makefile support to stay DRY
+ + Upgrade Python from 2.7.11 to 2.7.12
  + Image tag details:
   + Selenium: vTBD_SELENIUM_VERSION (TBD_SELENIUM_REVISION)
   + Chrome stable:  TBD_CHROME_STABLE

CONTRIBUTING.md

@@ -3,24 +3,24 @@
 ## Local
 For pull requests or local commits:
 
-    time (./test/before_install_build && ./test/install && ./test/script_start && ./test/script_with_retry && ./test/script_end) ; beep
+    time (./test/before_install_build && ./test/install && ./test/script_start && ./test/N_N && ./test/script_end) ; beep
     docker exec grid versions && ./test/after_script
     open ./images/grid_console.png #to verify the versions are correct
     git checkout ./images/grid_console.png && open ./videos/chrome/test.mkv
     travis lint #if you changed .travis.yml
-    git checkout -b tmp-2.53.1i #name your branch according to your changes
+    git checkout -b tmp-2.53.1j #name your branch according to your changes
     #git add ... git commit ... git push ... open pull request
 
 For repository owners only:
 
-    git commit -m "Upgrade dockertoolbox cask to 1.12.0-rc4"
+    git commit -m "Add Makefile support to stay DRY"
     git tag -d latest #tag latest will be updated from TravisCI
-    git tag 2.53.1i && git push origin tmp-2.53.1i && git push --tags
+    git tag 2.53.1j && git push origin tmp-2.53.1j && git push --tags
 
 -- Wait for Travis to pass OK
 -- Make sure changes got merged into master by elgalubot
 
-    git checkout master && git pull && git branch -d tmp-2.53.1i && git push origin --delete tmp-2.53.1i
+    git checkout master && git pull && git branch -d tmp-2.53.1j && git push origin --delete tmp-2.53.1j
 
 -- Re-add TBD_* section in CHANGELOG.md starting with TBD_DOCKER_TAG
 -- If Chrome version changed upload:
@@ -36,9 +36,9 @@ Keep certain bins if chrome version changed for example:
 ## Retry
 Failed in Travis? retry
 
-    git tag -d 2.53.1i && git push origin :2.53.1i
+    git tag -d 2.53.1j && git push origin :2.53.1j
     #git add ...
-    git commit --amend && git tag 2.53.1i && git push --force origin tmp-2.53.1i && git push --tags
+    git commit --amend && git tag 2.53.1j && git push --force origin tmp-2.53.1j && git push --tags
 
 ## Docker push from Travis CI
 Travis [steps](https://docs.travis-ci.com/user/docker/#Pushing-a-Docker-Image-to-a-Registry) involve `docker login` and docker credentials encryptions.

Dockerfile

@@ -1011,8 +1011,11 @@ ENV FIREFOX_VERSION="${FF_VER}" \
   VNC_PORT="${DEFAULT_VNC_PORT}" \
   # VNC_CLI_OPTS="-noipv6 -no6 -forever -shared" \
   VNC_CLI_OPTS="-forever -shared" \
-  # You can set the VNC password or leave null so a random password is generated:
-  # ENV VNC_PASSWORD topsecret
+  # VNC password options:
+  # - "no" means no password; less secure but not a big deal
+  # - "" (empty string) means it will be randomly generated
+  # - "some-value" will use that password "some-value" or whatever
+  VNC_PASSWORD=no \
   NOVNC_PORT="${DEFAULT_NOVNC_PORT}" \
   NOVNC="false" \
   SSHD="false" \

Makefile

@@ -0,0 +1,148 @@
+# Usage
+#  make setup #first time
+#  make chrome=3 firefox=5
+#   note is destructive, firsts `compose down`
+#   warns if your service is not listed in the `docker-compose.yml`
+#  make cleanup
+#
+# All in one
+#  make setup compose chrome=3 firefox=5 see browser=firefox node=5
+include .env
+
+ECHOERR=sh -c 'awk " BEGIN { print \"-- ERROR: $$1\" > \"/dev/fd/2\" }"' ECHOERR
+# TODO: Output warning in color: yellow
+ECHOWARN=sh -c 'awk " BEGIN { print \"-- WARN: $$1\" > \"/dev/fd/2\" }"' ECHOWARN
+
+default: compose
+
+docker-compose.yml:
+	wget -nv "https://raw.githubusercontent.com/elgalu/docker-selenium/latest/docker-compose.yml"
+
+.env:
+	wget -nv "https://raw.githubusercontent.com/elgalu/docker-selenium/latest/.env"
+
+mk/install_vnc.sh:
+	wget -nv "https://raw.githubusercontent.com/elgalu/docker-selenium/latest/mk/install_vnc.sh" \
+	  -O mk/install_vnc.sh
+
+mk/vnc_cask.rb:
+	wget -nv "https://raw.githubusercontent.com/elgalu/docker-selenium/latest/mk/vnc_cask.rb" \
+	  -O mk/vnc_cask.rb
+
+mk/see.sh:
+	wget -nv "https://raw.githubusercontent.com/elgalu/docker-selenium/latest/mk/see.sh" \
+	  -O mk/see.sh
+
+mk/wait.sh:
+	wget -nv "https://raw.githubusercontent.com/elgalu/docker-selenium/latest/mk/wait.sh" \
+	  -O mk/wait.sh
+
+install_vnc:
+	./mk/install_vnc.sh
+
+mk:
+	mkdir -p mk
+
+docker:
+	@if ! docker --version; then \
+	  ${ECHOERR} "We need docker installed" ; \
+	  ${ECHOERR} "google: 'install docker'" ; \
+	  exit 1; \
+	fi
+
+docker-compose:
+	@if ! docker-compose --version; then \
+	  ${ECHOERR} "We need docker installed" ; \
+	  ${ECHOERR} "google: 'install docker-compose'" ; \
+	  exit 1; \
+	fi
+
+pull:
+	# Only pull for end users
+	@if [ "${TESTING}" != "true" ]; then \
+	  docker pull elgalu/selenium:latest > mk/docker-pull.log ; \
+	fi
+
+warn_vncviewer:
+	# Only check if not in a CI server
+	@if [ "${BUILD_NUMBER}" = "" ]; then \
+	  if ! eval ${VNC_CHECK_CMD}; then \
+	    ${ECHOWARN} ${VNC_CLIENT_ERROR_MSG} ; \
+	    ${ECHOWARN} "  RUN: make install_vnc" ; \
+	  fi ; \
+	fi
+
+check_vncviewer:
+	@if ! eval ${VNC_CHECK_CMD}; then \
+	  ${ECHOERR} ${VNC_CLIENT_ERROR_MSG} ; \
+	  exit 4; \
+	fi
+
+see: check_vncviewer
+	./mk/see.sh &
+
+# Shortcut to VNC into Firefox
+seeff:
+	$(MAKE) see browser=firefox
+
+# Shortcut to VNC into Chrome
+seech:
+	$(MAKE) see browser=chrome
+
+env:
+	env
+
+basic_reqs: docker-compose.yml .env mk mk/wait.sh docker docker-compose
+
+# Gather all requisites
+setup: basic_reqs mk/install_vnc.sh mk/vnc_cask.rb mk/see.sh warn_vncviewer pull
+	@echo "Requirements checked."
+
+cleanup:
+	docker-compose -f ${COMPOSE_FILE} -p ${COMPOSE_PROJ_NAME} down \
+	  --remove-orphans >./mk/compose_down.log 2>&1
+
+# Alias
+down: cleanup
+
+scale:
+	docker-compose -f ${COMPOSE_FILE} -p ${COMPOSE_PROJ_NAME} scale \
+	  ${APP_NAME}=1 hub=1 chrome=${chrome} firefox=${firefox}
+	$(MAKE) wait chrome=${chrome} firefox=${firefox}
+
+compose: basic_reqs cleanup
+	docker-compose -f ${COMPOSE_FILE} -p ${COMPOSE_PROJ_NAME} up -d
+	$(MAKE) scale chrome=${chrome} firefox=${firefox}
+
+wait:
+	./mk/wait.sh
+
+# Run self tests
+test:
+	docker-compose -f ${COMPOSE_FILE} -p ${COMPOSE_PROJ_NAME} exec \
+	  --index 1 hub run_test
+
+# PHONY: Given make doesn't execute a task if there is an existing file
+# with that task name, .PHONY is used to skip that logic listing task names
+.PHONY: \
+	default \
+	docker \
+	docker-compose \
+	pull \
+	setup \
+	basic_reqs \
+	check_vncviewer \
+	warn_vncviewer \
+	vncviewer \
+	vnc \
+	see \
+	install_vnc \
+	scale \
+	seeff \
+	seech \
+	compose \
+	wait \
+	down \
+	cleanup \
+	env \
+	test

README.md

@@ -68,8 +68,7 @@ You will need to run the second `eval` command for every new terminal window.
         docker pull elgalu/selenium #upgrades to latest if a newer version is available
 
         docker run -d --name=grid -p 4444:24444 -p 5900:25900 \
-            -e TZ="US/Pacific" -e VNC_PASSWORD=no \
-            -v /dev/shm:/dev/shm elgalu/selenium
+            -e TZ="US/Pacific" -v /dev/shm:/dev/shm elgalu/selenium
 
 2. Wait until the grid starts properly before starting the tests _(Optional but recommended)_
 
@@ -79,7 +78,7 @@ After this, [Selenium][] should be up and running at `http://localhost:4444/wd/h
 If you are using Mac (OSX) or [Microsoft Windows](https://docs.docker.com/engine/installation/windows/) `localhost` won't work unless you are in Docker Beta (version >= 1.12) If you are using Docker version <= 1.11 please find out the correct IP through `docker-machine ip default`.
 
 **Notes:**
- * `VNC_PASSWORD=no` will make it VNC passwordless accessible.
+ * The new default `VNC_PASSWORD=no` will make it VNC passwordless accessible.
  * Once this [docker feature](https://github.com/docker/docker/pull/22719) is in place `wait_all_done` won't be necessary anymore.
 
 #### Stop
@@ -101,9 +100,9 @@ See [jenkins](./docs/jenkins.md)
 ### Parallel
 This image is designed to run one test on each docker container but if you still want to run multiple tests in parallel you can still do so, there are some ways to do this:
 
-1. The recommended way is via [docker-compose](./docs/docker-compose.md) and you should replace `adwords_mock` with your web service under test within the [docker-compose.yml][] file.
+1. The recommended way is via [docker-compose](./docs/docker-compose.md) and you should replace `mock` with your web service under test within the [docker-compose.yml][] file.
 
-        SELENIUM_HUB_PORT=4444 docker-compose -p selenium scale adwords_mock=1 hub=1 chrome=3 firefox=3
+        SELENIUM_HUB_PORT=4444 docker-compose -p grid scale mock=1 hub=1 chrome=3 firefox=3
 
 1. The _(not recommended)_ way is by increasing `MAX_INSTANCES` and `MAX_SESSIONS` which now [defaults](https://github.com/elgalu/docker-selenium/blob/2.53.1a/Dockerfile#L967) to 1.
 

browserstack/bin/start-browserstack.sh

@@ -3,10 +3,7 @@
 # set -e: exit asap if a command exits with a non-zero status
 set -e
 
-# echo fn that outputs to stderr http://stackoverflow.com/a/2990533/511069
-echoerr() {
-  cat <<< "$@" 1>&2;
-}
+echoerr() { awk " BEGIN { print \"$@\" > \"/dev/fd/2\" }" ; }
 
 # print error and exit
 die () {

browserstack/bin/wait-browserstack.sh

@@ -3,10 +3,7 @@
 # set -e: exit asap if a command exits with a non-zero status
 set -e
 
-# echo fn that outputs to stderr http://stackoverflow.com/a/2990533/511069
-echoerr() {
-  cat <<< "$@" 1>&2;
-}
+echoerr() { awk " BEGIN { print \"$@\" > \"/dev/fd/2\" }" ; }
 
 # print error and exit
 die () {