Question: how to script a grid node in Jenkins

[tnguyen14]

I am a bit confused by the documentation about how to use the images.
Once the hub container is running, it says to do something like

CH=$(docker run --rm --name=ch \
    --link selenium-hub:hub -v /e2e/uploads:/e2e/uploads \
    selenium/node-chrome:2.52.0)

I don't understand what is the use of the CH=$() syntax? The above is the same as just running docker run --rm ..., as both will run the container in the foreground and will exit only when pressing Ctrl-C.

It is advised that each node container should be run as a single process, but I wonder how I can continue in the CI flow (in Jenkins) to run tests that would connect to Selenium without sending the container into the background?

My knowledge of shell scripting is limited, so perhaps there is something I'm missing here. Any advice or clarification is greatly appreciated!

[alexkogon]

Hi, I'm not sure I understand your question, are you asking how to bring up nodes dynamically with Jenkins? If for instance you had Docker on your Jenkins machine, and you were running jobs one at a time, you could have your Jenkins job first launch a Docker Hub, then a Docker Node, then run your tests, then kill both the Hub and the Node.

More scaleable might be to have a Hub running, then launch a Node in your Jenkins, and bring it down after you use it. You wouldn't be able to guarantee necessarily that the node you launched was the one you were using if it was massively parallel though that might cause some problems in killing the node after...

Also remember to clean them up after or your disk will soon fill up with old containers :)

[tnguyen14]

@alexkogon I like the idea of having a Hub running in the background, and launch node as needed for the build step. But this kinda goes back to my original question of how do you keep a reference to the node you just spun up in order to shut it down? The example code in the readme CH=$(docker run --rm -... would not work as it does not put that docker container in the background.

[elgalu]

the example in the readme works, just change --rm with -d then you can stop the container later on docker stop $CH you can also use named containers e.g. docker run -d --name=build-99 and later on do docker stop build-99 and eventually 99 could come from an env var like $BUILD_NUMBER in Jenkins ;)

[tnguyen14]

Thanks @elgalu. That is an interesting suggestion. Let me try it out.

[tnguyen14]

@elgalu so one problem I'm running into is that if a build fails, the command to stop and remove the node container isn't executed, thus leaving them hanging around. Do you know of a way to solve this? This is strictly speaking not a docker or selenium issue. Just wondering if you have any suggestions.

[elgalu]

In Jenkins add a Post build action or task that executes Always, there either put the script to docker stop buid-$BUILD_NUMBER || true and also docker rm buid-$BUILD_NUMBER || true or trigger another job and call it cleanup that should do the docker stop; docker rm part but then you need to pass it the parameter of the container name.

[vbyjsue]

Before tests I trigger a jenkins job that creates a docker compose file with unique port for each testjob. After tests i trigger the same job with stop parameter to remove all containers. Each testjob has its own grid (which runs 6 threads in parallel).

if [ "$Action" == "start" ]; then
mkdir -p $TRIGGER_JOB
cd $TRIGGER_JOB
cat > docker-compose.yml <<EOF
hub:
  image: docker.vby.svenskaspel.se:8181/selenium/hub:2.48.2
  ports:
    - "$PORT:4444"
  volumes:
    - /home/byggarebob/workspace:/home/byggarebob/workspace
  environment:
    GRID_MAX_SESSION: 15
    JAVA_OPTS: "-Djava.security.egd=file:/dev/./urandom"

chrome:
  image: docker.vby.svenskaspel.se:8181/selenium/node-chrome:2.48.2
  links:
    - hub
  volumes:
    - /dev/shm:/dev/shm
  volumes_from:
    - hub
  environment:
    SCREEN_WIDTH: 1920
    SCREEN_HEIGHT: 1080
    JAVA_OPTS: "-Djava.security.egd=file:/dev/./urandom"     
EOF
fi 
export COMPOSE_PROJECT_NAME=$TRIGGER_JOB
export DOCKER_HOST="tcp://$TEST_HOST:2375"

if [ "$Action" == "start" ]; then
  #before test, start required nodes
  cd $TRIGGER_JOB
  docker-compose up -d --force-recreate
  docker-compose scale chrome=${NODES}
  sleep 20
fi
if [ "$Action" == "stop" ]; then
  #After test stop and remove nodes
  cd $TRIGGER_JOB
  docker-compose scale hub=0 chrome=0
fi

[elgalu]

Nice docker-compose approach @vbyjsue

[tnguyen14]

@vbyjsue how is the "$Action" set? Or how is this script actually used in your Jenkins job? Also, out of curiosity, what is JAVA_OPTS: "-Djava.security.egd=file:/dev/./urandom" used for?

[vbyjsue]

The testjob triggers a "restart_selenium" job and passes the parameters Action, PORT and TRIGGER_JOB. When tests are finished a postbuild action triggers the job again with Action=stop.

The JAVA_OPTS is recomended from this discussion #14

[emoshaya]

Hi @vbyjsue is there any way to get your docker_compose started in systemd or as a service in CentOS? Ideally, I want the grid to start when the machine starts up.

[vbyjsue]

I'm sure that can be done, but I don't configure docker that often. Googling "docker-compose on startup" would give you plenty of suggestions.

[diemol]

Hi @tnguyen14, were you successful?

I think that thanks to @vbyjsue suggestion we can close this issue, don't you think?