scalable_mandelbrot

A mandelbrot generator written in go to demonstrate architecture scalability, horizontal and vertical.

What does it consist of ?

• a single thread mandelbrot generator
• a multi thread mandelbrot generator
• a multi machines mandelbrot generator using http servers locally
• a multi machines mandelbrot generator using docker
• a multi machines mandelbrot generator using kubernetes

Single Thread Mandelbrot

This go script simply compute the mandelbrot on a single thread

Multi Thread Mandelbrot (vertical scalability)

This go script makes use of goroutines to compute the mandelbrot. The mandelbrot generating works in pixels such as to generate an iamge of a certain width and height. This means that the width equals the number of columns. That being said, the mandelbrot is generated by having a goroutine for each column.

Multi Machines Mandelbrot (horizontal scalability)

This part had two main steps. First we developed a system with a master and multiple slaves. This part is hardcoded, the master knows how many slaves are running and will divide the mandelbrot generation in the number of slaves that are available. The computation is thus done by the slaves which all run as http servers. When they are done calculating, they return the encoded bytes of de mandelbrot specific part that they have generated and the master script combines these arrays of bytes to generate the final image of the mandelbrot.

The second step was to migrate these script in docker images to simulate a horizontal scalability. This would keep the same structure of master and slaves except that we would have an additional player, a kind proxy. The slaves (servers) register to the proxy. Then when the client or master wants to generate a mandelbrot, it will first make a request for the servers that are up. Depending on the result, it will divide the work in the number of connected servers and start the generation of the mandelbrot.

This was then followed by a migration of the docker application to kubernetes which asked some refactoring as teh goal here was to have only a single connection between the kubernetes cluster and client. This connection was thus handled by the proxy. More information can found in the README in horizontal_k8s.