How to design the benchmarking

[certik]

We definitely need the functionality #5 and #6. I am thinking that perhaps there should be a script, that runs benchmarks for the given symengine commit and a certain way to install it. And then there should be a script that builds docker images A and B with two different configuration (either different symengine commits to do #5 or different way to build it to do #6). Then one can use these two images A and B to generate the comparison graphs.

1. So it seems the lower layer just needs to build a certain symengine version a certain way, run benchmarks and save the result to data.json.
2. The upper layer then runs the lower layer 1. for two different configurations and plots both in the same graph.

For two different symengine versions this can be automated. But for two different docker configurations, this might not be practical to automate, so rather the user would simply save the data.json, then change some docker configuration, rerun, save the new data.json, and then plot both in the same graph.