V0.5.10 (#88)

* Prepare next release

* TPC-H: Example for Minikube

* Include more Docker images

* Docs improved

* TPC-H: Example for Minikube

.github/workflows/publish-to-test-pypi.yml

@@ -44,3 +44,8 @@ jobs:
       with:
         password: ${{ secrets.TEST_PYPI_API_TOKEN }}
         repository_url: https://test.pypi.org/legacy/
+    - name: Publish distribution 📦 to PyPI
+      if: startsWith(github.ref, 'refs/tags')
+      uses: pypa/gh-action-pypi-publish@master
+      with:
+        password: ${{ secrets.PYPI_API_TOKEN }}

README.md

@@ -15,7 +15,8 @@ A more advanced workflow is: Plan a sequence of such experiments, run plan as a
 
 1. Download the repository: https://github.com/Beuth-Erdelt/Benchmark-Experiment-Host-Manager
 1. Install the package `pip install bexhoma`
-1. Make sure you have a working `kubectl` installed
+1. Make sure you have a working `kubectl` installed  
+  (Also make sure to have access to a running Kubernetes cluster - for example [Minikube](https://minikube.sigs.k8s.io/docs/start/))
 1. Adjust configuration [tbd in detail]
     1. Rename `k8s-cluster.config` to `cluster.config`
     1. Set name of context, namespace and name of cluster in that file
@@ -44,6 +45,7 @@ The repository contains a tool for running TPC-H (reading) queries at MonetDB an
 ## More Informations
 
 For full power, use this tool as an orchestrator as in [2]. It also starts a monitoring container using [Prometheus](https://prometheus.io/) and a metrics collector container using [cAdvisor](https://github.com/google/cadvisor). It also uses the Python package [dbmsbenchmarker](https://github.com/Beuth-Erdelt/DBMS-Benchmarker) as query executor [2] and evaluator [1].
+See the [images](images/) folder for more details.
 
 This module has been tested with Brytlyt, Citus, Clickhouse, DB2, Exasol, Kinetica, MariaDB, MariaDB Columnstore, MemSQL, Mariadb, MonetDB, MySQL, OmniSci, Oracle DB, PostgreSQL, SingleStore, SQL Server and SAP HANA.
 

bexhoma/scripts/tpch.py

@@ -17,6 +17,8 @@
 import logging
 import argparse
 import time
+from timeit import default_timer
+import datetime
 
 
 urllib3.disable_warnings()

experiments/tpch/queries-tpch.config

@@ -2397,6 +2397,7 @@ revenue{numRun}
 )
 order by
 s_suppkey""",
+"""drop view revenue{numRun}"""],
 				'MySQL': ["""create view revenue{numRun} as
 select
 l_suppkey supplier_no,

images/README.md

@@ -0,0 +1,26 @@
+# Benchmark Experiment Host Manager
+
+In this folder is a collection of useful images.
+
+## Orchestration of Benchmarking Experiments
+
+<p align="center">
+    <img src="https://raw.githubusercontent.com/Beuth-Erdelt/Benchmark-Experiment-Host-Manager/v0.5.6/docs/experiment-with-orchestrator.png" width="800">
+</p>
+
+For full power, use this tool as an orchestrator as in [2]. It also starts a monitoring container using [Prometheus](https://prometheus.io/) and a metrics collector container using [cAdvisor](https://github.com/google/cadvisor). It also uses the Python package [dbmsbenchmarker](https://github.com/Beuth-Erdelt/DBMS-Benchmarker) as query executor [2] and evaluator [1].
+
+
+## References
+
+[1] [A Framework for Supporting Repetition and Evaluation in the Process of Cloud-Based DBMS Performance Benchmarking](https://doi.org/10.1007/978-3-030-84924-5_6)
+
+> Erdelt P.K. (2021)
+> A Framework for Supporting Repetition and Evaluation in the Process of Cloud-Based DBMS Performance Benchmarking.
+> In: Nambiar R., Poess M. (eds) Performance Evaluation and Benchmarking. TPCTC 2020.
+> Lecture Notes in Computer Science, vol 12752. Springer, Cham.
+> https://doi.org/10.1007/978-3-030-84924-5_6
+
+
+[2] [Orchestrating DBMS Benchmarking in the Cloud with Kubernetes](https://www.researchgate.net/publication/353236865_Orchestrating_DBMS_Benchmarking_in_the_Cloud_with_Kubernetes)
+

images/dashboard/README.md

@@ -1,4 +1,10 @@
-# images
+# Image of the Evaluator Container
+
+This image contains
+* Dashboard of [DBMSBenchmarker](https://github.com/Beuth-Erdelt/DBMS-Benchmarker) at port 8050
+* A Jupyter Notebook Server at port 8888 (password = admin)
+
+## Build Commands
 
 ```
 docker build -t perdelt/bexhoma:dashboard --no-cache .

images/data/README.md

@@ -1,4 +1,10 @@
-# images
+# Image of TPC-H Data
+
+This image contains [TPC-H](http://www.tpc.org/tpch/) data at SF=1.
+Data will be copied to `/data/tpch/SF1`.
+This directory is supposed to be mounted to a volume or persistent volume in a shared filesystem.
+
+## Build Commands
 
 ```
 docker build -t perdelt/bexhoma:data-tpch-1 --no-cache .

images/dbmsbenchmarker/Dockerfile

@@ -9,22 +9,13 @@ RUN apt install default-jre -y
 
 RUN pip uninstall dbmsbenchmarker -y
 RUN pip install dbmsbenchmarker
-#RUN JAVA_HOME=/usr/lib/jvm/java-1.8.0/ pip install --no-cache-dir --upgrade --force-reinstall git+https://github.com/Beuth-Erdelt/DBMS-Benchmarker
-#@v0.11.4
-
-#COPY requirements.txt ./
-#RUN pip install --no-cache-dir -r requirements.txt
 
 RUN git clone https://github.com/Beuth-Erdelt/DBMS-Benchmarker
 
 WORKDIR /usr/src/app/DBMS-Benchmarker
 
 COPY . .
 
-#RUN git checkout --force v0.11.4
-
-RUN git pull
-
 RUN apt install nano -y
 
 RUN mkdir -p /results

images/dbmsbenchmarker/README.md

@@ -1,4 +1,22 @@
-# images
+# Image of the Query Executor Container
+
+This image contains an instance of [DBMSBenchmarker](https://github.com/Beuth-Erdelt/DBMS-Benchmarker).
+
+The container expects some environment variables:
+* `$DBMSBENCHMARKER_CODE`: Code of the DBMSBenchmarker experiment.
+* `$DBMSBENCHMARKER_CONNECTION`: Name of the connection (to the DBMS).
+* `$DBMSBENCHMARKER_SLEEP`: Number of seconds to wait before starting the benchmarking.
+* `$DBMSBENCHMARKER_CLIENT`: Number of parallel clients (i.e. Docker containers) for this connection.
+* `$DBMSBENCHMARKER_START`: Start time for container to start benchmarking. This is for synching several containers.
+
+
+This yields a run command as
+```
+CMD python ./benchmark.py run -b -d -w connection -mps -f /results/$DBMSBENCHMARKER_CODE -r /results/$DBMSBENCHMARKER_CODE -sf $DBMSBENCHMARKER_CONNECTION -cs -sl $DBMSBENCHMARKER_SLEEP -c "$DBMSBENCHMARKER_CONNECTION" -ca "$DBMSBENCHMARKER_ALIAS" -cf ${DBMSBENCHMARKER_CONNECTION}.config -ms $DBMSBENCHMARKER_CLIENT -st "$DBMSBENCHMARKER_START"
+```
+See [docs](https://github.com/Beuth-Erdelt/DBMS-Benchmarker) of DBMSBenchmarker for details.
+
+## Build Commands
 
 ```
 docker build -t perdelt/bexhoma:dbmsbenchmarker --no-cache .

images/monitoring/Dockerfile

@@ -2,23 +2,10 @@ FROM prom/prometheus:v2.24.1
 
 COPY prometheus.yml /etc/prometheus/prometheus.yml
 
-ENV BEXHOMA_SERVICE=bexhoma-sut.perdelt.svc.cluster.local
-
-#RUN [ "sh", "-c", "echo", "$BEXHOMA_SERVICE" ]
-#RUN [ "sh", "-c", "sed 's/localhost/'\"$BEXHOMA_SERVICE\"'/' -i /etc/prometheus/prometheus.yml" ]
-# sh -c "sed 's/localhost/'\"$BEXHOMA_SERVICE\"'/' -i /etc/prometheus/prometheus.yml"
-
 USER       nobody
 EXPOSE     9090
 VOLUME     [ "/prometheus" ]
 WORKDIR    /prometheus
-#ENTRYPOINT [ "/bin/prometheus" ]
-#CMD        [ "--config.file=/etc/prometheus/prometheus.yml", \
-#             "--storage.tsdb.path=/prometheus", \
-#             "--web.console.libraries=/etc/prometheus/console_libraries", \
-#             "--web.enable-lifecycle", \
-#             "--web.console.templates=/etc/prometheus/consoles" ]
-
 
 ENTRYPOINT [ "" ]
 CMD echo "$BEXHOMA_WORKERS" > /etc/prometheus/prometheus-bexhoma.yml ; echo "Test config" ; while [ ! -f /etc/prometheus/prometheus-bexhoma.yml ] ; do sleep 2 ; echo "Waiting for config" ; done ; /bin/prometheus \

images/monitoring/README.md

@@ -1,4 +1,10 @@
-# images
+# Image of Monitoring Container
+
+This image contains an instance of [Prometheus](https://prometheus.io/).
+
+It expects the config file (normally `/etc/prometheus/prometheus.yml`) in `$BEXHOMA_WORKERS`.
+
+## Build Commands
 
 ```
 docker build -t perdelt/bexhoma:monitoring --no-cache .