V0.4.4 (#52)

* Prepare next release

* Masterscript: K8s allow overwriting dialect

* Masterscript: K8s change setting outside of constructor

* Masterscript: get_connection_config()

* Masterscript: K8s Driver in K8s cluster first sketch

* Masterscript: K8s Driver in K8s cluster first sketch

* Masterscript: K8s Driver in K8s cluster first sketch

* Masterscript: K8s Driver in K8s cluster first sketch

* Masterscript: K8s Driver in K8s cluster first sketch

* Masterscript: K8s Driver in K8s cluster first sketch

* Masterscript: K8s Driver in K8s cluster first sketch

* Masterscript: K8s Driver in K8s cluster first sketch

* Masterscript: K8s Driver in K8s cluster first sketch

* Masterscript: K8s Driver in K8s cluster first sketch

* Masterscript: K8s Driver in K8s cluster first sketch - replace jar folder statically

* Masterscript: K8s Driver in K8s cluster first sketch - delete job and pod after success

* Masterscript: K8s Driver in K8s cluster first sketch - fixed code

* Masterscript: K8s Driver in K8s cluster first sketch - dynamic code and connection

* Masterscript: K8s Driver in K8s cluster first sketch - remove fixed code

* Masterscript: K8s Driver in K8s cluster first sketch - wait for client pod to run

* Masterscript: K8s Driver in K8s cluster first sketch - print status of client pod

* Masterscript: K8s Driver in K8s cluster first sketch - dbmsbenchmarker pod shall wait 60s (str)

* Masterscript: K8s Driver in K8s cluster first sketch - wait 60s for client pod to close

* Docs: Typo

* Docs: Reference to DBMSBenchmarker and Monitoring

* Masterscript: K8s wait without tqdm

* Masterscript: K8s copy Prometheus API URL

* Masterscript: K8s port forward to service name again

* Masterscript: K8s stop port forward in detached

* Masterscript: Add more labels to K8s objects

* Masterscript: Add more labels to K8s objects - benchmarker job

* Masterscript: Add more labels to K8s objects - sut deployment

* Masterscript: Add more labels to K8s objects - sut deployment configuration

* Masterscript: Add more labels to K8s objects - service name

* Masterscript: Add more labels to K8s objects - monitoring and dashboard

* Masterscript: Add more labels to K8s objects - monitoring and dashboard

* Masterscript: Add more labels to K8s objects - monitoring and dashboard

* Masterscript: Add more labels to K8s objects - filter by labels

* Masterscript: Add more labels to K8s objects - filter by labels

* Masterscript: Add more labels to K8s objects - filter by labels

* Masterscript: Add more labels to K8s objects - filter by labels

* Masterscript: Add more labels to K8s objects - delete by labels

* Masterscript: Add more labels to K8s objects - find active sut

* Masterscript: Add more labels to K8s objects - find active sut

* Masterscript: Add more labels to K8s objects - find active sut

* Masterscript: Add more labels to K8s objects - find active sut

* Masterscript: Add more labels to K8s objects - find active sut

* Masterscript: Add more labels to K8s objects - find active sut

* Masterscript: Add more labels to K8s objects - find active sut

* Masterscript: Cluster class

* Masterscript: Cluster class

* Masterscript: Cluster class

* Masterscript: Cluster class

* Masterscript: Start code

* Masterscript: Start code

* Masterscript: Add more labels to K8s objects - experiment

* Masterscript: Add more labels to K8s objects - experiment

* Masterscript: Add more labels to K8s objects - monitoring labels

* Masterscript: Add more labels to K8s objects - monitoring labels

* Masterscript: Add more labels to K8s objects - monitoring labels

* Masterscript: Add more labels to K8s objects - monitoring labels

* Masterscript: SUT deployment generated in result folder

* Masterscript: Set SUT deployment and service name

* Masterscript: Set SUT deployment and service name - lower and no _

* Masterscript: Set benchmarker job name

* Masterscript: Set SUT service name for benchmarker

* Masterscript: Delete all pods of a job

* Masterscript: K8s connection and configuration different

* Masterscript: K8s client for benchmarker jobs

* Masterscript: K8s client for benchmarker jobs

* Masterscript: K8s client for benchmarker jobs - connect to sut

* Masterscript: K8s connection and configuration different

* Masterscript: Message for no matching job

* Masterscript: K8s client for monitoring deployment - connect to sut

* Masterscript: K8s node selector should not be None

* Masterscript: K8s node selector should not be None

* Masterscript: K8s monitoring customized url

* Masterscript: K8s write pod logs

* Masterscript: K8s copy job yaml to result folder

* Masterscript: K8s sketch of configuration class

* Masterscript: K8s make result folder if not exists

* Masterscript: K8s sketch of merging results in dashboard pod

* Masterscript: K8s sketch of merging results in dashboard pod

* Masterscript: K8s sketch of merging results in dashboard pod

docs/API.md

@@ -1,13 +1,15 @@
 # API Details
 
-This document contains API details about
+This document contains API details about methods for **parameters**
 * [`set_code()`](#set-code)
 * [`set_experiment()`](#set-experiment)
 * [`set_workload()`](#set-workload)
 * [`set_connectionmanagement()`](#set-connection-management)
 * [`set_querymanagement()`](#set-query-management)
 * [`set_resources()`](#set-resources)
-* [`set_ddl_paramters()`](#set-ddl-parameters)
+* [`set_ddl_parameters()`](#set-ddl-parameters)
+
+and methods for the **workflow**
 * [`runExperiment()`](#run-experiment)
 * [`prepareExperiment()`](#prepare-experiment)
 * [`startExperiment()`](#start-experiment)

docs/Architecture-K8s.drawio

@@ -0,0 +1 @@
+<mxfile modified="2021-01-28T20:46:35.184Z" host="app.diagrams.net" agent="5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" etag="SgEVZvlzCICDDtghk_du" version="14.2.7" type="device"><diagram id="Eicp26NF3Bc-FzKGdtsJ" name="Architecture">7V1bd6M4Ev41OWf3AR8kxO2xE8fdvTvZzelMz+7MSw4G2aZjGw/GucyvX4mbQRIg28i37X7oGBkLUH36VFWqKm6Mu8X759hbzR6iAM9voB683xjDGwih47jkD235yFugC7OWaRwGWRvYNjyFf+G8Uc9bN2GA17UTkyiaJ+Gq3uhHyyX2k1qbF8fRW/20STSvX3XlTTHX8OR7c771P2GQzLJWB9rb9i84nM6KKwMrf+KFV5ycP8l65gXRW6XJuL8x7uIoSrJPi/c7PKejV4xL8nk0MYLfotGP4cvD9+d771vyqGWdjXb5SfkIMV4me3c9Xv14/ecf968PX5foJXx6GD+8Qg2YWd+v3nyTD1j+sMlHMYLTONqs8tNwnOB3kdy8cXG6Lnm3oBxCAj4cLXASf5Df5b1r0Mw7yoGnOVYusretGN3iarOqCPWi1cuxMy27344P+ZAP0S7DZXUPF+mGYJsc3L7NwgQ/rTyffvNG5hdpmyULcsUhIB/n3hjPbz3/hQ7wMriL5lGcdmEM0T0YmeSUSbRM8ilFLk2Ow/lcdN46iaMXXPkmsMaWSX/RKLeqfFqgwQuoMvzAEAw/NJCq0bcFo2/NyXVvxzUZWH9u6KRMh09bp+P3iZwArNX79kvyaUr/PkTLMInicDktOiM3Ny6+5cTrrVcZTU3Cd0zu9naF45A8H6Yjn8v+cdtUFbg3D6dL8tknA59+VzKKTg4Cbz2jHaYHa4Iacke/RgQzQ4M0hIuU54q/w3BBbnc0D8mNjjw/CV/xcxDG5M4iOtCjKV6Se/Cf1zgmsh+sX6fktxHtNKHzydQF4KIgCQlxfspvM6EXVwCgcm0ZmLUZbvLz2zYE09voYXaLGUk0uzNI0LHaH2IVlMphrmzLrnt9UEy57zFah0kY1e6jAOEvzAmLMAjoc3MoLb9ooNNltMQ89ndZowTob162AKwvW7auc6hGAta0gSJQo11WLIKCFW3059Em6F7BWEKpr06BiZ0AiVYnB44Ny1IpBkYKGjA4KZjI5sWAQA+Ll/AZJBQtac2Bm/gdekLgYWfiiyRh+Q4eTxRKAuouQ/N2yfsVYUDdEWgSRWPvwpBR4wJiRuSHUZzMomm09Ob329bblGZKmtye80uUUiUV1A+cJB+5YLxNEtXFyElDT/+l33hx8olaP1sGS9tGIX3Q9IIsEY6jJIkW5Asiwfjjv/k10oPf6U8GZnE4fM+7yI4+qkeV9SJrfA+Tsi/y+fdK+7YfelB0sxuS1tEm9nHLefnEJQ8/xW39odxso3JrBWaM5x5dpGrKjAhh+U8fozBd+3NA29CtwRmyxkb2RPmvtjgl0vQ+Kqet6Anrluu4oHYdkGu/W9hnPYp/baI6AQL2LrPx5O4ynU3lYO0/wXhNHQzI8WOMKTDIp/v3TFnJL1+deYRZkvpEiTHRsHILk0IsHztytnl7Yw5F+oxYmW02uUaje9N1BVQqCeeCUWQN5EbC1AeW4dp1iGVHBwLXAQwRawbTRzSZrLESODhnwrfdrNrMySwwlNCsXqXZgemAVqqlB2xPJ6Jf6ByHfpFbJzZklrA+ErW5HJYNSm3fNkvyP5GuP1t48cv6OmitmLgH05pG0AwKLbxXXmM61QyDBYQ6Xiuc3NdFbAUXZVSWs1E7E6lhHdVkYll1MjHZFbGBSnbW5ZApvI5Ql+sNm4DDJhrQhsdN+uMH0n/oH4mnurw0BI+T9N/+VFXOxR5UMAINtxdygqzOxXahkJuMa+SmPpWu3UirqgK9fL/94x/fR4+/fv/3i/0FT6ZfPr9r8DikZQLGtEOWpAbEd9XFfz3ZsuwtM/y36/mK+JL30KFUscMrAvsrmDrlsq4bTs3MsNvXdgnt8owMkWL3sXMabhVT3S5wf6hviMGt4br7Tk3bsupdWQ7bVcPk3HZ1hDVGtCPLTJRiT6HYNqoAXGzNNG0BlQ5OCR0ivdinwk2ui3zmxfbVLEloKMonOjhw5AdLMAiJwjAJyeSNBz65IhwFXuKRP7Sd0Nwowf5sSa46/dC8ZaDNvDh4IzinK8AI0g2f0Wu6/fVsWc+6pQHoDFZ0q0+Ri51hcSjYcjIEex1FW//7qBLunotDhd6JCkPfImEeLrFWfGERShp9GQ5VA8FAJqNuAtfhoGBCHgpFW/9Q4L0llw8F2AkFOniRtgmX9Fa1IPJfSCOgPrLRt08PajHA7rcBCAQbbgjwKEAtC95BKIC8m+Tz43cOCBXBMxuYJnDQyODUnVwnanGK7RJiUMKHD46ohkBoFqNcgeI4f5LiuHL/bvrvZrvBvnif0njLwdRfwcEMv3/16S3cruLsw3C62ihbLpjAumJjqcYRroAjVIXdQN5RwXNEKvcigBMyejIzqJN59OYTKk4GdFqOvcYtdWWTUK9PQQMKVmRBEAhStSIXI3piX0BpvVcM9t+r3+1ivUvunvtzb70OfcZ0Ei0u2w30raVk1y0lB558QyYXXPeGTN6hcneEWfe/l/HT0mZKuXIxG94Wo5Qo3teBvL8MDEib/kQ0eaoJQP23aL5ZYG7mnGRn51B3aUkJZ7xjzamzBhMQpM6yhTJxc5frApLf2dlyds3j2sHYfZOeJct6xyE9yDh6EBuT3NPGEUSsQ6klCKg36POOUDggQ6QPbx+ejkN+R9guKid4H1vbOjDqi5eG+vEoIj5ohwlQVUiBElGSF2e7o07bfU4oep369cYx+Z82mYCOrms/D4klj+Nnespzdl7h0tl/27IBhjtE1EKbt+AsgXlhKbPgrtHL0+3wi/yk+Ah0yicj/IFPgAhkGmwqjc7Hu4sMTmWpNIZEYMwVImK98OZzbalNCOnTvQCKCct2gK5riefPotQuPAVCOK+g4Yq8goYgCp/VufvDiITfJ9UdcHz/ijMVInf3NKQqNcbW55rz/i5BMSBFWVQ1T9Ri7Xt4kH67isM1HhBcZmcyDk5dB44zPOYq4jKmlSXIkBExhqsIDUhiK5HuqpSyLZyBNMMoyzfLZEaMCdpeBUM1L25M6GFM4FBNjOM1VNIHvdhdmZQunzDJgizOBk/e7TzHk0Sh5E29nsAgzMcBAtEDZfqDIXK9UIvjtwfy39clsXiXPu938Tfxazr3M/9hwLv8pJJtGmOMGC9ll80rY6vvJtROuzgXXHdyTH7iDkERpmkyPh1XiiCKtp4tbIMFn6xbkd0dN4AaU91A9eEynFo5hqZ8HfVmnCHhydp/SWXWMaJomNviBiqXWiFDs+vvK8beYgCDQZwFcR1vfWVD65BoC8gaCPQtUx8o07hkklCvw6lJBhHtvFXT4OvsjC7tmdWLwJyz8XayWHbkgtp2pVBgOvUVx24PE2XPR22U2xud8l4xMCAUq1drOlyJj7Tkiz58pAa0mJ3CXhQFqqrUunX14+X/GLzR0gKDn/EtZXxLWmDiOcCrefRBk4CfF97Sm5KbUbQcs2U5IBBZPbYgStJStRija3SRdbvXyWQMfawlId1Izz3rwK36xOpT49AMxt01N85X5vKqGxQgBaqK3kESjrKLQ0p3EGVAiH6qzaLNGr9F8YtGRJ1QcWbxtGm7BvQzQg5iIutMQ+CFP2YgNoJnoe9LJJAcxSRotwdOl1nWW+5LgT/lSfgOYxAUddoU580Cx1Cv4CNR3mRWkI3oScva7CkKvY1LcGt+hk9aDC6ejv8GTfIIZLTIveipCy3/DEzj73yhOIsmnN3NcXqZaomUsiBcdgsNBeEu1c4oeaoPOwMh0+rHsoA2G3wBzOPZFuZ5RKCJKpNIRoxdMJMi2SzCC2FSy61v+qB8+08pk5qijFq6rZNXIPglml5P+YFyuvYST+ZadddWAdxDE1TZsohq+EyYHn8e5QhE9ezOls66HcD2cWjKdthivQigAQQIOqad/s94SWR35xCb6W+r2Z0jD1C/Tp6l0Bf/CREvWsD3KSecAUfvS8k0qZL5RL0ADTpma9Hhk/MzP4Va6aaH9AbdgnXPdU9sDMAJ1UvE72N8ebxLzY/NOik8sa2C79wQzpvkxS9K0qtTfnuV5+ra7XBua0VuH4NZVW0+5xqIwnH7eEtBaylUGX9hjP/chDEOvlLRTNKBF/kOG72ER/PKgnp4BdD5mGdR0JKyQTZ3cN9f7CDbUDI2TN0wSyQhcE7nhtC/Jjpqjn4pvOlVAcAbcen7NNaT/CVWTV7kXpWYkMnoYkWmZMcGRR/R3sK1VoJvLi+sRHl511arYzfs7OP3EEryWFVcARxsbQcbmBYDaGNgNxsXsjWULFcfVHqx60qcYbptF+mr9Bm78Yyc1ntmzzfcfkufCaUuKGMKMyPBo5XPLrlYdytfnXNV2/pKrLHcrc5PI6gb2qjYXMxus9Gd9hcSwZOP2iTGWMvqd2UVnDRij9D6PQYd6yQm+oHqQk4mqntrgclvJEPRCzN6iFURYwJeISa6IxD8MPbneR2vbeQBgQCNYVENAkHOlskHoqjK2BK+0UvC5DnNywK5wou6MKlF2evq2nQuxe8KbFwLm6Jhut4V2Plet6Y6y7x4VIx9NP5B5zfU0yEpxpV5v1o/r217KGISGzyjTW8LbK+IBytjlcPoit7p1o7RtPvtKcy3veOviF+v+/6LCO4KOh0BOAEydgYnjW/M8LkDq0qsradhVcTaLoL158icKjFW5/j+1U5MwwbhHodTJd3Kouyhg1+/WrUt9S/ROmnm1eugyQPewtojjEpqPOFLWIWPIKE1nYYNDebNQ5YtYENRWLgy3zjcofq2+i2IVmlKA5LZgCheP3+q7Qe4Qynryxzibc3oU42xwS/qIH1Rzb9wQgP/uQE/8zIQdf/FLhlaDanOvQOhS9LqdqblctEveDoBcMz51PYemiseYxOcdoSvflEAouJQqoZY+AASRHEiVyOjBmqmSA9UZpkJB6vPd4sf0dMokFonsFH3ivZ/6GkUDpVVGaqfjkbe0dgj/C7D0ShhLp6GUyHjaNTK6Xo6TpV5ZfUZehr3AbV9PE6VG3tRPNnBjsanj3WCF+RrGmZGqFb/Ff90NrY4G3tE0tk6G2Xe33wmlFgWFDvdtARHUcpbBHVOHCUTJnUtCwQ4otYtOfoiJ8fBS8QwJqQa/1wTel0TmsBztosCbMbWYVZeO0Il0/vqnwXpfd/wejOnIckiFMvl+BXOKxoOtyZQkmAwho4CDzsTX0RHlu/g8aTNbS4FtT3WVCbVXrMdwYoKy+pxdRNNGdhMftVQnyqRXZOJ8t8v3UA85wXv62mZawfkFxw2+NYpBn/ft+ApFxq8DKGJ3Bgd9Xl6SKvu5F3ONfjNeyMnD9OQ4lYubq3pc51cbFtsaR2hy+fobOyclhBudko1U00IriwfNJj9x+GDosjeiWRGq7Xo9fcEDJCFOkTXUhxEnTwFbwtoWb1PJk/0cw7uIbNDEzUPk9lJtNiKIqWzWbqWtc8MFFUH2haRbMvUPT0CnAMBkP501/RWDTG14TVk1mo0SvwC6JAB4E4pruQwjqKkejotPf0QBXTT9v5/</diagram></mxfile>

docs/Config.md

@@ -2,7 +2,7 @@
 
 We need
 * a [config file](#clusterconfig) containing cluster information , say `cluster.config`
-* a [config folder](https://github.com/Beuth-Erdelt/GEO-GPU-DBMS-Benchmarks#config-folder) for the benchmark tool, say `experiments/tpch/`, containing a config file `queries.config` for the [queries](https://github.com/Beuth-Erdelt/DBMS-Benchmarker#query-file)
+* a [config folder](https://github.com/Beuth-Erdelt/DBMS-Benchmarker/docs/Options.md#config-folder) for the benchmark tool, say `experiments/tpch/`, containing a config file `queries.config` for the [queries](https://github.com/Beuth-Erdelt/DBMS-Benchmarker/docs/Options.md#query-file)
 * some additional data depending on if it is an [AWS](#on-aws) or a [k8s](#on-k8s) cluster
 * a python script managing the experimental workflow, say `experiment-tpch.py`
 
@@ -207,3 +207,5 @@ Monitoring requires
   * `grafanatoken` and `grafanaurl` to access this from DBMSBenchmarker
 * A dict of exporters given as docker commands
   * Will be installed and activated automatically at each instance when `cluster.prepareExperiment()` is invoked.
+
+More information can be found [here](Monitoring.md)

requirements.txt

@@ -5,4 +5,4 @@ requests>=2.21.0
 scp>=0.13.2
 kubernetes>=9.0.0
 psutil>=5.6.1
-dbmsbenchmarker>=0.10.2
+dbmsbenchmarker>=0.10.3

setup.py

@@ -8,7 +8,7 @@
 
 setuptools.setup(
     name="bexhoma",
-    version="0.4.3",
+    version="0.4.4",
     author="Patrick Erdelt",
     author_email="perdelt@beuth-hochschule.de",
     description="This Python tools helps managing DBMS Benchmarking experiments in a HPC cluster environment. It supports AWS and Kubernetes (K8s).",