V0.6.1 Refactoring, Naming and Docs (#113)

* Docs: PyPi badge * Masterscript: store parameters of loading and maintaining in connection for evaluation * Masterscript: store parameters of loading and maintaining in connection for evaluation, added to existing parameter dict * Masterscript: fill message queue just before lauching job (important for rerunning the same dbms) * Refactoring: Naming in class experiment * Experiments: Detached as default, cannot be overwritten by subclasses * Experiments: Detached as default, cannot be overwritten by subclasses * Refactoring: Naming in class experiment * Refactoring: Naming in class cluster * Refactoring: Removed deprecated methods from class experiment * Refactoring: Removed deprecated methods from class cluster (master testbeds) * Refactoring: Removed class for IaaS management at AWS, since this is deprecated * Refactoring: Typo in class experiment * Refactoring: Naming in class cluster (master testbed) * Refactoring: Doc in class experiment * Currently ignore API docs * Refactoring: Naming in class cluster (master testbed) * Refactoring: Remove deprecated methods (moved to experiment or configuration) * Refactoring: Docs in experiment class * Refactoring: Remove deprecated methods from experiment (do not send to cluster class anymore) * Refactoring: Docs in experiment class * Refactoring: Docs in experiment class * Refactoring: Naming in class configuration * Refactoring: Naming in class configuration and same indention * Refactoring: Naming in class configuration * Refactoring: Indention in prometheus YAML corrected * Refactoring: Naming in class configuration * Refactoring: Docs in configuration class * Refactoring: Docs in configuration class * Refactoring: Docs in configuration class * Refactoring: Indention in prometheus YAML corrected * Refactoring: Indention set to 4 spaces because of prometheus YAML * Master: cluster.start_messagequeue() * Master: merged into clusters * Refactoring: Docs in clusters class * Clusters: Name conflict kubernetes package * Clusters: Only start new dashboard if old is not running * Clusters: Test if dashboard is running * Clusters: Name conflict kubernetes package * Refactoring: Clean imports in configuration class * Readme: References * JOSS: First draft * JOSS: Contributing sections * JOSS: DBMS * JOSS: Intro * Masterscript: Hammerdb benchmarker in config * Masterscript: Hammerdb benchmarker in experiments as only benchmarker * Masterscript: Hammerdb benchmarker in experiments as only benchmarker in comment * Masterscript: Hammerdb as configuration class * Masterscript: Hammerdb as configuration class with number of vusers as parameter * Masterscript: Hammerdb collects results in df * Docs: Tests for API documentation in Sphinx * Docs: Tests for API documentation in Sphinx different theme * Docs: API documentation in Sphinx * Masterscript: Hammerdb more log messages * Docs: Intros nicer * Masterscript: Hammerdb collects results in df * Masterscript: Hammerdb as configuration class with number of client as parameter * Masterscript: Hammerdb as configuration class for MySQL * MySQL: 8.0.31 * Masterscript: Copy .pickle to /results * Masterscript: Store benchmarker latest jobname * Masterscript: Benchmarker jobname includes number of experiment * Masterscript: Print HammerDB results * Masterscript: Print self.loading_parameters * Masterscript: create_manifest_job() test for loading * Masterscript: create_manifest_job() test for loading including template * Masterscript: create_manifest_job() test for dbmsbenchmarker * Masterscript: create_manifest_job() test for hammerdb * Masterscript: create_manifest_job() test for benchmarking debug * Masterscript: create_manifest_job() test for benchmarking initContainers optional * Masterscript: Print HammerDB results * Masterscript: create_manifest_job() test for maintaining * Masterscript: create_manifest_job() test for maintaining debug * Masterscript: create_manifest() naming * Masterscript: create_manifest_job() test for maintaining debug * HammerDB: Use dbmsbenchmarker monitoring * HammerDB: TPC-C needs query file (although no queries are run by dbmsbenchmarker) * HammerDB: Use dbmsbenchmarker monitoring * HammerDB: Use dbmsbenchmarker monitoring debug * Masterscript: create_manifest_job() replaces other deprecated methods * HammerDB: Use dbmsbenchmarker monitoring in dashboard * Masterscript: delete maintaining pods when sut is stopped * Masterscript: Job always has experimentRun in name * Masterscript: delete maintaining pods when sut is stopped * Masterscript: volume as ENV and label * Masterscript: Download evaluations at end of experiment * Masterscript: Test result of experiment * Masterscript: Restart dashboard without deleting deployment * Masterscript: Number of parallel loading and maintaining pods can be different from total number * Masterscript: config.jobtemplate_loading * Masterscript: name of deployment and job files * Masterscript: config.jobtemplate_loading * Masterscript: number of loading and maintaining pods must be int * Masterscript: Evaluate correct pod for TPC-C kind of setting * Masterscript: Clean results when benchmarking in experiment ends * TPCC: Name result df like connection * TPCC: Sketch of test_results() * TSBS: Custome end_benchmarking() * TSBS: Custom end_loading() * Masterscript: BEXHOMA_URL in job manifest * Masterscript: BEXHOMA_USER and BEXHOMA_PASSWORD in job manifest * Masterscript: YCSB classes for experiment and benchmarking * Masterscript: YCSB classes include connectionmanagement * Masterscript: Comments * Masterscript: YCSB collect results * Masterscript: Print less env of jobs * Masterscript: BEXHOMA_DRIVER * Masterscript: Treat jobs that jump from pending to success * Masterscript: BEXHOMA_JAR * Masterscript: log_to_df() exception * Masterscript: nodegroup defaults for all job components * Masterscript: accept YCSB job to go into success within 10 seconds * Masterscript: Loading waits 30s for synch * Masterscript: Loading waits 30s for synch dump envs * Masterscript: Loading marks storage also for distributed setting * Masterscript: Loading marks storage also for distributed setting with custom storage name * Masterscript: No loading pods if PVC exists * Masterscript: Start monitoring also if PVC exists * Masterscript: Mark loading times at PVC corrected
Beuth-Erdelt · Dec 2, 2022 · 5cab3f4 · 5cab3f4
1 parent ec6bd6e
commit 5cab3f4
Show file tree

Hide file tree

Showing 34 changed files with 4,976 additions and 4,211 deletions.
diff --git a/.github/workflows/draft-pdf.yml b/.github/workflows/draft-pdf.yml
@@ -0,0 +1,23 @@
+on: [push]
+
+jobs:
+  paper:
+    runs-on: ubuntu-latest
+    name: Paper Draft
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: Build draft PDF
+        uses: openjournals/openjournals-draft-action@master
+        with:
+          journal: joss
+          # This should be the path to the paper within your repo.
+          paper-path: paper.md
+      - name: Upload
+        uses: actions/upload-artifact@v1
+        with:
+          name: paper
+          # This is the output path where Pandoc will write the compiled
+          # PDF. Note, this should be the same directory as the input
+          # paper.md
+          path: paper.pdf
diff --git a/.gitignore b/.gitignore
@@ -9,3 +9,5 @@ build/*
 bexhoma.egg-info/*
 dist/*
 .spyproject/*
+docs/html/*
+api/*
diff --git a/README.md b/README.md
@@ -1,13 +1,15 @@
 [![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://GitHub.com/Beuth-Erdelt/Benchmark-Experiment-Host-Manager/graphs/commit-activity)
 [![GitHub release](https://img.shields.io/github/release/Beuth-Erdelt/Benchmark-Experiment-Host-Manager.svg)](https://GitHub.com/Beuth-Erdelt/Benchmark-Experiment-Host-Manager/releases/)
+[![PyPI version](https://badge.fury.io/py/bexhoma.svg)](https://badge.fury.io/py/bexhoma)
 
-# Benchmark Experiment Host Manager
+# Benchmark Experiment Host Manager (Bexhoma)
 This Python tools helps **managing benchmark experiments of Database Management Systems (DBMS) in a Kubernetes-based High-Performance-Computing (HPC) cluster environment**.
 It enables users to configure hardware / software setups for easily repeating tests over varying configurations.
 
 It serves as the **orchestrator** [2] for distributed parallel benchmarking experiments in a Kubernetes Cloud.
-This has been tested at Amazon Web Services, Google Cloud, Microsoft Azure, IBM Cloud und Oracle Cloud and at Minikube installations,
-running with Citus, Clickhouse, DB2, Exasol, MariaDB, MariaDB Columnstore, MemSQL, MonetDB, MySQL, OmniSci, Oracle DB, PostgreSQL, SingleStore, SQL Server and SAP HANA.
+This has been tested at Amazon Web Services, Google Cloud, Microsoft Azure, IBM Cloud, Oracle Cloud, and at Minikube installations,
+running with Clickhouse, Exasol, Citus Data (Hyperscale), IBM DB2, MariaDB, MariaDB Columnstore, MemSQL (SingleStore), MonetDB, MySQL, OmniSci (HEAVY.AI), Oracle DB, PostgreSQL, SQL Server, SAP HANA, TimescaleDB, and Vertica.
+.
 
 <p align="center">
     <img src="https://raw.githubusercontent.com/Beuth-Erdelt/Benchmark-Experiment-Host-Manager/master/docs/experiment-with-orchestrator.png" width="800">
@@ -18,6 +20,8 @@ A more advanced workflow is: Plan a sequence of such experiments, run plan as a
 
 See the [homepage](https://github.com/Beuth-Erdelt/Benchmark-Experiment-Host-Manager) and the [documentation](https://bexhoma.readthedocs.io/en/latest/).
 
+If you encounter any issues, please report them to our [Github issue tracker](https://github.com/Beuth-Erdelt/Benchmark-Experiment-Host-Manager/issues).
+
 ## Installation
 
 1. Download the repository: https://github.com/Beuth-Erdelt/Benchmark-Experiment-Host-Manager
@@ -50,23 +54,42 @@ The repository contains a [tool](experiments/tpch/) for running TPC-H (reading)
 
 ## More Informations
 
-For full power, use this tool as an orchestrator as in [2]. It also starts a monitoring container using [Prometheus](https://prometheus.io/) and a metrics collector container using [cAdvisor](https://github.com/google/cadvisor). It also uses the Python package [dbmsbenchmarker](https://github.com/Beuth-Erdelt/Benchmark-Experiment-Host-Manager) as query executor [2] and evaluator [1].
+For full power, use this tool as an orchestrator as in [2]. It also starts a monitoring container using [Prometheus](https://prometheus.io/) and a metrics collector container using [cAdvisor](https://github.com/google/cadvisor). It also uses the Python package [dbmsbenchmarker](https://github.com/Beuth-Erdelt/DBMS-Benchmarker), [3], as query executor and evaluator as in [1,2].
 See the [images](https://github.com/Beuth-Erdelt/Benchmark-Experiment-Host-Manager/tree/master/images/) folder for more details.
 
+## Contributing, Bug Reports
+
+If you have any question or found a bug, please report them to our [Github issue tracker](https://github.com/Beuth-Erdelt/Benchmark-Experiment-Host-Manager/issues).
+In any bug report, please let us know:
+
+* Which operating system and hardware (32 bit or 64 bit) you are using
+* Python version
+* Bexhoma version (or git commit/date)
+* Traceback that occurs (the full error message)
+
+We are always looking for people interested in helping with code development, documentation writing, technical administration, and whatever else comes up.
+If you wish to contribute, please first read the [contribution section](https://github.com/Beuth-Erdelt/Benchmark-Experiment-Host-Manager/blob/master/docs/CONTRIBUTING.md) or visit the [documentation](https://bexhoma.readthedocs.io/en/latest/CONTRIBUTING.html).
+
 ## References
 
-[1] [A Framework for Supporting Repetition and Evaluation in the Process of Cloud-Based DBMS Performance Benchmarking](https://doi.org/10.1007/978-3-030-84924-5_6)
+If you use Bexhoma in work contributing to a scientific publication, we kindly ask that you cite our application note [2] or [1]:
 
+[1] [A Framework for Supporting Repetition and Evaluation in the Process of Cloud-Based DBMS Performance Benchmarking](https://doi.org/10.1007/978-3-030-84924-5_6)
 > Erdelt P.K. (2021)
 > A Framework for Supporting Repetition and Evaluation in the Process of Cloud-Based DBMS Performance Benchmarking.
 > In: Nambiar R., Poess M. (eds) Performance Evaluation and Benchmarking. TPCTC 2020.
 > Lecture Notes in Computer Science, vol 12752. Springer, Cham.
 > https://doi.org/10.1007/978-3-030-84924-5_6
 
-
-[2] [Orchestrating DBMS Benchmarking in the Cloud with Kubernetes](https://www.researchgate.net/publication/353236865_Orchestrating_DBMS_Benchmarking_in_the_Cloud_with_Kubernetes)
+[2] [Orchestrating DBMS Benchmarking in the Cloud with Kubernetes](https://doi.org/10.1007/978-3-030-94437-7_6)
 > Erdelt P.K. (2022)
 > Orchestrating DBMS Benchmarking in the Cloud with Kubernetes.
 > In: Nambiar R., Poess M. (eds) Performance Evaluation and Benchmarking. TPCTC 2021.
 > Lecture Notes in Computer Science, vol 13169. Springer, Cham.
 > https://doi.org/10.1007/978-3-030-94437-7_6
+
+[3] [DBMS-Benchmarker: Benchmark and Evaluate DBMS in Python](https://doi.org/10.21105/joss.04628)
+> Erdelt P.K., Jestel J. (2022).
+> DBMS-Benchmarker: Benchmark and Evaluate DBMS in Python.
+> Journal of Open Source Software, 7(79), 4628
+> https://doi.org/10.21105/joss.04628
diff --git a/bexhoma/__init__.py b/bexhoma/__init__.py
@@ -1,4 +1,4 @@
 """
 The clustermanager module
 """
-__all__ = ["masterK8s", "clusters", "experiments", "configurations"]
+__all__ = ["clusters", "experiments", "configurations"]
diff --git a/bexhoma/__pycache__/.gitignore b/bexhoma/__pycache__/.gitignore
@@ -1,3 +1,4 @@
 *.cpython-36.pyc
 *.cpython-37.pyc
 *.cpython-38.pyc
+*.cpython-39.pyc