Skip to content
This repository was archived by the owner on Mar 24, 2026. It is now read-only.

45216 continuousbenchmarking 20160926 asb 4#2292

Merged
knewmanTE merged 35 commits intoTechEmpower:masterfrom
ashawnbandy-te-tfb:45216-continuousbenchmarking-20160926-asb-4
Sep 27, 2016
Merged

45216 continuousbenchmarking 20160926 asb 4#2292
knewmanTE merged 35 commits intoTechEmpower:masterfrom
ashawnbandy-te-tfb:45216-continuousbenchmarking-20160926-asb-4

Conversation

@ashawnbandy-te-tfb
Copy link
Copy Markdown
Contributor

The work in this PR introduces a basic facility for continuous benchmarking. run-continuously is essentially a while-loop that makes 'life-cycle' calls in addition to initiating a benchmarking run. Additionally, an example of an upstart process configuration is included.

Below are some relevant notes from a recent commit:

toolset/run-continuously.sh is simply a while loop that
removes and rebuilds the framework benchmark suite and
then runs the benchmarks.

There are five environment variables to be set:

  • TFB_REPOPARENT the absolute path to the folder containing
    the repository
  • TFB_REPONAME the name of the repository folder
  • TFB_REPOURI the URI of the repository
  • TFB_REPOBRANCH the branch of the repository to use
  • TFB_LOGSFOLDER the folder to archive log files

There are four life cycle stages in run-continuously:

  • tear-down-environment.sh called upon to remove the existing
    environment
  • rebuild-environment.sh called upon to rebuild the environment
  • pre-run-tests scripts in this folder are called
    before the benchmarks are run
  • post-run-tests scripts in this folder are called
    after the benchmarks are run
  • Notes -

run-continuously.sh generally assumes that a clone of the
appropriate repo and branch exist and have the list
scripts available and an appropriate copy of benchmark.cfg
in place. Some effort is made to support starting states
that differ but those are not intended to be the general
case for use.

At the end of each run (post-run-tests) there are two python scripts. One zips the results.json file and sends it to a specified email address. The other makes copies of the logs for each frameworks (each are independently zipped while being copied).

Changes made following the previous PR for this work:

  1. Added @knewmanTE's work towards tagging the results with a commit.
  2. The mailing list environment variables are now split into two, on for TO: and one for FROM:
  3. I removed a duplicated block of code in email-results.py.

methane and others added 30 commits September 9, 2016 13:59
You are using a debug setup.
The proposed changes should give some benefit
TechEmpower#2237)

* Postgres/MySQL: Restrict database permissions to just what is required

Some frameworks were misbehaving and changing the schema.

Also remove upper-case verions of MySQL tables as those are not needed.

* Fix GRANT syntax error
toolset/run-continuously.sh is simply a while loop that
removes and rebuilds the framework benchmark suite and
then runs the benchmarks.

There are five environment variables to be set:

* TFB_REPOPARENT     the absolute path to the folder containing
  the repository
* TFB_REPONAME       the name of the repository folder
* TFB_REPOURI        the URI of the repository
* TFB_REPOBRANCH     the branch of the repository to use
* TFB_LOGSFOLDER     the folder to archive log files

There are four life cycle stages in run-continuously:

* tear-down-environment.sh  called upon to remove the existing
  environment
* rebuild-environment.sh    called upon to rebuild the environment
* pre-run-tests             scripts in this folder are called
                            before the benchmarks are run
* post-run-tests            scripts in this folder are called
                            after the benchmarks are run

- Notes -

run-continuously generally assumes that a clone of the
appropriate repo and branch exist and have the list
scripts available and an appropriate copy of benchmark.cfg
in place.  Some effort is made to support starting states
that differ but those are not intended to be the general
case for use.
…nto 45216-continuousbenchmarking-20160609-asb-2
@ashawnbandy-te-tfb
Copy link
Copy Markdown
Contributor Author

Also, I will defer the work related to @knewmanTE's PR #2283 until after it has been merged into master.

@knewmanTE knewmanTE merged commit 8ce974a into TechEmpower:master Sep 27, 2016
@knewmanTE
Copy link
Copy Markdown
Contributor

Merge'd!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants