Test Performance on CI #170
Conversation
JRJurman
force-pushed
the
performance-test-on-ci
branch
2 times, most recently
from
306119e
to
809793b
Compare
JRJurman
force-pushed
the
performance-test-on-ci
branch
from
809793b
to
e505603
Compare
| @@ -10,7 +10,7 @@ const start = require('./start') | |||
| * @package | |||
| * @public | |||
| * @description | |||
| * Tram-One is a light View Framework that comes with all the dependencies you need to start developing on the web. | |||
| * Tram-One is a Modern View Framework that has advance features like hooks, observables, and JSX-like template components, all in plain vanilla javascript. | |||
There was a problem hiding this comment.
needed something to test that the badge sizes would be updated, this worked, and this needed to be updated anyways.
| expect(stats['0050'].averageTime / 5).not.toBeGreaterThan(stats['0010'].averageTime + buffer) | ||
| expect(stats['0500'].averageTime / 5).not.toBeGreaterThan(stats['0100'].averageTime + buffer) | ||
| expect(stats['5000'].averageTime / 5).not.toBeGreaterThan(stats['1000'].averageTime + buffer) |
There was a problem hiding this comment.
using averageTime instead of medianTime, since that should be the least error prone (with the outliers removed)
| // number of times to run the test | ||
| const NUMBER_OF_RUNS = 50 | ||
| // the number of tests to toss (slowest and fastest) | ||
| const BUFFER = 10 |
There was a problem hiding this comment.
including a buffer now, which is the number of slowest and fastest runs that we'll toss when doing calculations. In reality, this means that we are only getting stats for 30 runs (50 runs, - 10 slowest, - 10 fastest), but these should be more consistent overall.
| - store_artifacts: | ||
| path: performance-tests/__snapshots__ |
There was a problem hiding this comment.
while not totally required, this does give us a historical view within a branch (since when we make our -artifacts branch, we always just force-push over the commit)
| if [[ -z "${CIRCLE_PULL_REQUEST}" ]] | ||
| then | ||
| echo "Master Build, checkout to master" | ||
| git checkout | ||
| else | ||
| echo "PR Build, checkout to new branch" | ||
| git checkout -b ${CIRCLE_BRANCH}-artifacts | ||
| fi |
There was a problem hiding this comment.
Some configurations only commit when on the master branch, but since we want information on PRs as well, this is allows us to account for both 👍
| echo "PR Build, checkout to new branch" | ||
| git checkout -b ${CIRCLE_BRANCH}-artifacts | ||
| fi | ||
| git commit --allow-empty -am "Update Snapshots & Badges [skip ci]" |
There was a problem hiding this comment.
[skip ci] is important here, this actually prevents CircleCI from infinitely triggering builds.
| git checkout -b ${CIRCLE_BRANCH}-artifacts | ||
| fi | ||
| git commit --allow-empty -am "Update Snapshots & Badges [skip ci]" | ||
| git push -fu origin HEAD |
There was a problem hiding this comment.
we force push the artifacts onto the branch, this is required for the Pull Request use case, less so for master, but I don't foresee that being an actual issue.
Summary
This PR adds the configuration to run the performance tests on CircleCI, makes them a little more consistent, and pushes the performance tests (and build badges!) back to the repo.
-artifactssuffix.Performance / Size
Performance Results - performance-test-on-ci...performance-test-on-ci-artifacts
Resources
Checklist