Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trade safety for speed on test database #1857

Merged
merged 2 commits into from Mar 12, 2019

Conversation

3 participants
@jim
Copy link
Contributor

jim commented Mar 12, 2019

Description

Postgres does a lot of work to make sure that data isn't lost in the event of a system failure. Some of this work comes at the expense of performance, however, and since our testing database is by definition ephemeral, we'd rather have things go as fast as possible.

This PR disables two settings on the milmove-db-test container, fsync and full_page_writes. Here are a few before/after numbers:

Before

$ env DB_PORT=$DB_PORT_TEST time go test -count=1 -test.v ./pkg/models
PASS
ok  	github.com/transcom/mymove/pkg/models	197.235s
      200.46 real        12.39 user         6.02 sys
$ time make server_test
...
      675.98 real        87.06 user        49.69 sys

After

$ env DB_PORT=$DB_PORT_TEST time go test -count=1 -test.v ./pkg/models
PASS
ok  	github.com/transcom/mymove/pkg/models	87.881s
       90.85 real        12.06 user         5.76 sys
$ time make server_test
...
      425.64 real        87.45 user        51.49 sys

Docker desktop is set to 4 CPUs and 8 gig of RAM on my machine.

Setup

You'll need to run make db_test_reset to destroy and recreate the test docker container if you already have one running.

jim added some commits Mar 12, 2019

@jim jim requested review from chrisgilmerproj , lynzt and reggieriser Mar 12, 2019

@jim

This comment has been minimized.

Copy link
Contributor Author

jim commented Mar 12, 2019

@jamesatheyDDS

This comment has been minimized.

Copy link
Contributor

jamesatheyDDS commented Mar 12, 2019

@codecov

This comment has been minimized.

Copy link

codecov bot commented Mar 12, 2019

Codecov Report

Merging #1857 into master will decrease coverage by 0.09%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master    #1857     +/-   ##
=========================================
- Coverage   49.61%   49.52%   -0.1%     
=========================================
  Files         428      428             
  Lines       18446    18444      -2     
  Branches     1632     1632             
=========================================
- Hits         9152     9134     -18     
- Misses       8492     8508     +16     
  Partials      802      802

@jim jim requested a review from pjdufour-truss Mar 12, 2019

@chrisgilmerproj
Copy link
Contributor

chrisgilmerproj left a comment

🚀 - If this works out we may want to consider using it in other places, like e2e.

@jim jim merged commit 6a46825 into master Mar 12, 2019

18 of 19 checks passed

codecov/project 49.52% (-0.1%) compared to 488e448
Details
Codacy/PR Quality Review Up to standards. A positive pull request.
Details
ci/circleci: acceptance_tests_experimental Your tests passed on CircleCI!
Details
ci/circleci: acceptance_tests_local Your tests passed on CircleCI!
Details
ci/circleci: acceptance_tests_staging Your tests passed on CircleCI!
Details
ci/circleci: build_app Your tests passed on CircleCI!
Details
ci/circleci: build_migrations Your tests passed on CircleCI!
Details
ci/circleci: build_tools Your tests passed on CircleCI!
Details
ci/circleci: client_test Your tests passed on CircleCI!
Details
ci/circleci: client_test_coverage Your tests passed on CircleCI!
Details
ci/circleci: integration_tests_mymove Your tests passed on CircleCI!
Details
ci/circleci: integration_tests_office Your tests passed on CircleCI!
Details
ci/circleci: integration_tests_tsp Your tests passed on CircleCI!
Details
ci/circleci: pre_deps_golang Your tests passed on CircleCI!
Details
ci/circleci: pre_deps_yarn Your tests passed on CircleCI!
Details
ci/circleci: pre_test Your tests passed on CircleCI!
Details
ci/circleci: server_test Your tests passed on CircleCI!
Details
ci/circleci: server_test_coverage Your tests passed on CircleCI!
Details
codecov/patch Coverage not affected when comparing 488e448...698108b
Details

@jim jim deleted the jb-db-test-no-fsync branch Mar 12, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.