Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run tests in pool_test.go as subtests #262

Merged
merged 3 commits into from
Oct 13, 2020

Conversation

jholdstock
Copy link
Member

@jholdstock jholdstock commented Oct 13, 2020

This updates the "parent" test in pool_test.go to utilize the go testing framework's concept of subtests.

A new database is created for each subtest. This removes the need for each test to have its own specific clean up, and guarantees each test will have a clean database which is not contaminated by any previous tests. The performance penalty for repeatedly creating new databases is negligible.

Utilizing subtests allows the testing framework to provide more detailed feedback about the test run. Information about each subtest is logged, rather than just one log line for the parent test.

Managing the creation, cleaning and deleting of test databases in the "parent" test, outside of the "child" test files, is going to be useful for #257 because it is a good step towards allowing the test files to be database agnostic.

As an immediate example of how the detailed feedback can be useful, the example below shows that of the 10 second runtime, almost all of it is consumed by testClient. Previously this fact was hidden because it just appeared that the parent test TestPool was taking 10 seconds.

Log before (click to expand)
$ go test -v -count 1 ./...
?       github.com/decred/dcrpool       [no test files]
?       github.com/decred/dcrpool/cmd/miner     [no test files]
?       github.com/decred/dcrpool/gui   [no test files]
=== RUN   TestErrorKindStringer
--- PASS: TestErrorKindStringer (0.00s)
=== RUN   TestError
--- PASS: TestError (0.00s)
=== RUN   TestErrorKindIsAs
--- PASS: TestErrorKindIsAs (0.00s)
=== RUN   TestPool
--- PASS: TestPool (9.93s)
=== RUN   TestUpgrades
=== PAUSE TestUpgrades
=== CONT  TestUpgrades
=== RUN   TestUpgrades/group
=== RUN   TestUpgrades/group/test0
=== PAUSE TestUpgrades/group/test0
=== RUN   TestUpgrades/group/test1
=== PAUSE TestUpgrades/group/test1
=== RUN   TestUpgrades/group/test2
=== PAUSE TestUpgrades/group/test2
=== CONT  TestUpgrades/group/test0
=== CONT  TestUpgrades/group/test2
=== CONT  TestUpgrades/group/test1
--- PASS: TestUpgrades (0.00s)
    --- PASS: TestUpgrades/group (0.00s)
        --- PASS: TestUpgrades/group/test2 (0.00s)
        --- PASS: TestUpgrades/group/test0 (0.00s)
        --- PASS: TestUpgrades/group/test1 (0.00s)
PASS
ok      github.com/decred/dcrpool/pool  9.957s
Log after (click to expand)
$ go test -v -count 1 ./...
?       github.com/decred/dcrpool       [no test files]
?       github.com/decred/dcrpool/cmd/miner     [no test files]
?       github.com/decred/dcrpool/gui   [no test files]
=== RUN   TestDifficulty
--- PASS: TestDifficulty (0.00s)
=== RUN   TestErrorKindStringer
--- PASS: TestErrorKindStringer (0.00s)
=== RUN   TestError
--- PASS: TestError (0.00s)
=== RUN   TestErrorKindIsAs
--- PASS: TestErrorKindIsAs (0.00s)
=== RUN   TestLimiter
--- PASS: TestLimiter (0.00s)
=== RUN   TestPool
=== RUN   TestPool/testPaymentMgr
=== RUN   TestPool/testHub
=== RUN   TestPool/testInitDB
=== RUN   TestPool/testDatabase
=== RUN   TestPool/testShares
=== RUN   TestPool/testPayment
=== RUN   TestPool/testEndpoint
=== RUN   TestPool/testChainState
=== RUN   TestPool/testFetchBucketHelpers
=== RUN   TestPool/testAcceptedWork
=== RUN   TestPool/testAccount
=== RUN   TestPool/testJob
=== RUN   TestPool/testClient
--- PASS: TestPool (10.00s)
    --- PASS: TestPool/testPaymentMgr (0.12s)
    --- PASS: TestPool/testHub (0.02s)
    --- PASS: TestPool/testInitDB (0.01s)
    --- PASS: TestPool/testDatabase (0.01s)
    --- PASS: TestPool/testShares (0.00s)
    --- PASS: TestPool/testPayment (0.01s)
    --- PASS: TestPool/testEndpoint (0.10s)
    --- PASS: TestPool/testChainState (0.03s)
    --- PASS: TestPool/testFetchBucketHelpers (0.00s)
    --- PASS: TestPool/testAcceptedWork (0.01s)
    --- PASS: TestPool/testAccount (0.00s)
    --- PASS: TestPool/testJob (0.01s)
    --- PASS: TestPool/testClient (9.60s)
=== RUN   TestUpgrades
=== PAUSE TestUpgrades
=== CONT  TestUpgrades
=== RUN   TestUpgrades/group
=== RUN   TestUpgrades/group/test0
=== PAUSE TestUpgrades/group/test0
=== RUN   TestUpgrades/group/test1
=== PAUSE TestUpgrades/group/test1
=== RUN   TestUpgrades/group/test2
=== PAUSE TestUpgrades/group/test2
=== CONT  TestUpgrades/group/test0
=== CONT  TestUpgrades/group/test2
=== CONT  TestUpgrades/group/test1
--- PASS: TestUpgrades (0.00s)
    --- PASS: TestUpgrades/group (0.00s)
        --- PASS: TestUpgrades/group/test0 (0.00s)
        --- PASS: TestUpgrades/group/test1 (0.00s)
        --- PASS: TestUpgrades/group/test2 (0.00s)
PASS
ok      github.com/decred/dcrpool/pool  10.035s

Also:

  • Remove Difficulty and Limiter tests from the parent test in pool_test.go. These can be run standalone with no issues.
  • Move emptyBucket func from production code into test code (it is only used by tests)

There is no need for these to be invoked in pool_test.go, they can be run in isolation.
Move the emptyBucket func out of production code because it is only used by tests.
This updates the "parent" test in pool_test.go to utilize the go testing framework's concept of subtests.

A new database is created for each subtest. This removes the need for each test to have its own specific clean up, and guarantees each test will have a clean database which is not contaminated by any previous tests. The performance penalty for repeatedly creating new databases is negligible.

Utilizing subtests allows the testing framework to provide more detailed feedback about the test run. Information about each subtest is logged, rather than just one log line for the parent test.
Copy link
Member

@dnldd dnldd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nicely done.

@jholdstock jholdstock merged commit 7b62039 into decred:master Oct 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants