Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable test result collection for go tests in CircleCI #2754

Merged
merged 8 commits into from Oct 4, 2019

Conversation

@gidjin
Copy link
Contributor

commented Oct 1, 2019

Description

The server_test job seems to fail irregularly with various test failures. In an effort to identify what's going on this PR changes server_test to output the test results so that CircleCI can read and collect the results. Once we start collecting these results we can see if the failures are common to specific tests or not

Reviewer Notes

This change makes make server_test run go test -v instead of just go test which means the output is different and more verbose. If you feel this is confusing to other engineers on the project we can do something else, like add another make command for running and outputting the results.

Second, this is to start collecting data, not resolve the problem.

Setup

This is primarily a change to CircleCI

Run make server_test and see the output is different
See build collect results as artifacts, example: https://circleci.com/gh/transcom/mymove/171523#artifacts/containers/0

Code Review Verification Steps

  • Request review from a member of a different team.

References

@gidjin gidjin force-pushed the gdjn_168881057_find_out_why_server_tests_fail branch 2 times, most recently from a442a92 to 92f24de Oct 2, 2019
@gidjin gidjin requested review from jim, reggieriser and chrisgilmerproj Oct 2, 2019
@gidjin gidjin marked this pull request as ready for review Oct 2, 2019
@gidjin gidjin changed the title WIP To fix server_test job Enable test result collection for go tests in CircleCI Oct 2, 2019
@gidjin gidjin added the A-Team label Oct 2, 2019
- run:
# This will eventually need to move to the base circleci image, but for now it is here for testing
name: Install go-junit-report
command: go get -u github.com/jstemmer/go-junit-report

This comment has been minimized.

Copy link
@chrisgilmerproj

chrisgilmerproj Oct 2, 2019

Contributor

Can we add this to pkg/tools/tools.go so that we get it as part of the pre_deps_golang step? I want to make sure we aren't getting any deps in any job that is not pre_deps_golang so we don't have network issues. I'm still trying to eliminate places where we get deps outside of that job.

@@ -402,7 +406,9 @@ jobs:
command: |
echo 'export LOGIN_GOV_SECRET_KEY=$(echo $E2E_LOGIN_GOV_SECRET_KEY | base64 --decode)' >> $BASH_ENV
source $BASH_ENV
make server_test
mkdir -p test-results/gotest

This comment has been minimized.

Copy link
@chrisgilmerproj

chrisgilmerproj Oct 2, 2019

Contributor

If you can eliminate making a new directory I would. This might play funny on developer machines. If you must make a new directory then let's use the tmp/ directory which we use for other stuff already.

This comment has been minimized.

Copy link
@gidjin

gidjin Oct 3, 2019

Author Contributor

The results need to go somewhere and it seems that CircleCI expects the structure to be parent_folder/language See the docs. I'll move it into tmp folder as that makes sense to me, but I'm not clear how this will do anything on a developer machine since it's only going to be running in a CircleCI environment.

Copy link
Contributor

left a comment

Just a few changes. Otherwise I'm excited to get this merged.

@gidjin gidjin force-pushed the gdjn_168881057_find_out_why_server_tests_fail branch 4 times, most recently from 3899061 to 732566f Oct 3, 2019
@gidjin gidjin requested a review from chrisgilmerproj Oct 4, 2019
Copy link
Contributor

left a comment

🚀

@gidjin gidjin force-pushed the gdjn_168881057_find_out_why_server_tests_fail branch from 732566f to c4ab9ce Oct 4, 2019
@gidjin gidjin merged commit cffa70a into master Oct 4, 2019
18 checks passed
18 checks passed
ci/circleci: acceptance_tests_experimental Your tests passed on CircleCI!
Details
ci/circleci: acceptance_tests_local Your tests passed on CircleCI!
Details
ci/circleci: acceptance_tests_staging Your tests passed on CircleCI!
Details
ci/circleci: build_app Your tests passed on CircleCI!
Details
ci/circleci: build_migrations Your tests passed on CircleCI!
Details
ci/circleci: build_storybook_app Your tests passed on CircleCI!
Details
ci/circleci: build_tasks Your tests passed on CircleCI!
Details
ci/circleci: build_tools Your tests passed on CircleCI!
Details
ci/circleci: check_generated_code Your tests passed on CircleCI!
Details
ci/circleci: client_test Your tests passed on CircleCI!
Details
ci/circleci: integration_tests Your tests passed on CircleCI!
Details
ci/circleci: pre_deps_golang Your tests passed on CircleCI!
Details
ci/circleci: pre_deps_yarn Your tests passed on CircleCI!
Details
ci/circleci: pre_test Your tests passed on CircleCI!
Details
ci/circleci: server_test Your tests passed on CircleCI!
Details
ci/circleci: server_test_coverage Your tests passed on CircleCI!
Details
codecov/patch/go Coverage not affected when comparing aa1ce71...c4ab9ce
Details
codecov/project/go 57.3% remains the same compared to aa1ce71
Details
@gidjin gidjin deleted the gdjn_168881057_find_out_why_server_tests_fail branch Oct 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
2 participants
You can’t perform that action at this time.