Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Remove CLI tests from make test #1643

Merged
merged 1 commit into from
Jul 17, 2023

Conversation

AndrewSisley
Copy link
Contributor

Relevant issue(s)

Resolves #1642

Description

Removes CLI tests from make test as they take ages to run and it is disruptive to dev-flow.

Also adds make test:short which is even leaner, skipping all the extras like the race detector.

Runtimes on my machine:
Old make test: 72s
New make test: 30s
New make test:short: 21s

@AndrewSisley AndrewSisley added code quality Related to improving code quality action/no-benchmark Skips the action that runs the benchmark. area/cli Related to the CLI binary labels Jul 17, 2023
@AndrewSisley AndrewSisley added this to the DefraDB v0.6 milestone Jul 17, 2023
@AndrewSisley AndrewSisley requested a review from a team July 17, 2023 11:57
@AndrewSisley AndrewSisley self-assigned this Jul 17, 2023
@AndrewSisley AndrewSisley changed the title tests: Remove CLI tests from make test test: Remove CLI tests from make test Jul 17, 2023
@codecov
Copy link

codecov bot commented Jul 17, 2023

Codecov Report

Patch coverage has no change and project coverage change: -0.11 ⚠️

Comparison is base (2f245d8) 75.64% compared to head (30493b5) 75.53%.

❗ Current head 30493b5 differs from pull request most recent head d24f8e9. Consider uploading reports for the commit d24f8e9 to get more accurate results

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #1643      +/-   ##
===========================================
- Coverage    75.64%   75.53%   -0.11%     
===========================================
  Files          200      200              
  Lines        20807    20807              
===========================================
- Hits         15738    15715      -23     
- Misses        3992     4008      +16     
- Partials      1077     1084       +7     
Flag Coverage Δ
all-tests 75.53% <ø> (-0.11%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 5 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2f245d8...d24f8e9. Read the comment docs.

Copy link
Collaborator

@fredcarle fredcarle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Makefile Outdated
Comment on lines 192 to 198
.PHONY: test\:short
test\:short:
go test $(DEFAULT_TEST_DIRECTORIES)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought: Possible conflict with the make:quick rule:

.PHONY: test\:quick
test\:quick:
	gotestsum --format pkgname -- $(DEFAULT_TEST_DIRECTORIES)

introduced in my #1609 which will merge shortly

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can just rename this to test:quick-go

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll check how much overhead gotestsum adds and then maybe drop this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Diff seems insignificant :) Dropping this.

  • drop

@@ -226,6 +232,10 @@ test\:lens:
@$(MAKE) deps:lens
gotestsum --format testname -- ./$(LENS_TEST_DIRECTORY)/... $(TEST_FLAGS)

.PHONY: test\:cli
test\:cli:
gotestsum --format testname -- ./$(CLI_TEST_DIRECTORY)/... $(TEST_FLAGS)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: For lens and cli I noticed we are also using testname over pkgname? Is that desired? I find testname to be useful in ci, but locally I prefer pgkname

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just copy-pasted the settings from the existing commands we had. I actually don't really care for either of them and other than for a quick sanity check that it has actually run the tests, I would probably be happy with just the count and any failure logs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it is being used in the ci command can leave as is

They are slow and it is distruptive to have to wait for them when working on core stuff
@AndrewSisley AndrewSisley force-pushed the 1642-test-cli branch 2 times, most recently from 30493b5 to d24f8e9 Compare July 17, 2023 20:51
@AndrewSisley AndrewSisley merged commit effd4eb into sourcenetwork:develop Jul 17, 2023
7 checks passed
@AndrewSisley AndrewSisley deleted the 1642-test-cli branch July 17, 2023 21:04
shahzadlone pushed a commit to shahzadlone/defradb that referenced this pull request Feb 23, 2024
## Relevant issue(s)

Resolves sourcenetwork#1642

## Description

Removes CLI tests from `make test` as they take ages to run and it is
disruptive to dev-flow.

Runtimes on my machine:
Old `make test`: 72s
New `make test`: 30s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
action/no-benchmark Skips the action that runs the benchmark. area/cli Related to the CLI binary code quality Related to improving code quality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove CLI tests from make test
3 participants