Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove TDD docs, add more info on running tests #2335

Merged
merged 1 commit into from Nov 7, 2019

Conversation

@cdrage
Copy link
Collaborator

cdrage commented Oct 30, 2019

What kind of PR is this?

/kind docs

What does does this PR do / why we need it:

  • Removes TDD documentation which is outdated / we no longer use it.
  • Adds documentation on how to run a single test

Which issue(s) this PR fixes:

Fixes #2295

How to test changes / Special notes to the reviewer:

N/A

Signed-off-by: Charlie Drage charlie@charliedrage.com

* Supply the name via command-line: `ginkgo -randomizeAllSpecs -slowSpecThreshold=120 -timeout 7200s -nodes=1 -progress -v tests/integration/ -F "When executing catalog list without component directory"`
* Modify the `It` statement to `Fit` and run `ginkgo -randomizeAllSpecs -slowSpecThreshold=120 -timeout 7200s -nodes=1 -progress -v tests/integration/`

Integration tests validates and focuses on specific fields of odo functionality or individual commands. For example, `cmd_app_test.go` or `generic_test.go`.

This comment has been minimized.

Copy link
@girishramnani

girishramnani Oct 31, 2019

Collaborator

validate and focus


To run ONE individual test, you can either:

* Supply the name via command-line: `ginkgo -randomizeAllSpecs -slowSpecThreshold=120 -timeout 7200s -nodes=1 -progress -v tests/integration/ -F "When executing catalog list without component directory"`

This comment has been minimized.

Copy link
@amitkrout

amitkrout Oct 31, 2019

Collaborator

Command does not work

$ ginkgo -randomizeAllSpecs -slowSpecThreshold=120 -timeout 7200s -nodes=1 -progress -v tests/integration/ -F "When executing catalog list without component directory"
Running Suite: Integration Suite
================================
Random Seed: 1572505443 - Will randomize all specs
Will run 122 of 122 specs         <--------------------- It running all specs

odo push command tests when push command is executed 
  should not build when changes are detected in a ignored file
[...]

This comment has been minimized.

Copy link
@amitkrout

amitkrout Oct 31, 2019

Collaborator

In this case let make it simple

* Supply the name via command-line: `ginkgo -focus="<spec title>" test_dir_path_of_spec_file`
+

----
ginkgo -focus="When executing catalog list without component directory" tests/integration/
----
+

In a NOTE mention that To customize output use ginkgo flags. see `ginkgo -h` for more details

This comment has been minimized.

Copy link
@amitkrout

amitkrout Oct 31, 2019

Collaborator

Verified the command locally

$ ginkgo -focus="When executing catalog list without component directory" tests/integration/
Running Suite: Integration Suite
================================
Random Seed: 1572506506
Will run 1 of 122 specs      <---------------------- Running only one spec 
[...]
To run ONE individual test, you can either:

* Supply the name via command-line: `ginkgo -randomizeAllSpecs -slowSpecThreshold=120 -timeout 7200s -nodes=1 -progress -v tests/integration/ -F "When executing catalog list without component directory"`
* Modify the `It` statement to `Fit` and run `ginkgo -randomizeAllSpecs -slowSpecThreshold=120 -timeout 7200s -nodes=1 -progress -v tests/integration/`

This comment has been minimized.

Copy link
@amitkrout

amitkrout Oct 31, 2019

Collaborator

It works, however you can make it simple and generic like

* Modify the `It` statement to `Fit` and run `ginkgo test_dir_path_of_spec_file`
+
----
For example:
ginkgo tests/integration/
----
+

As i mentioned above you can uses a common NOTE of ``` NOTE: To customize output use ginkgo flags. see `ginkgo -h` for more details``` for both the steps 

This comment has been minimized.

Copy link
@amitkrout

amitkrout Oct 31, 2019

Collaborator

Verified locally

$ ginkgo tests/integration/
Running Suite: Integration Suite
================================
Random Seed: 1572507116
Will run 1 of 122 specs    --------------- Running only one spec
[...]
@amitkrout

This comment has been minimized.

Copy link
Collaborator

amitkrout commented Oct 31, 2019

#2330 will fix the flake observed in ci/prow/v4.1-integration and ci/prow/v4.3-integration

Copy link
Collaborator

amitkrout left a comment

FYI..
@girishramnani @mik-dass @kadel @mohammedzee1000 @prietyc123 @dharmit This pr removes the TDD doc, i.e we are not going to follow TDD model, though we were not following it ;)

@cdrage cdrage force-pushed the cdrage:update-dev-docs branch from 3125f15 to 532d812 Oct 31, 2019
@cdrage

This comment has been minimized.

Copy link
Collaborator Author

cdrage commented Oct 31, 2019

/honk

@cdrage

This comment has been minimized.

Copy link
Collaborator Author

cdrage commented Nov 1, 2019

/test v4.3-integration

2 similar comments
@cdrage

This comment has been minimized.

Copy link
Collaborator Author

cdrage commented Nov 1, 2019

/test v4.3-integration

@cdrage

This comment has been minimized.

Copy link
Collaborator Author

cdrage commented Nov 4, 2019

/test v4.3-integration

@openshift openshift deleted a comment from openshift-ci-robot Nov 4, 2019
@cdrage

This comment has been minimized.

Copy link
Collaborator Author

cdrage commented Nov 4, 2019


We follow the Test Driven Development(TDD) workflow in our development process. You can read more about it link:tdd-workflow.adoc[here].
We use both unit, integration and e2e (End-2-End) tests.

This comment has been minimized.

Copy link
@dharmit

dharmit Nov 5, 2019

Collaborator

Aren't those three kind of tests? Referring to the word both being used in that statement.

@@ -181,7 +182,16 @@ $ MINISHIFT_ENABLE_EXPERIMENTAL=y minishift start --extra-clusterup-flags "--ena

*Integration tests:*

Integration tests utilize link:https://github.com/onsi/ginkgo[`Ginkgo`] and it's preferred matcher library link:https://github.com/onsi/gomega[`Gomega`] which define sets of test cases (spec). As per ginkgo test file comprises specs and these test file are controlled by test suite. Test and test suite files are located in `tests/integration` directory and can be called using `.PHONY` within `makefile`. Integration tests validates and focuses on specific fields of odo functionality or individual commands. For example, `cmd_app_test.go` or `generic_test.go`.
Integration tests utilize link:https://github.com/onsi/ginkgo[`Ginkgo`] and it's preferred matcher library link:https://github.com/onsi/gomega[`Gomega`] which define sets of test cases (spec). As per ginkgo test file comprises specs and these test file are controlled by test suite.

This comment has been minimized.

Copy link
@dharmit

dharmit Nov 5, 2019

Collaborator

s/it's preferred/its preferred?

**What kind of PR is this?**

/kind docs

**What does does this PR do / why we need it**:

- Removes TDD documentation which is outdated / we no longer use it.
- Adds documentation on how to run a single test

**Which issue(s) this PR fixes**:

Fixes #2295

**How to test changes / Special notes to the reviewer**:

N/A

Signed-off-by: Charlie Drage <charlie@charliedrage.com>
@cdrage cdrage force-pushed the cdrage:update-dev-docs branch from 532d812 to e7c3f01 Nov 5, 2019
@cdrage

This comment has been minimized.

Copy link
Collaborator Author

cdrage commented Nov 5, 2019

Update, thanks @dharmit !

@dharmit
dharmit approved these changes Nov 6, 2019
@dharmit

This comment has been minimized.

Copy link
Collaborator

dharmit commented Nov 6, 2019

/approve

@openshift-ci-robot

This comment has been minimized.

Copy link

openshift-ci-robot commented Nov 6, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dharmit

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@amitkrout

This comment has been minimized.

Copy link
Collaborator

amitkrout commented Nov 6, 2019

/lgtm

@amitkrout

This comment has been minimized.

Copy link
Collaborator

amitkrout commented Nov 6, 2019

/test v4.1-benchmark

 level=fatal msg="failed to initialize the cluster: Some cluster operators are still updating: authentication, console, monitoring: timed out waiting for the condition" 
@openshift-bot

This comment has been minimized.

Copy link

openshift-bot commented Nov 6, 2019

/retest

Please review the full test history for this PR and help us cut down flakes.

1 similar comment
@openshift-bot

This comment has been minimized.

Copy link

openshift-bot commented Nov 7, 2019

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot

This comment has been minimized.

Copy link

openshift-bot commented Nov 7, 2019

/retest

Please review the full test history for this PR and help us cut down flakes.

1 similar comment
@openshift-bot

This comment has been minimized.

Copy link

openshift-bot commented Nov 7, 2019

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit 70c5a6f into openshift:master Nov 7, 2019
12 checks passed
12 checks passed
Travis CI - Pull Request Build Passed
Details
ci/prow/v4.1-benchmark Job succeeded.
Details
ci/prow/v4.1-e2e-scenarios Job succeeded.
Details
ci/prow/v4.1-integration Job succeeded.
Details
ci/prow/v4.1-unit Job succeeded.
Details
ci/prow/v4.2-benchmark Job succeeded.
Details
ci/prow/v4.2-e2e-scenarios Job succeeded.
Details
ci/prow/v4.2-integration Job succeeded.
Details
ci/prow/v4.3-benchmark Job succeeded.
Details
ci/prow/v4.3-e2e-scenarios Job succeeded.
Details
ci/prow/v4.3-integration Job succeeded.
Details
tide In merge pool.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.