Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[APM] Test plan proposal #44887

Merged
merged 3 commits into from Sep 9, 2019
Merged

[APM] Test plan proposal #44887

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9c65525
[APM] Test plan proposal
sorenlouv Sep 5, 2019
e62830b
Fix version label
sorenlouv Sep 5, 2019
7e330d5
Update TEST_PLAN.md
sorenlouv Sep 9, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
20 changes: 20 additions & 0 deletions x-pack/legacy/plugins/TEST_PLAN.md
View file
Open in desktop
@@ -0,0 +1,20 @@
# Test plan process (7.4.0)

### What
Everything that is being released in the 7.4 should be tested by someone other than the person who implemented it

### When
The first week after feature freeze (FF) all known bugs and blockers will be fixed. The test plan will begin shortly after this (typically one week after FF date)

### Prerequesites
- Every pull request that is merged to 7.x (or backported to 7.4 after FF) must be given the label: `7.4.0`. [These pull requests](https://github.com/elastic/kibana/pulls?utf8=%E2%9C%93&q=is%3Apr+label%3Av7.4.0+label%3ATeam%3Aapm+-label%3Abackport) will provide the basis of our test plan.
sorenlouv marked this conversation as resolved.
Show resolved Hide resolved
- Immediately before starting the test plan all relevant PRs for the release are selected and the label `in-test-plan` is applied. The "assigned" field will be reset to nobody (read further why)

### Execution
- [Find the list](https://github.com/elastic/kibana/pulls?q=is%3Apr+label%3Av7.4.0+label%3Ain-test-plan+label%3ATeam%3Aapm+-label%3Atested-after-ff+no%3Aassignee) of unassigned, untested issues in the test plan
- Assign yourself to an issue as you start testing it to avoid duplicate work
- Testing a PR can have two outcomes:
- It worked as expected => Apply `tested-after-ff` label
- It did not work as expected => Create a bug report and link to it from the PR
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the bug have the label applied as well? The original issue's label should only be added after the bug is resolved.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the issue created for the bug should have the version label (in this case v7.4.0). But I don't think it makes sense to add in-test-plan since that's only for PRs that have already been merged (and can actually be tested)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With regards to the process when a bug is discovered I'm leaning towards still marking the PR as tested with tested-after-ff. Since an issue will be created (which should have high priority) it will eventually lead to a new PR which will end up in the test plan. And having duplicate PRs in the test plan is maybe confusing.