Simplify pull request template #11563

adbridge · 2019-09-25T10:58:59Z

Now looks like:

Description (required)

Summary of change (What the change is for and why)

Condense multiple options into the main three and summarize what
can go into those 3 types.

Add a test results section.

Added additional guidance

Documentation (Details of any document updates required)

Updates required to workflow docs

Pull request type (required)

[x] Patch update (Bug fix / Target update / Docs update / Test update / Refactor)
[] Feature update (New feature / Functionality change / New API)
[] Major update (Breaking change E.g. Return code change / API behaviour change)

Test results (required)

[x] No Tests required for this change (E.g docs only update)
[] Covered by existing mbed-os tests (Greentea or Unittest)
[] Tests / results supplied as part of this PR

Reviewers (optional)

Release Notes (required for feature/major PRs)

Summary of changes

Impact of changes

Migration actions required

Condense multiple options into the main three and summarise what can go into those 3 types. Add a test results section. Add compulsory sections to Release Notes.

ciarmcom · 2019-09-25T11:00:24Z

@adbridge, thank you for your changes.
@ARMmbed/mbed-os-maintainers please review.

bulislaw · 2019-09-25T11:10:50Z

.github/pull_request_template.md

-    [ ] Docs update
-    [ ] Test update
-    [ ] Breaking change
+    [ ] Patch update (Bug fix / Target update / Docs update / Test update / Refactor)


We should link to contribution guidelines with some further description of these.

I believe refactor should go into feature one as stated also in our guidelines

I believe refactor should go into feature one as stated also in our guidelines

Bartek thought it should go to patch unless the actual functionality is changed

We should link to contribution guidelines with some further description of these.

Don't we already do that in the non visible notes?

This section is compulsory for all Feature and Major PR types. This section is automatically added to release notes. Please fill in each sub-section with sufficient detail for a user. For more information, please see [the contributing guidelines](https://os.mbed.com/docs/mbed-os/latest/contributing/workflow.html#pull-request-types).

Although that comes under the Release Notes. Should we maybe put that right at the top or are you saying we should make that note visible?

Could be on the top as "Read our guidelines (url reference here) to understand pull request template" or similar note

Bartek thought it should go to patch unless the actual functionality is changed

@kjbracey-arm is that OK? I've seen this previously considered as refactors are internal changes (the worst thing there is introducing new bugs, but all backward compatible).

The reason to not include refactors (or any change not fixing a specific bug) would be to reduce the chance of unneccessarily introducing new bugs.

It increases confidence for users that the patch series is safe to accept general updates from - if they're on 5.14.x, they could feel confident that 5.14.y won't break anything.

In semantic versioning terms, certainly, a refactor would be a patch version increment, as it's supposed to be backwards compatible. But a Linux kernel patch release wouldn't accept it a backported kernel "refactor" unless it was to fix a specific bug, on grounds of risk.

So Bartek or someone needs to make the call on desired stability of backport patch series.

@bulislaw Please confirm this change is fine .

0xc0170

Otherwise looks good !

0xc0170 · 2019-09-25T12:27:43Z

.github/pull_request_template.md

-    [ ] Docs update
-    [ ] Test update
-    [ ] Breaking change
+    [ ] Patch update (Bug fix / Target update / Docs update / Test update / Refactor)


I believe refactor should go into feature one as stated also in our guidelines

0xc0170 · 2019-09-25T12:28:30Z

.github/pull_request_template.md

-    [ ] Breaking change
+    [ ] Patch update (Bug fix / Target update / Docs update / Test update / Refactor)
+    [ ] Feature update (New feature / Functionality change / New API)
+    [ ] Major update (Breaking change E.g. Return code change / API behaviour change)


should we add here - Release notes required , and also for feature update?

There is a comment under the Release Notes section that says it is compulsory for Feature and Major updates

There is a comment under the Release Notes section that says it is compulsory for Feature and Major updates

yes, just thinking if I fill this in if I get to Release notes :-) I would leave it in both places

jamesbeyond · 2019-09-25T12:39:28Z

.github/pull_request_template.md

-    [ ] Test update
-    [ ] Breaking change
+    [ ] Patch update (Bug fix / Target update / Docs update / Test update / Refactor)
+    [ ] Feature update (New feature / Functionality change / New API)


If a PR is a feature update, can we add a sub-question to ask the submitter to confirm where is the documentation and tests related to this new feature. @adbridge

as if "new API or feature, documentation and tests provided are required" ?

@jamesbeyond We already have a rest results (optional) section there. Are you saying we should expand on that to also show evidence for new feature testing ? I'm happy to do that. We could also have a section for 'Documentation Link'. What do @0xc0170 and @bulislaw think?

Test Result would make people think the CI that running would provide the test result.
The really question here is: if your PR is about a new feature, where is your tests ? are these test can be executed by current CI ?
Maybe we can make the section compulsory, but let them choose one from bellow?
[ ] No Tests required for this change
[ ] This PR changes will be covered by existing tests and CI
[ ] Tests supplied with in this PR
[ ] Other, specify bellow

BTW, I'd like to have the Documentation link section as well

Mmm the only problem I have with this is we are trying to simplify the basic PR template and this would start to add complexity again. Though I do like the idea @bulislaw @0xc0170 what are your thoughts ?

I think our goal is trying formalize and clarify the process, make the guidance easy for users to follow, and make it easy for us to manage the PRs. Not just only simplify it 😉

The problem with above is for many people CI tests are unkown and even testing infrastructure, they test it via their app or simple test case, so this would be mostly for internal teams rather than all of us.

What we often do is ask a simple question "How was this tested", maybe this type of question to be included in the template - it should make a person think about testing this. The answer could be one of the above "no tests done" , "tested on simple app", "run entire test suite", etc. I still feel we don't test enough would this question asked for all PRs help us to improve test coverage?

Yes @0xc0170 I agree with you. So the things I am a bit worried is about when people adding new feature, they more focus about the feature and forgot about tests and documentation.
e.g. the baremetal profile. Having this test compulsory section would let aware that tests are required as part of feature.

Also tested on simple app it is not acceptable to me for a new feature. we need them either in formal of Greentea, Unittest format or a test in their own team's CI, unless they'll need a really good explanation why there is no test.
For the changes to a existing feature, it is fairly enough for them to claim this is already covered by a exist greentea or unittest tests

Where is the test? is one of the question, and we know there are limitation about CI, e.g. USB test is is not running. I think that's another issue for CI/Infra team to resolve

Also add section separators

adbridge · 2019-09-25T14:50:43Z

Updated to add test results section and separating lines. Note to see them properly you have to look at the file in none edit mode. I've edited the template on this actual PR to demonstrate.

Made required/optional italic and removed a couple of superfluous lines

0xc0170

A comment to "Tests" section only, the rest looks fine to me.

Are "Tests" required or optional? the header states required, then in the comment it is optional?

I am not certain about "covered by existing tests and CI" - most of people have no idea what runs in CI, so would remove "and CI" but that might be just me being picky.

"Tests supplied below" - rather add "Test results" ?

jamesbeyond · 2019-09-26T09:01:23Z

Hi @adbridge @0xc0170

No Tests required for this change, shall we add (E.g. docs updates)
Covered by existing mbed-os tests (Greentea or Unittest) should be good to me.
tests supplied below sounds a bit ambiguous, maybe tests supplied part of this PR ?
why we dropped others, explained bellow option? I believe some PR will have some case, where it will not fall into any of the 3 options. like this is tested in their own team's CI, or test is submitted in separate PR.

BTW, how about documentation section? are we still having that section?

0xc0170 · 2019-09-26T10:03:50Z

BTW, how about documentation section? are we still having that section?

Docs covered in the Patch update.

tests supplied below sounds a bit ambiguous, maybe tests supplied part of this PR ?

Tests logs attached ?

why we dropped others, explained bellow option? I believe some PR will have some case, where it will not fall into any of the 3 options. like this is tested in their own team's CI, or test is submitted in separate PR.

If not any of these 3, the text below these 3 options is "others" - does it need own item in the list?

adbridge · 2019-09-26T10:10:46Z

If not any of these 3, the text below these 3 options is "others" - does it need own item in the list?

Not sure what 'others' would be? Pretty sure those 3 cover everything. Either no testing, testing covered by us or else they need to supply evidence...

adbridge · 2019-09-26T10:59:59Z

@jamesbeyond @0xc0170 @bulislaw please take another look

0xc0170

Not certain about separators

adbridge · 2019-09-26T11:23:32Z

Not certain about separators

As in the lines ?

0xc0170 · 2019-09-26T11:37:01Z

I forgot to remove my dated comment , as I firstly checked how it looks in the template then reviewed the file generated, all fine to me.

bulislaw · 2019-09-26T12:42:39Z

.github/pull_request_template.md

+    [] Major update (Breaking change E.g. Return code change / API behaviour change)
+
+----------------------------------------------------------------------------------------------------------------
+### Test results (*required*)


So is it required or optional? It says required in line 28 and optional in 30

Required should be updated, I thought I had changed line 30 in the last update - will check

bulislaw · 2019-09-26T12:44:02Z

.github/pull_request_template.md

-    Optional
-    In case of breaking changes, functionality changes or refactors, please add release notes here. 
-    For more information, please see [the contributing guidelines](https://os.mbed.com/docs/mbed-os/latest/contributing/workflow.html#pull-request-types).
+    This section is compulsory for all Feature and Major PR types.


For the features and major PRs we need only "summary of changes", for breaking changes we need all three subsections.

Major update is a breaking change?

OK updated the comments accordingly

jamesbeyond

Looks good to me, thanks!

0xc0170 · 2019-09-30T08:57:56Z

CI started

0xc0170 · 2019-09-30T09:22:17Z

Looks like this is ready, just need docs update reference here.

0xc0170 · 2019-10-25T11:57:01Z

Docs PR approved (ARMmbed/mbed-os-5-docs#1158), I'll merge this one

Simplify pull request template

2127b45

Condense multiple options into the main three and summarise what can go into those 3 types. Add a test results section. Add compulsory sections to Release Notes.

adbridge requested review from bulislaw, 0xc0170 and NeilJackson-AlifSemi September 25, 2019 10:58

ciarmcom requested a review from a team September 25, 2019 11:00

ciarmcom added the needs: review label Sep 25, 2019

bulislaw reviewed Sep 25, 2019

View reviewed changes

bulislaw approved these changes Sep 25, 2019

View reviewed changes

0xc0170 requested changes Sep 25, 2019

View reviewed changes

jamesbeyond reviewed Sep 25, 2019

View reviewed changes

Add test results section

4e505f6

Also add section separators

adbridge added 2 commits September 25, 2019 16:05

Add addional guidance

1725d5b

Improve formatting

580af4e

Made required/optional italic and removed a couple of superfluous lines

0xc0170 reviewed Sep 26, 2019

View reviewed changes

Add clarity to description and reword tests options

f8dc87c

0xc0170 self-requested a review September 26, 2019 11:10

0xc0170 approved these changes Sep 26, 2019

View reviewed changes

bulislaw reviewed Sep 26, 2019

View reviewed changes

Clarify release note section requirements

7349001

jamesbeyond approved these changes Sep 26, 2019

View reviewed changes

0xc0170 removed the needs: review label Sep 30, 2019

0xc0170 added the needs: CI label Sep 30, 2019

0xc0170 added needs: preceding PR and removed needs: CI labels Sep 30, 2019

0xc0170 added ready for merge and removed needs: preceding PR labels Oct 25, 2019

0xc0170 merged commit e5a728f into ARMmbed:master Oct 25, 2019

0xc0170 added release-version: 5.14.2 and removed ready for merge labels Oct 25, 2019

Simplify pull request template #11563

Simplify pull request template #11563

Conversation

adbridge commented Sep 25, 2019 • edited Loading

Description (required)

Summary of change (What the change is for and why)

Documentation (Details of any document updates required)

Pull request type (required)

Test results (required)

Reviewers (optional)

Release Notes (required for feature/major PRs)

Summary of changes

Impact of changes

Migration actions required

ciarmcom commented Sep 25, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adbridge Sep 25, 2019 • edited Loading

Choose a reason for hiding this comment

0xc0170 Sep 25, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kjbracey Sep 25, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

0xc0170 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jamesbeyond Sep 25, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jamesbeyond Sep 25, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jamesbeyond Sep 25, 2019 • edited Loading

Choose a reason for hiding this comment

adbridge commented Sep 25, 2019 • edited Loading

0xc0170 left a comment • edited Loading

Choose a reason for hiding this comment

jamesbeyond commented Sep 26, 2019

0xc0170 commented Sep 26, 2019

adbridge commented Sep 26, 2019

adbridge commented Sep 26, 2019

0xc0170 left a comment

Choose a reason for hiding this comment

adbridge commented Sep 26, 2019

0xc0170 commented Sep 26, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jamesbeyond left a comment

Choose a reason for hiding this comment

0xc0170 commented Sep 30, 2019

0xc0170 commented Sep 30, 2019

0xc0170 commented Oct 25, 2019

adbridge commented Sep 25, 2019 •

edited

Loading

adbridge Sep 25, 2019 •

edited

Loading

0xc0170 Sep 25, 2019 •

edited

Loading

kjbracey Sep 25, 2019 •

edited

Loading

jamesbeyond Sep 25, 2019 •

edited

Loading

jamesbeyond Sep 25, 2019 •

edited

Loading

jamesbeyond Sep 25, 2019 •

edited

Loading

adbridge commented Sep 25, 2019 •

edited

Loading

0xc0170 left a comment •

edited

Loading