Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-run of release pipeline results in loss of release notes information #440

Closed
wpschaub opened this issue Feb 25, 2019 · 21 comments

Comments

@wpschaub
Copy link

commented Feb 25, 2019

Azure DevOps Extension you are using

Generate Release Notes (Crossplatform)

Where are you running it?

Azure DevOps Service (VSTS)

Version of Extension/Task

2.8.1

Expected behaviour and actual behaviour

During a successful deployment the release notes are generated based on current and previous release, resulting in a release note as expected. If user re-deploys an environment, the release notes are re-generated (and potentially overwritten), but without any changes as extension now compares release to itself.

Steps to reproduce the problem

  1. Complete a pull request with valid changes, deploy an environment, and generate the release notes.
  2. Generated release notes are healthy and reflect the changes introduced by latest artefacts.
  3. Re-deploy the environment, which generates the release notes again.
  4. Changes introduced by the latest artefacts are not reflected in the release notes.

@wpschaub wpschaub changed the title Re-run of release pipeline results in loss of information Re-run of release pipeline results in loss of release notes information Feb 25, 2019

@rfennell

This comment has been minimized.

Copy link
Owner

commented Feb 26, 2019

I think this might be a limitation of the API call that is being made, as it compared between simiple numeric releases, ignoring and rerelease complexities. I need to investigate

@wpschaub

This comment has been minimized.

Copy link
Author

commented Feb 26, 2019

@rfennell I did the following test:

  1. Cloned pipeline and re-ran release.
  2. Run 1: Release = Release 1, Compare = Release 1, changes none.
  3. Run 2: Release = Release 2, Compare = Release 1
  4. Re-Run 2: Release = Release 2, Compare = Release 2

In step 1, should the Compare not be "None" and in step 4 be the last successful, but itself, i.e. "Release 1"? When it compares itself, there will be no differences.

@rfennell

This comment has been minimized.

Copy link
Owner

commented Feb 26, 2019

  1. Cloned pipeline and re-ran release.
  2. Run 1: Release = Release 1, Compare = Release 1, changes none. - I would expect all wi/cs associated with the trigger build (this is new feature), the detailed logs show a special route has been taken
  3. Run 2: Release = Release 2, Compare = Release 1 - Normal behavior
  4. Re-Run 2: Release = Release 2, Compare = Release 2 - this is the problem, I need to look into this this, but it won't be straight away
@rfennell

This comment has been minimized.

Copy link
Owner

commented Mar 19, 2019

I think the question is how is the task meant to know that step 4 (above) is really step 3.

It just looks for the last good release to compare against, and the step 3 one was good.

Any thoughts, how can we signal that step 4 is a redeploy?

@rfennell rfennell self-assigned this Mar 20, 2019

@wpschaub

This comment has been minimized.

Copy link
Author

commented Mar 20, 2019

@rfennell is there any way that the task can compare itself with the most recent successful build?
image

For example, release 31 compared itself with release 30. Release 30 compared itself with release 29, which was failed in post=deploy approvals.

I'm thinking aloud ... if we can configure the task to compare itself with the last successful release, for example Release 30 would compare itself with Release 26, we can fail a release, before re-deploying to fix this #440 issue. A bonus would be that the release notes would show the same results as what we see in the release summary when we manually compare with the last successful release.

Thoughts?

@rfennell

This comment has been minimized.

Copy link
Owner

commented Mar 20, 2019

Yes, I think that is the only way

I think the task needs to check that this release does match the last successful release's artifacts. if it does in your example run the compare for 30 > 26 as opposed to 31 >30

Think that works?

@wpschaub

This comment has been minimized.

Copy link
Author

commented Mar 20, 2019

@rfennell, sorry, my image is adding confusion. Ignore release 31 at the top. When Release 30 runs, it compares itself with Release 29 (which was failed in post-approval) according to the release notes. This is default behavior, even for the release summary view, so I would leave it "by default".

It would, however, be "cool" if we could configure the task not to process default behavior, but to compare itself with the most recent successful release. So, if we set this option, release notes would show that Release 31 compared itself with Release 26.

This behavior would unblock a few branching and release reporting issues we 're experiencing.

@wpschaub

This comment has been minimized.

Copy link
Author

commented Mar 20, 2019

@rfennell if we could take it one step further. If configured to NON-DEFAULT, release 30 looks for the last successful release (other than itself), which is 26. When Release 31 runs it finds 30. When release 31 is re-deployed, it still looks for the last (other than itself) successful release, which is 30.

@rfennell

This comment has been minimized.

Copy link
Owner

commented Mar 20, 2019

Seems reasonable, I will have a look

@rfennell

This comment has been minimized.

Copy link
Owner

commented Mar 20, 2019

Checked the code, it should do the compare against the last successful release. If you run with system.debug=true it lists the last successful release, seems Ok from my tests

@wpschaub

This comment has been minimized.

Copy link
Author

commented Mar 20, 2019

@wpschaub

This comment has been minimized.

Copy link
Author

commented Mar 21, 2019

@rfennell you're correct. release summary, by default, compares with the previous release, even if failed. Your extension skips the failed releases 👍 I'm stoked and cannot wait to go back to the office tomorrow :) That means the only thing that is still an issue is the re-deploy, which compares the release with itself.

@rfennell

This comment has been minimized.

Copy link
Owner

commented Mar 21, 2019

So my bug is good !

I won't fix it then, just the rerun issue and I have plan for that

@wpschaub

This comment has been minimized.

Copy link
Author

commented Mar 21, 2019

@wpschaub

This comment has been minimized.

Copy link
Author

commented Mar 21, 2019

@rfennell Test was successful. I failed 3 PR and 3 releases, then ran another release and the extension compared Current to Current - 6. Spot on!

In terms of a re-deploy a colleague had a great idea. How about the task skips release notes processing if Release.AttemptNumber > 1?

@rfennell

This comment has been minimized.

Copy link
Owner

commented Apr 15, 2019

@wpschaub been a while but I have been thinking on this one.

  • On the Release.AttemptNumber > 1 option where you thinking output a message somewhere?
  • I was looking at trying to detect of a release was off the same artifacts as the previous one and using the changes from the previous release, but this is confusing.

I like tracking the Release.AttemptNumber number , but not sure what to do with it.

@wpschaub

This comment has been minimized.

Copy link
Author

commented Apr 15, 2019

@rfennell I was thinking of allowing user to decide whether to re-run the release note generation if ReleaseAttemptNumber>1. By default I would not re-generate release notes, instead place a warning in the log that release notes were not regenerated due to the release being re-deployed.

@rfennell

This comment has been minimized.

Copy link
Owner

commented Apr 15, 2019

That sounds like I need both, if we are a re-run then need to go back and get the old artifact number for the checks

@wpschaub

This comment has been minimized.

Copy link
Author

commented Apr 15, 2019

rfennell added a commit that referenced this issue Apr 16, 2019
@rfennell

This comment has been minimized.

Copy link
Owner

commented Apr 17, 2019

Give 2..9.6 a try, it has a new advanced option that will skip generation of release notes on a re-deploy (of the stage the task is in). It completes with a 'pass with warnings'

The default is the old behavior, so it is a non-breaking change

Does that work for you?

@wpschaub

This comment has been minimized.

Copy link
Author

commented Apr 17, 2019

@wpschaub wpschaub closed this Apr 17, 2019

GenerateReleaseNotes (Xplat version) automation moved this from To do to Done Apr 17, 2019

rfennell added a commit that referenced this issue Apr 18, 2019
#440 stops generation if a redeploy (#471)
* Stops generation if a redeploy
* Added the parameter to the task.json
* Updated docs and fixed return type
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
2 participants
You can’t perform that action at this time.