Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Reboot scenarios in Release Management #1505

Closed
thomasdgx opened this issue Apr 13, 2018 · 17 comments
Closed

Support for Reboot scenarios in Release Management #1505

thomasdgx opened this issue Apr 13, 2018 · 17 comments

Comments

@thomasdgx
Copy link
Contributor

Workflow

During a release you come to the point that a reboot is required. Some usual workflow:

  1. Install MSI
  2. Reboot
  3. Run Integration Tests

Current solution

Currently we make use of agent phases but this is not reliable.

  1. We execute the installation and the reboot on the target machine
  2. jump to another agent (phase) and ping the target machine to check if reboot is finished
  3. jump back to target machine an execute the tests

The reboot can take some time. So we wait not long enough. Additionally the agent on "target machine" can be taken by some other build/release. As a workaround the "target machine" is exclusively used by one release.

Preferred solution

Native support of reboots. The release executes on the target machine. During the reboot the agent at the target machine remains reserved for the current release. The agents starts automatically after the reboot and continues the exeuction.

No more remote controll neither polling from outside.

@bryanmacfarlane
Copy link
Collaborator

Yes. This is on our backlog. We may still require multiple phases but the coordination should be built in as you outlined. It would need to coordinate with the server with a status of rebooting and the next phase would return to that machine.

I'll leave this open and track as an enhancement.

@chrispat
Copy link
Contributor

Specifically we are looking at reboot support for deployment groups not for generic build and release phases.

@flcdrg
Copy link

flcdrg commented Oct 14, 2018

I'd like to be able to reboot agent hosts from the Agent Pool page. That would be much more convenient than having to RDP to the specific machine

@bryanmacfarlane
Copy link
Collaborator

@flcdrg - that wouldn't be too hard to do if we did the other work above. Especially since there's no state issues (the service would be cooperative and not assign it another job till it came back up).

@AtOMiCNebula
Copy link
Member

Specifically we are looking at reboot support for deployment groups not for generic build and release phases.

@chrisrpatterson, is this to say that we shouldn't expect to get reboot support for build/release pipeline agents at all, or just that reboot support for deployment groups are being prioritized ahead of build/release?

@kevpar
Copy link
Member

kevpar commented May 8, 2019

To pile on here, I'm also hoping for generic reboot support as part of a release agent job (not deployment group). We run some tests that mess up the machine state, and would really like to reboot afterwards so the next pipeline execution can get a clean machine to run on.

Deployment groups are also not an option for us, as we are only allowed to create custom agent pools.

@bgoeppner
Copy link

I totally support this request. We face the same workflow that is outlined here, and struggle with the same problems:

  • install msi on a set of machines (via deployment group and tags)
  • reboot
  • wait for machines to come back
  • finish work (e.g. check if installation was successfull, run additional tests)

As we might deploy to many machines (VMs and physical ones) which might also be located in different subnets, we cannot use the concept of a "coordination agent" to check if all machines/agents are back online (this agent would need to have access to all machines, which is difficult due to firewall restrictions, etc).
Instead, it would be great if the server could remember which machines where executing the first part of the job, wait for each one to come back, and then allow them to complete the second part of the job. Maybe combined with a timeout, in case the machines do not come back.
It would be awesome, if some of those machines could already run phase 2, while others are still in phase 1 or rebooting (no need to wait for all machines to complete phase 1 and rebooting before a single machine can start with phase 2).

@ajklotz
Copy link

ajklotz commented Aug 15, 2019

My team needs this too.

@petermisovic
Copy link

Yes, this is very appreciated feature to have it. Is there any update according this issue? I use TFS server 2018, still no support :(

@jimhess
Copy link

jimhess commented Oct 22, 2019

Another vote for ability to reboot a hosted pipeline and resume the pipeline after the restart.

We must install MSI (https://package.chocolatey.org/packages/sqlserver-cmdlineutils ) to get sqlcmd added to path, but this now has KB dependencies requiring a reboot.

@kittenpoop
Copy link

kittenpoop commented Dec 9, 2019

Same here, would be nice to be able to do a complete new OS installation, with an agent "surviving" this. Implemented my own mechanism now, however this "homebrew" is not visible in agent status to end-user, resulting in incorrect expectations.

Is there anything I can do by voting or something to make this higher priority?

@ruicao93
Copy link

Any update on this enhancement? My team also needs this feature after some installation.

@github-actions
Copy link

This issue has had no activity in 180 days. Please comment if it is not actually stale

@github-actions github-actions bot added the stale label Aug 24, 2020
@nzbart
Copy link

nzbart commented Aug 25, 2020

By "stale", does your bot mean that nobody wants the feature anymore, or that you're not going to do any work on it anyway? Because I can assure you that the need hasn't gone away.

@github-actions github-actions bot closed this as completed Sep 1, 2020
@kittenpoop
Copy link

Sorry this was closed, since I also need your solution for this issue.

@vtbassmatt
Copy link
Member

The bot closed it because we're trying to centralize feedback (issues, suggestions) through Developer Community. We don't currently have plans to implement reboot support, but please feel free to file a ticket on DevComm. As more people pile onto it, it becomes easier to justify the investment.

@jamesmistry
Copy link

I couldn't see a corresponding DevComm feature request, so I created one: https://developercommunity.visualstudio.com/idea/1192446/support-test-signingreboots-in-azure-devops-pipeli.html

I've referred to this ticket but I'm sure a bump and/or a comment in support would help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests