Skip to content
This repository has been archived by the owner before Nov 9, 2022. It is now read-only.

bug: unable to cancel #71

Closed
stevecrozz opened this issue May 15, 2019 · 8 comments
Closed

bug: unable to cancel #71

stevecrozz opened this issue May 15, 2019 · 8 comments

Comments

@stevecrozz
Copy link

stevecrozz commented May 15, 2019

I managed to boil my issue down to the smallest possible failure scenario. Here is a minimal example:
https://github.com/stevecrozz/drone-issues/tree/cannot-cancel-with-depends_on

While the second step is running, I decide to cancel the build from the drone UI. The drone UI and the drone database is updated, the drone agent receives the cancel message, but the docker container that is running the build step does not exit.

I'm running drone using dockerhub images:

  • drone/drone:1.1.0
  • drone/agent:1.1.0
@bradrydzewski bradrydzewski transferred this issue from harness/drone May 16, 2019
@ashwilliams1 ashwilliams1 changed the title bug: unable to cancel when using depends_on bug: unable to cancel Oct 4, 2019
@GCorbel
Copy link

GCorbel commented Oct 4, 2019

@stevecrozz Did you found a solution to this issue? How do you know the agent received the message?

I saw than the payload of the DELETE request send to the server say than the process is killed but it isn't.

@GCorbel
Copy link

GCorbel commented Oct 4, 2019

I tried to do something similar, with curl, to what's drone is supposed to do :

docker-compose exec drone-agent curl --unix-socket /var/run/docker.sock -X POST http://v.1.24/containers/af0c5571da00/kill

It killed the container correctly so the problem doesn't comes from the communication the agent and the socket.

@GCorbel
Copy link

GCorbel commented Oct 4, 2019

I confirm the signal is received by the agent. There is a "done listening for cancel signals". It probably comes from this line. It should goes in the first condition of the if and trigger the cancel method.

@stevecrozz
Copy link
Author

fwiw: I did not find an answer and developed a terrible system of hacks using shell scripts and cron jobs to work around this problem

@GCorbel
Copy link

GCorbel commented Oct 4, 2019

Can you share your hack?

@stevecrozz
Copy link
Author

I don't mind, but it is very specific to my own implementation. I use an ec2 instance per pipeline in my environment and the hack involves shutting down the instance when it gets into this state. If you're still interested, I can dig up the cron job...

@GCorbel
Copy link

GCorbel commented Oct 4, 2019

I still can check what it is. I can give me some ideas.

The solution I think at the moment is to watch logs and do a docker-compose restart when there is "done listening for cancel signals" in it.

@bradrydzewski
Copy link
Member

this is no longer an issue in the newer drone/drone-runner-* images which replace the drone/agent containers.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants