Ctrl-C on `docker-compose up` raises thread.error #189

hunterloftis · 2014-04-13T00:33:52Z

I get this intermittently (sometimes it does shutdown gracefully):

^CGracefully stopping... (press Ctrl+C again to force)
Stopping dailylead_db_1...
Stopping dailylead_queue_1...
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/Users/ben/fig/build/fig/out00-PYZ.pyz/fig.cli.main", line 39, in main
  File "/Users/ben/fig/build/fig/out00-PYZ.pyz/fig.cli.docopt_command", line 21, in sys_dispatch
  File "/Users/ben/fig/build/fig/out00-PYZ.pyz/fig.cli.command", line 30, in dispatch
  File "/Users/ben/fig/build/fig/out00-PYZ.pyz/fig.cli.docopt_command", line 24, in dispatch
  File "/Users/ben/fig/build/fig/out00-PYZ.pyz/fig.cli.command", line 47, in perform_command
  File "/Users/ben/fig/build/fig/out00-PYZ.pyz/fig.cli.docopt_command", line 27, in perform_command
  File "/Users/ben/fig/build/fig/out00-PYZ.pyz/fig.cli.main", line 316, in up
  File "/Users/ben/fig/build/fig/out00-PYZ.pyz/fig.cli.log_printer", line 20, in run
  File "/Users/ben/fig/build/fig/out00-PYZ.pyz/fig.cli.multiplexer", line 20, in loop
  File "/Users/ben/fig/build/fig/out00-PYZ.pyz/Queue", line 182, in get
thread.error: release unlocked lock

The text was updated successfully, but these errors were encountered:

patrick91 · 2014-07-21T13:24:11Z

me too

aanand · 2014-07-21T17:36:12Z

I've seen this too - perhaps KeyboardInterrupt messes with Queue somehow.
Not sure if there's a clean solution - perhaps we should just catch
thread.error in this loop: https://github.com/orchardup/fig/blob/master/fig/cli/multiplexer.py#L24-L3

On Mon, Jul 21, 2014 at 6:24 AM, Patrick Arminio notifications@github.com
wrote:

me too

—
Reply to this email directly or view it on GitHub
#189 (comment).

d11wtq · 2014-07-25T01:04:39Z

So the combination of threads and signals (SIGINT in this case) doesn't play well together (IIRC, signals are dispatched to one thread and it's non-deterministic which thread that will be). I'm sure this can be fixed, though I may just change how the multiplexer works to use select() instead of threads and blocking IO. That should just make this problem just disappear, but we'll see. When you have multiple streams you want to read from, the idiomatic approach is to use select() in any case.

I should have time to do that this weekend, as this is biting us too.

mpe · 2014-10-14T11:44:08Z

Still seeing this on 0.5.2, is it fixed in mainline?

xiaods · 2014-10-20T11:56:19Z

not yet.

motdotla · 2014-11-12T14:32:17Z

happens to me too.

I'm new to fig. Anyone know how you end up clearing the process it leaves behind. It leaves behind my nodejs process running and with a state of Exit -1?

Edit: As soon as I asked, I figured it out. fig rm. Sorry about that.

pikeas · 2014-11-20T02:19:52Z

+1, happening to me as well. Using the passenger image.

abrutus · 2014-11-20T21:51:19Z

+1 while using spotify/cassandra

schmunk42 · 2014-12-02T16:03:19Z

Same here with 1.0.1 using custom image and mysql.

pwaller · 2015-01-16T11:56:04Z

I just hit this. It seems hard to reproduce. I ran it ten times and I didn't hit the "release unlocked lock" bug, but I did hit this:

Traceback (most recent call last):
  File "/code/build/fig/out00-PYZ.pyz/threading", line 552, in __bootstrap_inner
  File "/code/build/fig/out00-PYZ.pyz/threading", line 505, in run
  File "/code/build/fig/out00-PYZ.pyz/fig.cli.multiplexer", line 41, in _enqueue_output
  File "/code/build/fig/out00-PYZ.pyz/fig.cli.log_printer", line 56, in _make_log_generator
  File "/code/build/fig/out00-PYZ.pyz/fig.cli.utils", line 77, in split_buffer
  File "/code/build/fig/out00-PYZ.pyz/docker.client", line 332, in _multiplexed_socket_stream_helper
  File "/code/build/fig/out00-PYZ.pyz/docker.client", line 319, in recvall
error: [Errno 4] Interrupted system call

In my setup I'm using fig to just run one service.

wkruse · 2015-02-09T14:20:53Z

Same here with 1.1.0-rc2.

Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/Users/ben/fig/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 31, in main
  File "/Users/ben/fig/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch
  File "/Users/ben/fig/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 27, in dispatch
  File "/Users/ben/fig/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch
  File "/Users/ben/fig/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 59, in perform_command
  File "/Users/ben/fig/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 455, in up
  File "/Users/ben/fig/build/docker-compose/out00-PYZ.pyz/compose.cli.log_printer", line 22, in run
  File "/Users/ben/fig/build/docker-compose/out00-PYZ.pyz/compose.cli.multiplexer", line 25, in loop
  File "/Users/ben/fig/build/docker-compose/out00-PYZ.pyz/Queue", line 182, in get
thread.error: release unlocked lock

aidanlister · 2015-04-06T05:09:38Z

Yep same here = only seems to happen when I introduced a queuing server (rabbitmq)

delfuego · 2015-04-07T18:50:28Z

Definitely seeing it here; for me, it's when I ctrl-c to stop a multi-container app that includes couchdb, and it happens on the stop of the couchdb container.

emerleite · 2015-05-06T14:44:01Z

Any news on this?

aanand · 2015-05-06T16:56:54Z

Multiplexer still uses threading, so this still happens. If anyone wants to take a crack at refactoring it to use IO streams + select() instead of generators + threading, go for it.

mahnunchik · 2015-07-16T09:35:33Z

^CGracefully stopping... (press Ctrl+C again to force)
Stopping blog_phpmyadmin_1...
Stopping blog_web_1...
Stopping blog_mysql_1...
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 32, in main
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 34, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 66, in perform_command
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 481, in up
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.log_printer", line 22, in run
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.multiplexer", line 25, in loop
  File "/code/build/docker-compose/out00-PYZ.pyz/Queue", line 182, in get
thread.error: release unlocked lock

$ docker-compose -v
docker-compose version: 1.3.2
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013

$ docker -v
Docker version 1.7.1, build 786b29d

$ docker info
Containers: 3
Images: 117
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 190
 Dirperm1 Supported: false
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.13.0-24-generic
Operating System: Ubuntu 14.04.2 LTS
CPUs: 4
Total Memory: 14.69 GiB
Name: desktop
ID: MQKY:YIVZ:XCTH:DRAP:QYGC:SLX5:Z3HU:EZSW:E4T5:GYLP:4WMR:RSMI
WARNING: No swap limit support

hoIIer · 2015-08-05T06:48:24Z

+1

forbearer · 2015-08-18T18:39:14Z

👍

shyr · 2015-09-06T14:24:51Z

^CGracefully stopping... (press Ctrl+C again to force)
Stopping nginx_registry_1...
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 32, in main
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 34, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 66, in perform_command
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 481, in up
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.log_printer", line 22, in run
  File "/code/build/docker-compose/out00-PYZ.pyz/compose.cli.multiplexer", line 25, in loop
  File "/code/build/docker-compose/out00-PYZ.pyz/Queue", line 182, in get
thread.error: release unlocked lock

It also happen to me.

My docker version like following.

$ docker-compose -v
docker-compose version: 1.3.3
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013

$ docker -v
Docker version 1.8.1, build d12ea79

$ docker info
Containers: 78
Images: 360
Storage Driver: devicemapper
 Pool Name: docker-253:0-134331561-pool
 Pool Blocksize: 65.54 kB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 8.428 GB
 Data Space Total: 107.4 GB
 Data Space Available: 38.98 GB
 Metadata Space Used: 19.23 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.128 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.93-RHEL7 (2015-01-28)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.10.0-229.11.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
CPUs: 4
Total Memory: 7.585 GiB
Name: nexhub06.ndeploy
ID: CKJK:WFUB:JXXS:OG7K:U7ST:GUNU:HR3M:2J2A:LWTL:MVCW:PBWP:FXY3

christianbundy · 2015-09-11T02:55:12Z

Seeing it here too. Is system info helpful or superfluous?

dnephin · 2015-09-12T22:42:55Z

I've been using compose from pip install and never once saw this issue. I just recently tried the binary and I get it almost every time I ctrl-c the logs. I think it might have something to do with the version of python we're building against, or pyinstaller itself.

Signed-off-by: Daniel Nephin <dnephin@docker.com>

gustavosoares · 2015-11-09T10:23:33Z

on which version does this issue were solved? I am running compose 1.5.0 and I am still facing this.

dnephin · 2015-11-09T15:27:13Z

It should be fixed in 1.5.0. Could you include the stack trace?

gustavosoares · 2015-11-10T10:46:18Z

There is no stack trace actually. When I run docker-compose logs and then hit ctrl+c I receive the following message:

^CERROR:
Aborting.

hope that helps...

$ docker-compose version
docker-compose version: 1.5.0
docker-py version: 1.5.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013

$ docker version
Client:
Version: 1.8.2
API version: 1.20
Go version: go1.4.2
Git commit: 0a8c2e3
Built: Thu Sep 10 19:19:00 UTC 2015
OS/Arch: linux/amd64

Server:
Version: 1.8.2
API version: 1.20
Go version: go1.4.2
Git commit: 0a8c2e3
Built: Thu Sep 10 19:19:00 UTC 2015
OS/Arch: linux/amd64

dnephin · 2015-11-10T15:15:02Z

Ok, that's a different issue (#1040)

gustavosoares · 2015-11-10T16:18:06Z

ok, thanks. I have subscribed to it. perhaps the tilte of issue #1040 should be renamed... the problem occurs with docker-compose logs as well and not only docker-compose up.

andre487 · 2015-11-15T17:29:09Z

I have the same issue on 1.4.0 with this compose file:

web:
  image: andre487/apache-php
  ports:
    - 10001:80
  volumes:
    - ../..:/var/www
  environment:
    PROJECT_MODE: docker
  links:
    - db:database

db:
  image: mysql:5.5
  environment:
    MYSQL_ROOT_PASSWORD: swordfish
    MYSQL_DATABASE: db
    MYSQL_USER: user
    MYSQL_PASSWORD: swordfish

Trace:

^CGracefully stopping... (press Ctrl+C again to force)
Stopping docker_web_1... done
Stopping docker_db_1... done
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/compose/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 39, in main
  File "/compose/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 21, in sys_dispatch
  File "/compose/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 27, in dispatch
  File "/compose/build/docker-compose/out00-PYZ.pyz/compose.cli.docopt_command", line 24, in dispatch
  File "/compose/build/docker-compose/out00-PYZ.pyz/compose.cli.command", line 59, in perform_command
  File "/compose/build/docker-compose/out00-PYZ.pyz/compose.cli.main", line 503, in up
  File "/compose/build/docker-compose/out00-PYZ.pyz/compose.cli.log_printer", line 22, in run
  File "/compose/build/docker-compose/out00-PYZ.pyz/compose.cli.multiplexer", line 25, in loop
  File "/compose/build/docker-compose/out00-PYZ.pyz/Queue", line 182, in get
thread.error: release unlocked lock

Is it #1040 as well?

dnephin · 2015-11-15T19:39:34Z

It was fixed in #2055 which is in 1.5.0. Please try the latest version.

andre487 · 2015-11-15T20:08:59Z

Ok, thanks

lrkwz · 2015-11-18T12:15:14Z

A little bit better :-) using 1.5.1

^CGracefully stopping... (press Ctrl+C again to force)
Stopping dockersolostampa_varnish_1 ... done
Stopping dockersolostampa_nginx_1 ... done
Stopping dockersolostampa_php_1 ... done
Stopping dockersolostampa_db_1 ... done
Stopping dockersolostampa_application_1 ... done
ERROR: 
Aborting.

Environment:

~/D/docker-solostampa> docker-compose --version                                                                               1
docker-compose version: 1.5.1
~/D/docker-solostampa> docker --version
Docker version 1.9.0, build 76d6bc9
~/D/docker-solostampa> uname -r
3.13.0-68-generic

dnephin · 2015-11-18T15:40:28Z

#2364 may address the blank error

Error oom if container has not stopped

dnephin added the kind/bug label Dec 3, 2014

deviantony mentioned this issue Mar 14, 2015

thread.error: release unlocked lock #1069

Closed

1d3df9903ad mentioned this issue Apr 12, 2015

Release unlocked lock @ pool276.py:wait():549 andresriancho/w3af#9031

Closed

aanand changed the title ~~cmd + c sometimes crashes~~ Ctrl-C on docker-compose up raises thread.error Jul 16, 2015

bfirsh added the backlog label Jul 20, 2015

bfirsh removed the backlog label Sep 2, 2015

dnephin added a commit to dnephin/compose that referenced this issue Sep 16, 2015

Fixes docker#189 - stacktrace when ctrl-c stops logs

bdfb21f

Signed-off-by: Daniel Nephin <dnephin@docker.com>

dnephin self-assigned this Sep 16, 2015

dnephin mentioned this issue Sep 16, 2015

Fix stacktrace when ctrl-c stops logs #2055

Merged

aanand closed this as completed in #2055 Sep 17, 2015

mdaue pushed a commit to mdaue/compose that referenced this issue Oct 10, 2015

Fixes docker#189 - stacktrace when ctrl-c stops logs

44dc4a1

Signed-off-by: Daniel Nephin <dnephin@docker.com>

mdaue pushed a commit to mdaue/compose that referenced this issue Oct 10, 2015

Fixes docker#189 - stacktrace when ctrl-c stops logs

ec2e2d3

Signed-off-by: Daniel Nephin <dnephin@docker.com>

xulike666 pushed a commit to xulike666/compose that referenced this issue Jan 19, 2017

Merge pull request docker#189 from docker/oom

389483d

Error oom if container has not stopped

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ctrl-C on `docker-compose up` raises thread.error #189

Ctrl-C on `docker-compose up` raises thread.error #189

hunterloftis commented Apr 13, 2014

patrick91 commented Jul 21, 2014

aanand commented Jul 21, 2014

d11wtq commented Jul 25, 2014

mpe commented Oct 14, 2014

xiaods commented Oct 20, 2014

motdotla commented Nov 12, 2014

pikeas commented Nov 20, 2014

abrutus commented Nov 20, 2014

schmunk42 commented Dec 2, 2014

pwaller commented Jan 16, 2015

wkruse commented Feb 9, 2015

aidanlister commented Apr 6, 2015

delfuego commented Apr 7, 2015

emerleite commented May 6, 2015

aanand commented May 6, 2015

mahnunchik commented Jul 16, 2015

hoIIer commented Aug 5, 2015

forbearer commented Aug 18, 2015

shyr commented Sep 6, 2015

christianbundy commented Sep 11, 2015

dnephin commented Sep 12, 2015

gustavosoares commented Nov 9, 2015

dnephin commented Nov 9, 2015

gustavosoares commented Nov 10, 2015

dnephin commented Nov 10, 2015

gustavosoares commented Nov 10, 2015

andre487 commented Nov 15, 2015

dnephin commented Nov 15, 2015

andre487 commented Nov 15, 2015

lrkwz commented Nov 18, 2015

dnephin commented Nov 18, 2015

Ctrl-C on docker-compose up raises thread.error #189

Ctrl-C on docker-compose up raises thread.error #189

Comments

hunterloftis commented Apr 13, 2014

patrick91 commented Jul 21, 2014

aanand commented Jul 21, 2014

d11wtq commented Jul 25, 2014

mpe commented Oct 14, 2014

xiaods commented Oct 20, 2014

motdotla commented Nov 12, 2014

pikeas commented Nov 20, 2014

abrutus commented Nov 20, 2014

schmunk42 commented Dec 2, 2014

pwaller commented Jan 16, 2015

wkruse commented Feb 9, 2015

aidanlister commented Apr 6, 2015

delfuego commented Apr 7, 2015

emerleite commented May 6, 2015

aanand commented May 6, 2015

mahnunchik commented Jul 16, 2015

hoIIer commented Aug 5, 2015

forbearer commented Aug 18, 2015

shyr commented Sep 6, 2015

christianbundy commented Sep 11, 2015

dnephin commented Sep 12, 2015

gustavosoares commented Nov 9, 2015

dnephin commented Nov 9, 2015

gustavosoares commented Nov 10, 2015

dnephin commented Nov 10, 2015

gustavosoares commented Nov 10, 2015

andre487 commented Nov 15, 2015

dnephin commented Nov 15, 2015

andre487 commented Nov 15, 2015

lrkwz commented Nov 18, 2015

dnephin commented Nov 18, 2015

Ctrl-C on `docker-compose up` raises thread.error #189

Ctrl-C on `docker-compose up` raises thread.error #189