Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

erro[0149] error waiting for container: invalid character 'u' looking for beginning of value #5139

Open
2 tasks done
jellchen opened this issue Dec 16, 2020 · 55 comments
Open
2 tasks done

Comments

@jellchen
Copy link

jellchen commented Dec 16, 2020

  • I have tried with the latest version of my channel (Stable or Edge)
  • I have uploaded Diagnostics
  • Diagnostics ID: DB4CAD3F-E900-48DB-9F86-488066CD63AC/20201216123519

Expected behavior

I run google 'gclient sync' in ssh shell connecting to docker container running ubuntu18.04 based system.
it always exit and report erro[0149] error waiting for container: invalid character 'u' looking for beginning of value.
and the docker seem hang until restart docker

Actual behavior

docker hang unti restart

Information

  • macOS Version:

MACOS Big Sur 11.1 Beta版(20C5061b)

Diagnostic logs

Docker for Mac: version...
Docker version 20.10.0, build 7287ab3

Steps to reproduce the behavior

  1. ...
  2. ...
@Overbryd
Copy link

Overbryd commented Dec 21, 2020

I have the same problem.

v20.10.0

My steps to reproduce:

  • Start a container, and interactively bind to its shell
  • Start typing a command and hit <TAB>
  • The container completely freezes, cannot be stopped, killed or restart. I must restart Docker for mac to resolve the issue. After I restart Docker for mac, I am dropped back into my shell with the error message.
$ docker run -ti debian:slim /bin/bash
# apt<TAB>
ERRO[0101] error waiting for container: invalid character 'u' looking for beginning of value

@amirhmoradi
Copy link

same issue here, docker for mac v3.0.3, engine v20.10.0

@chenzx
Copy link

chenzx commented Jan 21, 2021

i encounter the same problem when trying to update code for chromium source:

root@f78e6bf0263a:/chromium/src# git rebase-update
ERRO[5887] error waiting for container: invalid character 'u' looking for beginning of value 
                                                                                             %    

@lttlfl
Copy link

lttlfl commented Jan 21, 2021

Hello,

I am getting a similar error. I had terminated one of the docker run command with ctrl c and then every container which I run next is stuck in created state and seems to be frozen. When I restarted docker desktop, I get the below error. can someone please suggest how to go about debugging this ?

OS : macOS Catalina
Docker desktop version : 3.1.0

ERRO[13699] error waiting for container: invalid character 'u' looking for beginning of value
docker: error during connect: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.41/containers/dec51ad1e9bfe045f03fc9f707bfeaa5cc8fd6bd000018ccb986a87682484c48/start: EOF.

@allisonburtch
Copy link

allisonburtch commented Jan 22, 2021

Similar to the commenter above, I had terminated a docker run command with CTRL C and now I also have this error when running docker.
ERRO[0114] error waiting for container: invalid character 'u' looking for beginning of value
The first error was this:
cc1: fatal error: ../extern/libwebsockets/libwebsockets/include/stdc-predef.h: Transport endpoint is not connected

EDIT: This issue was solved for me. I was using the make -j command. When I changed to the slower make I no longer ran into issues.

@JakkuSakura
Copy link

The same error for me. Some container can run but some cannot

@lttlfl
Copy link

lttlfl commented Jan 26, 2021

Hello,

I am getting a similar error. I had terminated one of the docker run command with ctrl c and then every container which I run next is stuck in created state and seems to be frozen. When I restarted docker desktop, I get the below error. can someone please suggest how to go about debugging this ?

OS : macOS Catalina
Docker desktop version : 3.1.0

ERRO[13699] error waiting for container: invalid character 'u' looking for beginning of value
docker: error during connect: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.41/containers/dec51ad1e9bfe045f03fc9f707bfeaa5cc8fd6bd000018ccb986a87682484c48/start: EOF.

Update: apparently the docker is waiting on response from the linux vm and linux seems to be stuck. The error messages posted above gets displayed only when the docker desktop is restarted. I did try to get into linux vm ( using docker run -it --privileged --pid=host debian nsenter -t 1 -m -u -n -I sh ) and was not able to kill the process running this container. Is there a way to find out why the linux vm is frozen ? got the below error I syslog while this happened.
localhost com.docker.hyperkit[577]: [664584.900697] INFO: task cc1:88959 blocked for more than 120 seconds.
localhost com.docker.hyperkit[577]: [664584.901760] Tainted: G T 4.19.76-linuxkit #1

@docker-robott
Copy link
Collaborator

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

@austinbutler
Copy link

/remove-lifecycle stale

@Reifier
Copy link

Reifier commented Apr 27, 2021

/remove-lifecycle stale

@Zodiase
Copy link

Zodiase commented May 5, 2021

My container runs npm ci and some build commands in it and currently it's never successfully completed a build today while I keep restarting Docker and retrying the command.

(the script runs fine on host machine)

Host OS: macOS 11.1
Docker Engine v20.10.5

@Bluefitdev
Copy link

Same issue for me

1 similar comment
@michal-kosyk
Copy link

Same issue for me

@copini
Copy link

copini commented May 20, 2021

Same here, but I could solve it by resetting to factory defaults from the Troubleshoot menu.
Restarting Docker and even purging data didn't help, only the factory reset made thing work normally again. Not ideal because all data and settings are lost, but at least I can use Docker again 🤷

@plaidman
Copy link

I'm having this same issue, running an ubuntu image, trying to compile a large project. resetting to factory defaults did not solve the issue for me.

@copini
Copy link

copini commented May 31, 2021

Happened again to me today, and this time resetting to factory defaults didn't solve it for me anymore either 😞 After reading this (related/same) issue I tried killing all Docker-related processes and even rebooted, nothing fixes it for me anymore...

UPDATE: got it working now by increasing memory in the resources settings (after resetting to factory defaults). Not sure if it's just a coincidence because it restarted Docker again (and it randomly works after some restarts?), but it's working for me now.

@giovapanasiti
Copy link

I made this script to restart docker and kill com.docker.vmnetd

#!/bin/bash

read -p "Are you sure you want to kill docker and com.docker.vmnetd? [y/n]" -n 1 -r

echo    # (optional) move to a new line
if [[ $REPLY =~ ^[Yy]$ ]]
then
    # do dangerous stuff
  $(killall Docker)

  pid=$(ps aux | grep com.docker.vmnetd | grep -v grep | awk '{print $2}')
  echo "Killing $pid"

  $(sudo kill -9 $pid)
  $(open /Applications/Docker.app)
fi

@michaeldong
Copy link

Similar to the commenter above, I had terminated a docker run command with CTRL C and now I also have this error when running docker.
ERRO[0114] error waiting for container: invalid character 'u' looking for beginning of value
The first error was this:
cc1: fatal error: ../extern/libwebsockets/libwebsockets/include/stdc-predef.h: Transport endpoint is not connected

EDIT: This issue was solved for me. I was using the make -j command. When I changed to the slower make I no longer ran into issues.

Yes!!!

@munapower
Copy link

I am having a similar issue. I am running on Mac OS
Docker Engine: Docker version 20.10.7, build f0df350
Docker Desktop Version 3.5.1 (3.5.1.7)

I am executing a make from inside a container. What has changed are the docker versions, the rest of my environment hasn't changed. When I execute:
make -C $FPC_PATH/utils/docker build
make: Entering directory '/project/src/github.com/hyperledger/fabric-private-chaincode/utils/docker'
DOCKER_BUILDKIT=1 docker build --build-arg FPC_VERSION=main -t hyperledger/fabric-private-chaincode-base-rt base-rt
failed to dial gRPC: cannot connect to the Docker daemon. Is 'docker daemon' running on this host?: dial unix /Users/mmunaro/.docker/run/docker.sock: connect: no such file or directory
make: *** [Makefile:175: base-rt] Error 1

I get an error that it can't find docker.sock. I usually click or unclick the Docker Compose v2 experimental feature and re run the command and I get stuck in the docker build:

make -C $FPC_PATH/utils/docker build
make: Entering directory '/project/src/github.com/hyperledger/fabric-private-chaincode/utils/docker'
DOCKER_BUILDKIT=1 docker build --build-arg FPC_VERSION=main -t hyperledger/fabric-private-chaincode-base-rt base-rt
make: *** [Makefile:175: base-rt] Terminated
ERRO[0395] error waiting for container: invalid character 'u' looking for beginning of value

The only way to get out of that is to restart Docker Desktop and that is when I get the invalid character.

I have reset factory settings, increased Resources memory, added DOCKER_BUILDKIT_PROGRESS=plain just to be sure where it hangs.
Still I can not get passed the docker build.

@lmarkes
Copy link

lmarkes commented Jul 12, 2021

@munapower I resolved a similar issue by updating to Docker Desktop Version 3.5.2 (3.5.2.18).
It's still using the same docker engine version you reference.

@vinnson
Copy link

vinnson commented Jul 20, 2021

same here and i see funny characters "^[[51;8R" when running shell

@erebosXx
Copy link

erebosXx commented Aug 5, 2021

I had the same issue, the problem was that I had no disk space left on my host machine.

@bzawada
Copy link

bzawada commented Aug 16, 2021

I have the same issue.

@tmtong
Copy link

tmtong commented Apr 22, 2022

/remove-lifecycle stale

@dlinten
Copy link

dlinten commented May 17, 2022

Known issue on Mac by default, either don't use a Mac, or ensure Docker engine is configured. make will return and will result in this error when memory is constrained for the container. See #5139 (comment)

Solution, always ensure there is enough memory for make when using -j option. Memory required 4GB Minimum for make to complete on 4 processor cores i.e. make -j4

/remove-lifecycle stale

@docker-robott
Copy link
Collaborator

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

@tmtong
Copy link

tmtong commented Aug 28, 2022

/lifecycle frozen

@keithgmitchell
Copy link

Is this still not resolved? the make -j doesnt seem to change it. Is there a certain version I can switch to where this works? I have plenty of disk space. How has nobody at docker fixed this?

@davidlukac
Copy link

This has started to happen for me randomly after last update of Docker Destop for Mac to version o 4.13.0 (89412) (Engine: 20.10.20). OS: Monterey 12.6 ... like during terraform apply

ERRO[0575] error waiting for container: invalid character 'c' looking for beginning of value

Use gRPC FUSE for file sharing already checked

@akpw
Copy link

akpw commented Oct 30, 2022

same thing here @davidlukac, have you found a solution?

@jamshid
Copy link

jamshid commented Oct 30, 2022

I doubt it's same root issue y'all are seeing but I'm seeing this error when vpnkit crashes on Docker for Mac (intel and arm64).
ERRO[0000] error waiting for container: invalid character 'c' looking for beginning of value
I have to Quit and reopen Docker but problem returns. Hopefully somebody responds in moby/vpnkit#610.

@max4ever
Copy link

max4ever commented Oct 31, 2022

I have this problem when using VNC to connect to selenium/standalone-chrome-debug

image
image

@mmikhalko
Copy link

I have the same issue!
/remove-lifecycle stale

@Yalchin403
Copy link

getting similar error on macos, invalid character 's' looking for beginning of value

m2 chip

@bientavu
Copy link

bientavu commented Nov 12, 2022

Hey guys, same here since I updated to MacOS ventura... ERRO[0193] error waiting for container: invalid character 'c' looking for beginning of value.

M1 chip, I have to restart as well Docker and sometimes do some factory reset...
Memory already extended to more than required and gRPC FUSE checked

@Yalchin403
Copy link

I have second machine which runs on windows 10, exact same docker-compose.yml file works flawlessly, however after running the same containers on my m2 macbook pro, I see that 2 of those containers restarting after some time, then the whole docker crashes and I see that error on the console. Then I'm unable to restart, quit the docker. I need to restart the whole laptop. For simple projects, i haven't noticed such a problem

@AmazingTurtle
Copy link

In my case it was due to CR line ending. Changing to CRLF resolved the issue for me!

@Yalchin403
Copy link

In my case it was due to CR line ending. Changing to CRLF resolved the issue for me!

Can you please elaborate on this, how can we achieve the same?

@Yalchin403
Copy link

Seems like there is no problem when I downgraded the version to 4.12.0

@gmossessian
Copy link

Not seeing this anymore after upgrading docker to 4.15.0, on an apple M1 with Ventura 13.0.

@Niek
Copy link

Niek commented Dec 21, 2022

I get this issue all the time, even on 4.15.0. The workaround for me is: pkill -f docker && open -a Docker

@yaroslav-zdanovskyi
Copy link

yaroslav-zdanovskyi commented Dec 30, 2022

When running docker build on Windows I get invalid character 'c' looking for beginning of value after which docker daemon dies...

changing FROM node:19 to FROM node:19-alpine fixed it for me

@wazum
Copy link

wazum commented Jan 17, 2023

I get this issue with Docker for Mac 4.16.1 on macOS 13.1 (22C65) randomly.

@ikreymer
Copy link

ikreymer commented Apr 1, 2023

Also seeing this consistently near 100% repro with using chrome/brave and playwright with 4.17.0, seems to be related to streaming a lot of data via the TCP connections?

@alordthorsen
Copy link

alordthorsen commented Apr 19, 2023

I'm currently hitting this 100% of the time when running on OSX 12.4 Monterey and Docker Desktop 4.18.0 and docker engine v20.10.24

docker run -i --rm -v "/Users/alexlordthorsen/git/rates/test-rates-generator/test_rate_generator":/code/test_rate_generator -v "/Users/alexlordthorsen/git/rates/test-rates-generator/tests":/code/tests docker-test-rate-event-unit-testing:latest /bin/bash

@Rany-yilian
Copy link

Rany-yilian commented Dec 4, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests