Deployment issues on M1 Mac

[TTitcombe]

The issue

I cannot deploy with VespaDocker multiple times without removing and re-pulling the image.

After pulling vespaengine/vespa (using latest), I can successfully deploy an instance using pyvespa on an M1 mac. If I stop and remove the container and then try to re-deploy, it timeouts on "waiting for application status". However, if I remove the image and then repull, the next deployment succeeds.

System Info

• Image: vespaengine/vespa:latest (I think 8.116-8.118 based on dates)
• Pyvespa: 0.22.0
• OS: macOS 12.3.1
• Chip: M1 pro

[kkraune]

removing the image sounds excessive - maybe the last container is not cleaned up - what does docker ps -a -q -f status=exited show?

[TTitcombe]

Only containers unrelated to vespa when I run that command and lsof shows that 8080 isn't in use.

I agree that removing the image is a ridiculous workaround seeing as images != containers, but yet it seems to work 🤷

[kkraune]

I am working on reproducing the problem now

[kkraune]

Hi again, we have found that the signal handling in the start script does not seem to properly trap the TERM signal, so stopping the container isn't correct. We are working to find out more, and a fix, will update

[kkraune]

vespa-engine/docker-image#44 will fix the Docker stop issue and be released later this week - so can test pyvespa again then - I will update

[kkraune]

https://hub.docker.com/r/vespaengine/vespa/tags - 8.122.27 (latest as of today) has a fix for stopping the Docker container without the wait (link in above comment). Maybe that will help fix your issue, I don't have your problem, but I'm on intel...

Can I ask you to test again with the latest image? if still an issue, I will find someone with an M1 and try to reproduce

[arnej27959]

copy-pasting the "getting started" sample python scripts and running it twice with stop/rm between works fine for me.
this is on M1 mac, pyvespa-0.31.0 and vespa-8.122.27 versions.

% python3 dd.py 
Waiting for configuration server, 0/300 seconds...
Waiting for configuration server, 5/300 seconds...
Waiting for application status, 0/300 seconds...
Waiting for application status, 5/300 seconds...
Waiting for application status, 10/300 seconds...
Waiting for application status, 15/300 seconds...
Waiting for application status, 20/300 seconds...
Waiting for application status, 25/300 seconds...
Finished deployment.
% docker stop qa
qa
% docker rm qa  
qa
% python3 dd.py 
Waiting for configuration server, 0/300 seconds...
Waiting for configuration server, 5/300 seconds...
Waiting for application status, 0/300 seconds...
Waiting for application status, 5/300 seconds...
Waiting for application status, 10/300 seconds...
Waiting for application status, 15/300 seconds...
Waiting for application status, 20/300 seconds...
Finished deployment.

[kkraune]

Thanks Arne!

[TTitcombe]

I can also confirm this works. Thank you @kkraune + team (and sorry for not testing the fix sooner)