Multiple 'Cannot start service' errors

[lumpysimon]

I'm running Lando v3.0.0-beta.45 on MacOS 10.13.4

Lando worked fine for the first few days, starting up projects went smoothly every time, but for the last couple of days I'm getting a load of errors almost every time I run lando start on any project. After getting the errors, I have to run lando rebuild (even though the .lando.yml file has not changed) and then everything is fine again.

Creating network "earl_default" with the default driver
Recreating earl_appserver_1 ...
Starting earl_mailhog_1     ...
Starting earl_database_1    ... error

Starting earl_mailhog_1     ... error
a7fbe not found
Recreating earl_appserver_1 ... done
ERROR: for earl_mailhog_1  Cannot start service mailhog: network b73b65c493adbd3962484fb35e275e7db055b169774f6f9b5f44a212244de4da not found

ERROR: for database  Cannot start service database: network a99c32bfd8d6c2015d46f1a8b0a22ce67f824fdbd1508d5613a358b6e63a7fbe not found

ERROR: for mailhog  Cannot start service mailhog: network b73b65c493adbd3962484fb35e275e7db055b169774f6f9b5f44a212244de4da not found
ERROR: Encountered errors while bringing up the project.
earl_appserver_1 is up-to-date
Starting earl_database_1 ...
Starting earl_mailhog_1  ... error

Starting earl_database_1 ... error
4da not found

ERROR: for earl_database_1  Cannot start service database: network a99c32bfd8d6c2015d46f1a8b0a22ce67f824fdbd1508d5613a358b6e63a7fbe not found

ERROR: for mailhog  Cannot start service mailhog: network b73b65c493adbd3962484fb35e275e7db055b169774f6f9b5f44a212244de4da not found

ERROR: for database  Cannot start service database: network a99c32bfd8d6c2015d46f1a8b0a22ce67f824fdbd1508d5613a358b6e63a7fbe not found
ERROR: Encountered errors while bringing up the project.
Starting earl_mailhog_1 ...
earl_appserver_1 is up-to-date
Starting earl_database_1 ... error

Starting earl_mailhog_1  ... error
de4da not found

ERROR: for earl_mailhog_1  Cannot start service mailhog: network b73b65c493adbd3962484fb35e275e7db055b169774f6f9b5f44a212244de4da not found

ERROR: for database  Cannot start service database: network b73b65c493adbd3962484fb35e275e7db055b169774f6f9b5f44a212244de4da not found

ERROR: for mailhog  Cannot start service mailhog: network b73b65c493adbd3962484fb35e275e7db055b169774f6f9b5f44a212244de4da not found
ERROR: Encountered errors while bringing up the project.
Starting earl_mailhog_1 ...
Starting earl_mailhog_1  ... error
earl_appserver_1 is up-to-date

ERROR: for earl_mailhog_1  Cannot start service mailhog: network a99c32bfd8d6c2015d46f1a8b0a22ce67f824fdbd1508d5613a358b6e63a7Starting earl_database_1 ... error

ERROR: for earl_database_1  Cannot start service database: network b73b65c493adbd3962484fb35e275e7db055b169774f6f9b5f44a212244de4da not found

ERROR: for mailhog  Cannot start service mailhog: network a99c32bfd8d6c2015d46f1a8b0a22ce67f824fdbd1508d5613a358b6e63a7fbe not found

ERROR: for database  Cannot start service database: network b73b65c493adbd3962484fb35e275e7db055b169774f6f9b5f44a212244de4da not found
ERROR: Encountered errors while bringing up the project.
earl_appserver_1 is up-to-date
Starting earl_mailhog_1 ...
Starting earl_database_1 ... error

Starting earl_mailhog_1  ... error
de4da not found

ERROR: for earl_mailhog_1  Cannot start service mailhog: network b73b65c493adbd3962484fb35e275e7db055b169774f6f9b5f44a212244de4da not found

ERROR: for database  Cannot start service database: network b73b65c493adbd3962484fb35e275e7db055b169774f6f9b5f44a212244de4da not found

ERROR: for mailhog  Cannot start service mailhog: network b73b65c493adbd3962484fb35e275e7db055b169774f6f9b5f44a212244de4da not found
ERROR: Encountered errors while bringing up the project.
error: Error

And here's my .lando.yml file (apart from the name and proxy URL, it's the same for every project):

name: earl
recipe: wordpress
config:
  php: '7.2'
  webroot: www
  xdebug: true
  conf:
    php: .vscode/php.ini
proxy:
  mailhog:
    - mail.earl.lndo.site
services:
  mailhog:
    type: mailhog
    hogfrom:
      - appserver

I'm using XDebug as per the instructions at https://docs.devwithlando.io/tutorials/lando-with-vscode.html

[dustinleblanc]

Hey @lumpysimon!

By chance, does this happen when you first start up a project for the day after leaving it dormant overnight? I had this happen to me several times and that seems to be a common pre-req for me

[dustinleblanc]

I think I've just tracked down the potential cause of this issue:

docker/compose#5745 (comment)

[dustinleblanc]

Repro steps that I just figured out:

1. lando start an app
2. lando stop an app
3. docker network prune <- kills that stopped app's network
4. lando start/restart <- error city
5. lando rebuild <- Turns out Lando rescues everyone from Vader even though it looked like he was an asshat

[lumpysimon]

Yep, that does seem to be the case, it seems to happen after I haven't used things for a while

[dustinleblanc]

@lumpysimon if you have time, I'd love if you can confirm that the steps in my comment above get you to the sad place as well

[pirog]

@dustinleblanc im going to try to repro this later because this is a nasty bug esp for people who have racked up a bunch of networks (which will happen if youve been using lando for awhile)

[lumpysimon]

@dustinleblanc Yep, following those steps takes me to the sad place too :(

[bianqui149]

@pirog docker network ls:

[mmenozzi]

Hi @pirog I think I have the same issue. This is my docker network ls output:

NETWORK ID          NAME                                        DRIVER              SCOPE
65c5de5b2dab        bridge                                      bridge              local
25f9388ea0ef        host                                        host                local
76b79e5c2fa9        lando_bridge_network                        bridge              local
8d363576e4cf        landoproxyhyperion5000gandalfedition_edge   bridge              local
2a91f8c50d62        manhattanshop_default                       bridge              local
4671adaf6c1e        none                                        null                local

[pirog]

ok i can definitely replicate this @dustinleblanc i think the corrective action we need to take here is

1. Filter out "lando-y" networks when we do this
2. Institute some sort of network tagging to make 1. easier in the future

[pirog]

Alright, i've pushed up a "should be good enough" resolution for this. going to roll a new release since this bug is somewhat opaque even if its only surfacing for a small subset of users