Manage ddev project list independently of docker container labels (yaml)

[rfay]

What happened (or feature request):

Over in #484 (comment) the discussion is underway about ddev start/stop/rm, and one of the key reasons we've kept containers around (the "stopped" state) is just because ddev doesn't know what projects it has unless docker tells it.

In reality we need to track projects separately from docker, because it's impossible for docker containers to represent the actual state of what a user may have set up.

We've talked at times about using a ddev-managed yaml file in ~/.ddev to track project state; It would probably be more appropriate to use a real database like sqlite instead. Basically every start would put the key information into the db, and ddev list would report the info from that database (whether sqlite or yaml or whatever). Additional docker container information would be used to supplement the information about project state.

This allows us to:

• Fully catalog the projects that a user has created (and show their state)
• Fully prevent name collisions on projects
• Fully prevent databases accidentally being shared by projects.

Essentially, way back in the day (drud local days) docker containers were the easiest way to get info about projects, and we used it. But we've outgrown that, and it was inadequate always. It's time to leave it behind.

I'd envision the following behavior:

• ddev start updates the database
• ddev stop (our current "remove") also updates the db
• ddev list uses the database and supplements with live docker info to present its list. But it would know of all projects. It could also know the status of db (size, and whether it exists or has been removed, which can be pretty important and useful)
• Probably we'd need a new and completely separate ddev rmdb or something to trash the database, as database existence isn't really about start/stop/rm
• There would probably need to be a new command to remove projects from the list that the user didn't want to show any more, and probably ddev list would need some new filter options, for example to not show projects that have no database, or to not show projects that are inactive.

Destruction of the list db would be nonfatal (just deletion of a file) and the database could be recreated naturally just by starting projects.

This is most likely a prerequisite to #484 (reorganize stop/rm commands)

Please use a complexity rating of 1-5 (5 is high) for a feature request. (High complexity implies more PR planning)

3 - This appears relatively easy to implement, but will likely expose weaknesses in our code that will have to be dealt with.

[rfay]

Note that #524 was the previous place this was being discussed, and there was an associated google doc with the various issues

[rickmanelius]

Initial read looks good. A few thoughts (and these are just thoughts/brainstorming).

• From a naming convention/organization, I wonder if it will make sense to switch from ddev list to ddev projects. I could see that then being used to target additions and removal of projects. Yes, running ddev config or ddev start assumes an insertion and update into the DB, but it would be possible to ddev project rm X
• Upgrades. One thing that would be neat is tracking the config.yml version and highlighting to the user that they are out of date and may need to recreate.
• This might also help address the long-standing feature request of global targetting As CLI user, I want the ability to interact with each site from any directory. #44

I realize that we probably need to keep this initial scope small and tight, but I can see wider implications and opportunities that I at least want to surface as we're shaping this to an actionable state.

[ultimike]

I think that having an alternate method to drive the results of ddev list (other than Docker containers) is a good idea. As someone who has taught DDEV to a few dozen folks at this point, I can attest that explaining that ddev list only shows projects that have containers built is counter-intuitive for users who aren't familiar with Docker.

More than once, after using ddev rm on a project, and then not seeing the project in ddev list, I've had students ask me (worried), "what happened to my project!?"

The idea to use a small database to keep track of projects with a .ddev/ directory is a good one, as I can envision all sorts of good project metadata that could be included and then output as part of ddev list 2.0:

• Name
• Type
• Location
• URLs
• Status (running, stopped, removed)
• Last time used
• Created date
• DDEV configuration version
• Database size
• Containers configured (web/db/dba/solr/etc...)

I'm sure there's more that I can come up with.

-mike

[rickmanelius]

Since this has been a long time since we've revisited/updated, we will also need to clarify the language around ddev rm/stop/start because for a while we were going to remove "stop" and now I don't believe we will. The question then becomes how we delineate removing the containers, the config, the metadata, and/or the database. Not an impossible problem to solve, but just issues to work through as we approach this.

[rfay]

With the pull of #1502 this one is now completely unblocked in honor of @rickmanelius .