ddev "api" for interacting with ddev-ui, other future applications, and normal user

[rfay]

What happened (or feature request):

For ddev-ui, we need to provide

• a straightforward, machine-consumable output technique
• A way to issue any command without any terminal-interactive behavior

We discussed this today and decided this overall plan:

• A command-line flag (with backup environment variable) will put ddev in json mode, so that all output is appropriate.
• A logging layer (or similar) will be used for all interaction with the end-user. Based on the above flag, it can output either end-user or json output. Brad suggested that we might find another project that has done this same thing and borrow their approach.
• Long-running processes like start/stop/rm must do status output that can be used for a progress indicator. For example, a json blob with "web container started X%", another with "db container started X%", etc.

These don't deal with the sudo problem, but that may be able to be bundled into a single ddev hostname call so that it can only happen in one place.

[andrew-c-tran]

The current UI expects the following JSON structure from the output of ddev list and describe:

ddev list is basically an object containing a string with the router status, as well as an array of site objects

EX:

{
  "routerStatus":"running",
  "sites": [
    {
      "name":"drupaltest",
      "type":"drupal8",
      "path":"~/Downloads/drupal",
      "url":"http://drupaltest.ddev.local",
      "state":"running"
    },
    {
      "name":"drupaltest2",
      "type":"drupal8",
      "path":"~/Downloads/drupal2",
      "url":"http://drupaltest2.ddev.local",
      "state":"running"
    }
  ]
}

ddev describe is an object in which various site details is contained in a child object with the site detail description as it's key.

EX:

{
   "MySQL Credentials":{
      "notes":[
         "To connect to mysql from your host machine, use port 32769 on 127.0.0.1.",
         "For example:mysql --host=127.0.0.1 --port=32769 --user=db --password=db --database=db"
      ],
      "Username":"db",
      "Password":"db",
      "Database name":"db",
      "Host":"db",
      "Port":"3306"
   },
   "Other Services":{
      "notes":[],
      "MailHog":"http://drupaltest.ddev.local:8025",
      "phpMyAdmin":"http://drupaltest.ddev.local:8036"
   }
}

[rickmanelius]

I apologize if these are naive, but I have a few follow-up questions:

• If we are using an environment variable as a backup, would a user know how to get out of json mode if all they are getting back is json?
• In the API response packets that @andrew-c-tran is suggesting, I notice there are things that are missing. For example, technically there are other bits of information, such as the ddev router status.
• I notice we're not noting commands like auth-pantheon.
• Do we need to handle error logging or always pass back a status?
• Administratively, what is the status of this ticket? Incubate? Actionable?

[andrew-c-tran]

I don't see a need to go into a different mode at all, i'd propose a flag (ddev list --json) that would instead output as json instead of the pretty text we're getting now.

ddev router status was indeed omitted, and would need to be added in. will update.

auth-pantheon in specific was not addressed simply because i haven't gotten to working with that command yet tbh.

[rickmanelius]

Hi @andrew-c-tran. Understood. I just want to be thorough so that this becomes something we can get an LOE and also get to an actionable state!

[rfay]

sirupsen/logrus has ways to do complete json output, so per Brad's suggestion I'm experimenting with it to see how we could use it for all user-oriented (or ddev-ui oriented) output.

[andrew-c-tran]

Updated format of DDEV List in post above to include RouterStatus

[rfay]

Example usage: https://github.com/rfay/logrustry/blob/master/cmd/logrustry/main.go

Proposed strategy:

• Add a UserOut logger (just a logrus.New())
• Set a custom formatter like UserTextFormatter to the UserOut - it will drop the timestamp and loglevel, can do other things as well. This is a fork of the logrus TextFormatter.
• Add a ddev command flag to specify json output, like --json. This would cause setting the UserOut formatter to log.JSONFormatter
• Use logrus (UserOut) for all output aimed at the user
• Remove all imports of the stdlib log
• Remove all usage of log in tests, they should be using t.log and family
• Remove all usages of fmt.Print* and the like
• ddev list and ddev describe need more attention than this because they painstakingly build the text as a simple output buffer. We'll need to refactor them so that they build a data structure and then either output the data structure as json or render the table and output it.

(There is an open question: logrus likes single-level fields with json. How can it do a table with rows and fields successfully? I'll take a look)