Add a local configuration source and a non-environmental ENV config source

Summary:
See discussion in T2221. Before we can move configuration to the database, we have a bootstrapping problem: we need database credentials to live //somewhere// if we can't guess them (and we can only really guess localhost / root / no password).

Some options for this are:

  - Have them live in ENV variables.
    - These are often somewhat unfamiliar to users.
    - Scripts would become a huge pain -- you'd have to dump a bunch of stuff into ENV.
    - Some environments have limited ability to set ENV vars.
    - SSH is also a pain.
  - Have them live in a normal config file.
    - This probably isn't really too awful, but:
    - Since we deploy/upgrade with git, we can't currently let them edit a file which already exists, or their working copy will become dirty.
    - So they have to copy or create a file, then edit it.
    - The biggest issue I have with this is that it will be difficult to give specific, easily-followed directions from Setup. The instructions need to be like "Copy template.conf.php to real.conf.php, then edit these keys: x, y, z". This isn't as easy to follow as "run script Y".
  - Have them live in an abnormal config file with script access (this diff).
    - I think this is a little better than a normal config file, because we can tell users 'run phabricator/bin/config set mysql.user phabricator' and such, which is easier to follow than editing a config file.

I think this is only a marginal improvement over a normal config file and am open to arguments against this approach, but I think it will be a little easier for users to deal with than a normal config file. In most cases they should only need to store three values in this file -- db user/host/pass -- since once we have those we can bootstrap everything else. Normal config files also aren't going away for more advanced users, we're just offering a simple alternative for most users.

This also adds an ENVIRONMENT file as an alternative to PHABRICATOR_ENV. This is just a simple way to specify the environment if you don't have convenient access to env vars.

Test Plan: Ran `config set x y`, verified writes. Wrote to ENVIRONMENT, ran `PHABRICATOR_ENV= ./bin/repository`.

Reviewers: btrahan, vrana, codeblock

Reviewed By: codeblock

CC: aran

Maniphest Tasks: T2221

Differential Revision: https://secure.phabricator.com/D4294

Author: epriestley

.gitignore

@@ -1,10 +1,6 @@
 .DS_Store
 ._*
-/docs/
-/src/.phutil_module_cache
-/conf/custom/*
 /webroot/rsrc/custom
-/.divinercache
 .#*
 *#
 *~
@@ -15,3 +11,15 @@
 
 # Arcanist scratch directory
 /.arc
+
+# Diviner
+/docs/
+/.divinercache
+
+# libphutil
+/src/.phutil_module_cache
+
+# Configuration
+/conf/custom/*
+/conf/local/local.json
+/conf/local/ENVIRONMENT

bin/config

@@ -0,0 +1 @@
+../scripts/setup/manage_config.php

conf/local/.keep

@@ -0,0 +1,22 @@
+#!/usr/bin/env php
+<?php
+
+$root = dirname(dirname(dirname(__FILE__)));
+require_once $root.'/scripts/__init_script__.php';
+
+$args = new PhutilArgumentParser($argv);
+$args->setTagline('manage configuration');
+$args->setSynopsis(<<<EOSYNOPSIS
+**config** __command__ [__options__]
+    Manage Phabricator configuration.
+
+EOSYNOPSIS
+  );
+$args->parseStandardArguments();
+
+$workflows = array(
+  new PhabricatorConfigManagementSetWorkflow(),
+  new PhutilHelpArgumentWorkflow(),
+);
+
+$args->parseWorkflows($workflows);

scripts/setup/manage_config.php

@@ -687,6 +687,9 @@
     'PhabricatorConfigEntryDAO' => 'applications/config/storage/PhabricatorConfigEntryDAO.php',
     'PhabricatorConfigFileSource' => 'infrastructure/env/PhabricatorConfigFileSource.php',
     'PhabricatorConfigListController' => 'applications/config/controller/PhabricatorConfigListController.php',
+    'PhabricatorConfigLocalSource' => 'infrastructure/env/PhabricatorConfigLocalSource.php',
+    'PhabricatorConfigManagementSetWorkflow' => 'infrastructure/env/management/PhabricatorConfigManagementSetWorkflow.php',
+    'PhabricatorConfigManagementWorkflow' => 'infrastructure/env/management/PhabricatorConfigManagementWorkflow.php',
     'PhabricatorConfigProxySource' => 'infrastructure/env/PhabricatorConfigProxySource.php',
     'PhabricatorConfigSource' => 'infrastructure/env/PhabricatorConfigSource.php',
     'PhabricatorConfigStackSource' => 'infrastructure/env/PhabricatorConfigStackSource.php',
@@ -2000,6 +2003,9 @@
     'PhabricatorConfigEntryDAO' => 'PhabricatorLiskDAO',
     'PhabricatorConfigFileSource' => 'PhabricatorConfigProxySource',
     'PhabricatorConfigListController' => 'PhabricatorConfigController',
+    'PhabricatorConfigLocalSource' => 'PhabricatorConfigProxySource',
+    'PhabricatorConfigManagementSetWorkflow' => 'PhabricatorConfigManagementWorkflow',
+    'PhabricatorConfigManagementWorkflow' => 'PhutilArgumentWorkflow',
     'PhabricatorConfigProxySource' => 'PhabricatorConfigSource',
     'PhabricatorConfigStackSource' => 'PhabricatorConfigSource',
     'PhabricatorContentSourceView' => 'AphrontView',

src/__phutil_library_map__.php

@@ -0,0 +1,51 @@
+<?php
+
+final class PhabricatorConfigLocalSource
+  extends PhabricatorConfigProxySource {
+
+  public function __construct() {
+    $config = $this->loadConfig();
+    $this->setSource(new PhabricatorConfigDictionarySource($config));
+  }
+
+  public function setKeys(array $keys) {
+    $result = parent::setKeys($keys);
+    $this->saveConfig();
+    return $result;
+  }
+
+  public function deleteKeys(array $keys) {
+    $result = parent::deleteKeys($keys);
+    $this->saveConfig();
+    return parent::deleteKeys($keys);
+  }
+
+  private function loadConfig() {
+    $path = $this->getConfigPath();
+    if (@file_exists($path)) {
+      $data = @file_get_contents($path);
+      if ($data) {
+        $data = json_decode($data, true);
+        if (is_array($data)) {
+          return $data;
+        }
+      }
+    }
+
+    return array();
+  }
+
+  private function saveConfig() {
+    $config = $this->getSource()->getAllKeys();
+    $json = new PhutilJSON();
+    $data = $json->encodeFormatted($config);
+    Filesystem::writeFile($this->getConfigPath(), $data);
+  }
+
+  private function getConfigPath() {
+    $root = dirname(phutil_get_library_root('phabricator'));
+    $path = $root.'/conf/local/local.json';
+    return $path;
+  }
+
+}

src/infrastructure/env/PhabricatorConfigLocalSource.php

@@ -29,15 +29,15 @@ public function getKeys(array $keys) {
   }
 
   public function canWrite() {
-    return $this->getSource->canWrite();
+    return $this->getSource()->canWrite();
   }
 
   public function setKeys(array $keys) {
-    return $this->getSource->setKeys();
+    return $this->getSource()->setKeys($keys);
   }
 
   public function deleteKeys(array $keys) {
-    return $this->getSource->deleteKeys();
+    return $this->getSource()->deleteKeys();
   }
 
 }

src/infrastructure/env/PhabricatorConfigProxySource.php

@@ -139,6 +139,14 @@ public static function getSelectedEnvironmentName() {
       $env = idx($_ENV, $env_var);
     }
 
+    if (!$env) {
+      $root = dirname(phutil_get_library_root('phabricator'));
+      $path = $root.'/conf/local/ENVIRONMENT';
+      if (Filesystem::pathExists($path)) {
+        $env = trim(Filesystem::readFile($path));
+      }
+    }
+
     return $env;
   }
 

src/infrastructure/env/PhabricatorEnv.php

@@ -0,0 +1,50 @@
+<?php
+
+final class PhabricatorConfigManagementSetWorkflow
+  extends PhabricatorConfigManagementWorkflow {
+
+  protected function didConstruct() {
+    $this
+      ->setName('set')
+      ->setExamples('**set** __key__ __value__')
+      ->setSynopsis('Set a local configuration value.')
+      ->setArguments(
+        array(
+          array(
+            'name'      => 'args',
+            'wildcard'  => true,
+          ),
+        ));
+  }
+
+  public function execute(PhutilArgumentParser $args) {
+    $console = PhutilConsole::getConsole();
+
+    $argv = $args->getArg('args');
+    if (count($argv) == 0) {
+      throw new PhutilArgumentUsageException(
+        "Specify a configuration key and a value to set it to.");
+    }
+
+    $key = $argv[0];
+
+    if (count($argv) == 1) {
+      throw new PhutilArgumentUsageException(
+        "Specify a value to set the key '{$key}' to.");
+    }
+
+    $value = $argv[1];
+
+    if (count($argv) > 2) {
+      throw new PhutilArgumentUsageException(
+        "Too many arguments: expected one key and one value.");
+    }
+
+    $config = new PhabricatorConfigLocalSource();
+    $config->setKeys(array($key => $value));
+
+    $console->writeOut(
+      pht("Set '%s' to '%s' in local configuration.", $key, $value)."\n");
+  }
+
+}

src/infrastructure/env/management/PhabricatorConfigManagementSetWorkflow.php

@@ -0,0 +1,10 @@
+<?php
+
+abstract class PhabricatorConfigManagementWorkflow
+  extends PhutilArgumentWorkflow {
+
+  final public function isExecutable() {
+    return true;
+  }
+
+}