Remove tokenizer.ondemand, and always load on demand

Summary:
Ref T4420. Tokenizers currently operate in "preload" or "ondemand" modes. In the former mode, which is default, they'll try to load the entire result list when a page loads.

The theory here was that this would slightly improve the experience for small installs, and once they got big enough they could switch to "ondemand". In practice, several issues have arisen:

  - We generally don't have a good mechanism for telling installs that they should tweak perf config -- `metamta.send-immediately` is the canonical example here. Some large installs are probably affected negatively by not knowing to change this setting, and having settings like this is generally annoying.
  - We have way way too much config now.
  - With the advent of ApplicationSearch, pages like Maniphest make many redundant loads to prefill sources like projects. Most of the time, this data is not used. It's far simpler to switch everything to ondemand than try to deal with this, and dealing with this would mean creating two very complex divergent pathways in the codebase for a mostly theoretical performance benefit which only impacts tiny installs.
  - We've been using `tokenizer.ondemand` forever on `secure.phabricator.com` since we have many thousands of user accounts, and it doesn't seem sluggish and works properly.

Removing this config is an easy fix which makes the codebase simpler.

I've retained the ability to use preloaded sources, since they may make sense in some cases (in at least one case -- task priorities -- adding a static source pathway might make sense), and they're part of Javelin itself. However, the code will no longer ever go down that pathway.

Test Plan: Used `secure.phabricator.com` for years with this setting enabled.

Reviewers: btrahan, chad

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T4420

Differential Revision: https://secure.phabricator.com/D8232

Author: epriestley

conf/default.conf.php

@@ -638,21 +638,6 @@
     'https' => true,
   ),
 
-  // Tokenizers are UI controls which let the user select other users, email
-  // addresses, project names, etc., by typing the first few letters and having
-  // the control autocomplete from a list. They can load their data in two ways:
-  // either in a big chunk up front, or as the user types. By default, the data
-  // is loaded in a big chunk. This is simpler and performs better for small
-  // datasets. However, if you have a very large number of users or projects,
-  // (in the ballpark of more than a thousand), loading all that data may become
-  // slow enough that it's worthwhile to query on demand instead. This makes
-  // the typeahead slightly less responsive but overall performance will be much
-  // better if you have a ton of stuff. You can figure out which setting is
-  // best for your install by changing this setting and then playing with a
-  // user tokenizer (like the user selectors in Maniphest or Differential) and
-  // seeing which setting loads faster and feels better.
-  'tokenizer.ondemand'          => false,
-
   // By default, Phabricator includes some silly nonsense in the UI, such as
   // a submit button called "Clowncopterize" in Differential and a call to
   // "Leap Into Action". If you'd prefer more traditional UI strings like

resources/celerity/map.php

@@ -8,7 +8,7 @@
   'names' =>
   array(
     'core.pkg.css' => '044c2f0c',
-    'core.pkg.js' => 'ba6e232d',
+    'core.pkg.js' => '0dc59a05',
     'darkconsole.pkg.js' => 'ca8671ce',
     'differential.pkg.css' => '6aef439e',
     'differential.pkg.js' => '322ea941',
@@ -431,7 +431,7 @@
     'rsrc/js/core/KeyboardShortcutManager.js' => 'ad7a69ca',
     'rsrc/js/core/MultirowRowManager.js' => 'e7076916',
     'rsrc/js/core/Notification.js' => '95944043',
-    'rsrc/js/core/Prefab.js' => '1e644329',
+    'rsrc/js/core/Prefab.js' => '979f864d',
     'rsrc/js/core/ShapedRequest.js' => 'dfa181a4',
     'rsrc/js/core/TextAreaUtils.js' => 'b3ec3cfc',
     'rsrc/js/core/ToolTip.js' => '0a81ea29',
@@ -701,7 +701,7 @@
     'phabricator-object-list-view-css' => '1a1ea560',
     'phabricator-object-selector-css' => '029a133d',
     'phabricator-phtize' => 'd254d646',
-    'phabricator-prefab' => '1e644329',
+    'phabricator-prefab' => '979f864d',
     'phabricator-profile-css' => '3a7e04ca',
     'phabricator-project-tag-css' => '095c9404',
     'phabricator-remarkup-css' => 'ca7f2265',
@@ -928,19 +928,6 @@
       5 => 'phabricator-drag-and-drop-file-upload',
       6 => 'phabricator-draggable-list',
     ),
-    '1e644329' =>
-    array(
-      0 => 'javelin-install',
-      1 => 'javelin-util',
-      2 => 'javelin-dom',
-      3 => 'javelin-typeahead',
-      4 => 'javelin-tokenizer',
-      5 => 'javelin-typeahead-preloaded-source',
-      6 => 'javelin-typeahead-ondemand-source',
-      7 => 'javelin-dom',
-      8 => 'javelin-stratcom',
-      9 => 'javelin-util',
-    ),
     '1f595fb0' =>
     array(
       0 => 'javelin-install',
@@ -1411,6 +1398,19 @@
       2 => 'javelin-view-visitor',
       3 => 'javelin-util',
     ),
+    '979f864d' =>
+    array(
+      0 => 'javelin-install',
+      1 => 'javelin-util',
+      2 => 'javelin-dom',
+      3 => 'javelin-typeahead',
+      4 => 'javelin-tokenizer',
+      5 => 'javelin-typeahead-preloaded-source',
+      6 => 'javelin-typeahead-ondemand-source',
+      7 => 'javelin-dom',
+      8 => 'javelin-stratcom',
+      9 => 'javelin-util',
+    ),
     '9b9197be' =>
     array(
       0 => 'javelin-behavior',

src/applications/config/check/PhabricatorSetupCheckExtraConfig.php

@@ -178,6 +178,8 @@ public static function getAncientConfig() {
         'Mail is now always delivered by the daemons.'),
       'auth.sessions.conduit' => $session_reason,
       'auth.sessions.web' => $session_reason,
+      'tokenizer.ondemand' => pht(
+        'Phabricator now manages typeahead strategies automatically.'),
     );
 
     return $ancient_config;

src/applications/config/option/PhabricatorCoreConfigOptions.php

@@ -137,32 +137,6 @@ public function getOptions() {
             "  %s", $path))
         ->addExample('/usr/local/bin', pht('Add One Path'))
         ->addExample("/usr/bin\n/usr/local/bin", pht('Add Multiple Paths')),
-       $this->newOption('tokenizer.ondemand', 'bool', false)
-        ->setBoolOptions(
-          array(
-            pht("Query on demand"),
-            pht("Query on page load"),
-          ))
-        ->setSummary(
-          pht("Query for tokenizer fields on demand."))
-        ->setDescription(
-          pht(
-            "Tokenizers are UI controls which let the user select other ".
-            "users, email addresses, project names, etc., by typing the ".
-            "first few letters and having the control autocomplete from a ".
-            "list. They can load their data in two ways: either in a big ".
-            "chunk up front, or as the user types. By default, the data is ".
-            "loaded in a big chunk. This is simpler and performs better for ".
-            "small datasets. However, if you have a very large number of ".
-            "users or projects, (in the ballpark of more than a thousand), ".
-            "loading all that data may become slow enough that it's ".
-            "worthwhile to query on demand instead. This makes the typeahead ".
-            "slightly less responsive but overall performance will be much ".
-            "better if you have a ton of stuff. You can figure out which ".
-            "setting is best for your install by changing this setting and ".
-            "then playing with a user tokenizer (like the user selectors in ".
-            "Maniphest or Differential) and seeing which setting loads ".
-            "faster and feels better.")),
       $this->newOption('config.lock', 'set', array())
         ->setLocked(true)
         ->setDescription(pht('Additional configuration options to lock.')),

src/applications/differential/view/DifferentialAddCommentView.php

@@ -121,7 +121,6 @@ public function render() {
             'src' => '/typeahead/common/usersorprojects/',
             'value' => $this->reviewers,
             'row' => 'add-reviewers',
-            'ondemand' => PhabricatorEnv::getEnvConfig('tokenizer.ondemand'),
             'labels' => $add_reviewers_labels,
             'placeholder' => pht('Type a user or project name...'),
           ),
@@ -130,7 +129,6 @@ public function render() {
             'src' => '/typeahead/common/mailable/',
             'value' => $this->ccs,
             'row' => 'add-ccs',
-            'ondemand' => PhabricatorEnv::getEnvConfig('tokenizer.ondemand'),
             'placeholder' => pht('Type a user or mailing list...'),
           ),
         ),

src/applications/diffusion/controller/DiffusionCommitController.php

@@ -780,14 +780,12 @@ private function renderAddCommentPanel(
             'actions' => array('add_auditors' => 1),
             'src' => '/typeahead/common/users/',
             'row' => 'add-auditors',
-            'ondemand' => PhabricatorEnv::getEnvConfig('tokenizer.ondemand'),
             'placeholder' => pht('Type a user name...'),
           ),
           'add-ccs-tokenizer' => array(
             'actions' => array('add_ccs' => 1),
             'src' => '/typeahead/common/mailable/',
             'row' => 'add-ccs',
-            'ondemand' => PhabricatorEnv::getEnvConfig('tokenizer.ondemand'),
             'placeholder' => pht('Type a user or mailing list...'),
           ),
         ),

src/applications/maniphest/controller/ManiphestTaskDetailController.php

@@ -308,21 +308,18 @@ public function processRequest() {
       ManiphestTransaction::TYPE_PROJECTS => array(
         'id'          => 'projects-tokenizer',
         'src'         => '/typeahead/common/projects/',
-        'ondemand'    => PhabricatorEnv::getEnvConfig('tokenizer.ondemand'),
         'placeholder' => pht('Type a project name...'),
       ),
       ManiphestTransaction::TYPE_OWNER => array(
         'id'          => 'assign-tokenizer',
         'src'         => '/typeahead/common/users/',
         'value'       => $default_claim,
         'limit'       => 1,
-        'ondemand'    => PhabricatorEnv::getEnvConfig('tokenizer.ondemand'),
         'placeholder' => pht('Type a user name...'),
       ),
       ManiphestTransaction::TYPE_CCS => array(
         'id'          => 'cc-tokenizer',
         'src'         => '/typeahead/common/mailable/',
-        'ondemand'    => PhabricatorEnv::getEnvConfig('tokenizer.ondemand'),
         'placeholder' => pht('Type a user or mailing list...'),
       ),
     );

src/view/form/control/AphrontFormTokenizerControl.php

@@ -64,7 +64,6 @@ protected function renderInput() {
         'value'       => mpull($values, 'getFullName', 'getPHID'),
         'icons'       => mpull($values, 'getTypeIcon', 'getPHID'),
         'limit'       => $this->limit,
-        'ondemand'    => PhabricatorEnv::getEnvConfig('tokenizer.ondemand'),
         'username'    => $username,
         'placeholder' => $this->placeholder,
       ));

webroot/rsrc/js/core/Prefab.js

@@ -58,7 +58,15 @@ JX.install('Prefab', {
       }
 
       var datasource;
-      if (config.ondemand) {
+
+      // Default to an ondemand source if no alternate configuration is
+      // provided.
+      var ondemand = true;
+      if ('ondemand' in config) {
+        ondemand = config.ondemand;
+      }
+
+      if (ondemand) {
         datasource = new JX.TypeaheadOnDemandSource(config.src);
       } else {
         datasource = new JX.TypeaheadPreloadedSource(config.src);