chore(NA): upgrade to lodash@4 (elastic#69868)

* chore(NA): upgrade oss to lodash4

chore(NA): migrate cli, cli_plugin, cli_keystore,  dev, test_utils and apm src script to lodash4

chore(NA): missing file for cli plugin

chore(NA): add src core

chore(NA): es archiver and fixtures

chore(NA): try to fix functional test failure

chore(NA): migrate src/legacy entirely to lodash4 except src/legacy/core_plugins

chore(NA): move legacy core plugins to lodash4

chore(NA): upgrade optimize to lodash4

chore(NA): upgrade to lodash4 on advanced_settings, charts, console and dashboard

chore(NA): migrate to lodash4 on dev_tools, discover, embeddable, es_ui)shared, expressions, home plugins

chore(NA): upgrade data plugin to lodash4

chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4

chore(NA): missing data upgrades to lodash4

Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4"

This reverts commit 137055c.

Revert "Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4""

This reverts commit f7e7368.

Revert "chore(NA): missing data upgrades to lodash4"

This reverts commit 92b85bf.

Revert "chore(NA): upgrade data plugin to lodash4"

This reverts commit 88fdb07.

chore(NA): upgrade idx_pattern_mgt, input_control_vis, inspector, kbn_legacy, kbn_react, kbn_usage_collections, kbn_utils, management and maps_legacy to lodash4

chore(NA): map src plugin data to lodash3

chore(NA): missing lodash.clonedeep dep

chore(NA): change packages kbn-config-schema deps

chore(NA): update renovate config

chore(NA): upgrade vis_type plugins to lodash4

chore(NA): move vis_type_vislib to lodash3

chore(NA): update visualizations and visualize to lodash4

chore(NA): remove lodash 3 types from src and move test to lodash4

chore(NA): move home, usage_collection and management to lodash 3

Revert "chore(NA): move home, usage_collection and management to lodash 3"

This reverts commit f86e858.

chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3

chore(NA): update x-pack test to mock lodash4

Revert "chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3"

This reverts commit 2d10fe4.

* chore(NA): move x-pack and packages to lodash 4

* chore(NA): remove mention to lodash from main package.json

* chore(NA): remove helper alias for lodash4 and make it the default lodash

* chore(NA): fix last failing types in the repo

* chore(NA): fix public api

* chore(NA): fix types for agg_row.tsx

* chore(NA): fix increment of optimizer modules in the rollup plugin

* chore(NA): migrate `src/core/public/http/fetch.ts` (#5)

* omit undefined query props

* just remove merge usage

* fix types

* chore(NA): fixes for feedback from apm team

* chore(NA): recover old behaviour on apm LoadingIndeicatorContext.tsx

* chore(NA): fixes for feedback from watson

* Platform lodash4 tweaks (#6)

* chore(NA): fix types and behaviour on src/core/server/elasticsearch/errors.ts

* Canvas fixes for lodash upgrade

* [APM] Adds unit test for APM service maps transform (#7)

* Adds a snapshot unit test for getConnections and rearranges some code to make testing easier

* reverts `ArrayList` back to `String[]` in the painless script within `fetch_service_paths_from_trace_ids.ts`

* chore(NA): update yarn.lock

* chore(NA): remove any and use a real type for alerts task runner

Co-authored-by: Gidi Meir Morris <github@gidi.io>

* chore(NA): used named import for triggers_actions_ui file

* chore(NA): fix eslint

* chore(NA): fix types

* Delete most uptime lodash references.

* Simplify. Clean up types.

* [Uptime] Delete most uptime lodash references (#8)

* Delete most uptime lodash references.

* Simplify. Clean up types.

* chore(NA): add eslint rule to avoid using lodash3

* chore(NA): apply changes on feedback from es-ui team

* fix some types (#9)

* Clean up some expressions types.

* chore(NA): missing ts-expect-error statements

* Upgrade lodash 4 vislib (#11)

* replace lodash 3 with lodash 4 on vislib plugin

* Further changes

* further replacement of lodash3 to 4

* further work on upgrading to lodash 4

* final changes to update lodash

* chore(NA): upgrade data plugin to lodash4

chore(NA): upgrade data plugin public to lodash4

chore(NA): fix typecheck task

chore(NA): fix agg_config with hasIn

chore(NA): assign to assignIn and has to hasIn

chore(NA): upgrade data plugin server to lodash4

chore(NA): new signature for core api

fix(NA): match behaviour between lodash3 and lodash4 for set in search_source

* chore(NA): remove lodash3 completely from the repo

* chore(NA): fix x-pack/test/api_integration/apis/metrics_ui/snapshot.ts missing content

* chore(NA): fix lodash usage on apm

* chore(NA): fix typecheck for maps

* Patch lodash template (#12)

* Applying changes from elastic#64985

* Using isIterateeCall, because it seems less brittle

* Also patching `lodash/template` and `lodash/fp/template`

* Reorganizing some files...

* Revising comment

* Ends up `_` is a function also... I hate JavaScript

Co-authored-by: Pierre Gayvallet <pierre.gayvallet@gmail.com>
Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: Clint Andrew Hall <clint.hall@elastic.co>
Co-authored-by: Oliver Gupte <ogupte@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
Co-authored-by: Justin Kambic <justin.kambic@elastic.co>
Co-authored-by: Stratoula Kalafateli <stratoula1@gmail.com>
Co-authored-by: Luke Elmers <luke.elmers@elastic.co>
Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com>
Co-authored-by: kobelb <brandon.kobel@elastic.co>

docs/development/plugins/data/public/kibana-plugin-plugins-data-public.fieldformat.tojson.md

@@ -12,14 +12,14 @@ Serialize this format to a simple POJO, with only the params that are not defaul
 
 ```typescript
 toJSON(): {
-        id: unknown;
-        params: _.Dictionary<unknown> | undefined;
+        id: any;
+        params: any;
     };
 ```
 <b>Returns:</b>
 
 `{
-        id: unknown;
-        params: _.Dictionary<unknown> | undefined;
+        id: any;
+        params: any;
     }`
 

package.json

@@ -86,8 +86,9 @@
     "**/@types/angular": "^1.6.56",
     "**/@types/hoist-non-react-statics": "^3.3.1",
     "**/@types/chai": "^4.2.11",
+    "**/cypress/@types/lodash": "^4.14.155",
     "**/typescript": "3.9.5",
-    "**/graphql-toolkit/lodash": "^4.17.13",
+    "**/graphql-toolkit/lodash": "^4.17.15",
     "**/hoist-non-react-statics": "^3.3.2",
     "**/isomorphic-git/**/base64-js": "^1.2.1",
     "**/image-diff/gm/debug": "^2.6.9",
@@ -213,8 +214,7 @@
     "leaflet.heat": "0.2.0",
     "less": "npm:@elastic/less@2.7.3-kibana",
     "less-loader": "5.0.0",
-    "lodash": "npm:@elastic/lodash@3.10.1-kibana4",
-    "lodash.clonedeep": "^4.5.0",
+    "lodash": "^4.17.15",
     "lru-cache": "4.1.5",
     "markdown-it": "^10.0.0",
     "mini-css-extract-plugin": "0.8.0",
@@ -355,8 +355,7 @@
     "@types/json5": "^0.0.30",
     "@types/license-checker": "15.0.0",
     "@types/listr": "^0.14.0",
-    "@types/lodash": "^3.10.1",
-    "@types/lodash.clonedeep": "^4.5.4",
+    "@types/lodash": "^4.14.155",
     "@types/lru-cache": "^5.1.0",
     "@types/markdown-it": "^0.0.7",
     "@types/minimatch": "^2.0.29",

packages/kbn-config-schema/package.json

@@ -14,6 +14,7 @@
     "tsd": "^0.7.4"
   },
   "peerDependencies": {
+    "lodash": "^4.17.15",
     "joi": "^13.5.2",
     "moment": "^2.24.0",
     "type-detect": "^4.0.8"

packages/kbn-interpreter/package.json

@@ -11,8 +11,7 @@
   "dependencies": {
     "@babel/runtime": "^7.10.2",
     "@kbn/i18n": "1.0.0",
-    "lodash": "npm:@elastic/lodash@3.10.1-kibana4",
-    "lodash.clone": "^4.5.0",
+    "lodash": "^4.17.15",
     "uuid": "3.3.2"
   },
   "devDependencies": {

packages/kbn-interpreter/src/common/lib/registry.js

@@ -17,7 +17,7 @@
  * under the License.
  */
 
-import clone from 'lodash.clone';
+import { clone } from 'lodash';
 
 export class Registry {
   constructor(prop = 'name') {

packages/kbn-plugin-generator/index.js

@@ -23,7 +23,7 @@ const dedent = require('dedent');
 const sao = require('sao');
 const chalk = require('chalk');
 const getopts = require('getopts');
-const snakeCase = require('lodash.snakecase');
+const { snakeCase } = require('lodash');
 
 exports.run = function run(argv) {
   const options = getopts(argv, {
@@ -41,7 +41,7 @@ exports.run = function run(argv) {
   if (options.help) {
     console.log(
       dedent(chalk`
-        # {dim Usage:} 
+        # {dim Usage:}
         node scripts/generate-plugin {bold [name]}
         Generate a fresh Kibana plugin in the plugins/ directory
       `) + '\n'

packages/kbn-plugin-generator/package.json

@@ -8,10 +8,7 @@
     "dedent": "^0.7.0",
     "execa": "^4.0.2",
     "getopts": "^2.2.4",
-    "lodash.camelcase": "^4.3.0",
-    "lodash.kebabcase": "^4.1.1",
-    "lodash.snakecase": "^4.1.1",
-    "lodash.startcase": "^4.4.0",
+    "lodash": "^4.17.15",
     "sao": "^0.22.12"
   }
 }

packages/kbn-plugin-generator/sao_template/sao.js

@@ -20,9 +20,7 @@
 const { relative, resolve } = require('path');
 const fs = require('fs');
 
-const startCase = require('lodash.startcase');
-const camelCase = require('lodash.camelcase');
-const snakeCase = require('lodash.snakecase');
+const { camelCase, startCase, snakeCase } = require('lodash');
 const chalk = require('chalk');
 const execa = require('execa');
 

packages/kbn-pm/package.json

@@ -22,7 +22,7 @@
     "@types/glob": "^5.0.35",
     "@types/globby": "^6.1.0",
     "@types/has-ansi": "^3.0.0",
-    "@types/lodash.clonedeepwith": "^4.5.3",
+    "@types/lodash": "^4.14.155",
     "@types/log-symbols": "^2.0.0",
     "@types/ncp": "^2.0.1",
     "@types/node": ">=10.17.17 <10.20.0",
@@ -46,7 +46,7 @@
     "globby": "^8.0.1",
     "has-ansi": "^3.0.0",
     "is-path-inside": "^3.0.2",
-    "lodash.clonedeepwith": "^4.5.0",
+    "lodash": "^4.17.15",
     "log-symbols": "^2.2.0",
     "multimatch": "^4.0.0",
     "ncp": "^2.0.0",

packages/kbn-pm/src/test_helpers/absolute_path_snapshot_serializer.ts

@@ -17,7 +17,7 @@
  * under the License.
  */
 
-import cloneDeepWith from 'lodash.clonedeepwith';
+import { cloneDeepWith } from 'lodash';
 import { resolve, sep as pathSep } from 'path';
 
 const repoRoot = resolve(__dirname, '../../../../');

packages/kbn-storybook/lib/webpack.dll.config.js

@@ -54,7 +54,6 @@ module.exports = {
     'highlight.js',
     'html-entities',
     'jquery',
-    'lodash.clone',
     'lodash',
     'markdown-it',
     'mocha',

packages/kbn-test/package.json

@@ -14,6 +14,7 @@
     "@kbn/babel-preset": "1.0.0",
     "@kbn/dev-utils": "1.0.0",
     "@types/joi": "^13.4.2",
+    "@types/lodash": "^4.14.155",
     "@types/parse-link-header": "^1.0.0",
     "@types/puppeteer": "^3.0.0",
     "@types/strip-ansi": "^5.2.1",
@@ -28,6 +29,7 @@
     "getopts": "^2.2.4",
     "glob": "^7.1.2",
     "joi": "^13.5.2",
+    "lodash": "^4.17.15",
     "parse-link-header": "^1.0.1",
     "puppeteer": "^3.3.0",
     "rxjs": "^6.5.5",

packages/kbn-test/src/functional_test_runner/lib/config/config.ts

@@ -18,10 +18,7 @@
  */
 
 import { Schema } from 'joi';
-import { cloneDeep, get, has } from 'lodash';
-
-// @ts-ignore internal lodash module is not typed
-import toPath from 'lodash/internal/toPath';
+import { cloneDeepWith, get, has, toPath } from 'lodash';
 
 import { schema } from './schema';
 
@@ -114,15 +111,15 @@ export class Config {
       throw new Error(`Unknown config key "${key}"`);
     }
 
-    return cloneDeep(get(this[$values], key, defaultValue), (v) => {
+    return cloneDeepWith(get(this[$values], key, defaultValue), (v) => {
       if (typeof v === 'function') {
         return v;
       }
     });
   }
 
   public getAll() {
-    return cloneDeep(this[$values], (v) => {
+    return cloneDeepWith(this[$values], (v) => {
       if (typeof v === 'function') {
         return v;
       }

packages/kbn-test/src/legacy_es/legacy_es_test_cluster.js

@@ -19,8 +19,7 @@
 
 import { resolve } from 'path';
 import { format } from 'url';
-import { get } from 'lodash';
-import toPath from 'lodash/internal/toPath';
+import { get, toPath } from 'lodash';
 import { Cluster } from '@kbn/es';
 import { CI_PARALLEL_PROCESS_PREFIX } from '../ci_parallel_process_prefix';
 import { esTestConfig } from './es_test_config';

packages/kbn-test/src/page_load_metrics/navigation.ts

@@ -19,7 +19,6 @@
 
 import Fs from 'fs';
 import Url from 'url';
-import _ from 'lodash';
 import puppeteer from 'puppeteer';
 import { resolve } from 'path';
 import { ToolingLog } from '@kbn/dev-utils';

packages/kbn-ui-framework/Gruntfile.js

@@ -21,7 +21,7 @@ const sass = require('node-sass');
 const postcss = require('postcss');
 const postcssConfig = require('../../src/optimize/postcss.config');
 const chokidar = require('chokidar');
-const debounce = require('lodash/function/debounce');
+const { debounce } = require('lodash');
 
 const platform = require('os').platform();
 const isPlatformWindows = /^win/.test(platform);

packages/kbn-ui-framework/package.json

@@ -17,7 +17,7 @@
   "dependencies": {
     "classnames": "2.2.6",
     "focus-trap-react": "^3.1.1",
-    "lodash": "npm:@elastic/lodash@3.10.1-kibana4",
+    "lodash": "^4.17.15",
     "prop-types": "15.6.0",
     "react": "^16.12.0",
     "react-ace": "^5.9.0",

renovate.json5

@@ -557,22 +557,6 @@
           '@types/lodash',
         ],
       },
-      {
-        groupSlug: 'lodash.clonedeep',
-        groupName: 'lodash.clonedeep related packages',
-        packageNames: [
-          'lodash.clonedeep',
-          '@types/lodash.clonedeep',
-        ],
-      },
-      {
-        groupSlug: 'lodash.clonedeepwith',
-        groupName: 'lodash.clonedeepwith related packages',
-        packageNames: [
-          'lodash.clonedeepwith',
-          '@types/lodash.clonedeepwith',
-        ],
-      },
       {
         groupSlug: 'log-symbols',
         groupName: 'log-symbols related packages',

src/apm.js

@@ -20,7 +20,7 @@
 const { join } = require('path');
 const { readFileSync } = require('fs');
 const { execSync } = require('child_process');
-const merge = require('lodash.merge');
+const { merge } = require('lodash');
 const { name, version, build } = require('../package.json');
 
 const ROOT_DIR = join(__dirname, '..');

src/cli/cluster/cluster_manager.test.ts

@@ -93,7 +93,7 @@ describe('CLI cluster manager', () => {
     }
 
     const football = {};
-    const messenger = sample(manager.workers);
+    const messenger = sample(manager.workers) as any;
 
     messenger.emit('broadcast', football);
     for (const worker of manager.workers) {

src/cli/cluster/worker.ts

@@ -177,7 +177,7 @@ export class Worker extends EventEmitter {
   }
 
   flushChangeBuffer() {
-    const files = _.unique(this.changes.splice(0));
+    const files = _.uniq(this.changes.splice(0));
     const prefix = files.length > 1 ? '\n - ' : '';
     return files.reduce(function (list, file) {
       return `${list || ''}${prefix}"${file}"`;

src/cli/help.js

@@ -72,7 +72,7 @@ function commandsSummary(program) {
   }, 0);
 
   return cmds.reduce(function (help, cmd) {
-    return `${help || ''}${_.padRight(cmd[0], cmdLColWidth)} ${cmd[1] || ''}\n`;
+    return `${help || ''}${_.padEnd(cmd[0], cmdLColWidth)} ${cmd[1] || ''}\n`;
   }, '');
 }
 

src/core/public/http/fetch.ts

@@ -17,7 +17,7 @@
  * under the License.
  */
 
-import { merge } from 'lodash';
+import { omitBy } from 'lodash';
 import { format } from 'url';
 import { BehaviorSubject } from 'rxjs';
 
@@ -42,6 +42,10 @@ interface Params {
 const JSON_CONTENT = /^(application\/(json|x-javascript)|text\/(x-)?javascript|x-json)(;.*)?$/;
 const NDJSON_CONTENT = /^(application\/ndjson)(;.*)?$/;
 
+const removedUndefined = (obj: Record<string, any> | undefined) => {
+  return omitBy(obj, (v) => v === undefined);
+};
+
 export class Fetch {
   private readonly interceptors = new Set<HttpInterceptor>();
   private readonly requestCount$ = new BehaviorSubject(0);
@@ -119,32 +123,31 @@ export class Fetch {
       asResponse,
       asSystemRequest,
       ...fetchOptions
-    } = merge(
-      {
-        method: 'GET',
-        credentials: 'same-origin',
-        prependBasePath: true,
-      },
-      options,
-      {
-        headers: {
-          'Content-Type': 'application/json',
-          ...options.headers,
-          'kbn-version': this.params.kibanaVersion,
-        },
-      }
-    );
+    } = {
+      method: 'GET',
+      credentials: 'same-origin',
+      prependBasePath: true,
+      ...options,
+      // options can pass an `undefined` Content-Type to erase the default value.
+      // however we can't pass it to `fetch` as it will send an `Content-Type: Undefined` header
+      headers: removedUndefined({
+        'Content-Type': 'application/json',
+        ...options.headers,
+        'kbn-version': this.params.kibanaVersion,
+      }),
+    };
+
     const url = format({
       pathname: shouldPrependBasePath ? this.params.basePath.prepend(options.path) : options.path,
-      query,
+      query: removedUndefined(query),
     });
 
     // Make sure the system request header is only present if `asSystemRequest` is true.
     if (asSystemRequest) {
       fetchOptions.headers['kbn-system-request'] = 'true';
     }
 
-    return new Request(url, fetchOptions);
+    return new Request(url, fetchOptions as RequestInit);
   }
 
   private async fetchResponse(fetchOptions: HttpFetchOptionsWithPath): Promise<HttpResponse<any>> {

src/core/public/plugins/plugins_service.test.ts

@@ -91,22 +91,23 @@ describe('PluginsService', () => {
       context: contextServiceMock.createSetupContract(),
       fatalErrors: fatalErrorsServiceMock.createSetupContract(),
       http: httpServiceMock.createSetupContract(),
-      injectedMetadata: pick(injectedMetadataServiceMock.createStartContract(), 'getInjectedVar'),
+      injectedMetadata: injectedMetadataServiceMock.createStartContract(),
       notifications: notificationServiceMock.createSetupContract(),
       uiSettings: uiSettingsServiceMock.createSetupContract(),
     };
     mockSetupContext = {
       ...mockSetupDeps,
       application: expect.any(Object),
       getStartServices: expect.any(Function),
+      injectedMetadata: pick(mockSetupDeps.injectedMetadata, 'getInjectedVar'),
     };
     mockStartDeps = {
       application: applicationServiceMock.createInternalStartContract(),
       docLinks: docLinksServiceMock.createStartContract(),
       http: httpServiceMock.createStartContract(),
       chrome: chromeServiceMock.createStartContract(),
       i18n: i18nServiceMock.createStartContract(),
-      injectedMetadata: pick(injectedMetadataServiceMock.createStartContract(), 'getInjectedVar'),
+      injectedMetadata: injectedMetadataServiceMock.createStartContract(),
       notifications: notificationServiceMock.createStartContract(),
       overlays: overlayServiceMock.createStartContract(),
       uiSettings: uiSettingsServiceMock.createStartContract(),
@@ -117,6 +118,7 @@ describe('PluginsService', () => {
       ...mockStartDeps,
       application: expect.any(Object),
       chrome: omit(mockStartDeps.chrome, 'getComponent'),
+      injectedMetadata: pick(mockStartDeps.injectedMetadata, 'getInjectedVar'),
     };
 
     // Reset these for each test.