Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Reporting]: Move router + license checks to new platform #66331

Merged
merged 50 commits into from
May 29, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
004dd70
WIP: Move routes to new API, license and other checks inbound
joelgriffith May 8, 2020
938e14a
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
joelgriffith May 11, 2020
500b933
Move license checks over to np licensing observable
joelgriffith May 12, 2020
b9cc20c
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
joelgriffith May 12, 2020
4feb02f
Fix license checks + remove older modules
joelgriffith May 14, 2020
02394d5
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
joelgriffith May 14, 2020
2b18fe2
Fixing check_license tests, move to TS/Jest
joelgriffith May 19, 2020
dfd62f3
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
joelgriffith May 19, 2020
ac361a8
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
joelgriffith May 19, 2020
7fab2c6
Fix licensing setup for mocks
joelgriffith May 20, 2020
48c88f8
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
joelgriffith May 20, 2020
15ece89
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
joelgriffith May 20, 2020
d58fc14
Move job.test.ts over to np
joelgriffith May 21, 2020
6e1667e
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
joelgriffith May 21, 2020
007dd57
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
joelgriffith May 26, 2020
ed1ac8c
WIP: move user checks to higher-order func
joelgriffith May 26, 2020
c08ea7b
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
joelgriffith May 26, 2020
756551f
Move more handler logic over to Response factory vs Boom
joelgriffith May 26, 2020
7e6efa2
Major refactor to consolidate types, remove facades, and udpate helpers
joelgriffith May 27, 2020
82a1767
Fix validation for dates in immediate exports
joelgriffith May 27, 2020
bb0e7ee
Linter fix on check license test
joelgriffith May 27, 2020
ec94204
Fix job generation tests
joelgriffith May 27, 2020
39a1313
Move deps => setupDeps
joelgriffith May 27, 2020
c7cba25
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
joelgriffith May 27, 2020
b4e54e1
fix api test
tsullivan May 26, 2020
ee7db9a
fix jobs test
tsullivan May 27, 2020
e7262f4
authorized_user_pre_routing and tests
tsullivan May 27, 2020
d69be5f
Fixing duplicate identifiers
joelgriffith May 28, 2020
107eca0
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
joelgriffith May 28, 2020
ca58c89
Merge pull request #1 from tsullivan/chore/np-route-migration
joelgriffith May 28, 2020
a8f4bcd
Fix licensing implementation changes
joelgriffith May 28, 2020
54863d0
Merge branch 'chore/np-route-migration' of https://github.com/joelgri…
joelgriffith May 28, 2020
86920d3
WIP: Moving license over to async/observables
joelgriffith May 28, 2020
7532d55
Fix disabled-security case
joelgriffith May 28, 2020
dc57a6f
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
joelgriffith May 28, 2020
f764bf8
finish auth_user_pre_routing cleanup - no more license check
tsullivan May 28, 2020
01fc733
Merge pull request #2 from tsullivan/chore/np-route-migration
joelgriffith May 28, 2020
874222f
WIP: Fixing final api tests
joelgriffith May 28, 2020
dce4083
Trying to get schema differences in alignment
joelgriffith May 28, 2020
76904b7
Merge branch 'chore/np-route-migration' of https://github.com/joelgri…
joelgriffith May 28, 2020
2a5f9a0
Reverting back to previous generation handler
joelgriffith May 28, 2020
f7d1134
Fix final API tests
joelgriffith May 28, 2020
0aca67c
Final API test fixes, few more hardening tests and better error messages
joelgriffith May 29, 2020
7554127
Merge remote-tracking branch 'upstream/master' into chore/np-route-mi…
joelgriffith May 29, 2020
27040a9
Simplify lower-level module implementation (core only interface) + te…
joelgriffith May 29, 2020
8ad57ff
Push some core logic into plugin
joelgriffith May 29, 2020
4f12c8a
Move some core logic up to plugin
joelgriffith May 29, 2020
3ab8095
Marking private setupDeps + downstream fixes
joelgriffith May 29, 2020
1cf1546
revert logger as a param
tsullivan May 29, 2020
fcf9f58
Merge pull request #3 from tsullivan/chore/np-route-migration
joelgriffith May 29, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions x-pack/legacy/plugins/reporting/common/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ export const WHITELISTED_JOB_CONTENT_TYPES = [
'application/pdf',
CONTENT_TYPE_CSV,
'image/png',
'text/plain',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is needed now so we can pass long certain job-status messages (they were incorrectly being sent as application/json, and I think hapi was fixing that under-the-hood)

];

// See:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,30 +4,27 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { KibanaRequest, RequestHandlerContext } from 'src/core/server';
import { ReportingCore } from '../../../server';
import { cryptoFactory } from '../../../server/lib';
import {
ConditionalHeaders,
CreateJobFactory,
ESQueueCreateJobFn,
RequestFacade,
} from '../../../server/types';
import { CreateJobFactory, ESQueueCreateJobFn } from '../../../server/types';
import { JobParamsDiscoverCsv } from '../types';

export const createJobFactory: CreateJobFactory<ESQueueCreateJobFn<
JobParamsDiscoverCsv
>> = function createJobFactoryFn(reporting: ReportingCore) {
const config = reporting.getConfig();
const crypto = cryptoFactory(config.get('encryptionKey'));
const setupDeps = reporting.getPluginSetupDeps();

return async function createJob(
jobParams: JobParamsDiscoverCsv,
headers: ConditionalHeaders['headers'],
request: RequestFacade
context: RequestHandlerContext,
request: KibanaRequest
) {
const serializedEncryptedHeaders = await crypto.encrypt(headers);
const serializedEncryptedHeaders = await crypto.encrypt(request.headers);

const savedObjectsClient = request.getSavedObjectsClient();
const savedObjectsClient = context.core.savedObjects.client;
const indexPatternSavedObject = await savedObjectsClient.get(
'index-pattern',
jobParams.indexPatternId!
Expand All @@ -36,7 +33,7 @@ export const createJobFactory: CreateJobFactory<ESQueueCreateJobFn<
return {
headers: serializedEncryptedHeaders,
indexPatternSavedObject,
basePath: request.getBasePath(),
basePath: setupDeps.basePath(request),
...jobParams,
};
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@

import { notFound, notImplemented } from 'boom';
import { get } from 'lodash';
import { KibanaRequest, RequestHandlerContext } from 'src/core/server';
import { CSV_FROM_SAVEDOBJECT_JOB_TYPE } from '../../../../common/constants';
import { ReportingCore } from '../../../../server';
import { cryptoFactory, LevelLogger } from '../../../../server/lib';
import { CreateJobFactory, RequestFacade, TimeRangeParams } from '../../../../server/types';
import { CreateJobFactory, TimeRangeParams } from '../../../../server/types';
import {
JobDocPayloadPanelCsv,
JobParamsPanelCsv,
Expand All @@ -23,8 +24,9 @@ import { createJobSearch } from './create_job_search';

export type ImmediateCreateJobFn<JobParamsType> = (
jobParams: JobParamsType,
headers: Record<string, string>,
req: RequestFacade
headers: KibanaRequest['headers'],
context: RequestHandlerContext,
req: KibanaRequest
) => Promise<{
type: string | null;
title: string;
Expand All @@ -46,21 +48,21 @@ export const createJobFactory: CreateJobFactory<ImmediateCreateJobFn<

return async function createJob(
jobParams: JobParamsPanelCsv,
headers: any,
req: RequestFacade
headers: KibanaRequest['headers'],
context: RequestHandlerContext,
req: KibanaRequest
): Promise<JobDocPayloadPanelCsv> {
const { savedObjectType, savedObjectId } = jobParams;
const serializedEncryptedHeaders = await crypto.encrypt(headers);
const client = req.getSavedObjectsClient();

const { panel, title, visType }: VisData = await Promise.resolve()
.then(() => client.get(savedObjectType, savedObjectId))
.then(() => context.core.savedObjects.client.get(savedObjectType, savedObjectId))
.then(async (savedObject: SavedObject) => {
const { attributes, references } = savedObject;
const {
kibanaSavedObjectMeta: kibanaSavedObjectMetaJSON,
} = attributes as SavedSearchObjectAttributesJSON;
const { timerange } = req.payload as { timerange: TimeRangeParams };
const { timerange } = req.body as { timerange: TimeRangeParams };

if (!kibanaSavedObjectMetaJSON) {
throw new Error('Could not parse saved object data!');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,11 @@
*/

import { i18n } from '@kbn/i18n';
import { KibanaRequest, RequestHandlerContext } from 'src/core/server';
import { CONTENT_TYPE_CSV, CSV_FROM_SAVEDOBJECT_JOB_TYPE } from '../../../common/constants';
import { ReportingCore } from '../../../server';
import { cryptoFactory, LevelLogger } from '../../../server/lib';
import {
ExecuteJobFactory,
JobDocOutput,
JobDocPayload,
RequestFacade,
} from '../../../server/types';
import { ExecuteJobFactory, JobDocOutput, JobDocPayload } from '../../../server/types';
import { CsvResultFromSearch } from '../../csv/types';
import { FakeRequest, JobDocPayloadPanelCsv, JobParamsPanelCsv, SearchPanel } from '../types';
import { createGenerateCsv } from './lib';
Expand All @@ -25,7 +21,8 @@ import { createGenerateCsv } from './lib';
export type ImmediateExecuteFn<JobParamsType> = (
jobId: null,
job: JobDocPayload<JobParamsType>,
request: RequestFacade
context: RequestHandlerContext,
req: KibanaRequest
) => Promise<JobDocOutput>;

export const executeJobFactory: ExecuteJobFactory<ImmediateExecuteFn<
Expand All @@ -39,7 +36,8 @@ export const executeJobFactory: ExecuteJobFactory<ImmediateExecuteFn<
return async function executeJob(
jobId: string | null,
job: JobDocPayloadPanelCsv,
realRequest?: RequestFacade
context,
req
): Promise<JobDocOutput> {
// There will not be a jobID for "immediate" generation.
// jobID is only for "queued" jobs
Expand All @@ -58,10 +56,11 @@ export const executeJobFactory: ExecuteJobFactory<ImmediateExecuteFn<

jobLogger.debug(`Execute job generating [${visType}] csv`);

let requestObject: RequestFacade | FakeRequest;
if (isImmediate && realRequest) {
let requestObject: KibanaRequest | FakeRequest;

if (isImmediate && req) {
jobLogger.info(`Executing job from Immediate API using request context`);
requestObject = realRequest;
requestObject = req;
} else {
jobLogger.info(`Executing job async using encrypted headers`);
let decryptedHeaders: Record<string, unknown>;
Expand Down Expand Up @@ -103,6 +102,7 @@ export const executeJobFactory: ExecuteJobFactory<ImmediateExecuteFn<
let size = 0;
try {
const generateResults: CsvResultFromSearch = await generateCsv(
context,
requestObject,
visType as string,
panel,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,16 @@
*/

import { badRequest } from 'boom';
import { ReportingCore } from '../../../../server';
import { KibanaRequest, RequestHandlerContext } from 'src/core/server';
import { LevelLogger } from '../../../../server/lib';
import { RequestFacade } from '../../../../server/types';
import { ReportingCore } from '../../../../server';
import { FakeRequest, JobParamsPanelCsv, SearchPanel, VisPanel } from '../../types';
import { generateCsvSearch } from './generate_csv_search';

export function createGenerateCsv(reporting: ReportingCore, logger: LevelLogger) {
return async function generateCsv(
request: RequestFacade | FakeRequest,
context: RequestHandlerContext,
request: KibanaRequest | FakeRequest,
visType: string,
panel: VisPanel | SearchPanel,
jobParams: JobParamsPanelCsv
Expand All @@ -26,11 +27,12 @@ export function createGenerateCsv(reporting: ReportingCore, logger: LevelLogger)
switch (visType) {
case 'search':
return await generateCsvSearch(
request as RequestFacade,
reporting,
logger,
context,
request as KibanaRequest,
panel as SearchPanel,
jobParams
jobParams,
logger
);
default:
throw badRequest(`Unsupported or unrecognized saved object type: ${visType}`);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { IUiSettingsClient, KibanaRequest } from '../../../../../../../../src/core/server';
import {
IUiSettingsClient,
KibanaRequest,
RequestHandlerContext,
} from '../../../../../../../../src/core/server';
import {
esQuery,
EsQueryConfig,
Expand All @@ -13,23 +17,16 @@ import {
Query,
} from '../../../../../../../../src/plugins/data/server';
import { CancellationToken } from '../../../../../../../plugins/reporting/common';
import { ReportingCore } from '../../../../server';
import { LevelLogger } from '../../../../server/lib';
import { RequestFacade } from '../../../../server/types';
import { ReportingCore } from '../../../../server';
import { createGenerateCsv } from '../../../csv/server/lib/generate_csv';
import {
CsvResultFromSearch,
GenerateCsvParams,
JobParamsDiscoverCsv,
SearchRequest,
} from '../../../csv/types';
import {
IndexPatternField,
QueryFilter,
SavedSearchObjectAttributes,
SearchPanel,
SearchSource,
} from '../../types';
import { IndexPatternField, QueryFilter, SearchPanel, SearchSource } from '../../types';
import { getDataSource } from './get_data_source';
import { getFilters } from './get_filters';

Expand All @@ -54,17 +51,16 @@ const getUiSettings = async (config: IUiSettingsClient) => {
};

export async function generateCsvSearch(
req: RequestFacade,
reporting: ReportingCore,
logger: LevelLogger,
context: RequestHandlerContext,
req: KibanaRequest,
searchPanel: SearchPanel,
jobParams: JobParamsDiscoverCsv
jobParams: JobParamsDiscoverCsv,
logger: LevelLogger
): Promise<CsvResultFromSearch> {
const savedObjectsClient = await reporting.getSavedObjectsClient(
KibanaRequest.from(req.getRawRequest())
);
const savedObjectsClient = context.core.savedObjects.client;
const { indexPatternSavedObjectId, timerange } = searchPanel;
const savedSearchObjectAttr = searchPanel.attributes as SavedSearchObjectAttributes;
const savedSearchObjectAttr = searchPanel.attributes;
const { indexPatternSavedObject } = await getDataSource(
savedObjectsClient,
indexPatternSavedObjectId
Expand Down Expand Up @@ -153,9 +149,7 @@ export async function generateCsvSearch(

const config = reporting.getConfig();
const elasticsearch = await reporting.getElasticsearchService();
const { callAsCurrentUser } = elasticsearch.dataClient.asScoped(
KibanaRequest.from(req.getRawRequest())
);
const { callAsCurrentUser } = elasticsearch.dataClient.asScoped(req);
const callCluster = (...params: [string, object]) => callAsCurrentUser(...params);
const uiSettings = await getUiSettings(uiConfig);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ export function getFilters(
let timezone: string | null;

if (indexPatternTimeField) {
if (!timerange) {
if (!timerange || !timerange.min || !timerange.max) {
throw badRequest(
`Time range params are required for index pattern [${indexPatternId}], using time field [${indexPatternTimeField}]`
);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,19 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { RequestFacade } from '../../../../server/types';
import { KibanaRequest } from 'src/core/server';
import { JobParamsPanelCsv, JobParamsPostPayloadPanelCsv } from '../../types';

export function getJobParamsFromRequest(
request: RequestFacade,
request: KibanaRequest,
opts: { isImmediate: boolean }
): JobParamsPanelCsv {
const { savedObjectType, savedObjectId } = request.params;
const { timerange, state } = request.payload as JobParamsPostPayloadPanelCsv;
const { savedObjectType, savedObjectId } = request.params as {
savedObjectType: string;
savedObjectId: string;
};
const { timerange, state } = request.body as JobParamsPostPayloadPanelCsv;

const post = timerange || state ? { timerange, state } : undefined;

return {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,28 +5,23 @@
*/

import { validateUrls } from '../../../../common/validate_urls';
import { ReportingCore } from '../../../../server';
import { cryptoFactory } from '../../../../server/lib';
import {
ConditionalHeaders,
CreateJobFactory,
ESQueueCreateJobFn,
RequestFacade,
} from '../../../../server/types';
import { CreateJobFactory, ESQueueCreateJobFn } from '../../../../server/types';
import { JobParamsPNG } from '../../types';

export const createJobFactory: CreateJobFactory<ESQueueCreateJobFn<
JobParamsPNG
>> = function createJobFactoryFn(reporting: ReportingCore) {
>> = function createJobFactoryFn(reporting) {
const config = reporting.getConfig();
const setupDeps = reporting.getPluginSetupDeps();
const crypto = cryptoFactory(config.get('encryptionKey'));

return async function createJob(
{ objectType, title, relativeUrl, browserTimezone, layout }: JobParamsPNG,
headers: ConditionalHeaders['headers'],
request: RequestFacade
{ objectType, title, relativeUrl, browserTimezone, layout },
context,
req
) {
const serializedEncryptedHeaders = await crypto.encrypt(headers);
const serializedEncryptedHeaders = await crypto.encrypt(req.headers);

validateUrls([relativeUrl]);

Expand All @@ -37,7 +32,7 @@ export const createJobFactory: CreateJobFactory<ESQueueCreateJobFn<
headers: serializedEncryptedHeaders,
browserTimezone,
layout,
basePath: request.getBasePath(),
basePath: setupDeps.basePath(req),
forceNow: new Date().toISOString(),
};
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,33 +5,28 @@
*/

import { validateUrls } from '../../../../common/validate_urls';
import { ReportingCore } from '../../../../server';
import { cryptoFactory } from '../../../../server/lib';
import {
ConditionalHeaders,
CreateJobFactory,
ESQueueCreateJobFn,
RequestFacade,
} from '../../../../server/types';
import { CreateJobFactory, ESQueueCreateJobFn } from '../../../../server/types';
import { JobParamsPDF } from '../../types';

export const createJobFactory: CreateJobFactory<ESQueueCreateJobFn<
JobParamsPDF
>> = function createJobFactoryFn(reporting: ReportingCore) {
>> = function createJobFactoryFn(reporting) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for cleaning up this inferrable types!

const config = reporting.getConfig();
const setupDeps = reporting.getPluginSetupDeps();
const crypto = cryptoFactory(config.get('encryptionKey'));

return async function createJobFn(
{ title, relativeUrls, browserTimezone, layout, objectType }: JobParamsPDF,
headers: ConditionalHeaders['headers'],
request: RequestFacade
context,
req
) {
const serializedEncryptedHeaders = await crypto.encrypt(headers);
const serializedEncryptedHeaders = await crypto.encrypt(req.headers);

validateUrls(relativeUrls);

return {
basePath: request.getBasePath(),
basePath: setupDeps.basePath(req),
browserTimezone,
forceNow: new Date().toISOString(),
headers: serializedEncryptedHeaders,
Expand Down
Loading