New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Reporting] Fix Generating Reports with long jobParams
RISON
#45603
Changes from 18 commits
7791970
0640cab
8dd3241
df344d0
2cbfe96
68db0d4
fc7fbe5
1b7b3d8
411178e
7039ee2
a6d2df2
d3fba77
317212b
ce9433c
451e80a
9319e87
1703ccb
074982f
547c59d
5e6735b
0419325
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
/* | ||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
* or more contributor license agreements. Licensed under the Elastic License; | ||
* you may not use this file except in compliance with the Elastic License. | ||
*/ | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🙇 |
||
import expect from '@kbn/expect'; | ||
import supertest from 'supertest'; | ||
import { JOB_PARAMS_IN_POSTBODY, JOB_PARAMS_IN_QUERYSTRING } from './fixtures'; | ||
|
||
// eslint-disable-next-line import/no-default-export | ||
export default function({ getService }: { getService: any }) { | ||
const esArchiver = getService('esArchiver'); | ||
const supertestSvc = getService('supertest'); | ||
const generateAPI = { | ||
getCsvFromParamsInPayload: async (jobParams: object = {}) => { | ||
return await supertestSvc | ||
.post(`/api/reporting/generate/csv`) | ||
.set('kbn-xsrf', 'xxx') | ||
.send(jobParams); | ||
}, | ||
getCsvFromParamsInQueryString: async (jobParams: string = '') => { | ||
return await supertestSvc | ||
.post(`/api/reporting/generate/csv?jobParams=${jobParams}`) | ||
.set('kbn-xsrf', 'xxx'); | ||
}, | ||
}; | ||
|
||
describe('Generation from Job Params', () => { | ||
before(async () => { | ||
await esArchiver.load('reporting/logs'); | ||
await esArchiver.load('logstash_functional'); | ||
}); // prettier-ignore | ||
after(async () => { | ||
await esArchiver.unload('reporting/logs'); | ||
await esArchiver.unload('logstash_functional'); | ||
}); // prettier-ignore | ||
|
||
it('Rejects bogus jobParams', async () => { | ||
const { status: resStatus, text: resText } = (await generateAPI.getCsvFromParamsInPayload({ | ||
jobParams: 0, | ||
})) as supertest.Response; | ||
|
||
expect(resStatus).to.eql(400); | ||
expect(resText).to.match(/\\\"jobParams\\\" must be a string/); | ||
}); | ||
|
||
it('Rejects empty jobParams', async () => { | ||
const { | ||
status: resStatus, | ||
text: resText, | ||
} = (await generateAPI.getCsvFromParamsInPayload()) as supertest.Response; | ||
|
||
expect(resStatus).to.eql(400); | ||
expect(resText).to.match(/jobParams RISON string is required/); | ||
}); | ||
|
||
it('Accepts jobParams in POST payload', async () => { | ||
const { status: resStatus, text: resText } = (await generateAPI.getCsvFromParamsInPayload({ | ||
jobParams: JOB_PARAMS_IN_POSTBODY, | ||
})) as supertest.Response; | ||
expect(resStatus).to.eql(200); | ||
expect(resText).to.match(/"jobtype":"csv"/); | ||
}); | ||
|
||
it('Accepts jobParams in query string', async () => { | ||
const { status: resStatus, text: resText } = (await generateAPI.getCsvFromParamsInQueryString( | ||
JOB_PARAMS_IN_QUERYSTRING | ||
)) as supertest.Response; | ||
expect(resStatus).to.eql(200); | ||
expect(resText).to.match(/"jobtype":"csv"/); | ||
}); | ||
}); | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -158,3 +158,27 @@ export const CSV_RESULT_NANOS = `date,message,"_id" | |
"2015-01-01T12:10:30.123456789Z","Hello 2", | ||
"2015-01-01T12:10:30","Hello 1", | ||
`; | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe we could use something like the backtick operator with There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I like it! It turns out I also realized there does not need to be 2 consts here, because the query string one is just the same rison, just URI-encoded. |
||
export const JOB_PARAMS_IN_QUERYSTRING = | ||
`(conflictedTypesFields:!(),fields:!(%27@timestamp%27,clientip,extens` + | ||
`ion),indexPatternId:%27logstash-*%27,metaFields:!(_source,_id,_type,_index,_scor` + | ||
`e),searchRequest:(body:(_source:(excludes:!(),includes:!(%27@timestamp%27,client` + | ||
`ip,extension)),docvalue_fields:!(),query:(bool:(filter:!((match_all:()),(range:(` + | ||
`%27@timestamp%27:(gte:%272015-09-20T10:19:40.307Z%27,lt:%272015-09-20T10:26:56.2` + | ||
`21Z%27))),(range:(%27@timestamp%27:(format:strict_date_optional_time,gte:%272004` + | ||
`-09-17T21:19:34.213Z%27,lte:%272019-09-17T21:19:34.213Z%27)))),must:!(),must_not` + | ||
`:!(),should:!())),script_fields:(),sort:!((%27@timestamp%27:(order:desc,unmapped` + | ||
`_type:boolean))),stored_fields:!(%27@timestamp%27,clientip,extension),version:!t` + | ||
`),index:%27logstash-*%27),title:%27A%20Saved%20Search%20With%20a%20DATE%20FILTER` + | ||
`%27,type:search)`; | ||
export const JOB_PARAMS_IN_POSTBODY = | ||
`(conflictedTypesFields:!(),fields:!('@timestamp',clientip,extension),indexPatt` + | ||
`ernId:'logstash-*',metaFields:!(_source,_id,_type,_index,_score),searchRequest:(` + | ||
`body:(_source:(excludes:!(),includes:!('@timestamp',clientip,extension)),docvalu` + | ||
`e_fields:!(),query:(bool:(filter:!((match_all:()),(range:('@timestamp':(gte:'201` + | ||
`5-09-20T10:19:40.307Z',lt:'2015-09-20T10:26:56.221Z'))),(range:('@timestamp':(fo` + | ||
`rmat:strict_date_optional_time,gte:'2004-09-17T21:19:34.213Z',lte:'2019-09-17T21` + | ||
`:19:34.213Z')))),must:!(),must_not:!(),should:!())),script_fields:(),sort:!(('@t` + | ||
`imestamp':(order:desc,unmapped_type:boolean))),stored_fields:!('@timestamp',clie` + | ||
`ntip,extension),version:!t),index:'logstash-*'),title:'A Saved Search With a DAT` + | ||
`E FILTER',type:search)`; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice I like this method better
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have an unrelated small fix I can make to validate the exportType param is registered in the exportTypesRegistry as an ID of one of the types