Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bench mark report #280

Merged
merged 46 commits into from
Apr 7, 2020
Merged

Bench mark report #280

Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
40c22e0
test(raw data changes): add bench marks end point
OrFrenkelZooz Mar 22, 2020
736c36b
test(unit-tests): add unit-tests bench mark
OrFrenkelZooz Mar 31, 2020
a56e1dc
test(unit-tests): fix change
OrFrenkelZooz Mar 31, 2020
829cd38
Merge branch 'master' into banch-mark
OrFrenkelZooz Mar 31, 2020
594e5fd
test(): add integration tests
OrFrenkelZooz Mar 31, 2020
3a38dc8
test(integration test): add integration tests and swagger validations
OrFrenkelZooz Mar 31, 2020
956ab34
test(integration test): add unit-tests
OrFrenkelZooz Mar 31, 2020
db5e642
test(integration test): temp commit
OrFrenkelZooz Mar 31, 2020
f115e70
test(temp push): temp push
OrFrenkelZooz Apr 1, 2020
17b9cbc
test(fix): fix typo
OrFrenkelZooz Apr 1, 2020
2e94c2b
test(fix): fix typo and all review changes
OrFrenkelZooz Apr 1, 2020
63b0aa9
Merge branch 'banch-mark' into bench-mark-report
OrFrenkelZooz Apr 1, 2020
fe0e4dd
test(fix): temp push reset mac
OrFrenkelZooz Apr 2, 2020
f16ff6e
Merge branch 'master' into bench-mark-report
OrFrenkelZooz Apr 5, 2020
9a1c6af
feat(benchmark): add benchmark
OrFrenkelZooz Apr 5, 2020
745d38d
feat(benchmark): fix lint
OrFrenkelZooz Apr 5, 2020
5529aed
test(unit-tests): add unit-tests
OrFrenkelZooz Apr 5, 2020
a454730
test(integration tests): fix swagger for integration tests
OrFrenkelZooz Apr 5, 2020
62f4e42
test(integration tests): fix swagger for integration tests
OrFrenkelZooz Apr 5, 2020
7223a1a
feat(code review): code review cahnges and add calaualte func
OrFrenkelZooz Apr 6, 2020
d931d68
test(unit-tests): fix unit-tests
OrFrenkelZooz Apr 6, 2020
fb6bb00
test(unit-tests): fix unit-tests
OrFrenkelZooz Apr 6, 2020
abded93
test(unit-tests): fix unit-tests
OrFrenkelZooz Apr 6, 2020
e51d011
test(unit-tests): add integration tests
OrFrenkelZooz Apr 6, 2020
8c30db1
test(unit-tests): fix integration tests
OrFrenkelZooz Apr 6, 2020
9e75918
test(unit-tests): add integration tests
OrFrenkelZooz Apr 6, 2020
1c1ad7b
test(unit-tests): add integration tests
OrFrenkelZooz Apr 6, 2020
64fce3b
feat(data-type): change type to blob
OrFrenkelZooz Apr 7, 2020
f2c5726
feat(data-type): change type Sequelize.DataTypes.TEXT('long')
OrFrenkelZooz Apr 7, 2020
d4583ba
test(sqllite): fix sqlite
OrFrenkelZooz Apr 7, 2020
59c12eb
feat(bench-mark): add default value to bench_mark config
OrFrenkelZooz Apr 7, 2020
9af9943
test(sqllite): fix last report score
OrFrenkelZooz Apr 7, 2020
c08df43
test(sqllite): fix last report score
OrFrenkelZooz Apr 7, 2020
b14418f
test(sqllite): fix last report score
OrFrenkelZooz Apr 7, 2020
2c2fc14
test(sqllite): revert marked changes
OrFrenkelZooz Apr 7, 2020
1d55280
test(): triger push again
OrFrenkelZooz Apr 7, 2020
b604cbb
test(): remove comments
OrFrenkelZooz Apr 7, 2020
b04d47e
test(): fix config unit tests
OrFrenkelZooz Apr 7, 2020
3b39f33
feat(): change to percentile_ninety_five
OrFrenkelZooz Apr 7, 2020
a68dd42
Merge branch 'default-config-benchmark' into bench-mark-report
OrFrenkelZooz Apr 7, 2020
1ef217a
feat(): change to percentile_ninety_five
OrFrenkelZooz Apr 7, 2020
4dc11f0
feat(): change to percentile_ninety_five
OrFrenkelZooz Apr 7, 2020
20e079b
feat(): change to percentile_ninety_five
OrFrenkelZooz Apr 7, 2020
65895d2
Merge branch 'default-config-benchmark' into bench-mark-report
OrFrenkelZooz Apr 7, 2020
1edc554
feat(): change to percentile_ninety_five
OrFrenkelZooz Apr 7, 2020
961d665
feat(): change to percentile_ninety_five
OrFrenkelZooz Apr 7, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
161 changes: 40 additions & 121 deletions docs/devguide/docs/swagger-docs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -312,13 +312,39 @@ paths:
application/json:
schema:
$ref: '#/components/schemas/error_response'
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/benchmark_request'
description: The bench mark to add
required: true
get:
enudler marked this conversation as resolved.
Show resolved Hide resolved
tags:
- Tests
parameters:
- in: path
name: test_id
description: The test id.
required: true
schema:
type: string
format: uuid
example: 4bf5d7ab-f310-4a64-8ec2-d65c06188ec1
summary: get a bench mark for test
description: Create a bench mark for test
responses:
'200':
description: Success
content:
application/json:
schema:
$ref: '#/components/schemas/benchmark_request'
'404':
description: Not found
content:
application/json:
schema:
$ref: '#/components/schemas/error_response'
'500':
description: Internal server error
content:
application/json:
schema:
$ref: '#/components/schemas/error_response'
'/v1/tests/{test_id}':
get:
operationId: retrieve-a-test
Expand Down Expand Up @@ -879,55 +905,6 @@ paths:
application/json:
schema:
$ref: '#/components/schemas/error_response'
put:
operationId: edit-a-report
tags:
- Reports
summary: edit a Report
description: edit a specific report for the specified test.
parameters:
- in: path
name: test_id
required: true
description: The id of the test for which to retrieve the report.
schema:
type: string
example: 81a27853-0db5-4e57-ad63-4b637528398a
- in: path
name: report_id
description: The id of the report to retrieve.
required: true
schema:
type: string
responses:
'204':
description: Success

'400':
description: Bad request
content:
application/json:
schema:
$ref: '#/components/schemas/error_response'
'404':
description: Not found
content:
application/json:
schema:
$ref: '#/components/schemas/error_response'
'500':
description: Internal server error
content:
application/json:
schema:
$ref: '#/components/schemas/error_response'
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/edit_report'
description: report data
required: true
/v1/tests/last_reports:
get:
operationId: retrieve-recent-reports
Expand Down Expand Up @@ -1046,12 +1023,6 @@ paths:
application/json:
schema:
$ref: "#/components/schemas/config_body"
"422":
description: Unprocessable entity
content:
application/json:
schema:
$ref: '#/components/schemas/error_response'
"500":
description: Internal server error
content:
Expand Down Expand Up @@ -1652,13 +1623,6 @@ components:
run_id:
type: string
description: The run id. Only shown if the job is ran immediately.
edit_report:
additionalProperties: false
properties:
notes:
type: string
description: notes for specific report

post_report:
required:
- report_id
Expand Down Expand Up @@ -1779,6 +1743,13 @@ components:
notes:
type: string
description: notes about the test
score:
type: number
description: The score of report comapring to test benchmark
example: 9.25
weights_data:
type: object
description: The way score calualated
config_body:
additionalProperties: false
properties:
Expand Down Expand Up @@ -1892,55 +1863,3 @@ components:
timeout:
description: timout to smtp server in milliseconds
type: number
benchmark_threshold:
type: number
description: benchmark threshold. Test below this threshold considered as failed and webhook will be sent to the threshold_webhook_url.
minimum: 0
maximum: 100
benchmark_threshold_webhook_url:
type: string
format: uri
description: webhook url to alert when coverage less than threshold
benchmark_weights:
additionalProperties: false
type: object
description: benchmark_weights
required:
- percentile_ninety
- percentile_fifty
- server_errors
- client_errors
- rps
properties:
percentile_ninety:
description: P90 benchmark weight
allOf:
- $ref: '#/components/schemas/benchmark_weights'
percentile_fifty:
description: P50 benchmark weight
allOf:
- $ref: '#/components/schemas/benchmark_weights'
server_errors:
description: 5xx server errors benchmark weight
allOf:
- $ref: '#/components/schemas/benchmark_weights'
client_errors:
description: 4xx server errors benchmark weight
allOf:
- $ref: '#/components/schemas/benchmark_weights'
rps:
description: requests per second benchmark weight
allOf:
- $ref: '#/components/schemas/benchmark_weights'
benchmark_weights:
type: object
required:
- factor
- percentage
properties:
factor:
type: number
description: benchmark factor weight
percentage:
type: number
description: benchmark percentage weight
48 changes: 44 additions & 4 deletions docs/openapi3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,6 @@ paths:
application/json:
schema:
$ref: '#/components/schemas/error_response'
# Tests
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please revert (# Tests

/v1/tests:
post:
operationId: create-a-test
Expand Down Expand Up @@ -323,6 +322,40 @@ paths:
$ref: '#/components/schemas/benchmark_request'
description: The bench mark to add
required: true
get:
tags:
- Tests
parameters:
- in: path
name: test_id
description: The test id.
required: true
schema:
type: string
format: uuid
example: 4bf5d7ab-f310-4a64-8ec2-d65c06188ec1
summary: get a bench mark for test
description: Create a bench mark for test
enudler marked this conversation as resolved.
Show resolved Hide resolved
responses:
'200':
description: Success
content:
application/json:
schema:
$ref: '#/components/schemas/benchmark_request'
'404':
description: Not found
content:
application/json:
schema:
$ref: '#/components/schemas/error_response'
'500':
description: Internal server error
content:
application/json:
schema:
$ref: '#/components/schemas/error_response'

'/v1/tests/file/{file_id}':
get:
operationId: retrieve-a-file
Expand Down Expand Up @@ -2047,6 +2080,13 @@ components:
type: number
description: The duration of the test in seconds.
example: 60
score:
type: number
description: The score of report comapring to test benchmark
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The score of a report, calculated based on a benchmark and benchmark weights

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

example: 9.25
weights_data:
enudler marked this conversation as resolved.
Show resolved Hide resolved
type: object
description: The way score calualated
arrival_rate:
type: number
description: The arrival rate that was set for the test. This is the number of times per second that the test scenarios will run.
Expand Down Expand Up @@ -2203,15 +2243,15 @@ components:
- $ref: '#/components/schemas/benchmark_weights'
server_errors:
description: 5xx server errors benchmark weight
allOf:
allOf:
- $ref: '#/components/schemas/benchmark_weights'
client_errors:
description: 4xx server errors benchmark weight
allOf:
allOf:
- $ref: '#/components/schemas/benchmark_weights'
rps:
description: requests per second benchmark weight
allOf:
allOf:
- $ref: '#/components/schemas/benchmark_weights'
benchmark_weights:
type: object
Expand Down
3 changes: 3 additions & 0 deletions src/database/cassandra-handler/init-scripts/19__bench_mark_data_reports.cql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
ALTER TABLE reports_summary ADD weights_data text;
enudler marked this conversation as resolved.
Show resolved Hide resolved
---
ALTER TABLE reports_summary ADD score float;
Loading