Skip to content
This repository has been archived by the owner on Mar 28, 2022. It is now read-only.

fix: incorrect assertion func and json number parse rule #83

Merged
merged 12 commits into from Feb 15, 2022
18 changes: 9 additions & 9 deletions docs/BUILTIN.md
Expand Up @@ -16,13 +16,13 @@ HttpRunner+ validation should follow the following format. `check`, `assert` and
The `assert` method name will be mapped to a built-in function with the following function signature.

```go
func(t assert.TestingT, expected interface{}, actual interface{}, msgAndArgs ...interface{}) bool
func(t assert.TestingT, actual interface{}, expected interface{}, msgAndArgs ...interface{}) bool
```

Currently, HttpRunner+ has the following built-in assertion functions.

| `assert` | Description | A(check), B(expect) | examples |
| -- | -- | -- | -- |
| --- | --- | --- | --- |
| `eq`, `equals`, `equal` | value is equal | A == B | 9 eq 9 |
| `lt`, `less_than` | less than | A < B | 7 lt 8 |
| `le`, `less_or_equals` | less than or equals | A <= B | 7 le 8, 8 le 8 |
Expand All @@ -31,16 +31,16 @@ Currently, HttpRunner+ has the following built-in assertion functions.
| `ne`, `not_equal` | not equals | A != B | 6 ne 9 |
| `str_eq`, `string_equals` | string equals | str(A) == str(B) | 123 str_eq '123' |
| `len_eq`, `length_equals`, `length_equal` | length equals | len(A) == B | 'abc' len_eq 3, [1,2] len_eq 2 |
| `len_gt`, `count_gt` | length greater than | len(A) > B | 'abc' len_gt 2, [1,2,3] len_gt 2 |
| `len_ge`, `count_ge` | length greater than or equals | len(A) >= B | 'abc' len_ge 3, [1,2,3] len_gt 3 |
| `len_lt`, `count_lt` | length less than | len(A) < B | 'abc' len_lt 4, [1,2,3] len_lt 4 |
| `len_le`, `count_le` | length less than or equals | len(A) <= B | 'abc' len_le 3, [1,2,3] len_le 3 |
| `len_gt`, `count_gt`, `length_greater_than` | length greater than | len(A) > B | 'abc' len_gt 2, [1,2,3] len_gt 2 |
| `len_ge`, `count_ge`, `length_greater_or_equals` | length greater than or equals | len(A) >= B | 'abc' len_ge 3, [1,2,3] len_gt 3 |
| `len_lt`, `count_lt`, `length_less_than` | length less than | len(A) < B | 'abc' len_lt 4, [1,2,3] len_lt 4 |
| `len_le`, `count_le`, `length_less_or_equals` | length less than or equals | len(A) <= B | 'abc' len_le 3, [1,2,3] len_le 3 |
| `contains` | contains | [1, 2] contains 1 | 'abc' contains 'a', [1,2,3] len_lt 4 |
| `contained_by` | contained by | A in B | 'a' contained_by 'abc', 1 contained_by [1,2] |
| `type_match` | A and B are in the same type | type(A) == type(B) | 123 type_match 1 |
| `regex_match` | regex matches | re.match(B, A) | 'abcdef' regex 'a\w+d' |
| `startswith` | starts with | A.startswith(B) is True | 'abc' startswith 'ab' |
| `endswith` | ends with | A.endswith(B) is True | 'abc' endswith 'bc' |
| `regex_match` | regex matches | re.match(B, A) | 'abcdef' regex_match 'a\w+d' |
| `starts_with` | starts with | A.starts_with(B) is True | 'abc' starts_with 'ab' |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

还是保留 startswith/endswith 吧;更自然些,而且跟 Python 版本保持一致

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

| `ends_with` | ends with | A.ends_with(B) is True | 'abc' ends_with 'bc' |

## Builtin functions

Expand Down
4 changes: 4 additions & 0 deletions docs/CHANGELOG.md
@@ -1,5 +1,9 @@
# Release History

## v0.6.1 (2022-02-11)

- fix: assertion function errors and json number parse rule

## v0.6.0 (2022-02-08)

- feat: implement `rendezvous` mechanism for data driven
Expand Down
2 changes: 1 addition & 1 deletion docs/cmd/hrp.md
Expand Up @@ -33,4 +33,4 @@ Copyright 2021 debugtalk
* [hrp run](hrp_run.md) - run API test
* [hrp startproject](hrp_startproject.md) - create a scaffold project

###### Auto generated by spf13/cobra on 8-Feb-2022
###### Auto generated by spf13/cobra on 14-Feb-2022
2 changes: 1 addition & 1 deletion docs/cmd/hrp_boom.md
Expand Up @@ -39,4 +39,4 @@ hrp boom [flags]

* [hrp](hrp.md) - One-stop solution for HTTP(S) testing.

###### Auto generated by spf13/cobra on 8-Feb-2022
###### Auto generated by spf13/cobra on 14-Feb-2022
2 changes: 1 addition & 1 deletion docs/cmd/hrp_har2case.md
Expand Up @@ -23,4 +23,4 @@ hrp har2case $har_path... [flags]

* [hrp](hrp.md) - One-stop solution for HTTP(S) testing.

###### Auto generated by spf13/cobra on 8-Feb-2022
###### Auto generated by spf13/cobra on 14-Feb-2022
2 changes: 1 addition & 1 deletion docs/cmd/hrp_run.md
Expand Up @@ -33,4 +33,4 @@ hrp run $path... [flags]

* [hrp](hrp.md) - One-stop solution for HTTP(S) testing.

###### Auto generated by spf13/cobra on 8-Feb-2022
###### Auto generated by spf13/cobra on 14-Feb-2022
2 changes: 1 addition & 1 deletion docs/cmd/hrp_startproject.md
Expand Up @@ -16,4 +16,4 @@ hrp startproject $project_name [flags]

* [hrp](hrp.md) - One-stop solution for HTTP(S) testing.

###### Auto generated by spf13/cobra on 8-Feb-2022
###### Auto generated by spf13/cobra on 14-Feb-2022
2 changes: 1 addition & 1 deletion examples/demo.json
Expand Up @@ -48,7 +48,7 @@
},
{
"check": "headers.\"Content-Type\"",
"assert": "startswith",
"assert": "starts_with",
"expect": "application/json"
},
{
Expand Down
2 changes: 1 addition & 1 deletion examples/demo.yaml
Expand Up @@ -33,7 +33,7 @@ teststeps:
expect: 200
msg: check response status code
- check: headers."Content-Type"
assert: startswith
assert: starts_with
expect: application/json
- check: body.args.foo1
assert: length_equals
Expand Down
3 changes: 0 additions & 3 deletions examples/extract_test.go
Expand Up @@ -58,12 +58,9 @@ func TestCaseExtractStepAssociation(t *testing.T) {
WithHeaders(map[string]string{"User-Agent": "$agent"}).
Extract().
WithJmesPath("status_code", "statusCode").
WithJmesPath("headers.\"Content-Type\"", "contentType").
WithJmesPath("body.args.foo1", "varFoo1").
Validate().
AssertEqual("$statusCode", 200, "check status code").
AssertEqual("headers.Connection", "keep-alive", "check header Connection").
AssertEqual("$contentType", "application/json; charset=utf-8", "check header Content-Type").
AssertEqual("$varFoo1", "bar1", "check args foo1").
AssertEqual("body.args.foo2", "bar2", "check args foo2").
AssertEqual("body.headers.\"user-agent\"", "HttpRunnerPlus", "check header user agent"),
Expand Down
81 changes: 81 additions & 0 deletions examples/httpbingo_test.json
@@ -0,0 +1,81 @@
{
"config": {
"name": "request methods testcase: validate with parameters",
"base_url": "https://httpbingo.bytedance.net"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

代码中不要带字节的内网域名。示例代码还是使用 postman echo 的吧

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

},
"teststeps": [
{
"name": "get with params",
"request": {
"method": "GET",
"url": "/get",
"params": {
"a": 1,
"foo": "I'm a happy guy"
}
},
"validate": [
{
"check": "body.args.a",
"assert": "eq",
"expect": "1",
"msg": "check argument a"
},
{
"check": "status_code",
"assert": "lt",
"expect": 201,
"msg": "check status code"
},
{
"check": "status_code",
"assert": "gt",
"expect": 199,
"msg": "check status code"
},
{
"check": "status_code",
"assert": "ge",
"expect": 200,
"msg": "check status code"
},
{
"check": "status_code",
"assert": "le",
"expect": 200,
"msg": "check status code"
},
{
"check": "status_code",
"assert": "ne",
"expect": 199,
"msg": "check status code"
},
{
"check": "body.args.foo",
"assert": "contains",
"expect": "happy",
"msg": "check status code"
},
{
"check": "body.args.foo",
"assert": "type_match",
"expect": "string",
"msg": "check status code"
},
{
"check": "body.args.foo",
"assert": "starts_with",
"expect": "I'm",
"msg": "check status code"
},
{
"check": "body.args.foo",
"assert": "type_match",
"expect": "string",
"msg": "check status code"
}
]
}
]
}
2 changes: 0 additions & 2 deletions examples/request_test.go
Expand Up @@ -20,8 +20,6 @@ func TestCaseBasicRequest(t *testing.T) {
}).
Validate().
AssertEqual("status_code", 200, "check status code").
AssertEqual("headers.Connection", "keep-alive", "check header Connection").
AssertEqual("headers.\"Content-Type\"", "application/json; charset=utf-8", "check header Content-Type").
AssertEqual("body.args.foo1", "bar1", "check args foo1").
AssertEqual("body.args.foo2", "bar2", "check args foo2"),
hrp.NewStep("post raw text").
Expand Down
14 changes: 5 additions & 9 deletions examples/validate_test.go
Expand Up @@ -24,10 +24,8 @@ func TestCaseValidateStep(t *testing.T) {
Extract().
WithJmesPath("body.args.foo1", "varFoo1").
Validate().
AssertEqual("status_code", "$expectedStatusCode", "check status code"). // assert status code
AssertEqual("headers.Connection", "keep-alive", "check header Connection"). // assert response header
AssertEqual("headers.\"Content-Type\"", "application/json; charset=utf-8", "check header Content-Type"). // assert response header, with double quotes
AssertEqual("body.args.foo1", "bar1", "check args foo1"). // assert response json body with jmespath
AssertEqual("status_code", "$expectedStatusCode", "check status code"). // assert status code
AssertEqual("body.args.foo1", "bar1", "check args foo1"). // assert response json body with jmespath
AssertEqual("body.args.foo2", "bar2", "check args foo2").
AssertEqual("body.headers.\"user-agent\"", "HttpRunnerPlus", "check header user agent"),
hrp.NewStep("get with params").
Expand All @@ -40,12 +38,10 @@ func TestCaseValidateStep(t *testing.T) {
WithHeaders(map[string]string{"User-Agent": "$agent"}).
Extract().
WithJmesPath("status_code", "statusCode").
WithJmesPath("headers.\"Content-Type\"", "contentType").
Validate().
AssertEqual("$statusCode", 200, "check status code"). // assert with extracted variable from current step
AssertEqual("$contentType", "application/json; charset=utf-8", "check header Content-Type"). // assert with extracted variable from current step
AssertEqual("$varFoo1", "bar1", "check args foo1"). // assert with extracted variable from previous step
AssertEqual("body.args.foo2", "bar2", "check args foo2"), // assert response json body with jmespath
AssertEqual("$statusCode", 200, "check status code"). // assert with extracted variable from current step
AssertEqual("$varFoo1", "bar1", "check args foo1"). // assert with extracted variable from previous step
AssertEqual("body.args.foo2", "bar2", "check args foo2"), // assert response json body with jmespath
},
}

Expand Down
6 changes: 0 additions & 6 deletions examples/variables_test.go
Expand Up @@ -22,8 +22,6 @@ func TestCaseConfigVariables(t *testing.T) {
WithHeaders(map[string]string{"User-Agent": "$agent"}).
Validate().
AssertEqual("status_code", "$expectedStatusCode", "check status code").
AssertEqual("headers.Connection", "keep-alive", "check header Connection").
AssertEqual("headers.\"Content-Type\"", "application/json; charset=utf-8", "check header Content-Type").
AssertEqual("body.args.foo1", "bar1", "check args foo1").
AssertEqual("body.args.foo2", "bar2", "check args foo2").
AssertEqual("body.headers.\"user-agent\"", "HttpRunnerPlus", "check header user agent"),
Expand Down Expand Up @@ -53,8 +51,6 @@ func TestCaseStepVariables(t *testing.T) {
WithHeaders(map[string]string{"User-Agent": "$agent"}).
Validate().
AssertEqual("status_code", "$expectedStatusCode", "check status code").
AssertEqual("headers.Connection", "keep-alive", "check header Connection").
AssertEqual("headers.\"Content-Type\"", "application/json; charset=utf-8", "check header Content-Type").
AssertEqual("body.args.foo1", "bar1", "check args foo1").
AssertEqual("body.args.foo2", "bar2", "check args foo2").
AssertEqual("body.headers.\"user-agent\"", "HttpRunnerPlus", "check header user agent"),
Expand Down Expand Up @@ -88,8 +84,6 @@ func TestCaseOverrideConfigVariables(t *testing.T) {
WithHeaders(map[string]string{"User-Agent": "$agent"}).
Validate().
AssertEqual("status_code", "$expectedStatusCode", "check status code").
AssertEqual("headers.Connection", "keep-alive", "check header Connection").
AssertEqual("headers.\"Content-Type\"", "application/json; charset=utf-8", "check header Content-Type").
AssertEqual("body.args.foo1", "bar1", "check args foo1").
AssertEqual("body.args.foo2", "bar2", "check args foo2").
AssertEqual("body.headers.\"user-agent\"", "HttpRunnerPlus", "check header user agent"),
Expand Down