Task doesn't fail when status code is incorrect #436
Comments
|
Please enable Offhand I'm not seeing a problem in what you're doing, hence why collecting logs will give us more insight. I imagine you've already found the relevant documentation for using the request builder: Note that there's already logic in the function you refer to where we identify whether or not the individual request failed, and this will show up in the logs and in the metrics: |
|
The error is in the request log see below. The error does appear in the per request numbers and in the "ERRORS" section in the end. This issue that the error doesn't show up in the per task details and an error isn't logged to the console when the error occurs. Logging to the console seems important to me because as I can see that something is failing, maybe because on an incorrect task. See details below. |
|
In the current release you must include |
|
It is hard to tell as the -v logs everything to the console including debug statements so it is very hard to find errors. Should the per task metrics include the failure count? |
|
Which version of Goose are you using? If a single In any case, I reviewed the code in question and we don't seem to be emitting an |
In order for a task to show as an error, you must return Err() from your task. Per-request errors will only show up in the per-request section of the metrics. |
|
I was using 0.15.2 but are on the current main branch. info, warn or error? I also find it helpful to include the request body as in this case and others the issue is often caused by invalid payload. Example code in the location use suggested: if !request_metric.success {
if let Ok(r) = response {
warn!("{:?} {}: {}", &path, &request_metric.error, r.text().await.unwrap_or("".to_string()));
} else {
warn!("{:?} {}", &path, &request_metric.error);
}
return Err(GooseTaskError::RequestFailed{raw_request: request_metric});
} |
So I should be doing |
|
It sounds like you're looking for the debug log. Try adding the following options: Now review the You can also store the body of the request made in the request log if you enable |
Take a look at the provided Goose examples. You need to decide what qualifies as a failure of your task. For example, see the For much simpler error handling, see the |
|
I think the session sample needs to be updated, that is what I started with. It doesn't check the response so the task will always succeed even if it fails. /// A very simple task that simply loads the front page.
async fn authenticated_index(user: &mut GooseUser) -> GooseTaskResult {
// This will panic if the session is missing or if the session is not of the right type.
// Use `get_session_data` to handle a missing session.
let session = user.get_session_data_unchecked::<Session>();
// Create a Reqwest RequestBuilder object and configure bearer authentication when making
// a GET request for the index.
let reqwest_request_builder = user
.get_request_builder(&GooseMethod::Get, "/")?
.bearer_auth(&session.jwt_token);
// Add the manually created RequestBuilder and build a GooseRequest object.
let goose_request = GooseRequest::builder()
.set_request_builder(reqwest_request_builder)
.build();
// Make the actual request.
user.request(goose_request).await?;
Ok(())
} |
|
The simple example also doesn't check for the response error so the task will always succeed. |
|
Mostly correct. It will fail if There's a number of lines where this function many return an Error:
As explained in the documentation, the For a more fleshed-out example, see the |
|
The issue with the examples (simple and session) is that the request could fail because of a typo or invalid payload and the task will be marked as successful. From the task how do you actually check for an error? I just checked In my case I am trying to get the simplest example working. I did that by copying the sample and going from there. When I provided the wrong JSON for the payload of the task it wasn't obvious that it failed because it was saying it succeeded. I only knew it failed because I wasn't getting an incorrect log statement in my back-end. This isn't criticism, I just want to understand how to use it better because it works great, I am able to send 100s of request with very little overhead which is awesome. |
|
Tasks and Requests are different things. (It may be helpful to see this PR which is working to clarify this #435.) In Goose, there are Requests of which one or more are contained in a Task of which one or more are contained in a TaskSet. It is up to you when writing your load test to determine if a single failed Request results in the entire Task failing or not. For example: if you have a Task called "load front page" it may 1) load index.html, 2) validate the HTML title and returned Headers, 3) scrape locally hosted assets from the returned html, and 4) load each of those scraped assets (ie images, JS and CSS). It's common for one or more of these assets to be missing (resulting in a 404) or otherwise to generate an error: but even though that Request failed I would consider the Task to have failed. When writing load tests with Goose, I always enable the error log and the debug log to collect as much information as possible about errors. This will capture all 400+ responses (unless you override this, ie if you WANT to load test a 404 page) and allow you to quickly track down things like typos. Of course, if you Really want a 1:1 mapping between a single Request and a single Task, you can do this easily enough by parsing the I referred you to the Umami example as it is the most complete load test example included with Goose and demonstrates how to do things like this and much more. I personally tend to leverage the Goose Eggs validate functions to avoid writing boilerplate. |
|
It's a little outdated now, but we tried to capture some real-world examples of using Goose in this blog: |
|
Please also see this similar discussion: #437 |
|
It would be good to be able to easily return an error from a request if the status is invalid. In my case I am testing an API not a web page. Something like: let req = user.get_request_builder(&GooseMethod::Post, "api/view/Flight/execute")?
.basic_auth("admin", Option::<&str>::None)
.json(&view);
let req = GooseRequest::builder()
.set_request_builder(req)
.error_on_fail()
.build();Is that something you would consider? |
|
Yes, adding flexibility is always welcome, so long as it doesn't negatively impact performance. This would be a welcome addition. |
|
Fixed in #438 |
Hi,
I am using GooseRequest builder and the task aren't recorded as "failed" even if an error is returned (status code 400+). The code for my task is below, in this case the JSON is invalid a 400 error is returned. Not sure if there is something wrong in how I am using it:
I have put together a fix to return the error and also return the body as part of the error message (goose.rs 1631-1638):
I can create a pull request if this solves the issue I have.
The text was updated successfully, but these errors were encountered: