Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return HTTP 404 instead of 400 for supervisor/task endpoints #11724

Merged
merged 11 commits into from
Nov 25, 2021

Conversation

FrankChen021
Copy link
Member

@FrankChen021 FrankChen021 commented Sep 19, 2021

Description

We're using supervisor/task HTTP APIs to manage supervisor and tasks in a independent system. Current supervisor/task endpoint return 400 when a supervisor or a task is not found. This status code is not friendly and confusing for the 3rd system. And according to the definition of HTTP status code, 404 is right code for such case. So this PR changes the status code from 400 to 404 to eliminate the ambigiuty.

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

Copy link
Contributor

@jihoonson jihoonson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change seems reasonable. @FrankChen021 can you add some tests for this change?

@FrankChen021
Copy link
Member Author

This change seems reasonable. @FrankChen021 can you add some tests for this change?

Sure, I will add some tests later this week.

Copy link
Contributor

@suneet-s suneet-s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 from me after tests. Returning 404 instead of 400 seems more like a bug fix vs introducing an incompatible change so I'm also ok with this without a feature flag to fall back to the old behavior for any users who wouldn't want to incorporate this change on an upgrade.

@@ -386,6 +386,7 @@ public Response specGetAllHistory(@Context final HttpServletRequest req)
@GET
@Path("/{id}/history")
@Produces(MediaType.APPLICATION_JSON)
@ResourceFilters(SupervisorResourceFilter.class)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are using the SupervisorResourceFilter here, I think it would be safe to remove the filtering done by AuthorizationUtils.filterAuthorizedResources() later in this method?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing out this, this causes problem. And since SupervisorResourceFilter requires WRITE permission, and the function requires READ permission, I will delete the filter. I added this filter to hope the tests can have same exception message handling.


import static org.easymock.EasyMock.expect;

public class OverlordResourceFilterTest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should call this class TaskResourceFilterTest as TaskResourceFilter is the one used in the tests here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@@ -159,4 +159,6 @@
public static final String SHUFFLE_DEEP_STORE = "shuffle-deep-store";

public static final String CUSTOM_COORDINATOR_DUTIES = "custom-coordinator-duties";

public static final String OVERLORD_RESOURCE = "overlord-resources";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use a more generic group name here that could later include other tests, say for other resources / HTTP endpoints?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about http-endpoint ?

@@ -136,6 +136,9 @@ public TaskStatusPlus getTaskStatus(String taskID)
);
return taskStatusResponse.getStatus();
}
catch (ISE e) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to catch all RuntimeExceptions here (not just ISE) and re-throw them as is?

(same comment in other places too)

Copy link
Member Author

@FrankChen021 FrankChen021 Oct 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ISE is a specific exception thrown by makeRequest function when HTTP response status code is not 200. And when this exceptio occurs, its exception message contains the HTTP status code. Here we catch and re-throw the exception to simplify the exception handling to check the HTTP status code in the message.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, thanks for the clarification. 👍

Copy link
Member

@asdf2014 asdf2014 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 LGTM

@FrankChen021
Copy link
Member Author

@jihoonson Do you have any other comments?

Copy link
Contributor

@jihoonson jihoonson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @FrankChen021!

@FrankChen021
Copy link
Member Author

@jihoonson @asdf2014 @suneet-s @kfaraz Thank you for your review.

@FrankChen021 FrankChen021 merged commit 98957be into apache:master Nov 25, 2021
@FrankChen021 FrankChen021 deleted the supervisor_status_code branch November 25, 2021 05:09
@abhishekagarwal87 abhishekagarwal87 added this to the 0.23.0 milestone May 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants