Skip to content

[#4259] feat(core): support custom rest api#4261

Merged
mchades merged 7 commits intoapache:mainfrom
mchades:custom-rest-api
Jul 31, 2024
Merged

[#4259] feat(core): support custom rest api#4261
mchades merged 7 commits intoapache:mainfrom
mchades:custom-rest-api

Conversation

@mchades
Copy link
Contributor

@mchades mchades commented Jul 24, 2024

What changes were proposed in this pull request?

  • add static method fromString for Catalog.Type and Namespace
  • make GravitinoEnv extensional
  • make GravitinoServer use passed in GravitinoEnv
  • support custom rest api

Why are the changes needed?

Fix: #4259

Does this PR introduce any user-facing change?

yes, users now can use custom REST API packages

How was this patch tested?

test locally

@mchades mchades requested a review from jerryshao July 24, 2024 08:47
@mchades mchades self-assigned this Jul 24, 2024
@mchades mchades requested a review from yuqi1129 July 30, 2024 09:17
@mchades
Copy link
Contributor Author

mchades commented Jul 30, 2024

@yuqi1129 @jerryshao Could you please review this when you have time? Thank you!

# The response header size of the built-in web server
gravitino.server.webserver.responseHeaderSize = 131072
# Comma separated list of REST API packages to scan
gravitino.restApiPackages = org.apache.gravitino.server.web.rest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this affect the iceberg rest server, did you test it, @FANNG1 please also verify it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides, I think this should not be a configuration for users. "org.apache.gravitino.server.web.rest" should always be configured, right? What is the behavior if users configure others?

You'd better configure like "additionalPackages" to support additional rest packages, and make Gravitino's package as a always loaded package.

Copy link
Contributor Author

@mchades mchades Jul 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we use configure like "additionalPackages" that means we don't allow user to overwrite the existing REST APIs?

I'm now set org.apache.gravitino.server.web.rest as a default value and allow user to overwrite it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have such requirements to overwrite? I don't think people have such requirements.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, maybe we can make it additionalPackages and support overwrite when necessary

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

* @return The EventBus instance.
*/
public EventBus eventBus() {
Preconditions.checkNotNull(eventBus, "GravitinoEnv is not initialized.");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remembered that for Iceberg rest service, some modules will not enabled, so it will be null, can you please confirm this with @FANNG1 , also use checkArgument not checkNotNull.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this doesn't seem to conflict with the fact that if the variable is not initialized it is not allowed to get it.

If the Iceberg REST service disables the modules, it does not need to call the method.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the Iceberg REST service disables the modules, it does not need to call the method.
+1

@mchades mchades requested review from FANNG1 and jerryshao July 31, 2024 01:49

public static final ConfigEntry<Optional<String>> REST_API_EXTENSION_PACKAGES =
new ConfigBuilder("gravitino.extension.restApiPackages")
public static final ConfigEntry<Optional<List<String>>> REST_API_EXTENSION_PACKAGES =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need Optional<List<String>>? I think we can use an empty list as a default value?

@FANNG1
Copy link
Contributor

FANNG1 commented Jul 31, 2024

LGTM,except minor comments

| `gravitino.server.webserver.responseHeaderSize` | Maximum size of HTTP responses. | `131072` | No | 0.1.0 |
| `gravitino.server.shutdown.timeout` | Time in milliseconds to gracefully shut down of the Gravitino webserver. | `3000` | No | 0.2.0 |
| `gravitino.server.webserver.customFilters` | Comma-separated list of filter class names to apply to the API. | (none) | No | 0.4.0 |
| `gravitino.extension.restApiPackages` | Comma-separated list of REST API packages to expand | (none) | No | 0.6.0 |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs should also be updated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@mchades mchades closed this Jul 31, 2024
@mchades mchades reopened this Jul 31, 2024
@mchades mchades requested a review from jerryshao July 31, 2024 11:08
@mchades
Copy link
Contributor Author

mchades commented Jul 31, 2024

CI failure seems unrelated to the changes, do you have any other comments? @jerryshao @FANNG1

@jerryshao
Copy link
Contributor

I don't have further comments, just waiting for CI to pass.

@mchades mchades merged commit 4d8e0f3 into apache:main Jul 31, 2024
@mchades mchades deleted the custom-rest-api branch April 12, 2025 11:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] support custom REST API

3 participants