Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add schema admin api get schema info with schema version #4877

Merged
merged 8 commits into from
Aug 16, 2019
Merged

Add schema admin api get schema info with schema version #4877

merged 8 commits into from
Aug 16, 2019

Conversation

congbobo184
Copy link
Contributor

Motivation

To fix #4854 and support get keyValueSchema

Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (yes)
The default values of configurations: (no)
The wire protocol: (no)
The rest endpoints: (no)
The admin cli options: (no)
Anything that affects deployment: (no)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
If a feature is not applicable for documentation, explain why?
If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

Copy link
Member

@sijie sijie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@congbobo184 overall looks good. Can you please add tests?

@@ -461,11 +465,28 @@ public void getVersionBySchema(
}

private static GetSchemaResponse convertSchemaAndMetadataToGetSchemaResponse(SchemaAndMetadata schemaAndMetadata) {
String schemaData;
if (schemaAndMetadata.schema.getType() == SchemaType.KEY_VALUE) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add test cases for this code path as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add test cases for this code path as well?

OK, I will add the tests

@@ -461,11 +465,28 @@ public void getVersionBySchema(
}

private static GetSchemaResponse convertSchemaAndMetadataToGetSchemaResponse(SchemaAndMetadata schemaAndMetadata) {
String schemaData;
if (schemaAndMetadata.schema.getType() == SchemaType.KEY_VALUE) {
ByteBuf byteBuf = ByteBufAllocator.DEFAULT.heapBuffer().writeBytes(schemaAndMetadata.schema.getData());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think KeyValueSchemaInfo or KeyValueSchema provides the until functions to split the key schema and value schema. We should try to use those util functions rather than reimplementing the logic here again.

@sijie sijie added area/admin type/feature The PR added a new feature or issue requested a new feature labels Aug 5, 2019
@sijie sijie added this to the 2.5.0 milestone Aug 5, 2019
@congbobo184
Copy link
Contributor Author

run Integration Tests

@congbobo184
Copy link
Contributor Author

run cpp tests

@congbobo184
Copy link
Contributor Author

run java8 tests

@congbobo184
Copy link
Contributor Author

run Integration Tests

}

@Test(dataProvider = "schemas")
public void testSchemaInfoWithVersionApi(Schema<?> schema) throws Exception {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the difference with line 113?

@@ -236,4 +245,36 @@ public String getTopicName() {
}
});
}


public static String getKeyValueSchemaString(SchemaData schemaData) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@congbobo184

Use KeyValueSchemaInfo.decodeKeyValueSchemaInfo:

  • covert SchemaData to SchemaInfo
  • KeyValue<SchemaInfo, SchemaInfo> kvsSchemaInfo = KeyValueSchemaInfo.decodeKeyValueSchemaInfo(schemaInfo);
  • Use ObjectMapper to convert kvSchemaInfo into a json string.

The schemaInfo serializer and deserializer is already defined in SchemaUtils. You can modify it to handle KEY_VALUE correctly.

@congbobo184
Copy link
Contributor Author

run cpp tests

Copy link
Member

@sijie sijie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@congbobo184 it seems that you made changes to support key/value schema info. but you didn't add tests for key/value schema info. Can you please add tests for it?

@NoArgsConstructor
@Accessors(chain = true)
@Builder
public class KeyValueSchemaInfoData {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@congbobo any reason creating a new class? Can't you just KeyValue<Object, Object>?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@congbobo any reason creating a new class? Can't you just KeyValue<Object, Object>?

yes, there should use keyValue<Object, Object>, and then i will change it and add the test for it .

2.Delete keyValueSchemaInfoData then use KeyValue<Object, Object>
@sijie sijie merged commit f859da9 into apache:master Aug 16, 2019
@wolfstudy wolfstudy modified the milestones: 2.5.0, 2.4.2 Nov 19, 2019
@wolfstudy
Copy link
Member

Change the Milestone to 2.4.2, because of conflict.

wolfstudy pushed a commit that referenced this pull request Nov 20, 2019
### Motivation

To fix #4854 and support get keyValueSchema

(cherry picked from commit f859da9)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/admin type/feature The PR added a new feature or issue requested a new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot get a schema version
5 participants