Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: indicate header columns in source descriptions #8475

Merged
merged 2 commits into from
Dec 10, 2021
Merged

Conversation

jzaralim
Copy link
Contributor

@jzaralim jzaralim commented Dec 8, 2021

Description

DESCRIBE statements will indicate which columns are headers, for example

 Field  | Type                                                      
--------------------------------------------------------------------
 ROWKEY | VARCHAR(STRING)  (primary key)                            
 f_0    | BOOLEAN                                                   
 f_1    | INTEGER                                                   
 f_2    | BIGINT                                                    
 f_3    | DOUBLE                                                    
 f_4    | VARCHAR(STRING)                                           
 f_5    | ARRAY<VARCHAR(STRING)>                                    
 f_6    | MAP<STRING, BIGINT>                                       
 f_7    | STRUCT<a DOUBLE>                                          
 HEAD   | ARRAY<STRUCT<KEY VARCHAR(STRING), VALUE BYTES>> (headers) 
--------------------------------------------------------------------

Testing done

Manual, unit, console tests

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")

@jzaralim jzaralim requested a review from a team as a code owner December 8, 2021 23:27
@jzaralim jzaralim requested a review from spena December 8, 2021 23:27
@@ -31,22 +31,26 @@

public enum FieldType {
SYSTEM, // To be removed in the future. 0.9 saw this value no longer used.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be removed, right? I don't see being used either.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose of keeping this was so that pre-0.9 servers could still work with newer CLIs. I'm not sure how many people are still using 0.9, but I think this is easy enough to just keep anyway. This does raise a good point though - the changes in this PR would make HEADERS require the newest CLI. Are we okay with that?


@JsonCreator
public FieldInfo(
@JsonProperty(value = "name", required = true) final String name,
@JsonProperty(value = "schema", required = true) final SchemaInfo schema,
@JsonProperty("fieldType") final Optional<FieldType> type
@JsonProperty("fieldType") final Optional<FieldType> type,
@JsonProperty("headerKey") final Optional<String> headerKey
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems this field is optional in different places. What if you create another constructor that defaults to Optional.empty()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


Queries that read from this TABLE
-----------------------------------
readId (ERROR) : read query
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this ERROR ok?

Copy link
Contributor Author

@jzaralim jzaralim Dec 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, all the source description tests have query errors to test printing errors

Comment on lines 507 to 512
final String headerType;
if (field.getHeaderKey().isPresent()) {
headerType = "(header('" + field.getHeaderKey().get() + "'))";
} else {
headerType = "(headers)";
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

final String headerType = field.getHeaderKey()
  .map(k -> "(header('" + k + "'))")
  .orElse(() -> "(headers)");

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@jzaralim jzaralim merged commit 065de82 into master Dec 10, 2021
@jzaralim jzaralim deleted the describe branch December 10, 2021 03:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants