Skip to content

Conversation

@lostluck
Copy link
Contributor

@lostluck lostluck commented Aug 5, 2020

Adds initial Schema to Go type conversions and vice versa.

In particular, handles the "easy" stuff: basic struct to Row conversion, Slices, and maps, basic field name conversions.

Doesn't implement struct+schema compatibility comparisons, which are necessary at pipeline construction time, however, this is more than sufficient to allow other immediately incoming code to unlock the row coder tests in standard_coders.yaml.


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Choose reviewer(s) and mention them in a comment (R: @username).
  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang SDK Dataflow Flink Samza Spark Twister2
Go Build Status --- Build Status --- Build Status ---
Java Build Status Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status Build Status
Build Status
Build Status
Build Status
Python Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
--- Build Status ---
XLang Build Status --- Build Status --- Build Status ---

Pre-Commit Tests Status (on master branch)

--- Java Python Go Website
Non-portable Build Status Build Status
Build Status
Build Status
Build Status Build Status
Portable --- Build Status --- ---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels

See CI.md for more information about GitHub Actions CI.

@probot-autolabeler probot-autolabeler bot added the go label Aug 5, 2020
@lostluck
Copy link
Contributor Author

lostluck commented Aug 5, 2020

R: @youngoli

Copy link
Contributor

@youngoli youngoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just some small comments not worth blocking on.

@lostluck lostluck merged commit 8d03dc6 into apache:master Aug 6, 2020
}, nil
case reflect.Interface, reflect.Chan, reflect.UnsafePointer, reflect.Complex128, reflect.Complex64:
return nil, errors.Errorf("unable to convert unsupported type %v to schema", ot)
default: // must be an atomic type
Copy link
Contributor

@youngoli youngoli Aug 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the change, and the full error propogation is even better than my original suggestion, so two thumbs up for that.

But I think you missed my suggestion to merge these unsupported type and default cases. I.E. combine them into one default case that tries using reflectTypeToAtomicTypeMap, and if that fails, then it's an unsupported type.

Having a specific case for unsupported types seems brittle and doesn't serve much of a purpose other than having a slightly different error message than the one under default. If one of these types gets added, this is just one more spot that needs to be changed and will break if you forget.

Obviouslt it's not so important it needs an immediate fix, but I'd say it's worth bundling into whatever schema PR is coming next.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants