-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Spark: Refactor SparkReadConf use primitive type for confs with default values #7429
Spark: Refactor SparkReadConf use primitive type for confs with default values #7429
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly looks good to me. But wanted to double check, is it a backward compatible change?
User will ever get NoSuchMethodError?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good to me
Under what case would user have this error? @szehon-ho |
we can get the values in both long, Long, so using the function call directly would not be problematic but let's say if some one is extending SparkReadConf and over-riding the Would like to know your thoughts and make the changes accordingly, as |
Yea I am thinking, if our caller does not recompile their library on newer Iceberg version , they will hit it, but I think its not the normal flow. Thanks for double checking. |
Thanks, @singhpk234! |
Thanks @singhpk234 ! i was thinkiing about @szehon-ho point , I don't think we can set an expectation that people can extend SparkReadConf and have signatures align. So overall, this change looks good to me |
Co-authored-by: Prashant Singh <psinghvk@amazon.com>
About Change
Address review feedback #7422 (comment) .
cc @aokolnychyi