New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[tf.data] move StructuredFunctionWrapper
into a separate module
#51947
[tf.data] move StructuredFunctionWrapper
into a separate module
#51947
Conversation
8b24120
to
ebee4b5
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @kvignesh1420.
Regarding the circular dependency with DatasetV2
, lazy-loading DatasetV2
at the start of each dataset transformation sgtm.
StructuredFunctionWrapper
into a common moduleStructuredFunctionWrapper
into a separate module
@aaudiber I have the changes to the current PR. Please take a look. P.S: Thanks for the suggestions related to the future PRs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @kvignesh1420, one last comment and this LGTM
efc4dea
to
c6dd3d3
Compare
refactor using structured_function module use the new API location of structured_function move DEBUG_MODE to dataset_ops
c6dd3d3
to
0224cb4
Compare
@aaudiber any updates on this? Please let me know if any changes are required. Thanks! |
This PR refactors
dataset_ops.py
by moving theStructuredFunctionWrapper
class into acommon
module. This enables further refactoring of function-specificDataset
s.side note for future PR's: after further examination of the function-specific
Dataset
classes (for ex:MapDataset
), I see that they inherit fromUnaryDataset
which itself inherits fromDatasetV2
. And since theDatasetV2
class itself needsMapDataset
for thedef map()
functionality, this poses a circular dependency challenge even if we split the classes into different files. Thus, lazy loading would be imminent and the files might get a bit messy.cc: @aaudiber What do you think?