Skip to content

[BEAM-1340] Add __all__ tags to modules in package apache_beam/transforms#3075

Closed
charlesccychen wants to merge 3 commits intoapache:masterfrom
charlesccychen:fix-all-1
Closed

[BEAM-1340] Add __all__ tags to modules in package apache_beam/transforms#3075
charlesccychen wants to merge 3 commits intoapache:masterfrom
charlesccychen:fix-all-1

Conversation

@charlesccychen
Copy link
Contributor

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

  • Make sure the PR title is formatted like:
    [BEAM-<Jira issue #>] Description of pull request
  • Make sure tests pass via mvn clean verify.
  • Replace <Jira issue #> in the title with the actual Jira issue
    number, if there is one.
  • If this contribution is large, please file an Apache
    Individual Contributor License Agreement.

@charlesccychen
Copy link
Contributor Author

R: @robertwb



__all__ = [
'DoFnContext',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DoFnContext should not be public.

'DoFnContext',
'DoFnProcessContext',
'DoFn',
'CallableWrapperDoFn',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CallableWrapper* classes shouldn't be public.

'CombineGlobally',
'CombinePerKey',
'CombineValues',
'CombineValuesDoFn',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CombineValuesDoFn should not be public.

'CombineValues',
'CombineValuesDoFn',
'GroupByKey',
'ReifyWindows',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ReifyWindows, GroupAlsoByWIndows, and GroupByKeyOnly should not be public.

'GroupAlsoByWindow',
'GroupByKeyOnly',
'Partition',
'ApplyPartitionFnFn',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ApplyPartitonFn should not be public.



__all__ = [
'GetPValues',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only exported things from here should be PTransform and possibly label_from_callable. Maybe ptransform_fn, but mark it clearly as experimental (CloudML had some closure issues that were never fully resolved). The rest are internal implementation details.



__all__ = [
'default_window_mapping_fn',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't expose anything from this module (yet).



__all__ = [
'TimeDomain',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only TimeDomain should be exported from this module.

'AfterAll',
'AfterEach',
'OrFinally',
'TriggerContext',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TriggerContext and below are an implementation details.

'FixedWindows',
'SlidingWindows',
'Sessions',
]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aaltay Was the intent to export WindowedValue from here rather than from utils?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or should we keep it Experimental for now (it's not an exposed concept in Java).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should move WindowedValue out of util to here and also mark it as experimental. (cc: @sb2nov)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marked it is experimental in #3087

@charlesccychen
Copy link
Contributor Author

Thanks, PTAL.

Copy link
Contributor

@robertwb robertwb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one comment, otherwise LGTM.

def ptransform_fn(fn):
"""A decorator for a function-based PTransform.

For internal use only; no backwards-compatibility guarantees.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not for internal use, just note that it's expiremental (and, yes, not backwards compatible).

from apache_beam.transforms import window


__all__ = []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may cause lint errors. Remove.

@robertwb
Copy link
Contributor

robertwb commented May 11, 2017

[ignore]

@charlesccychen charlesccychen changed the title [BEAM-1340] Add __all__ tags to modules in package apache_beam/tranforms [BEAM-1340] Add __all__ tags to modules in package apache_beam/transforms May 11, 2017
@charlesccychen
Copy link
Contributor Author

Thanks, PTAL.

@robertwb
Copy link
Contributor

LGTM

@robertwb
Copy link
Contributor

There were a couple of trivial test failures (e.g. due to no beam.GroupByKeyOnly). Fixing as part of the merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants