Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Data] Make name a required argument for AggregateFn #44880

Merged
merged 4 commits into from
Apr 22, 2024

Conversation

scottjlee
Copy link
Contributor

@scottjlee scottjlee commented Apr 19, 2024

Why are these changes needed?

Currently, name is an optional arg for AggregateFn, which can cause errors when the user defines a custom AggregateFn without a name. For example, here we will get an empty row due to the AggregateFn.name not being set.

This PR makes the name parameter required for AggregateFn, which resolves the issue.

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

Signed-off-by: Scott Lee <sjl@anyscale.com>
Signed-off-by: Scott Lee <sjl@anyscale.com>
accumulate_row: Callable[[AggType, T], AggType] = None,
accumulate_block: Callable[[AggType, Block], AggType] = None,
finalize: Callable[[AggType], U] = lambda a: a,
name: Optional[str] = None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did AggregateFn always error if name isn't specified? If there are cases where AggregateFn worked without a name, this'd be a breaking change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all the existing constructor usages in ray specify a name. i think the error occurs when we call aggregate on a GroupedDataset without providing a name.

if there are external users using this without a name, it could potentially break, but i think throwing a TypeError specific for this input should suffice.

accumulate_row: Callable[[AggType, T], AggType] = None,
accumulate_block: Callable[[AggType, Block], AggType] = None,
finalize: Callable[[AggType], U] = lambda a: a,
name: Optional[str] = None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should also add a check. Changing the type hint doesn't have any effect at runtime.

Signed-off-by: Scott Lee <sjl@anyscale.com>
Signed-off-by: Scott Lee <sjl@anyscale.com>
@bveeramani bveeramani merged commit 1ccf925 into ray-project:master Apr 22, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants