Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhancement] IncrementalBasicStatistics interface #1630

Merged
merged 21 commits into from
Apr 11, 2024

Conversation

olegkkruglov
Copy link
Contributor

@olegkkruglov olegkkruglov commented Jan 2, 2024

Changes proposed in this pull request:

  • Added IncrementalBasicStatistics interface to onedal and sklearnex Python modules

Copy link
Contributor

@samir-nasibli samir-nasibli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please separate BasicStatistics refactoring and IncrementalBasicStatistics addition

@olegkkruglov
Copy link
Contributor Author

Please separate BasicStatistics refactoring and IncrementalBasicStatistics addition

done

@olegkkruglov
Copy link
Contributor Author

/intelci: run

@olegkkruglov
Copy link
Contributor Author

/intelci: run

Copy link

@Alexandr-Solovev Alexandr-Solovev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general LGTM with minor comments

@olegkkruglov olegkkruglov changed the title IncrementalBasicStatistics interface for onedal4py/sklearnex [enhancement] IncrementalBasicStatistics interface Jan 24, 2024
@olegkkruglov olegkkruglov added enhancement New feature or request examples Modify examples testing Tests for sklearnex/daal4py/onedal4py & patching sklearn labels Jan 24, 2024
@olegkkruglov
Copy link
Contributor Author

/intelci: run

1 similar comment
@olegkkruglov
Copy link
Contributor Author

/intelci: run

@samir-nasibli samir-nasibli dismissed their stale review April 8, 2024 16:04

Request is addressed. Review is required

Copy link
Contributor

@ethanglaser ethanglaser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly minor revisions needed here - seems to be close to ready

Comment on lines +43 to +45
print(f"Mean:\n{result.mean}")
print(f"Max:\n{result.max}")
print(f"Sum:\n{result.sum}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can make this more specific: "Mean via automatic splitting via batch size:"

onedal/basic_statistics/basic_statistics.cpp Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great coverage here

self._need_to_finalize = False

def _onedal_partial_fit(self, X, weights, queue):
first_pass = not hasattr(self, "n_samples_seen_") or self.n_samples_seen_ == 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are first_pass and n_samples_seen_ needed for incremental basic stats?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first_pass is used below. and n_samples_seen_ is added for conformance. I admit there can be scenarios where user needs it.

@ethanglaser
Copy link
Contributor

/intelci: run

@olegkkruglov
Copy link
Contributor Author

/intelci: run

@olegkkruglov olegkkruglov merged commit 88fc7a2 into intel:main Apr 11, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request examples Modify examples testing Tests for sklearnex/daal4py/onedal4py & patching sklearn
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants