Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merge ModelLoader and InferenceRunner into same class. #21795

Merged
merged 8 commits into from
Jun 10, 2022
Merged

Merge ModelLoader and InferenceRunner into same class. #21795

merged 8 commits into from
Jun 10, 2022

Conversation

robertwb
Copy link
Contributor

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests

See CI.md for more information about GitHub Actions CI.

@robertwb
Copy link
Contributor Author

R: @yeandy , @ryanthompson591 or @TheNeuralBit

TheNeuralBit
TheNeuralBit approved these changes Jun 10, 2022
View reviewed changes
sdks/python/apache_beam/ml/inference/base.py Outdated
class InferenceRunner(Generic[ExampleT, PredictionT, ModelT]):
"""Implements running inferences for a framework."""
class ModelHandler(Generic[ExampleT, PredictionT, ModelT]):
"""Has the ability to load an ML model."""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe update this docstring?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly for the implementations, consider if the the docstrings need to be merged.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. Done.

ryanthompson591
ryanthompson591 approved these changes Jun 10, 2022
View reviewed changes
sdks/python/apache_beam/ml/inference/base.py Outdated
class InferenceRunner(Generic[ExampleT, PredictionT, ModelT]):
"""Implements running inferences for a framework."""
class ModelHandler(Generic[ExampleT, PredictionT, ModelT]):
"""Has the ability to load an ML model."""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has the abiltity to load ML models and run inferences on that model.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

sdks/python/apache_beam/ml/inference/sklearn_inference.py
@@ -26,8 +26,7 @@

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might have to merge this file since I just merged a change. Should be straight forward.

sdks/python/apache_beam/ml/inference/sklearn_inference_test.py Outdated
@@ -71,7 +70,7 @@ def tearDown(self):

def test_predict_output(self):
fake_model = FakeModel()
inference_runner = SklearnInferenceRunner()
inference_runner = SklearnModelHandler('unused')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SklearnModelHandler(uri='unused')?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

yeandy
yeandy approved these changes Jun 10, 2022
View reviewed changes
Copy link
Contributor

@yeandy yeandy left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't add a Github comment since these lines weren't changed, but can we change the name test_inference_runner_X to something like test_model_handler_X in the pytorch_inference.py file? Lines 142, 169, 182, 218

sdks/python/apache_beam/ml/inference/sklearn_inference.py Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Jun 10, 2022 •
edited

Codecov Report

Merging #21795 (3eaccaf) into master (913c81e) will increase coverage by 0.03%.
The diff coverage is 51.42%.

❗ Current head 3eaccaf differs from pull request most recent head 758bc7a. Consider uploading reports for the commit 758bc7a to get more accurate results

@@            Coverage Diff             @@
##           master   #21795      +/-   ##
==========================================
+ Coverage   74.01%   74.05%   +0.03%     
==========================================
  Files         698      698              
  Lines       92224    92330     +106     
==========================================
+ Hits        68263    68377     +114     
+ Misses      22710    22702       -8     
  Partials     1251     1251
Flag Coverage Δ
python 83.64% <51.42%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...thon/apache_beam/ml/inference/pytorch_inference.py 0.00% <0.00%> (ø)
sdks/python/apache_beam/ml/inference/base.py 94.30% <88.88%> (+0.60%) ⬆️
sdks/python/apache_beam/ml/inference/api.py 90.00% <100.00%> (ø)
...thon/apache_beam/ml/inference/sklearn_inference.py 91.66% <100.00%> (-0.84%) ⬇️
...eam/transforms/py_dataflow_distribution_counter.py 91.42% <0.00%> (-4.87%) ⬇️
sdks/python/apache_beam/utils/counters.py 85.39% <0.00%> (-1.36%) ⬇️
...eam/runners/portability/fn_api_runner/execution.py 92.44% <0.00%> (-0.65%) ⬇️
...hon/apache_beam/runners/worker/bundle_processor.py 93.42% <0.00%> (-0.25%) ⬇️
...ks/python/apache_beam/runners/worker/sdk_worker.py 88.94% <0.00%> (-0.16%) ⬇️
sdks/python/apache_beam/transforms/window.py 87.20% <0.00%> (+0.15%) ⬆️
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 913c81e...758bc7a. Read the comment docs.

@robertwb
Copy link
Contributor Author

Run PythonLint PreCommit

@robertwb robertwb merged commit 997623d into apache:master Jun 10, 2022
@Abacn
Copy link
Contributor

Abacn commented Jun 10, 2022

Seems Unit tests are brocken:

self = <apache_beam.ml.inference.sklearn_inference_test.SkLearnRunInferenceTest testMethod=test_infer_invalid_data_type>
    def test_infer_invalid_data_type(self):
      with self.assertRaises(ValueError):
        unexpected_input_type = [[1, 2, 3, 4], [5, 6, 7, 8]]
>       inference_runner = SklearnModelLoader(model_uri=unused)
E       NameError: name 'SklearnModelLoader' is not defined

bullet03 pushed a commit to akvelon/beam that referenced this pull request Jun 20, 2022
@robertwb @bullet03
Merge ModelLoader and InferenceRunner into same class. (apache#21795)
4d617dd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@TheNeuralBit TheNeuralBit TheNeuralBit approved these changes

@ryanthompson591 ryanthompson591 ryanthompson591 approved these changes

@yeandy yeandy yeandy approved these changes

Assignees
No one assigned
Labels
python
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@robertwb @Abacn @TheNeuralBit @ryanthompson591 @yeandy