[air] Add batch predictor class #23808
Conversation
|
I don't really like the |
python/ray/ml/scorer.py
Outdated
| batch_size: Split dataset into batches of this size for prediction. | ||
| max_scoring_actors: If set, specify the maximum number of scoring actors. | ||
| ray_remote_args: Additional resource requirements to request from | ||
| ray (e.g., num_gpus=1 to request GPUs for the map tasks). |
There was a problem hiding this comment.
I'm wondering if num_gpus should be a top-level arg, since GPU-based batch inference is pretty fundamental. 🤔
There was a problem hiding this comment.
Agreed, this should be easily accessible. I've added it to the top level API
| scorer = BatchScorer(DummyPredictor, Checkpoint.from_dict({"factor": 2.0})) | ||
|
|
||
| test_dataset = ray.data.from_items([1.0, 2.0, 3.0, 4.0]) | ||
| assert scorer.score(test_dataset).to_pandas().to_numpy().squeeze().tolist() == [ |
There was a problem hiding this comment.
If looking for a more concise conversion + comparison, content should be assertable without this chain, something like:
assert scorer.score(test_dataset).take() == [2.0, 4.0, 6.0, 8.0]There was a problem hiding this comment.
This leads to
E AssertionError: assert [{'value': 8.0}, {'value': 6.0}, {'value': 4.0}, {'value': 2.0}] == [2.0, 4.0, 6.0, 8.0]
how do I get them as a series and not dicts?
Also, the order problem seems to remain, any insights on that?
|
Regarding the naming, I don't think that "scoring" implies it's only 1 measure, but yeah, it's not ideal because we're not actually scoring a dataset, but perform inference on it. I wouldn't want to go with BatchPredictor or anything *Predictor, as we do have a top-level Predictor interface and this wrapper does not implement it. Maybe |
|
|
|
Actually, we can go with |
krfricke
changed the title
|
|
ericl
added
the
@author-action-required
krfricke
removed
the
@author-action-required
ericl
added
the
@author-action-required
| @@ -89,6 +89,9 @@ Predictors | |||
| .. autoclass:: ray.ml.predictor.Predictor | |||
| :members: | |||
|
|
|||
| .. autoclass:: ray.ml.batch_predictor.BatchPredictor | |||
amogkam
changed the title
Why are these changes needed?
What: This class adds a generic
BatchPredictorclass that offers an interface to run batch inference on Ray datasets. It takes a Predictor class and checkpoint as an input, and provides apredict(dataset)method to run scalable scoring inference.Why: Currently users have to implement scorers themselves. This is mostly boilerplate and prone to errors, so we should provide a simple solution instead.
Note that this predictor also implements the
Predictorinterface.Related issue number
Checks
scripts/format.shto lint the changes in this PR.