-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PI improvements #6229
PI improvements #6229
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor polish/docs, otherwise LGTM 👍
import java.util.concurrent.locks.ReentrantReadWriteLock; | ||
|
||
/** | ||
* This ParallelInference implementation provides inference functionality without launching additional threads, so inference happens in the calling thread |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
More javadoc - limitations, when to use.
Also how to build one (i.e., "use PI builder configured with X, not this class") - that might not be obvious to users at first glance (it wasn't for me)
} | ||
|
||
|
||
public static class ModelSelector { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make protected/private, unless there's a reason for it to be public?
BATCHED, | ||
|
||
/** | ||
* Inference will applied in the calling thread instead of workers. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
More javadoc - limitations, etc
ROUND_ROBIN, | ||
|
||
/** | ||
* in this mode we'll be |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
javadoc
WIP; DO NOT MERGE;
This PR adds: