Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement] Add support for batches #78

Open
KennethEnevoldsen opened this issue Oct 14, 2020 · 0 comments
Open

[Improvement] Add support for batches #78

KennethEnevoldsen opened this issue Oct 14, 2020 · 0 comments

Comments

@KennethEnevoldsen
Copy link
Contributor

Currently, the framework does not seem to allow for batching inputs, which reduce its usability on larger datasets.

The intuitive approach would be:

from danlp.models import load_bert_tone_model
classifier = load_bert_tone_model()
classifier.predict(["I am very happy", "I am very very happy"])
# {'analytic': 'objective', 'polarity': 'positive'}

While you would expect:

from danlp.models import load_bert_tone_model
classifier = load_bert_tone_model()
classifier.predict(["I am very happy", "I am very very happy"], batch_size=2)
#[{'analytic': 'objective', 'polarity': 'positive'},
# {'analytic': 'objective', 'polarity': 'positive'}]

The reason why you would add the batch_size=2 is to distinguish between looping through each text and batching them for faster computation on GPUs

Interestingly the first approach does not throw an error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant