Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: do not spawn a thread for num_thread=1 #216

Merged

Conversation

vikmary
Copy link

@vikmary vikmary commented Apr 16, 2021

The spawning of a single thread on my computer produces a huge time overhead.

Thus I propose to add an if clause that would lead to no multithreading when num_thread=1 is used. This is already implemented for the case when multiple inputs are used, but isn't for the case when a single string is being input to .augment().

I have benchmarked with the following code:

import random
import time
import nlpaug.augmenter.char as nac

aug = nac.OcrAug()
times = []
for i in range(100):
    start = time.time()
    aug.augment(''.join(random.choice('qwertyuiopsdfghj klzxcv bnm,') for _ in range(random.randint(10, 30))))
    times.append(time.time() - start)
print(sum(times) / len(times))

For the dev version of nlpaug it results in the delta time of 0.10184303998947143, while the proposed version results in 9.496212005615235e-05, which makes huge difference.

@vikmary vikmary changed the title feat: do not spawn a thread in augmenter feat: do not spawn a thread for num_thread=1 Apr 16, 2021
@makcedward makcedward merged commit d6dc361 into makcedward:dev Apr 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants