Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faster fetching with parallel API request #34

Merged
merged 2 commits into from
Jun 24, 2015
Merged

Conversation

uu59
Copy link
Contributor

@uu59 uu59 commented Jun 24, 2015

Jiralicious.search invokes HTTP request 3 times per issue, if you get 50 issues from search API, you will finally do 150 requests to JIRA API.
This PR solves that problem by parallelize HTTP requests with parallel gem.

Before

$ /usr/bin/time bundle exec embulk preview -L ./ guessed.yml > /dev/null
      678.17 real        41.11 user         3.16 sys

After (with 50 threads)

$ /usr/bin/time bundle exec embulk preview -L ./ guessed.yml > /dev/null
       30.08 real        28.97 user         1.71 sys

@yoshihara
Copy link
Contributor

LGTM

uu59 added a commit that referenced this pull request Jun 24, 2015
Faster fetching with parallel API request
@uu59 uu59 merged commit 94ffa99 into master Jun 24, 2015
@uu59 uu59 deleted the faster_issue_fetching branch June 24, 2015 07:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants