Worker: Create article on dealing with rate-limits (using the blog article) #97

Open
frommww opened this Issue May 21, 2012 · 1 comment

Projects

None yet

1 participant

@frommww
Contributor
frommww commented May 21, 2012

Show how delay can be used to queue up lots of workers at once but use IW to manage the running of.

Assaf M.
14:10
@andrew - can you elaborate on the fb limits - and how I can make sure the ironworkers don't spawn too many to run into this wall OR alternatively, some suggest recovery/retry mechanism from this error?
BTW - 2nd error is also related to fb? or redis?
Andrew K.
14:11
redis
I need to look at fb errors...
Assaf M.
14:12
@travis - any suggestion on the killing of the stuck workers?
Assaf M.
14:15
@andrew ? @travis?
Ken F.
14:16
@assaf running some tests on the hud to verify. we had an issue before but it appears to have been acted on and closed.
Travis R.
14:17
hey sorry
Assaf M.
14:17
@ken you are talking regarding the stuck worker killing or my fb limit errors....?
Ken F.
14:17
stuck worker killing.
Travis R.
14:18
we'll look into the stuck worker thing and get back to you. Can you give me your email address so we can email you?
And regarding fb limits
Basically the same thing I just told Steve
try this: http://blog.iron.io/2012/02/rate-limiting-your-workers-to-play.html
Assaf M.
14:21
@travis thanks. Feel free to email me at assaf@fbhatchery.com. Please CC my server developer at andrey@fbhatchery.com.
Assaf M.
14:25
@travis I looked at the blog entry. Two questions in this regard: 1. do you happen to know where I can see the fb limits? and if they are "fixed"? 2. If we use the feature you recommended last time whereby we enqueue one job which they spawns child workers on its own --> How can we make sure it does not produce to many too fast?
Travis R.
14:26
will it be one worker?
doing the spawning?
Assaf M.
14:30
we have three different worker tasks started in parallel. One is a single task. The two others spawn tasks based on the number of fb friends you have. Currently we do the spawning by setting each task with between 5 and 50 fb friends to handle. However, we are looking at enqueueing one worker, giving it the list of friends, and having it spawn child workers to handle the list. This latest optimization may be problematic to control, given the fb limits and a user with 1000+ friends.....
Here's a piece of code we use for one of the tasks:
Assaf M.
14:31
def enqueue_get_connections
%w(likes groups events).each do |connection_type|
([facebook_id] + friends_and_fofs).tap do |people_|
people_.each_slice(50) do |friends_ids_slice|
JustaWorker.new.tap do |j|
j.action_ = :get_connections
...
Show full text
Travis R.
14:32
are those started on a schedule?
Assaf M.
14:33
Yes and no - they are part of initial user setup AND later are ran periodically.
the code example is one of the three tasks
Travis R.
14:34
would using delay like in that blog post work for your situation?
Assaf M.
14:37
Well, If I continue to "manually" do the spawning of workers as above, I can either delay after spawning the X worker, or divide the no. of friends to slices ensuring a number of workers < X. In both cases, I may lose some critical time. In both cases, I still manually enqueue the workers, losing some critical time on the enqueueing process itself. In the latter case, I prefer to enqueue a single worker and have it spawn the child-workers, but somehow control their number via embedded delay or some other. What do you think?
Travis R.
14:39
you don't need to delay after spawning the worker, just use the "delay" field
Assaf M.
14:40
Not following you
Travis R.
14:41
well in your example above
j.enqueue delay: 10 * j, recursive: true
for instance
will ensure that only one gets run every 10 seconds
you can queue them all up as fast as you want

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment