Proposal: Octopoller - here to solve all your polling needs

[BenEmdon]

Octopoller 🦑

This is a proposal to add Octopoller to octokit.
Feedback/ideas would be appreciated 😄

The code for Octopoller can be found in education/classroom at: https://github.com/education/classroom/blob/master/lib/octopoller.rb

What is Octopoller

I created Octopoller, a polling micro module for education/classroom to make repeating requests to the GitHub API. After adding it to classroom @tarebyte said in a review education/classroom#1375 (comment):

I'd be interested in having this inside of https://github.com/octokit/octokit.rb 😄

After refining Octopoller since that initial PR I feel its ready to move over to octokit. Either inside octokit.rb or as a tiny gem under the octokit org.

What was Octopoller used for

Octopoller was created for the purpose of polling the Source Imports API for a repo's import status. Octopoller is now the backbone of classrooms import starter code process.

Show me the code

Octopoller exposes a single function poll. Here is what what the API looks like:

# Polls until success
# Re-runs when the block returns `:re_poll`
#
# wait      - The time delay in seconds between polls (default is 1 second)
#           - When given the argument `:exponentially` the action will be retried with exponetial backoff
# timeout   - The maximum number of seconds the poller will execute
# retries   - The maximum number of times the action will be retried
# yield     - A block that will execute, and if `:re_poll` is returned it will re-run
#           - Re-runs until something is returned or the timeout/retries is reached
# raise     - Raises an Octopoller::TimeoutError if the timeout is reached
# raise     - Raises an Octopoller::TooManyAttemptsError if the retries is reached
#
def poll(wait: 1.second, timeout: nil, retries: nil)

Using it might look like this:

Octopoller.poll(timeout: 15.seconds) do 
  begin
    make_that_request() # raises an Error sometimes
  rescue Error
    :re_poll
  end
end

# => { "status" => 200,  "body" => "🦑" }

Usage

Octopoller has 3 use cases:

• Poll something with a set timeout
• Poll something with a set number of retries
• Poll something with exponential backoff

Here's what using Octpoller is like for each of the use cases listed above:

• Poll with a timeout:

  Octopoller.poll(timeout: 15.seconds) do 
    puts "🦑"
    :re_poll
  end
  
  # => "🦑"
  # => "🦑"
  # ... (for 15 seconds)
  # Timed out patiently (Octopoller::TimeOutError)

• Poll with retries:

  Octopoller.poll(retries: 2) do 
    puts "🦑"
    :re_poll
  end
  
  # => "🦑"
  # => "🦑"
  # => "🦑"
  # Tried maximum number of attempts (Octopoller::TooManyAttemptsError)

• Poll with exponential backoff:

  start = Time.now
  Octopoller.poll(wait: :exponentially, retries: 4) do 
    puts Time.now - start
    :re_poll
  end
  
  # => 0.5 seconds
  # => 1 second
  # => 2 seconds
  # => 4 seconds
  # => 8 seconds
  # Tried maximum number of attempts (Octopoller::TooManyAttemptsError)

What benefit is this to the repo/org?

Octopoller is perfect for developers who need to poll an endpoint or attempt a something a set number of times until success. Octopoller is also unopinionated such that any ruby developer could use this.

Maintainability and work

Octopoller is a micro module and will change very little. I am happy to do all the work to write documentation (and possibly publish as a ruby gem).

[BenEmdon]

I drafted up a ruby gem repo for you two to look at:
https://github.com/BenEmdon/octopoller

CC: @kytrinyx @tarebyte

[BenEmdon]

Friendly bump 🙂 @tarebyte @kytrinyx

[kytrinyx]

@BenEmdon thanks so much for this!

I chatted with @tarebyte about this and we have a few thoughts.

We think that the behavior is very useful, and should be made available to folks using octokit.

The current project that I'm working on in the GitHub REST API, is going to allow us to do a bunch of code generation in this gem. As part of that we're going to want to take a step back and think about the overall architecture of the gem. Since we have a few things up in the air right now, we don't want to add octopoller to octokit.rb directly, just yet, though we think that is what should happen once we get our duckies in a row.

How would you feel about the following:

• transfer octopoller to the octokit org
• rename it to octopoller.rb
• update the README of the octokit.rb gem to refer to the octopoller gem

You would remain the main maintainer on the octopoller gem, but for redundancy, we'd probably want to add both me and tarebyte to the repo, and give us both the ability to cut new releases and publish them to rubygems.

[BenEmdon]

@kytrinyx @tarebyte thanks for considering Octopoller. I feel very good with adding the gem to the Octokit org and renaming it to octopoller.rb!

I have some question about how we should move forward:

• Does this warrant adding me to the octokit org? If so would someone mind adding me?

update the README of the octokit.rb gem to refer to the octopoller gem

• Would this entail a description of what octopoller is? Or just a link?

• Should the ruby gem name (on rubygems.org) change? Currently the ruby gem name on rubygems.org is octopoller (similar to how octokit.rb's ruby gem name is octokit). I will look into how I can transfer the ownership of the ruby gem on rubygems.org.

[tarebyte]

• Does this warrant adding me to the octokit org? If so would someone mind adding me?

Sure does, you have an invite in your inbox 😄

Would this entail a description of what octopoller is? Or just a link?

Probably it's own section talking about when polling would be useful and using the gem as an example with a link.

Should the ruby gem name (on rubygems.org) change? Currently the ruby gem name on rubygems.org is octopoller (similar to how octokit.rb's ruby gem name is octokit). I will look into how I can transfer the ownership of the ruby gem on rubygems.org.

I think the current naming is fine.

[BenEmdon]

I tried to transfer ownership of octopoller.rb to the octokit org but I was met with this error message:

You don’t have the permission to create repositories on octokit

@tarebyte how should I transfer ownership?

[kytrinyx]

how should I transfer ownership?

It looks like this is sorted, but for the future: you can only transfer to (a) another user, or (b) an org you are admin of. My standard approach is to transfer to a user who is an admin, who then transfers it on to the org :)

[BenEmdon]

Since octopoller.rb is already under the management of octokit I think we are good to close this proposal 🎉
I have made issues with the remaining actions for ownership transfer:

• Figure out how to transfer rubygems.org ownership octopoller.rb#3
• Write small section on octokit/octokit.rb  octopoller.rb#4