While Oceana Sleeps

Changes

This release adds an opt-in way to pace requests for multi-threaded use, makes connection handling faster, and fixes an endless-retry problem when listing many pages.

Big thanks to @ajslater for all his help with this release.

🚀 Features

• Reuse a single requests.Session for connection pooling @bpepple (#169)
• Add opt-in rate_limiter pacing gate to Session @bpepple (#167)

🐛 Bug Fixes

• Bound pagination 429 retries and defer to the rate_limiter @bpepple (#168)

Highlights for threaded / concurrent use

New rate_limiter option that waits instead of racing.
By default, mokkari's rate-limit check is advisory. With several threads, a few requests can get past it before the X-RateLimit-* headers show the limit is used up. You can now pass a rate_limiter to mokkari.api(). Every request goes through it before it is sent. If there's no capacity, the caller blocks until there is.

from concurrent.futures import ThreadPoolExecutor

import mokkari
from mokkari.rate_limit import HeaderPacedRateLimiter

m = mokkari.api(username, password, rate_limiter=HeaderPacedRateLimiter())

with ThreadPoolExecutor(max_workers=20) as executor:
    issues = list(executor.map(m.issue, issue_ids))

• HeaderPacedRateLimiter is the ready-made implementation. It reads the per-minute limit from Metron's headers and spaces requests evenly. At the 20/min minimum that is one every 3 seconds. It keeps its own monotonic clock, so a wrong system clock doesn't affect it.
• If Metron still returns a 429, every waiting thread backs off for the server's Retry-After.
• The daily limit is not waited out. When it is used up, acquire() raises RateLimitError with retry_after set, because the wait could be hours. Your app can then choose to wait or quit. The wait is a minimum, not a guarantee. If you retry after it and get another 429, the limiter backs off again.
• You can supply your own limiter by implementing the new RateLimiter protocol (acquire, release, on_rate_limited).
• Without a rate_limiter, behavior is unchanged.

Connections are now reused (#169).
Each call used to open and close its own connection, so every request paid a new TCP and TLS handshake. Session now keeps one pooled connection set. This helps most with repeated or threaded calls.

• The pool holds up to 32 connections, so threaded callers don't hit urllib3's "Connection pool is full" warnings.

• Cookies are never stored or sent back, so authentication is unchanged: Basic or Bearer on every request.

• Session can now be used as a context manager, and it has a close() method. Closing is optional, and a closed session reopens connections when you use it again:

  with mokkari.api(api_token="your-token") as m:
      issue = m.issue(1)

• close() only releases HTTP connections. A cache you passed in is left open.

• Don't share one Session across forked processes. Create one per process.

• Rarely, if the server closes a connection at the moment it is reused, that request raises ApiError. mokkari doesn't retry it automatically.

Fixes

• List calls no longer retry a 429 forever (#168). When a paginated list call hit a 429 with no Retry-After, it could loop indefinitely. It now raises RateLimitError after 3 header-less 429s in a row. It also raises after 20 consecutive 429s on the same page when Retry-After is present. The count resets after any successful page.
• Pagination no longer sleeps on top of the rate_limiter. With a rate_limiter, list calls let the limiter pace the retry. Before, they slept as well and waited twice. A RateLimitError that the limiter raises itself, such as for the daily limit, now reaches your code. It used to be swallowed and slept on.
• Without a rate_limiter, pagination sleeps exactly the server's Retry-After. It no longer adds 2 seconds, because Metron always sends the header now.
• Fixed a rate-limiter slot leak. Some request errors, such as too many redirects, could leave a slot marked in use, which eventually blocked later requests.