Skip to content

[FEATURE] Slack App Improvements #15953

@danhsiung

Description

@danhsiung

Is your feature request related to a problem? Please describe.
Slack API calls are subject to rate limits, and we need to improve our handling of those limits

  1. Pagination: Slack API calls that support pagination are rate limited more heavily when called without limits. Unfortunately the default behavior for some of these calls (e.g. users.list) is to make a non-limited request. We should, by default, always pass a reasonable limit in our requests to endpoints that support pagination. Example fix
  2. SDK Rate Limit Handling: By default, the Slack Node SDK handles rate limited requests by blocking for N seconds (N is determined by the server's 429 backoff response), and then retrying the request.
    Anyc Options: We should have our default behavior be rejectRateLimitedCalls for any async options calls, and display a notification to the user.
    Workflow Executions: we may want to block and retry workflow executions automatically, but we should handle that explicitly, and try to notify the user when it happens.
  3. UserNames: The userNames() function recursively calls users.list until it retrieves the full list of users in a workspace. We should not be using this function anymore.
    Listing Users: For the list-users action (and any other actions/sources that use the users/list API endpoint), we should instead be calling with pagination, default to only retrieving one page, and providing an option to retrieve subsequent pages (but only if the user configures it).
    Resolving User Names from ID's: In most cases we're currently calling userNames(), it's to translate Slack User ID's to human readable User Names. In these cases, we should call users.profile.get if we have a small number of users to look up, or provide a function that pages through users.list calls and looks for a given set of UserID's, stopping when they are all resolved.

Task List:

  • Find all API calls we're making that support pagination, and make sure we default to using pagination
  • Switch our sdk client to rejectRateLimitedCalls by default.
  • For async options calls, handle rate limits by notifying the user that they've been rate limited
  • For workflow executions, handle rate limits explicitly by catching rate limit errors and sleeping (e.g. using setTimeout), but notify user (e.g. via logging to standard output) that execution was delayed for this reason.
  • (Nice to Have) Use $.flow.rerun to pause and retry rather than setTimeout.
  • (Nice to Have) Provide an option to fail and notify on workflow executions that hit rate limits.
  • Remove the userNames() function
  • Change list-users (and any other components that use users.list) to use pagination, and provide options for page size and number of pages to retrieve (defaulting to 1 page, and a reasonable page size).
  • Where possible/reasonable, use users.profile.get (tier 4 rate limit) instead of users.list (tier 2 rate limit)
  • If still needed, provide a helper function userNameLookup(list_of_ids) which pages through users.list, and returns a map of ID's to Names, stopping when all ID's are found. By default, this function should immediately fail if it's rate limited.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions