Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add support for user limits #93

Merged
merged 3 commits into from
Sep 17, 2019
Merged

[WIP] Add support for user limits #93

merged 3 commits into from
Sep 17, 2019

Conversation

jcrist
Copy link
Member

@jcrist jcrist commented Sep 16, 2019

Adds support for server-side limits of the following:

  • Total cores per user
  • Total memory per user
  • Total clusters per user

From a user, hitting the limits results in:

  • An informative error if creating a new cluster would exceed a limit
  • A warning if scaling to n workers would exceed a limit

Fixes #25.

Still needs tests.

Adds support for server-side limits of the following:
- Total cores per user
- Total memory per user
- Total clusters per user

From a user, hitting the limits results in:
- An informative error if creating a new cluster would exceed a limit
- A warning if scaling to ``n`` workers would exceed a limit
@jcrist
Copy link
Member Author

jcrist commented Sep 16, 2019

From a admin perspective, adding limits looks like:

# Each user is limited to no more than 50 cores
c.UserLimits.max_cores = 50

# Each user is limited to no more than 100 GiB
c.UserLimits.max_memory = "100 GiB"

# Each user can have no more than 2 active clusters
c.UserLimits.max_clusters = 2

If a user tries to create a cluster, and doing so would exceed a limit, the operation errors:

In [6]: cluster = g.new_cluster()
...

ValueError: Cannot start new cluster, would exceed user limit of 5 active clusters.

If a user tries to scale a cluster to ``n` workers, and doing so would exceed a limit, the user gets the maximum number of workers, and a warning is shown:

In [10]: cluster.scale(2)
/Users/jcrist/Code/dask-gateway/dask-gateway/dask_gateway/client.py:576: UserLimitWarning: Adding 2 workers to cluster be8844f83a2b4773921ae7db8f3f3a9e would exceed user cores limit of 2.0, adding 1 workers instead.
  warnings.warn(UserLimitWarning(msg["message"]))

@jcrist jcrist mentioned this pull request Sep 16, 2019
@jcrist
Copy link
Member Author

jcrist commented Sep 17, 2019

Tests and configuration docs added, merging.

@jcrist jcrist merged commit 554434b into master Sep 17, 2019
@jcrist jcrist deleted the add-user-limits branch September 17, 2019 15:00
kyprifog pushed a commit to kyprifog/dask-gateway that referenced this pull request Oct 16, 2020
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

User restrictions
1 participant