Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(app): use 3 scenarios for session cpu limits #828

Merged
merged 3 commits into from
Nov 30, 2021
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions helm-chart/renku-notebooks/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,8 @@ spec:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: ENFORCE_CPU_LIMITS
value: {{ .Values.enforceCPULimits | quote }}
ports:
- name: http
containerPort: 8000
Expand Down
2 changes: 2 additions & 0 deletions helm-chart/renku-notebooks/templates/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ spec:
value: |
{{- toYaml . | nindent 12 }}
{{- end }}
- name: ENFORCE_CPU_LIMITS
value: {{ $.Values.enforceCPULimits | quote }}
command:
- pipenv
- run
Expand Down
7 changes: 7 additions & 0 deletions helm-chart/renku-notebooks/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,13 @@ serverDefaults:
gpu_request: 0
lfs_auto_fetch: false

## How to enforce CPU limits for sessions, options are "lax", "off" or "strict"
## - "strict" = CPU limit equals cpu request
## - "lax" = CPU limit equals maximum from server_options, if CPU is not in server options then
## CPU limit is set to the CPU request
## - "off" = no CPU limits at all
enforceCPULimits: "off"

## Default image used when the requested image for a user session cannot be found.
defaultSessionImage: "renku/renkulab-py:3.9-0.10.1"

Expand Down
15 changes: 12 additions & 3 deletions renku_notebooks/api/classes/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -210,14 +210,23 @@ def _get_registry_secret(self, b64encode=True):
return output

def _get_session_k8s_resources(self):
cpu = float(self.server_options["cpu_request"])
cpu_request = float(self.server_options["cpu_request"])
mem = self.server_options["mem_request"]
gpu_req = self.server_options.get("gpu_request", {})
gpu = {"nvidia.com/gpu": str(gpu_req)} if gpu_req else None
resources = {
"requests": {"memory": mem, "cpu": cpu},
"limits": {"memory": mem, "cpu": cpu},
"requests": {"memory": mem, "cpu": cpu_request},
"limits": {"memory": mem},
}
if current_app.config["ENFORCE_CPU_LIMITS"] == "lax":
if "cpu_request" in current_app.config["SERVER_OPTIONS_UI"]:
resources["limits"]["cpu"] = max(
current_app.config["SERVER_OPTIONS_UI"]["cpu_request"]["options"]
)
else:
resources["limits"]["cpu"] = cpu_request
elif current_app.config["ENFORCE_CPU_LIMITS"] == "strict":
resources["limits"]["cpu"] = cpu_request
if gpu:
resources["requests"] = {**resources["requests"], **gpu}
resources["limits"] = {**resources["limits"], **gpu}
Expand Down
1 change: 1 addition & 0 deletions renku_notebooks/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,3 +118,4 @@
SESSION_NODE_SELECTOR = safe_load(os.environ.get("SESSION_NODE_SELECTOR", "{}"))
SESSION_AFFINITY = safe_load(os.environ.get("SESSION_AFFINITY", "{}"))
SESSION_TOLERATIONS = safe_load(os.environ.get("SESSION_TOLERATIONS", "[]"))
ENFORCE_CPU_LIMITS = os.getenv("ENFORCE_CPU_LIMITS", "off")