feat(api,ui,sdk): Make CPU limits configurable

[deadlycoconuts]

Context

Similar to caraml-dev/merlin#586, this PR aims to make CPU limits configurable for the end user.

As of present, users are not able to configure the CPU limits of the pods in which Turing routers/enrichers/ensemblers (docker and pyfunc) are deployed in - they are instead determined automatically on the platform-level (Turing API server). Depending on how the API server has been configured, one of the following happens:

• the CPU limit of a component is set as its CPU request value, multiplied by a scaling factor (e.g. 2 CPU * 1.5) or,
  • Note that this is the existing way memory limits are automatically set by the Turing API server
• the CPU limit is left unset

This PR introduces a new workflow which would allow users to instead override the platform-level CPU limits (described in the paragraph above) set on a component. This workflow is available via the UI, SDK and by extension, directly calling the API endpoint of the API server.

UI:

SDK:

config = router.config
config.resource_request = ResourceRequest(
    min_replica=1,
    max_replica=3,
    cpu_request="100m",
    cpu_limit="2",
    memory_request="512Mi",
)

router.update(config)

In addition, this PR adds a new configuration, DefaultEnvVarsWithoutCPULimits, which is a list of env vars that automatically get added to all Turing routers/enrichers/ensemblers (docker and pyfunc) when CPU limits are not set. This allows the Turing API server's operators to set env vars platform-wide that can potentially improve these deployments' performance, e.g. env vars involving concurrency.

Modifications

• api/turing/cluster/knative_service.go - Removal of platform-level fields from the KnativeService struct
• api/turing/cluster/servicebuilder/service_builder.go - Addition of platform-level configs to clusterSvcBuilder and new helper methods to set default env vars when cpu limits are not explicitly set and when the cpu limit scaling factor is set as 0
• api/turing/config/config.go - Addition of the new field DefaultEnvVarsWithoutCPULimits
• sdk/turing/router/config/resource_request.py - Addition of a new cpu limit field to the resource request class
• ui/src/router/components/form/components/CPULimitsFormGroup.js - Addition of a new form group to allow cpu limits to be specified on the UI

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.06%. Comparing base (0d6f578) to head (1f9187e).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #381      +/-   ##
==========================================
+ Coverage   62.19%   68.06%   +5.87%     
==========================================
  Files         124      149      +25     
  Lines        9755    11809    +2054     
==========================================
+ Hits         6067     8038    +1971     
- Misses       2954     3033      +79     
- Partials      734      738       +4     

Flag	Coverage Δ
api-test	62.26% <ø> (+0.07%)	⬆️
sdk-test-3.10	95.92% <100.00%> (?)
sdk-test-3.8	95.92% <100.00%> (?)
sdk-test-3.9	95.92% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[leonlnj]

thanks LGTM. +1 for moving platform default out and putting it as part of the service builder