[k8s] Add Kubernetes support for ports #2713

hemildesai · 2023-10-17T05:42:37Z

Background for this can be found here

This PR adds Kubernetes support for ports. It uses Ingress to accomplish this. For each port, it creates a new service and a new ingress rule. The port can then be accessed by visiting the url obtained by running sky status {cluster_name} --ports. In the initial PR, we assume that the ingress controller is already created by the Kubernetes cluster admin, but in the future we will add support for creating one automatically if none exists. We can add documentation on how to manually install the ingress controller for now. Currently, the ingress controller of choice is nginx.

The new cli command looks like

sky status {cluster_name} --ports              
33828: http://{external_ip}/skypilot/{cluster_name_on_cloud}_33828, https://{external_ip}/skypilot/{cluster_name_on_cloud}_33828

Tested (run the relevant ones):

Code formatting: bash format.sh
Any manual or new tests for this PR (please specify below)
All smoke tests: pytest tests/test_smoke.py
Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
Backward compatibility tests: bash tests/backward_comaptibility_tests.sh

cblmemo

Thanks for this exciting feature! 🚀 Left several comments on the ports related part 👀

sky/cli.py

cblmemo · 2023-10-17T19:54:51Z

sky/backends/cloud_vm_ray_backend.py

-            # so that our backend do not need to know the specific details
-            # for different clouds.
-            if isinstance(cloud, (clouds.AWS, clouds.GCP, clouds.Azure)):
+            if provision_lib.supports(repr(cloud), 'cleanup_ports'):


Great! 🚀

sky/cli.py

cblmemo · 2023-10-17T20:00:29Z

sky/cli.py

@@ -1750,17 +1761,23 @@ def status(all: bool, refresh: bool, ip: bool, show_spot_jobs: bool,
                          show_all=False,
                          limit_num_jobs_to_show=not all,
                          is_called_by_user=False))
-        if ip:
+        if ip or ports:


To discuss: Probably we could have an enum containing ip and ports, and use sth like --target ip/--target ports in the CLI?

sky/clouds/kubernetes.py

sky/provision/kubernetes/__init__.py

sky/provision/kubernetes/network.py

cblmemo · 2023-10-17T20:09:48Z

sky/templates/kubernetes-ray.yml.j2

@@ -25,6 +28,18 @@ provider:

    ssh_jump_image: {{k8s_ssh_jump_image}}

+    ingress_name: {{cluster_name_on_cloud}}-skypilot-ingress
+
+    {%- if ports is not none %}


Current ports implementation won't put ports in the provider config. Instead, we use handle.launched_resources.ports to keep track of all opened ports. Reference:

skypilot/sky/backends/cloud_vm_ray_backend.py

Lines 3067 to 3077 in bf33602

prev_ports = None

if prev_handle is not None:

prev_ports = prev_handle.launched_resources.ports

current_ports = handle.launched_resources.ports

open_new_ports = bool(

resources_utils.port_ranges_to_set(current_ports) -

resources_utils.port_ranges_to_set(prev_ports))

if open_new_ports:

with rich_utils.safe_status(

'[bold cyan]Launching - Opening new ports'):

self._open_ports(handle)

Can we change to the same implementation logic to support sky launch --ports <new-ports> to open new ports on an existing cluster? It seems like we can construct all other fields using port and cluster_name so keeping track of ports should be enough.

Addressed in 186193c

sky/templates/kubernetes-ray.yml.j2

romilbhardwaj

Thanks @hemildesai! This is super cool. Tested some functionality and seems to work nicely for a simple http server. Jupyter seems to have some problems, see comments below. Will take a closer look at the code soon.

sky/provision/kubernetes/network_utils.py

romilbhardwaj · 2023-10-19T17:36:02Z

sky/provision/kubernetes/network_utils.py

+def delete_namespaced_service(namespace: str, service_name: str) -> None:
+    """Deletes a service resource."""
+    core_api = kubernetes.core_api()
+    core_api.delete_namespaced_service(service_name, namespace)


I ran into a bug here - my previous provisioning had failed due to a library mismatch and ended up not creating the service in k8s.

When I tried to terminate the skypilot cluster (stuck in a bad INIT state), I got:

kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'f463ef5b-3fd6-488d-8c32-1ef4bcce7240', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': 'c57dbea3-732c-471b-840e-3fd589a06584', 'X-Kubernetes-Pf-Prioritylevel-Uid': '7187fedc-79b8-4e30-933f-705406be228f', 'Date': 'Thu, 19 Oct 2023 17:28:55 GMT', 'Content-Length': '240'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"services \"test-2ea4-skypilot-service--8888\" not found","reason":"NotFound","details":{"name":"test-2ea4-skypilot-service--8888","kind":"services"},"code":404}

We should probably handle any deletion failures due to port not existing by wrapping them into a new sky.exceptions.PortDoesNotExist class here, re-raising it and handling it upstream (perhaps by simply pass) when provision_lib.cleanup_ports is called in cloud_vm_ray_backend. cc @cblmemo - does that make sense to you?

Yes! The current implementation just prints some warning messages in the provision lib API. I think it is a good idea to handle them in a unified place 🙂

Addressed in 8a70791

romilbhardwaj · 2023-10-19T17:49:07Z

sky/cli.py

@@ -1672,6 +1677,12 @@ def _get_spot_jobs(
                    'clusters, the returned IP address is the internal IP '
                    'of the head pod, and may not be accessible from outside '
                    'the cluster.'))
+@click.option('--ports',


I'm for some reason getting two endpoints listed:

$ sky status --ports test2 8888: http://35.222.205.70/skypilot/test2-2ea4_8888, https://35.222.205.70/skypilot/test2-2ea4_8888

Repro:

# task.yaml resources: ports: 8888 run: python -m http.server 8888

sky launch -c test2 task.yaml --cloud kubernetes

Discussed offline, this currently returns HTTP and HTTPS url but will default to HTTP for now.

Done in 3d411a1

romilbhardwaj · 2023-10-19T17:53:20Z

sky/provision/__init__.py

+def query_ports(
+    provider_name: str,
+    cluster_name_on_cloud: str,
+    provider_config: Optional[Dict[str, Any]] = None,
+) -> Dict[str, Tuple[str, str]]:
+    """Query details about ports on a cluster.
+
+    Returns a dict with port as the key and the value as the http and https url to access the port.
+    """
+    raise NotImplementedError


Nice! We should also probably add something for gcp and aws too, simply returning <ip>:<port> so sky status --endpoints can be used consistently across clouds.

Done in 3d411a1

romilbhardwaj · 2023-10-19T17:59:41Z

sky/cli.py

@@ -1672,6 +1677,12 @@ def _get_spot_jobs(
                    'clusters, the returned IP address is the internal IP '
                    'of the head pod, and may not be accessible from outside '
                    'the cluster.'))
+@click.option('--ports',


A thought - we should support this API for easy programmatic use too. For example, many users query their skypilot endpoints with:

$ curl http://$(sky status mycluster --ip):8888/myapi

Since the current output is of the format 8888: http://35.222.205.70/skypilot/test2-2ea4_8888, it would require some parsing on the user's part.

Can we:

Rename --ports to --endpoints

If a port number is specified to --endpoints, then we simply return the URL. E.g.,:

$sky status test2 --endpoint 8888 http://35.222.205.70/skypilot/test2-2ea4_8888

If a port number is not specified, then we list all ports open on the cluster:

$ sky status --endpoints test2 8888: http://35.222.205.70/skypilot/test2-2ea4_8888 8889: https://35.222.205.70/skypilot/test2-2ea4_8889

Should the latest CLI command be sky status --endpoints test2?

Yes - that should work. Would be nice to change --ports to --endpoint(s).

romilbhardwaj · 2023-10-19T18:05:07Z

sky/provision/kubernetes/network_utils.py

+from sky.adaptors import kubernetes
+
+
+def create_namespaced_ingress(namespace: str, ingress_name: str,


Jupyter does not seems to play nicely with ingress:

#jupyter.yaml resources: ports: 8888 setup: pip install jupyter run: jupyter lab --port 8888 --no-browser --ip=0.0.0.0

On accessing https://35.222.205.70/skypilot/test-2ea4_8888/ in Chrome, I get 404:

Note that it tried to redirect me to https://35.222.205.70/lab, which is incorrect.

On accessing the exact suburl (/lab?token=....), I get an empty white page but loading some js/css is likely failing with 404:

Can we configure the ingress to correctly redirect to the sub-path we have under the ingress?

cblmemo

Thanks for the prompt fix! Left some other comments ;)

sky/cli.py

sky/clouds/kubernetes.py

sky/provision/aws/instance.py

sky/provision/gcp/instance.py

sky/provision/kubernetes/network.py

cblmemo · 2023-10-23T06:28:15Z

sky/provision/kubernetes/network.py

+    assert provider_config is not None, cluster_name_on_cloud
+    for port in ports:
+        service_name = f"{cluster_name_on_cloud}-skypilot-service--{port}"
+        ingress_name = f"{cluster_name_on_cloud}-skypilot-ingress--{port}"


Just curious, is there any way in k8s that we can register multiple ports in a single service? That way, we could get rid of the ports argument; But if there is no such API I'm happy with the current implementation 🙂

I will look into this.

This is possible. But, thinking about it, this becomes tricky for maintenance and modification. If we want to add ports or remove some, we will have to edit the service in place.

The benefit of multiple ports under a single service is that it shares the IP, but since the services are internal and the ingress routes are port specific, I don't think a shared IP provides much benefit.

Actually, this wasn't so bad and is the default way for opening ports using the LoadBalancer service type.

sky/provision/kubernetes/network.py

hemildesai · 2023-11-02T04:28:43Z

To resolve the issue with Jupyter and similar services, we decided to support LoadBalancer service type (which creates an external LB per cluster) so that each cluster can have a dedicated IP. The user can configure whether to use LoadBalancer or Ingress using skypilot config. Related changes in 55f2ece and 430e6a7

romilbhardwaj

Awesome @hemildesai! This works very nicely - I had a quick go with both nginx ingress and loadbalancer. There are a few edge cases to handle, mentioned in comments below. We should also update the --ports flag to --endpoint(s).

Will take a closer look at the code soon.

sky/provision/kubernetes/network.py

romilbhardwaj · 2023-11-02T23:35:30Z

sky/cli.py

            head_ip = handle.external_ips()[0]
+            if ports:


We should perhaps add some error handling when clusters have different port modes. E.g., if user switches their global config.yaml from loadbalancer to ingress or vice-versa.

For instance, I launched one cluster with ports: loadbalancer, then changed my config to ports: ingress. On running sky status --ports jupyter, I get this error. It would be nice to either fix this or cleanly ask the user to not mix port modes.

(base) ➜ ~ sky status --ports jupyter D 11-02 16:31:37 skypilot_config.py:157] Using config path: /Users/romilb/.sky/config.yaml D 11-02 16:31:37 skypilot_config.py:160] Config loaded: D 11-02 16:31:37 skypilot_config.py:160] {'kubernetes': {'ports': 'loadbalancer'}} D 11-02 16:31:37 skypilot_config.py:166] Config syntax check passed. D 11-02 16:31:38 skypilot_config.py:93] User config: kubernetes.ports -> loadbalancer D 11-02 16:31:38 skypilot_config.py:157] Using config path: /Users/romilb/.sky/config.yaml D 11-02 16:31:38 skypilot_config.py:160] Config loaded: D 11-02 16:31:38 skypilot_config.py:160] {'kubernetes': {'ports': 'loadbalancer'}} D 11-02 16:31:38 skypilot_config.py:166] Config syntax check passed. Traceback (most recent call last): File "/Users/romilb/tools/anaconda3/bin/sky", line 8, in <module> sys.exit(cli()) File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/click/core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/utils/common_utils.py", line 311, in _record return f(*args, **kwargs) File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/cli.py", line 1163, in invoke return super().invoke(ctx) File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/click/core.py", line 754, in invoke return __callback(*args, **kwargs) File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/utils/common_utils.py", line 332, in _record return f(*args, **kwargs) File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/cli.py", line 1821, in status port_details = provision_lib.query_ports( File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/provision/__init__.py", line 41, in _wrapper return impl(*args, **kwargs) File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/provision/kubernetes/network.py", line 162, in query_ports return _query_ports_for_loadbalancer( File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/provision/kubernetes/network.py", line 184, in _query_ports_for_loadbalancer external_ip = network_utils.get_loadbalancer_ip( File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/provision/kubernetes/network_utils.py", line 169, in get_loadbalancer_ip service = core_api.read_namespaced_service(service_name, namespace) File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/kubernetes/client/api/core_v1_api.py", line 25141, in read_namespaced_service return self.read_namespaced_service_with_http_info(name, namespace, **kwargs) # noqa: E501 File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/kubernetes/client/api/core_v1_api.py", line 25228, in read_namespaced_service_with_http_info return self.api_client.call_api( File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/kubernetes/client/api_client.py", line 348, in call_api return self.__call_api(resource_path, method, File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/kubernetes/client/api_client.py", line 180, in __call_api response_data = self.request( File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/kubernetes/client/rest.py", line 238, in request raise ApiException(http_resp=r) kubernetes.client.exceptions.ApiException: (404) Reason: Not Found HTTP response headers: HTTPHeaderDict({'Audit-Id': 'ba277f1b-fb86-41c8-888b-32896a6f1e9c', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': 'cf43db27-7222-4741-adea-8b7f8f0e681e', 'X-Kubernetes-Pf-Prioritylevel-Uid': '17aff9c6-0fa7-4909-a811-4644194e9001', 'Date': 'Thu, 02 Nov 2023 23:31:38 GMT', 'Content-Length': '244'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"services \"jupyter-2ea4-skypilot-loadbalancer\" not found","reason":"NotFound","details":{"name":"jupyter-2ea4-skypilot-loadbalancer","kind":"services"},"code":404}

I think we can persist the mode for each cluster in the provider config during creation time. This will avoid this issue and allow users to have separate modes for separate clusters. Wdyt?

This is fixed in 4ae6602

romilbhardwaj · 2023-11-02T23:52:50Z

sky/provision/kubernetes/network_utils.py

+class KubernetesPortMode(enum.Enum):
+    """Enum for the different types of modes supported for opening
+    ports on Kubernetes.
+    """
+    INGRESS = 'ingress'
+    LOADBALANCER = 'loadbalancer'


This is fantastic! I love the flexibility this provides. Would be nice to document this in docs/source/reference/config.rst :)

For sure, will update.

romilbhardwaj · 2023-11-02T23:54:19Z

sky/cli.py

@@ -1672,6 +1677,12 @@ def _get_spot_jobs(
                    'clusters, the returned IP address is the internal IP '
                    'of the head pod, and may not be accessible from outside '
                    'the cluster.'))
+@click.option('--ports',


Yes - that should work. Would be nice to change --ports to --endpoint(s).

cblmemo

Thanks for the awesome work!! 🚀 Left one comment to discuss 🫡

cblmemo · 2023-11-03T00:38:21Z

sky/provision/kubernetes/network.py

+            selector_key='skypilot-cluster',
+            selector_value=cluster_name_on_cloud,
+        )
+        network_utils.create_namespaced_service(


Current ports implementation is incremental, i.e. if I do sky launch -c mycluster --ports 8080 and then sky launch -c mycluster --ports 8081, the second sky launch will update the ports to '8080-8081'. In

skypilot/sky/backends/cloud_vm_ray_backend.py

Lines 3009 to 3017 in 53ba415

def _open_ports(self, handle: CloudVmRayResourceHandle) -> None:

cloud = handle.launched_resources.cloud

logger.debug(

f'Opening ports {handle.launched_resources.ports} for {cloud}')

config = common_utils.read_yaml(handle.cluster_yaml)

provider_config = config['provider']

provision_lib.open_ports(repr(cloud), handle.cluster_name_on_cloud,

handle.launched_resources.ports,

provider_config)

We passed the updated list to the open_ports API ('8080-8081' in the example above). Therefore, the open_ports API should update the ports to what ports describe. I'm not very familiar with k8s API, but the API is named create_xxx rather than create_or_update_xxx; so just want to make sure, does this API support update semantic?

Great point, I will update the ingress implementation to be create_or_update. LoadBalancer is already create_or_update.

Updated in 1d95434

Co-authored-by: Tian Xia <cblmemo@gmail.com>

LoadBalancer service

romilbhardwaj · 2023-11-06T22:07:29Z

sky/clouds/kubernetes.py

-        deploy_vars = {
+        return {


should we retain the deploy_vars so the return statement remains clean?

Done in 4ae6602

romilbhardwaj · 2023-11-06T22:53:56Z

sky/provision/__init__.py

@@ -123,12 +131,28 @@ def open_ports(
 def cleanup_ports(
    provider_name: str,
    cluster_name_on_cloud: str,
+    ports: List[str],
    provider_config: Optional[Dict[str, Any]] = None,
 ) -> None:
    """Delete any opened ports."""


Should we update the docstr and make this an optional arg? Then the semantics can be:

If ports are specified, clean up only the specific ports

If not specified, clean up all ports

That makes sense. However, for loadbalancer based services, a single service contains all ports. In that case, we'll have to get all_ports - cleanup_ports and recreate the service using that set.

Also, we'll have to support this for other clouds as well. Maybe we can leave it out of scope for this PR and do it in a followup PR. Wdyt?

Sounds good - we should update the docstr and mark it as a TODO to implement the desired behavior

romilbhardwaj · 2023-11-06T23:16:36Z

sky/provision/__init__.py

+) -> Dict[str, Tuple[str, str]]:
+    """Query details about ports on a cluster.
+
+    Returns a dict with port as the key and the value as the http and https url to access the port.
+    """


Ports may not necessarily be exposing both HTTP and HTTPS urls. Should we change the return type to Dict[str, List[Endpoint]], which is a dict of {port: [endpoint]}?

Endpoint is new class like so:

class Endpoint: pass class SocketEndpoint(Endpoint): def __init__(self, host, port): self.host = host self.port = port def __str__(self): return f'{self.host}:{self.port}' class HTTPEndpoint(Endpoint): def __init__(self, url): self.url = url def __str__(self): return self.url class HTTPSEndpoint(Endpoint): def __init__(self, url): self.url = url def __str__(self): return self.url

Example return values:

# Example for k8s {8080: [ HTTPEndpoint("http://www.example.com/mycluster/8080"), HTTPSEndpoint("https://www.example.com/mycluster/8080") ] } # Example for raw socket {8080: [ SocketEndpoint(ip='1.1.1.1', port=8080), ] }

cc @cblmemo

Done in 4ae6602

sky/provision/kubernetes/network_utils.py

sky/provision/kubernetes/network.py

sky/templates/kubernetes-ray.yml.j2

sky/provision/kubernetes/network_utils.py

hemildesai · 2024-01-03T23:22:30Z

sky/backends/cloud_vm_ray_backend.py

-            if (cloud.PROVISIONER_VERSION >= clouds.ProvisionerVersion.
-                    RAY_PROVISIONER_SKYPILOT_TERMINATOR or


Looks like this was recently added, not sure if it's necessary along with if provision_lib.supports(repr(cloud), 'cleanup_ports')

cc @cblmemo - can you take a quick look at this?

This is added in #2681, https://github.com/skypilot-org/skypilot/pull/2681/files#diff-f14940b9d929546868fb41cb0b27a1db08b321b7bbc0cb3ab3b864589aa82aa8
I still think the provision_lib.supports(cloud, 'cleanup_ports') is needed since clouds.ProvisionerVersion is not related to what cloud supports open ports. The new condition here is essentially still isinstance(cloud, (GCP, AWS, Azure)). The current implementation LGTM; cc @Michaelvll for a double-check

How about we change the check to the following:

try: cloud.check_features_are_supported(handle.launched_resources, {clouds.CloudImplementationFeatures.OPEN_PORTS}) provision_lib.cleanup_ports(repr(cloud), cluster_name_on_cloud, handle.launched_resources.ports, config['provider']) except exceptions.NotSupportedError: pass except ...

The provision_lib.supports seems duplicated with the CloudImplementationFeatures in Cloud class.

Agreed! Let's use cloud.check_features_are_supported. This should be fine since all ports related functions is implemented in new provisioned API.

This reverts commit 7302e45.

romilbhardwaj

Thanks @hemildesai! Pushed some additional fixes for error msg handling. Ran smoke tests, LGTM!

EDIT - testing some final local kubernetes integration before merging

This reverts commit 846d93c.

romilbhardwaj · 2024-01-11T23:36:50Z

Made some changes to error handling and disallowed opening ports sky local up clusters for now till we figure out a good solution there. Re-running smoke tests, will merge when they pass.

romilbhardwaj · 2024-01-12T19:56:09Z

Fixed other comments and removed provision_lib.supports(). Smoke tests pass, merging now!

cblmemo reviewed Oct 17, 2023

View reviewed changes

romilbhardwaj reviewed Oct 18, 2023

View reviewed changes

sky/templates/kubernetes-ray.yml.j2 Outdated Show resolved Hide resolved

romilbhardwaj changed the title ~~Add Kubernetes support for ports~~ [k8s] Add Kubernetes support for ports Oct 18, 2023

romilbhardwaj reviewed Oct 19, 2023

View reviewed changes

cblmemo reviewed Oct 23, 2023

View reviewed changes

romilbhardwaj reviewed Nov 2, 2023

View reviewed changes

cblmemo reviewed Nov 3, 2023

View reviewed changes

hemildesai and others added 13 commits November 3, 2023 00:34

Add Kubernetes support for ports

00382a9

Fix formatting

5a2035a

Apply suggestions from code review

2ae1960

Co-authored-by: Tian Xia <cblmemo@gmail.com>

PR feedback part 1

6492f71

Use ports from handle

500fde4

Make path prefix a constant

afb029f

Pin kubernetes to >20.0.0

d636d64

PR feedback part 2

3613cd9

Handle port does not exist in cleanup_ports

bd140a7

query_ports for AWS, GCP

b782de3

Use template

4f96be8

Add support for opening ports using Kubernetes

e0bc5b9

LoadBalancer service

Add query_ports support for LoadBalancer

da8fad6

hemildesai force-pushed the hd/k8s-ports branch from 430e6a7 to da8fad6 Compare November 3, 2023 07:35

create or update services and ingress in K8s

1d95434

romilbhardwaj reviewed Nov 7, 2023

View reviewed changes

hemildesai added 6 commits November 7, 2023 23:15

Fix bugs, store port mode in provider config and add Endpoint class

4ae6602

Move KubernetesPortMode enum to sky.utils.kubernetes_enums

e155a8c

Add sky status --endpoint/s

bf895d9

Add _request_timeout

6dd2de2

Fix issue when external ip is pending

1e86dcf

Use ingress.hostname if available

8eaeb1a

hemildesai added 3 commits December 30, 2023 18:45

Pylint fixes

d4e3018

Merge branch 'master' into hd/k8s-ports

a597025

isort fix

0cbb8e5

hemildesai commented Jan 3, 2024

View reviewed changes

hemildesai and others added 9 commits January 3, 2024 15:47

PR feedback - get correct ingress nginx svc

0bce7db

Fix kind loadbalancer services

7302e45

Add retries for flaky curl

a4a279b

Revert "Fix kind loadbalancer services"

b568cfd

This reverts commit 7302e45.

Increase timeout for EKS clusters

4200cd4

nicer error handling

7d57a30

revert eks_test_cluster.yaml

889b5e0

Add check if ports should actually be open and show relevant msg

721b037

lint

cd33c74

romilbhardwaj approved these changes Jan 10, 2024

View reviewed changes

romilbhardwaj added 8 commits January 10, 2024 11:53

refactor get_port_mode

846d93c

Revert "refactor get_port_mode"

2d01490

This reverts commit 846d93c.

better error handling

df659dc

lint

328681f

disallow sky local up port opening

5f9b077

message fixes

520a906

lint

8413921

move port check to classmethod

e8fcda8

romilbhardwaj added 3 commits January 11, 2024 23:10

remove "supports"

48cb26b

lint

f958196

lint

cbf3acf

romilbhardwaj merged commit 26900ff into skypilot-org:master Jan 12, 2024
19 checks passed

hemildesai mentioned this pull request Jan 18, 2024

Add docs for opening ports on Kubernetes #2997

Merged

romilbhardwaj mentioned this pull request Jan 23, 2024

[k8s] sky down --purge fails on Kubernetes clusters that no longer exist #3013

Closed

	prev_ports = None
	if prev_handle is not None:
	prev_ports = prev_handle.launched_resources.ports
	current_ports = handle.launched_resources.ports
	open_new_ports = bool(
	resources_utils.port_ranges_to_set(current_ports) -
	resources_utils.port_ranges_to_set(prev_ports))
	if open_new_ports:
	with rich_utils.safe_status(
	'[bold cyan]Launching - Opening new ports'):
	self._open_ports(handle)

		from sky.adaptors import kubernetes


		def create_namespaced_ingress(namespace: str, ingress_name: str,

	def _open_ports(self, handle: CloudVmRayResourceHandle) -> None:
	cloud = handle.launched_resources.cloud
	logger.debug(
	f'Opening ports {handle.launched_resources.ports} for {cloud}')
	config = common_utils.read_yaml(handle.cluster_yaml)
	provider_config = config['provider']
	provision_lib.open_ports(repr(cloud), handle.cluster_name_on_cloud,
	handle.launched_resources.ports,
	provider_config)

		if (cloud.PROVISIONER_VERSION >= clouds.ProvisionerVersion.
		RAY_PROVISIONER_SKYPILOT_TERMINATOR or

[k8s] Add Kubernetes support for ports #2713

[k8s] Add Kubernetes support for ports #2713

Conversation

hemildesai commented Oct 17, 2023 • edited

cblmemo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

romilbhardwaj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

romilbhardwaj Oct 19, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cblmemo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hemildesai Oct 23, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hemildesai commented Nov 2, 2023

romilbhardwaj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cblmemo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Michaelvll Jan 10, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

romilbhardwaj left a comment • edited

Choose a reason for hiding this comment

romilbhardwaj commented Jan 11, 2024

romilbhardwaj commented Jan 12, 2024

hemildesai commented Oct 17, 2023 •

edited

romilbhardwaj Oct 19, 2023 •

edited

hemildesai Oct 23, 2023 •

edited

Michaelvll Jan 10, 2024 •

edited

romilbhardwaj left a comment •

edited