get_namespaced_custom_object 404

[bsiara]

Hi,

I try to get HTTPRoute definition from k8s and this is my script which return 404:

k8s_contexts, active_context = config.list_kube_config_contexts()
active_context = k8s_context['name']
k8s_api_client = config.new_client_from_config(context=active_context)
k8s_client = client.CustomObjectsApi(api_client=k8s_api_client)

k8s_object = k8s_client.get_namespaced_custom_object(group='gateway.networking.k8s.io', version='v1', namespace='name', plural='httproute', name='route1')

log.info("descriptor:\n%s", str(yaml.safe_dump(k8s_object.to_dict())))

error:

Traceback (most recent call last):
  File "/app/k8s.py", line 520, in <module>
    sys.exit(main(sys.argv[1:]))
             ^^^^^^^^^^^^^^^^^^
  File "/app/k8s.py", line 501, in main
    k8s_object = k8s_client.get_namespaced_custom_object(group='gateway.networking.k8s.io', version='v1', namespace='name', plural='httproute', name='route1')
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/venv/lib/python3.11/site-packages/kubernetes/client/api/custom_objects_api.py", line 1627, in get_namespaced_custom_object
    return self.get_namespaced_custom_object_with_http_info(group, version, namespace, plural, name, **kwargs)  # noqa: E501
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/venv/lib/python3.11/site-packages/kubernetes/client/api/custom_objects_api.py", line 1734, in get_namespaced_custom_object_with_http_info
    return self.api_client.call_api(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/venv/lib/python3.11/site-packages/kubernetes/client/api_client.py", line 348, in call_api
    return self.__call_api(resource_path, method,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/venv/lib/python3.11/site-packages/kubernetes/client/api_client.py", line 180, in __call_api
    response_data = self.request(
                    ^^^^^^^^^^^^^
  File "/app/venv/lib/python3.11/site-packages/kubernetes/client/api_client.py", line 373, in request
    return self.rest_client.GET(url,
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/venv/lib/python3.11/site-packages/kubernetes/client/rest.py", line 244, in GET
    return self.request("GET", url,
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/venv/lib/python3.11/site-packages/kubernetes/client/rest.py", line 238, in request
    raise ApiException(http_resp=r)
kubernetes.client.exceptions.ApiException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Audit-Id': 'aabdeebb-7bde-434f-b0ef-c1a7bedda7c7', 'Cache-Control': 'no-cache, private', 'Content-Type': 'text/plain; charset=utf-8', 'X-Content-Type-Options': 'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid': '1b0bef5a-d410-4601-94f8-f8d43cafa43e', 'X-Kubernetes-Pf-Prioritylevel-Uid': '98716d07-5e25-4074-be86-14f146715ca2', 'Date': 'Wed, 26 Feb 2025 15:17:42 GMT', 'Content-Length': '19'})
HTTP response body: 404 page not found

python version is 3.11.11
k8s version 1.29.5
python kubernetes version 29.0.0
What I'm doing wrong? When I use kubectl I get this descriptor:

$ kubectl --context name -n name get httproutes.gateway.networking.k8s.io route1 -o yaml
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"gateway.networking.k8s.io/v1","kind":"HTTPRoute","metadata":{"annotations":{},"labels":{"app":"route1"},"name":"route1","namespace":"name"},"spec":{"hostnames":["route1.name.cluster.com"],"parentRefs":[{"name":"nginxga-gateway","namespace":"nginx-gateway"}],"rules":[{"backendRefs":[{"name":"route1","port":8080}]},{"backendRefs":[{"name":"route1","port":8080}],"matches":[{"headers":[{"name":"header1","value":"active"}]}]}]}}
  creationTimestamp: "2025-02-17T12:53:55Z"
  generation: 1
  labels:
    app: route1
  name: route1
  namespace: name
  resourceVersion: "181756847"
  uid: 172f37c1-7f04-4be7-9860-871e45b46215
spec:
  hostnames:
  - route1.name.cluster.com
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: nginxga-gateway
    namespace: nginx-gateway
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: route1
      port: 8080
      weight: 1
    matches:
    - path:
        type: PathPrefix
        value: /
  - backendRefs:
    - group: ""
      kind: Service
      name: route1
      port: 8080
      weight: 1
    matches:
    - headers:
      - name: header1
        type: Exact
        value: active
      path:
        type: PathPrefix
        value: /
status:
  parents:
  - conditions:
    - lastTransitionTime: "2025-02-17T12:53:56Z"
      message: The route is accepted
      observedGeneration: 1
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2025-02-17T12:53:56Z"
      message: All references are resolved
      observedGeneration: 1
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    controllerName: gateway.nginx.org/nginx-gateway-controller
    parentRef:
      group: gateway.networking.k8s.io
      kind: Gateway
      name: nginxga-gateway
      namespace: nginx-gateway

[roycaihw]

Could you try using the debug log to see if the request is targeting the same endpoint as kubectl?

[bsiara]

thanks @roycaihw for your response, this is log with debug enabled:

2025-02-27 10:28:46,261 - DEBUG - Starting new HTTPS connection (1): lb-test.region.amazonaws.com:6443
/app/venv/lib/python3.11/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'lb-test.region.amazonaws.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
send: b'GET /apis/gateway.networking.k8s.io/v1/namespaces/name/httproute/route1 HTTP/1.1\r\nHost: lb-test.region.amazonaws.com:6443\r\nAccept-Encoding: identity\r\nAccept: application/json\r\nUser-Agent: OpenAPI-Generator/29.0.0/python\r\nContent-Type: application/json\r\n\r\n'
reply: 'HTTP/1.1 404 Not Found\r\n'
header: Audit-Id: ce5b57eb-86a1-4675-a77b-e49f012f7fc7
header: Cache-Control: no-cache, private
header: Content-Type: text/plain; charset=utf-8
header: X-Content-Type-Options: nosniff
header: X-Kubernetes-Pf-Flowschema-Uid: 1b0bef5a-d410-4601-94f8-f8d43cafa43e
header: X-Kubernetes-Pf-Prioritylevel-Uid: 98716d07-5e25-4074-be86-14f146715ca2
header: Date: Thu, 27 Feb 2025 10:28:46 GMT
header: Content-Length: 19
2025-02-27 10:28:46,446 - DEBUG - https://lb-test.region.amazonaws.com:6443 "GET /apis/gateway.networking.k8s.io/v1/namespaces/name/httproute/route1 HTTP/1.1" 404 0
2025-02-27 10:28:46,446 - DEBUG - response body: 404 page not found

Traceback (most recent call last):
  File "/app/k8s.py", line 548, in <module>
    sys.exit(main(sys.argv[1:]))
             ^^^^^^^^^^^^^^^^^^
  File "/app/k8s.py", line 529, in main
    k8s_object = k8s_client.get_namespaced_custom_object(group='gateway.networking.k8s.io', version='v1', namespace='name', plural='httproute', name='route1')
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/venv/lib/python3.11/site-packages/kubernetes/client/api/custom_objects_api.py", line 1627, in get_namespaced_custom_object
    return self.get_namespaced_custom_object_with_http_info(group, version, namespace, plural, name, **kwargs)  # noqa: E501
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/venv/lib/python3.11/site-packages/kubernetes/client/api/custom_objects_api.py", line 1734, in get_namespaced_custom_object_with_http_info
    return self.api_client.call_api(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/venv/lib/python3.11/site-packages/kubernetes/client/api_client.py", line 348, in call_api
    return self.__call_api(resource_path, method,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/venv/lib/python3.11/site-packages/kubernetes/client/api_client.py", line 180, in __call_api
    response_data = self.request(
                    ^^^^^^^^^^^^^
  File "/app/venv/lib/python3.11/site-packages/kubernetes/client/api_client.py", line 373, in request
    return self.rest_client.GET(url,
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/venv/lib/python3.11/site-packages/kubernetes/client/rest.py", line 244, in GET
    return self.request("GET", url,
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/venv/lib/python3.11/site-packages/kubernetes/client/rest.py", line 238, in request
    raise ApiException(http_resp=r)
kubernetes.client.exceptions.ApiException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Audit-Id': 'ce5b57eb-86a1-4675-a77b-e49f012f7fc7', 'Cache-Control': 'no-cache, private', 'Content-Type': 'text/plain; charset=utf-8', 'X-Content-Type-Options': 'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid': '1b0bef5a-d410-4601-94f8-f8d43cafa43e', 'X-Kubernetes-Pf-Prioritylevel-Uid': '98716d07-5e25-4074-be86-14f146715ca2', 'Date': 'Thu, 27 Feb 2025 10:28:46 GMT', 'Content-Length': '19'})
HTTP response body: 404 page not found

[bsiara]

this is log form kubectl:

I0227 11:38:53.562055   33131 round_trippers.go:463] GET https://lb-test.region.amazonaws.com:6443/apis/gateway.networking.k8s.io/v1/namespaces/name/httproutes/route1
I0227 11:38:53.562082   33131 round_trippers.go:469] Request Headers:
I0227 11:38:53.562097   33131 round_trippers.go:473]     User-Agent: kubectl/v1.29.5 (linux/amd64) kubernetes/59755ff
I0227 11:38:53.562110   33131 round_trippers.go:473]     Accept: application/json
I0227 11:38:53.606520   33131 round_trippers.go:574] Response Status: 200 OK in 44 milliseconds
I0227 11:38:53.606544   33131 round_trippers.go:577] Response Headers:
I0227 11:38:53.606561   33131 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 1b0bef5a-d410-4601-94f8-f8d43cafa43e
I0227 11:38:53.606573   33131 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: 98716d07-5e25-4074-be86-14f146715ca2
I0227 11:38:53.606583   33131 round_trippers.go:580]     Content-Length: 2586
I0227 11:38:53.606592   33131 round_trippers.go:580]     Date: Thu, 27 Feb 2025 10:38:53 GMT
I0227 11:38:53.606602   33131 round_trippers.go:580]     Audit-Id: 48e404ca-621a-4d71-95cf-b881904fb124
I0227 11:38:53.606612   33131 round_trippers.go:580]     Cache-Control: no-cache, private
I0227 11:38:53.606622   33131 round_trippers.go:580]     Content-Type: application/json
I0227 11:38:53.610298   33131 request.go:1212] Response Body: {"apiVersion":"gateway.networking.k8s.io/v1","kind":"HTTPRoute","metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"gateway.networking.k8s.io/v1\",\"kind\":\"HTTPRoute\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"route1\"},\"name\":\"route1\",\"namespace\":\"name\"},\"spec\":{\"hostnames\":[\"route1.name.cluster.com\"],\"parentRefs\":[{\"name\":\"nginxga-gateway\",\"namespace\":\"nginx-gateway\"}],\"rules\":[{\"backendRefs\":[{\"name\":\"route1\",\"port\":8080}]},{\"backendRefs\":[{\"name\":\"route1\",\"port\":8080}],\"matches\":[{\"headers\":[{\"name\":\"header1\",\"value\":\"active\"}]}]}]}}\n"},"creationTimestamp":"2025-02-17T12:53:55Z","generation":1,"labels":{"app":"route1"},"managedFields":[{"apiVersion":"gateway.networking.k8s.io/v1","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}},"f:labels":{".":{},"f:app":{}}},"f:spec":{".":{},"f [truncated 1562 chars]

[bsiara]

ok, I see the difference, this is my mistake in plural which should be httproutes, not httproute