Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid HTTP port #441

Closed
laoshancun opened this issue May 12, 2020 · 8 comments
Closed

Invalid HTTP port #441

laoshancun opened this issue May 12, 2020 · 8 comments

Comments

@laoshancun
Copy link

Invalid HTTP port: tcp://172.31.14.133:9000 when upgrade from 0.8.5 to 0.9.0.

@moliware
Copy link
Collaborator

Could you elaborate a bit more on this?

Not sure if related but we introduced these changes regarding the port:

@laoshancun
Copy link
Author

Server:Docker
Env:none
Image:lmenezes/cerebro:0.9.0

@laoshancun
Copy link
Author

Container Stdout

20200512153015

Pod yaml

apiVersion: v1
kind: Pod
metadata:
  labels:
    app: cerebro
    pod-template-hash: "6954484579"
  name: cerebro-6954484579-kdnhj
spec:
  containers:
  - image: lmenezes/cerebro:0.9.0
    imagePullPolicy: Always
    livenessProbe:
      failureThreshold: 3
      httpGet:
        path: /
        port: 9000
        scheme: HTTP
      initialDelaySeconds: 10
      periodSeconds: 5
      successThreshold: 1
      timeoutSeconds: 2
    name: cerebro
    ports:
    - containerPort: 9000
      name: http
      protocol: TCP
    readinessProbe:
      failureThreshold: 3
      httpGet:
        path: /
        port: 9000
        scheme: HTTP
      initialDelaySeconds: 10
      periodSeconds: 5
      successThreshold: 2
      timeoutSeconds: 2
    resources:
      limits:
        cpu: 500m
        memory: 800Mi
      requests:
        cpu: 100m
        memory: 800Mi
    securityContext:
      capabilities: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-b4655
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30

@BitProcessor
Copy link

Can also confirm that 0.8.5 works, while 0.9.1 is giving the Invalid HTTP Port error.
Also running on Kubernetes.

@vmercierfr
Copy link

We experienced similar issue during upgrade to Cerebro 9.0
We identified the root cause with a workaround.

Root cause:

Cerebro 9.0 introduced CEREBRO_PORT environment variable to set the daemon port (See 04b5ca8).
This variable could conflicts with Kubernetes deployment.

Kubernetes generates (by default) environment variables for services deployed in same namespace (see https://kubernetes.io/docs/concepts/services-networking/service/#environment-variables).
If you deploy Cerebro with a Kubernetes service called cerebro, the CEREBRO_PORT environment variable will be automatically defined to tcp://<pod_id_address>:9000.

When Cerebro starts, it execute server.http.port = tcp://<pod_id_address>:9000 and crash with the following error:

Invalid HTTP port: tcp://<pod_id_address>:9000

Work-around

We identified 2 workarounds:

  1. Override CEREBRO_PORT to force Cerebro port. You may add the following lines to your container:
    env:
    - name: "CEREBRO_PORT"
      value: "9000"
  2. Set enableServiceLinks=false in your pod deployment (See https://kubernetes.io/docs/concepts/services-networking/connect-applications-service/#accessing-the-service) to avoid environment variables conflicts

@BitProcessor
Copy link

BitProcessor commented Jun 1, 2020

@vmercierfr - tried it with work-around #2 and can confirm this works fine with the 0.9.1 image.
Thanks! 👍
For others encountering this issue:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cerebro
  labels:
    app: cerebro
spec:
  selector:
    matchLabels:
      app: cerebro
  replicas: 1
  template:
    metadata:
      labels:
        app: cerebro
    spec:
      enableServiceLinks: false
      containers:
      - name: cerebro   
        image: lmenezes/cerebro:0.9.1
        ports:
        - containerPort: 9000
          name: http
        resources:
          requests:
            cpu: 50m
            memory: 512Mi
          limits:
            memory: 512Mi

@laoshancun
Copy link
Author

We experienced similar issue during upgrade to Cerebro 9.0
We identified the root cause with a workaround.

Root cause:

Cerebro 9.0 introduced CEREBRO_PORT environment variable to set the daemon port (See 04b5ca8).
This variable could conflicts with Kubernetes deployment.

Kubernetes generates (by default) environment variables for services deployed in same namespace (see https://kubernetes.io/docs/concepts/services-networking/service/#environment-variables).
If you deploy Cerebro with a Kubernetes service called cerebro, the CEREBRO_PORT environment variable will be automatically defined to tcp://<pod_id_address>:9000.

When Cerebro starts, it execute server.http.port = tcp://<pod_id_address>:9000 and crash with the following error:

Invalid HTTP port: tcp://<pod_id_address>:9000

Work-around

We identified 2 workarounds:

  1. Override CEREBRO_PORT to force Cerebro port. You may add the following lines to your container:
    env:
    - name: "CEREBRO_PORT"
      value: "9000"
  2. Set enableServiceLinks=false in your pod deployment (See https://kubernetes.io/docs/concepts/services-networking/connect-applications-service/#accessing-the-service) to avoid environment variables conflicts

Thx,it works.

@moliware
Copy link
Collaborator

moliware commented Jun 2, 2020

Thank you very much for the investigation @vmercierfr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants