Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP Azure Function Trigger not scaling up #34

Open
rahulpr17 opened this issue Jul 12, 2019 · 7 comments

Comments

@rahulpr17
Copy link

commented Jul 12, 2019

Environment:

  • Kubernetes distribution - AKS - 3 node cluster
  • Kubernetes version - v1.15.0
  • Osiris version: Latest
  • Install method: Azure Core Tools

What happened?
I was trying to deploy an HTTP Trigger to keda. I have installed Osiris components for the same. It helped me to scale to zero when no request is coming, but it is not scaling up from 1 instance. I have removed all replica-constraints from deploy.yaml file still no effects. Can you help me with any supportive links?

What you expected to happen?
I expect the nodes to be scaled up when doing a load test for 100 users. But it always show 1 instance
How to reproduce it (as minimally and precisely as possible):
Deploy a Http trigger to Keda and load test >100 user
Anything else that we need to know?

@krancour

This comment has been minimized.

Copy link
Member

commented Jul 12, 2019

Honestly, I was a little surprised to discover that Azure Functions Core Tools now includes Osiris since Osiris isn't GA. In fact, there hasn't even been a single release, GA or otherwise. You reported using latest, and since we haven't had a release, I equate "latest" with "head of the master branch." I had to try this for myself and discovered that the Osiris that's bundled with Azure Functions Core Tools lags the head of master by a little bit-- but enough that I'm sure you're missing some bugs that were incidentally fixed when we refactored things a bit to include HTTP/2 support. And this is assuming your Azure Functions Core Tools itself is the latest release. If you installed using an older version, your Osiris could be even older.

I'm curious what kind of luck you have if you install Osiris that is built from the head of the master branch using instructions here:

https://github.com/deislabs/osiris#install-osiris

@rahulpr17

This comment has been minimized.

Copy link
Author

commented Jul 15, 2019

I have tried the above link and installed osiris using the helm package. But the result is same. No scaling upwards. Do you have any document and working sample apart from the above link which i can use and confirm if its working fine?

@krancour

This comment has been minimized.

Copy link
Member

commented Jul 18, 2019

There's just the hello-osiris app.

Can you share steps to reproduce? Is there an ingress controller or anything involved here that isn't assumed by the install / hello-osiris instructions?

Do you see anything in the activator logs to indicate it is / isn't receiving traffic for scaled down deployments? Or anything in the activator logs that explicitly indicates activation is failing?

@cwoolum

This comment has been minimized.

Copy link

commented Jul 19, 2019

In your service, you not only need the enabled and deployment annotations but you also need osiris.deislabs.io/loadBalancerHostname or Osiris won't know how to respond to messages.

@krancour, is there a reason that annotation isn't included in the usage section of the main page since this doesn't seem to work without it? Am I missing another use case where it's not needed?

@krancour

This comment has been minimized.

Copy link
Member

commented Jul 23, 2019

In your service, you not only need the enabled and deployment annotations but you also need osiris.deislabs.io/loadBalancerHostname or Osiris won't know how to respond to messages.

This isn't universally true. It may be true depending on how the service is being addressed. It's the reason I asked this:

Is there an ingress controller or anything involved here that isn't assumed by the install / hello-osiris instructions?

is there a reason that annotation isn't included in the usage section... ?

It shouldn't be required to make the example work. That said, the annotations absolutely should be documented somewhere and they currently are not. PRs are welcome.

@rahulpr17

This comment has been minimized.

Copy link
Author

commented Jul 24, 2019

Hi Thanks for looking into the issue. I am copying the deployment.yaml file which i was using to deploy the service. The service is a basic service which we create from azure function http trigger.

data:
AzureWebJobsStorage: ''
FUNCTIONS_WORKER_RUNTIME: ZG90bmV0

apiVersion: v1
kind: Secret
metadata:
name: init-sample
namespace: default

apiVersion: v1
kind: Service
metadata:
name: init-sample-http
namespace: default
annotations:
osiris.deislabs.io/enabled: true
osiris.deislabs.io/deployment: init-sample-http
spec:
selector:
app: init-sample-http
ports:

  • protocol: TCP
    port: 80
    targetPort: 80
    type: LoadBalancer

apiVersion: apps/v1
kind: Deployment
metadata:
name: init-sample-http
namespace: default
labels:
app: init-sample-http
annotations:
osiris.deislabs.io/enabled: true
spec:
selector:
matchLabels:
app: init-sample-http
template:
metadata:
labels:
app: init-sample-http
spec:
containers:
- name: init-sample-http
image: docker4rahul/init-sample
ports:
- containerPort: 80
env:
- name: AzureFunctionsJobHost__functions__0
value: Function1
envFrom:
- secretRef:
name: init-sample

Below is the screen shot of the deployment for osisris available in the namespace.

keda 1/1 1 1 18d
osiris-osiris-edge-activator 1/1 1 1 18d
osiris-osiris-edge-endpoints-controller 1/1 1 1 18d
osiris-osiris-edge-endpoints-hijacker 1/1 1 1 18d
osiris-osiris-edge-proxy-injector 1/1 1 1 18d
osiris-osiris-edge-zeroscaler 1/1 1 1 18d

Please let me know if i am missing any components here.

@krancour

This comment has been minimized.

Copy link
Member

commented Jul 26, 2019

@rahulpr17 logs from the activator would be helpful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.