New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose memory usage for each function execution #1451

Open
davidebbo opened this Issue Apr 24, 2017 · 30 comments

Comments

Projects
None yet
@davidebbo
Copy link
Contributor

davidebbo commented Apr 24, 2017

We should expose this since the billing is based on that. Since App Insights is the future, that would be the right place for users to see this, alongside the rest of the data about each execution (duration, logs, ...).

@paulbatum

This comment has been minimized.

@davidebbo

This comment has been minimized.

Copy link
Contributor

davidebbo commented Apr 24, 2017

I think the main ask that came in (internal alias thread) is to get the memory info for a specific function invocation, rather than aggregation over time (though that is good to have as well).

@martell

This comment has been minimized.

Copy link

martell commented Apr 26, 2017

Great timing on this thread.
I was actually looking for a way to pull out the usage to calculate correct cost to pass on to a client.

2017-04-25T17:02:06.131 Function started (Id=12345678-1234-1234-1234-123456789012)
2017-04-25T17:02:06.989 Auth Successful, Hello world! from user 12345
2017-04-25T17:02:07.005 Function completed (Success, Id=12345678-1234-1234-1234-123456789012, Duration=9877ms)

Would it be possible to add an argument to the Function completed line which does something like this (Success, Id=12345678-1234-1234-1234-123456789012, Duration=9877ms, Memory=862MB)

@paulbatum

This comment has been minimized.

Copy link
Member

paulbatum commented Apr 26, 2017

@martell I agree that would be nice. Unfortunately we do not have an accurate measure of per-function memory usage within the runtime component that is emitting these logs.

For your scenario I suggest you create a function app per client and then use the monitor API discussed here to get function execution units per function app and use those as the basis of your cost calculation.

@paulbatum

This comment has been minimized.

Copy link
Member

paulbatum commented May 30, 2017

Closing as answered.

@paulbatum paulbatum closed this May 30, 2017

@Fabsi110

This comment has been minimized.

Copy link

Fabsi110 commented Jul 15, 2017

@paulbatum Hmm, I don't think this is very satisfying. In AWS Lambda the log contains that information:

RequestId: 1df6eacf-xxxx Duration: 15000.37 ms Billed Duration: 15000 ms Memory Size: 512 MB Max Memory Used: 71 MB

This is really nice transparency for every single request, so I can reproduce how my bill was generated. They don't even need to include the "Max Memory Used", because their pricing model is based on the duration only. So there is a price list for every second of execution time for the different memory sizes I can choose for my Function, so it is sufficient for me to take that price per second of execution for the memory size of the Function and I have my price.
Your pricing model is way more flexible as you don't have this fixed memory size I need to assign to a Function as AWS has. So I cannot calculate a price of a single function invocation at all with the information you provide...
Sure I can use something you somehow already aggregated, but do you see the problem regarding billing transparency?

@paulbatum paulbatum added the feature label Jul 17, 2017

@paulbatum paulbatum added this to the Backlog milestone Jul 17, 2017

@paulbatum

This comment has been minimized.

Copy link
Member

paulbatum commented Jul 17, 2017

Makes sense, I'll leave this open as a feature request for per-execution billing information to be exposed.

@paulbatum paulbatum reopened this Jul 17, 2017

@ataylorm

This comment has been minimized.

Copy link

ataylorm commented Aug 17, 2017

I would agree this is highly needed. Right now I'm using Metrics to see the cost of a single execution. I've got a function that simply downloads the HTML of a page sends it back as it's response. According to VS 2017 Diagnostic Tools, running this function locally is using all of 81 MB of memory (I am assuming that's mostly overhead, as the snapshot shows a total of only about 200KB data memory usage). However executing the function one time on Azure shows 1 execution of 760.06K memory usage which if I understand the math correctly is approximately 760 ms of 1 GB memory usage. My function returned in 723 milliseconds. So I can only assume that means somehow from running locally to running on azure I have a 1200% increase in memory usage of my function on Azure. It would be great to confirm this in the logs somehow.

@Fabsi110

This comment has been minimized.

Copy link

Fabsi110 commented Dec 13, 2017

@paulbatum do you have any update to this?

@paulbatum

This comment has been minimized.

Copy link
Member

paulbatum commented Dec 13, 2017

No updates at this time. We have received occasional feedback on the desirability of per execution billing info but not enough to prioritize this over other work.

@stap123

This comment has been minimized.

Copy link

stap123 commented Jan 1, 2018

+1 This would be useful

@JeffHughes

This comment has been minimized.

Copy link

JeffHughes commented Jan 30, 2018

+1

@NichUK

This comment has been minimized.

Copy link

NichUK commented Mar 12, 2018

Definitely +1 need to be able to break down costs for clients, and better predict what costs will be when functions scale up through use.

@Fabsi110

This comment has been minimized.

Copy link

Fabsi110 commented Mar 12, 2018

@paulbatum since your last comment, three people upvoted this issue and a similar issue was closed as duplicate. What do you think about the priority of this issue?

@paulbatum

This comment has been minimized.

Copy link
Member

paulbatum commented Mar 12, 2018

@Fabsi110 It is not a high priority for us right now. Our efforts are focused on issues that block adoption of functions (running cross platform / perf / reliability / more language support) and the data we have so far indicates this feature is more of a nice to have than a blocker.

@Fabsi110

This comment has been minimized.

Copy link

Fabsi110 commented Mar 12, 2018

@paulbatum I understand your argumentation and I think all those features are very important. Unfortunately, I think this feature is not just nice-to-have. Nobody can verify your bills right now and this is not appropriate. When you want money from the people, they should be able to understand and verify the cost. Your competitors (AWS and Google) are a lot more transparent and you should follow them.

@JeffHughes

This comment has been minimized.

Copy link

JeffHughes commented Mar 12, 2018

@paulbatum

This comment has been minimized.

Copy link
Member

paulbatum commented Mar 12, 2018

@JeffHughes @Fabsi110 thanks for providing your feedback, I'll make sure this issue gets some visibility with product owners.

@stap123

This comment has been minimized.

Copy link

stap123 commented Mar 12, 2018

I completely agree, it is VERY difficult to get any kind of buy in when your asked how much is it likely to cost to run XYZ. and you have to respond no idea, let just do it anyway and we'll know after a month.

You just can't quantify "a function" so quantifying a project that likely to contain several is exponentially harder.

The things you've listed are absolutely important @paulbatum and will be blocking some people adopting functions but I'm pretty sure the next thing on their list of "can we use this" will be cost.

@NichUK

This comment has been minimized.

Copy link

NichUK commented Mar 12, 2018

Just to chime in as well, I completely agree with @Fabsi110. It's about transparency as much as letting us be able to budget from some test runs, without jumping through a lot of hoops to try and work out locally what it might cost in reality.

@wlloyduw

This comment has been minimized.

Copy link

wlloyduw commented May 24, 2018

If the memory usage per function is not available, does this suggest that Azure Function's billing implementation is based on accumulated memory usage for a function over time?

In other words, there is a meter tracking accumulated memory usage at the function level (what the graphs show), but not at the call-level?

image

Thank you!

@rocklan

This comment has been minimized.

Copy link

rocklan commented Jun 8, 2018

Very surprised that I have to sit here and try to make an argument that it's helpful for customers to be able to predict their costs and that it's not just a nice to have. This is a vital feature and without it makes it impossible to me to justify Azure Functions to anyone inside the business. In the meantime I'm going to AWS where I can tell my boss how much it will save us to move a bunch of our code to this new serverless architecture.

@paulbatum

This comment has been minimized.

Copy link
Member

paulbatum commented Jun 8, 2018

I understand why this feature is often requested and am not debating its usefulness. I've directly pointed my team at this issue to make sure that the level of interest behind this feature request is understood.

However I personally think that claiming that its absence means you cannot predict your costs is misleading. Here's why:

  1. You can just write your function and run it once with input that is representative of the expected workload and use the metrics that Azure exposes with the appropriate time filter and granularity to find out the cost of that execution.
  2. Any type of solution you put into production should undergo some type of performance testing beforehand and when you do this performance testing you'll generate load similar to what you expect when you go into production and the truly valuable metric is what your bill looks like under that load. Again you can use the metrics API to retrieve that data.

Is it as simple as running a function and having the UX directly tell you how much it cost? No. See my first comment above!

Any feedback on why the approaches I outlined are problematic is of course welcome.

@JeffHughes

This comment has been minimized.

Copy link

JeffHughes commented Jun 8, 2018

@paulbatum

This comment has been minimized.

Copy link
Member

paulbatum commented Jun 8, 2018

I don't understand how being able to query the system for the cost of an individual function execution helps with that problem. If anything, it backs up my statement that collecting the billing metrics based on a load/performance test is a better approach.

@manusis

This comment has been minimized.

Copy link

manusis commented Jun 26, 2018

+1. We are a SaaS platform and we need this information to bill our clients according to their functions usage.

@witek1902

This comment has been minimized.

Copy link

witek1902 commented Oct 31, 2018

Any changes in Functions 2.0?

@paulbatum

This comment has been minimized.

Copy link
Member

paulbatum commented Nov 1, 2018

No changes.

@witek1902

This comment has been minimized.

Copy link

witek1902 commented Nov 2, 2018

Any plans?

@fhtino

This comment has been minimized.

Copy link

fhtino commented Jan 10, 2019

In my humble opinion, Azure billing has always been a problem. A lot of services lack a clear and easy way to verify the costs placed in the monthly bill. Azure Functions is one of these. Please, be more transparent and point out memory usage on every request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment