New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide high-level monitoring experience (similar to Data Factory) #759

Closed
tomkerkhove opened this Issue Nov 9, 2016 · 21 comments

Comments

Projects
None yet
6 participants
@tomkerkhove

tomkerkhove commented Nov 9, 2016

It would be great to have high-level monitoring available that visualizes all the functions in an instance, similar to how Azure Data Factory does it in the Monitor-feature.

Following information would be interesting:

  • Failure rate per function
  • Dependency metrics per binding, per function (similar to AI Dependency)
  • Function state (enabled/disabled)
  • ...

Feature request is also available on UserVoice

@tomkerkhove

This comment has been minimized.

Show comment
Hide comment
@tomkerkhove

tomkerkhove Feb 7, 2017

What is the status of this feature?

tomkerkhove commented Feb 7, 2017

What is the status of this feature?

@lindydonna

This comment has been minimized.

Show comment
Hide comment
@lindydonna

lindydonna Mar 18, 2017

Contributor

This will be handled through first-class app insights integration: Azure/azure-functions-host#727

Closing this issue.

Contributor

lindydonna commented Mar 18, 2017

This will be handled through first-class app insights integration: Azure/azure-functions-host#727

Closing this issue.

@lindydonna lindydonna closed this Mar 18, 2017

@tomkerkhove

This comment has been minimized.

Show comment
Hide comment
@tomkerkhove

tomkerkhove Mar 21, 2017

@lindydonna please re-open this issue as this is not covered by Application Insights.

This issue is to provide a high-level overview of all your Functions in a single Function-instance that gives you an at a glance overview of the health à la Azure Data Factory monitoring experience.
adf_walkthrough_variation-v2_thumb

A pipeline would be replaced with a function that has linked triggers/outputs that shows me all what is going on.

tomkerkhove commented Mar 21, 2017

@lindydonna please re-open this issue as this is not covered by Application Insights.

This issue is to provide a high-level overview of all your Functions in a single Function-instance that gives you an at a glance overview of the health à la Azure Data Factory monitoring experience.
adf_walkthrough_variation-v2_thumb

A pipeline would be replaced with a function that has linked triggers/outputs that shows me all what is going on.

@lindydonna

This comment has been minimized.

Show comment
Hide comment
@lindydonna

lindydonna Mar 21, 2017

Contributor

@tomkerkhove We plan to enable the Application Insights application map, which provides a very similar view. See https://docs.microsoft.com/en-us/azure/application-insights/app-insights-app-map#open-the-application-map

Contributor

lindydonna commented Mar 21, 2017

@tomkerkhove We plan to enable the Application Insights application map, which provides a very similar view. See https://docs.microsoft.com/en-us/azure/application-insights/app-insights-app-map#open-the-application-map

@tomkerkhove

This comment has been minimized.

Show comment
Hide comment
@tomkerkhove

tomkerkhove Mar 22, 2017

What will it look like then? How will it indicate an Azure Function? I don't see how AI will display failed runs then and the timeslots it ran for example while the Azure Data Factory clearly shows this for the same use-case as Azure Functions in my opinion.

tomkerkhove commented Mar 22, 2017

What will it look like then? How will it indicate an Azure Function? I don't see how AI will display failed runs then and the timeslots it ran for example while the Azure Data Factory clearly shows this for the same use-case as Azure Functions in my opinion.

@christopheranderson

This comment has been minimized.

Show comment
Hide comment
@christopheranderson

christopheranderson Mar 27, 2017

Member

Going to reopen this.

We are planning on leveraging App Insights App Map because they will be able to handle cross app/dependency stuff WAY better than we could ourselves.

That said, what they have right now isn't sufficient. I'm working with them to get a proper, cross service DFD built that would show all the steps and dependencies of a given execution, similar to the behavior of Data Factory. Using App Insights means we can have more than 1 Function App (different services or different version of the same service running side by side), plus other compute services like Service Fabric or App Service all working in a single diagram. AI has already started work on improving their data model to support this, so I'm pretty confident we'll have a good solution, probably Summer-ish, but can't commit until we have a proper engineering design.

Self-assigning this one since it's tracking a partner team ask, for the time being.

Member

christopheranderson commented Mar 27, 2017

Going to reopen this.

We are planning on leveraging App Insights App Map because they will be able to handle cross app/dependency stuff WAY better than we could ourselves.

That said, what they have right now isn't sufficient. I'm working with them to get a proper, cross service DFD built that would show all the steps and dependencies of a given execution, similar to the behavior of Data Factory. Using App Insights means we can have more than 1 Function App (different services or different version of the same service running side by side), plus other compute services like Service Fabric or App Service all working in a single diagram. AI has already started work on improving their data model to support this, so I'm pretty confident we'll have a good solution, probably Summer-ish, but can't commit until we have a proper engineering design.

Self-assigning this one since it's tracking a partner team ask, for the time being.

@tomkerkhove

This comment has been minimized.

Show comment
Hide comment
@tomkerkhove

tomkerkhove Mar 28, 2017

Thanks for the clarification @christopheranderson, sounds very useful to me - Thanks!

tomkerkhove commented Mar 28, 2017

Thanks for the clarification @christopheranderson, sounds very useful to me - Thanks!

@tomkerkhove

This comment has been minimized.

Show comment
Hide comment
@tomkerkhove

tomkerkhove Aug 9, 2017

What's the status of this? Eager to see this coming

tomkerkhove commented Aug 9, 2017

What's the status of this? Eager to see this coming

@tomkerkhove

This comment has been minimized.

Show comment
Hide comment
@tomkerkhove

tomkerkhove commented Aug 30, 2017

Any update on this @lindydonna @christopheranderson?

@tomkerkhove

This comment has been minimized.

Show comment
Hide comment

tomkerkhove commented Oct 19, 2017

@tomkerkhove

This comment has been minimized.

Show comment
Hide comment
@tomkerkhove

tomkerkhove Dec 4, 2017

Any updates on this? /cc @jeffhollan

tomkerkhove commented Dec 4, 2017

Any updates on this? /cc @jeffhollan

@btardif

This comment has been minimized.

Show comment
Hide comment
@btardif

btardif Mar 26, 2018

Member

@nertim consider this your main item for AI integration

Member

btardif commented Mar 26, 2018

@nertim consider this your main item for AI integration

@tomkerkhove

This comment has been minimized.

Show comment
Hide comment
@tomkerkhove

tomkerkhove Mar 28, 2018

For what it's worth - Do we really need a dependency on AI for this? And if you do, please allow me to configure what you guys will start emitting because I don't want my AI instance to be swamped with other information if I only want to have the overview.

Always happy to review something if you want

tomkerkhove commented Mar 28, 2018

For what it's worth - Do we really need a dependency on AI for this? And if you do, please allow me to configure what you guys will start emitting because I don't want my AI instance to be swamped with other information if I only want to have the overview.

Always happy to review something if you want

@tomkerkhove

This comment has been minimized.

Show comment
Hide comment
@tomkerkhove

tomkerkhove May 5, 2018

tomkerkhove commented May 5, 2018

@nertim

This comment has been minimized.

Show comment
Hide comment
@nertim

nertim May 5, 2018

Contributor

There are several changes related to monitoring that have been checked in.. So many of the items related to the monitoring could have been closed. The route taken was to have deeper AI integration, and provide the ability to allow user to easily navigate to the AI for more detailed logs and streams.

Please feel free to reopen if needed.

Contributor

nertim commented May 5, 2018

There are several changes related to monitoring that have been checked in.. So many of the items related to the monitoring could have been closed. The route taken was to have deeper AI integration, and provide the ability to allow user to easily navigate to the AI for more detailed logs and streams.

Please feel free to reopen if needed.

@tomkerkhove

This comment has been minimized.

Show comment
Hide comment
@tomkerkhove

tomkerkhove May 7, 2018

Don't really want to reopen with the currnet information, but if I would want to do it - I can't :)

However, can you please elaborate on what you mentioned? Is this already available or coming? Are there other issues that are related here? AI is good but I don't want to be enforced to use AI to simply have a high-level overview of all my functions, their sources and sink.

Would be really great to have this in the landing page of Azure functions:
image

tomkerkhove commented May 7, 2018

Don't really want to reopen with the currnet information, but if I would want to do it - I can't :)

However, can you please elaborate on what you mentioned? Is this already available or coming? Are there other issues that are related here? AI is good but I don't want to be enforced to use AI to simply have a high-level overview of all my functions, their sources and sink.

Would be really great to have this in the landing page of Azure functions:
image

@nertim

This comment has been minimized.

Show comment
Hide comment
@nertim

nertim May 7, 2018

Contributor

Ah okay.. sorry my brain waves got crossed on this one.. So what you're asking for is not just logs and worker/instance information.. but also deeper visuals of how your functions executions. @btardif what are your thoughts on this?

Contributor

nertim commented May 7, 2018

Ah okay.. sorry my brain waves got crossed on this one.. So what you're asking for is not just logs and worker/instance information.. but also deeper visuals of how your functions executions. @btardif what are your thoughts on this?

@tomkerkhove

This comment has been minimized.

Show comment
Hide comment
@tomkerkhove

tomkerkhove May 7, 2018

That's right but mainly a high-level visualization of all functions in my function apps and what sinks & sources they have, would be great to see how functions are chained.

tomkerkhove commented May 7, 2018

That's right but mainly a high-level visualization of all functions in my function apps and what sinks & sources they have, would be great to see how functions are chained.

@btardif

This comment has been minimized.

Show comment
Hide comment
@btardif

btardif May 10, 2018

Member

While this is a good idea, I think we need a different item to track it as a feature request, I'm not sure we have all the necessary backed support for this.

Also the view might be limited to only the function within a given function app, if we need to show functions in different function apps that might go beyond what we can/should do.

Member

btardif commented May 10, 2018

While this is a good idea, I think we need a different item to track it as a feature request, I'm not sure we have all the necessary backed support for this.

Also the view might be limited to only the function within a given function app, if we need to show functions in different function apps that might go beyond what we can/should do.

@jeffhollan

This comment has been minimized.

Show comment
Hide comment
@jeffhollan

jeffhollan May 10, 2018

Member

@alexkarcher-msft do you know the status of app-map support with Azure Functions? Now that App Insights is enabled by default with new functions if we emitted the right resources we could have some visualization. I don't think we would build a full design pages like data factory pipelines or logic apps workflows though - though I understand the desire for one.

Member

jeffhollan commented May 10, 2018

@alexkarcher-msft do you know the status of app-map support with Azure Functions? Now that App Insights is enabled by default with new functions if we emitted the right resources we could have some visualization. I don't think we would build a full design pages like data factory pipelines or logic apps workflows though - though I understand the desire for one.

@tomkerkhove

This comment has been minimized.

Show comment
Hide comment
@tomkerkhove

tomkerkhove May 10, 2018

Also the view might be limited to only the function within a given function app, if we need to show functions in different function apps that might go beyond what we can/should do.

@btardif Just to clarify - I'm not expecting this to be across Function Apps, but only across Functions inside one Function App, but that's what you mean as well I think?

@jeffhollan I get that Application Insights could be a fit but I'm just not a fan of having to use Application Insights, simply because I want to have a high-level overview that should be out-of-the-box, and also makes my pay more because you are sending telemetry - Which will implicitly defeat the purpose of "serverless". That said, I kinda get that you don't want to reinvent the wheel.

If it would be valuable, I'd be willing to jump on a call.

tomkerkhove commented May 10, 2018

Also the view might be limited to only the function within a given function app, if we need to show functions in different function apps that might go beyond what we can/should do.

@btardif Just to clarify - I'm not expecting this to be across Function Apps, but only across Functions inside one Function App, but that's what you mean as well I think?

@jeffhollan I get that Application Insights could be a fit but I'm just not a fan of having to use Application Insights, simply because I want to have a high-level overview that should be out-of-the-box, and also makes my pay more because you are sending telemetry - Which will implicitly defeat the purpose of "serverless". That said, I kinda get that you don't want to reinvent the wheel.

If it would be valuable, I'd be willing to jump on a call.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment