-
Notifications
You must be signed in to change notification settings - Fork 170
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
V2 Planning - Ideas and Feedback Needed! #85
Comments
I have an odd perspective on this. My desire is to ship go errors straight to the db, and view them via opserver. A few things I'd love to see for that, recognizing I'm not going to be a top-level supported use case:
|
I like what you're doing, @NickCraver For the One other thing that I've done for my own use (though I have thought about releasing it as an independent add-on package) is I created an Sorry for the above wall of text. I just couldn't think of a simpler way of explaining it. Let me know if you're interested in it though. 😉 |
@captncraig I think the story there would have to be go sending an error to an endpoint (I'd be happy to package an ASP.NET Core endpoint in the Exceptional repo) that logs to the DB, accepting JSON (or Opserver could do this). In short, it doesn't make sense not to have .NET in the flow if we're using it to view errors anyway. That and Go's packaging story is a complete trainwreck, so there's no "update the library" when the schema changes, it's a lot more work, a lot more practical skew, and just more problems overall. But throwing JSON fields that rarely change: that we can work with. It's actually how we supposed Node.js back in the day. There's a related issue here: .NET will have The stack trace aspect is an interesting one, as that'll need some handling. The prettification is very much for .NET format stack traces, conventions, classes, etc. It's not pluggable nor could it be. The overall pluggability of which prettifier to use is a thing maybe, but the error would have to indicate which platform it was from somehow. I'd hate to record ".NET" a billion times in a table for most shops, but an enum doesn't work. Recognizing stacks or conventions seems brittle, thoughts? As for the individual questions:
|
@CptRobby I've got the date bits locally, it's pretty simple - not worried about that piece. Thanks though! I admit I'm not following the rest - the abstraction is what |
@NickCraver No problem. I just thought I'd offer to help on that part first since it was a low hanging branch for me. Got any other pieces I could try my hand at? 😄 Regarding the rest, sorry about the confusion. The key point that I used was the I probably could have just done it all from within my website's code without the abstraction layer. Then I would just have a class that inherited from The reason I wanted to use EntityFramework for storing the errors was twofold. First, I wanted to tweak the structure of the table a bit without having to modify the SQL strings in the Hopefully that helps explain why I did it the way I did. I'm not expecting you to take this and do the same thing, that would probably be pointless. If you did want to take anything away from this though, you could change the I know though that it would largely be pointless for the majority of users who just use the built in error stores, so I'm not really expecting anything. Just sharing with you how I've been using it. 😉 |
Addresses comments in #85 and adds flexibility.
Rleated to #85, this adds the last date an exception occured. While CreationDate is needed for de-dupe rollups, we want to show the actual last time it happened in the dashboard...this does that. Schema scripts for SQL and MySQL also made to be re-runnable and update schemas to v2 (more changes incoming). INFORMATION_SCHEMA is used in SQL to make it more portable for more platforms later.
@CptRobby Understood! I've just push a commit that changes things up. The original reasoning was based on current plans for ASP.NET Core and AppDomain which have pulled a 180 since. I changed the implementation to look at all assemblies in the domain for any inheritors of |
@NickCraver I saw the commit. My only question regarding the change is how it would behave with the The situation I'm thinking about (convoluted yes, but not unreasonably so) is a user who has their website all set up and running. It currently is using the I honestly don't know the answer. I tried searching to see how assemblies are loaded in ASP.NET, but got conflicting answers. Obviously if the Also, this question needs to be considered for native .NET applications. One of the results that I found said that assemblies are loaded differently in ASP.NET from the way they're loaded in native .NET applications. If in either case the assemblies aren't just loaded automatically without any code references, then maybe just add a section before the point where the loaded assemblies are searched to make sure that any files in the current folder that match "StackExchange.Exceptional.*.dll" are loaded. |
Just found a really well written blog post by Rick Strahl that seems to answer the question definitively: https://weblog.west-wind.com/posts/2012/Nov/03/Back-to-Basics-When-does-a-NET-Assembly-Dependency-get-loaded In that article, it says that "ASP.NET pre-loads all assemblies referenced in the GAC assembly list and the /bin folder." But for .NET applications, the JIT compiler avoids loading any assemblies until the very moment they are needed. This means that to enable the easy drop-in approach for .NET applications, we should probably insert the following code: var path = Path.GetDirectoryName(typeof(ErrorStore).Assembly.Location);
//Note that I added another . before the *, since we know that we already have
//StackExchange.Exceptional.dll loaded
foreach (var filename in Directory.GetFiles(path, "StackExchange.Exceptional.*.dll"))
{
Assembly.LoadFrom(filename);
} This will ensure that StackExchange.Exceptional.MySql.dll (or any other assembly following the naming convention) is loaded even though it isn't referenced in code. |
I really like the idea of having an API that we could use to log from other platforms (ie mobile apps). Our devs have been asking for this for a while to centralize our error logging. One question though, where would this "API" (or single endpoint rather?) live? In an application that uses Exceptional? Or would we essentially spin up a new instance of it to act as just an API? I'd be happy to help contribute to this if it's a route you'd like to pursue. While not straightforward, depending on how you were planning on the |
Can we implement a custom module or a simple module which we can add it into dashboard page with simple information |
@minhhungit What type of module are you talking about exactly? Just saying that you could add it into a dashboard page doesn't help since there are numerous types of dashboards (including StackExchange's Opserver). And are you asking for an API to allow you to build a custom module or are you asking to be given a module ready to go? |
@CptRobby some APIs that we can create a simple panel to contain a list top error as table, and link it to exception main page when we click on them (we also should have setting to disable the link) |
Thanks for the help everyone (especially contributors!), 2.0 alpha packages are now up on NuGet: https://www.nuget.org/packages?q=StackExchange.Exceptional Special props to @StefanKert on lots of work for ASP.NET Core configuration and @CptRobby for fixes in 2.0 as well. |
Just a heads up, your example and documents claim you can call In order to exclusively pull the configuration from the appsettings file , you have to use I'm actually trying to re-add Exceptional to my project, since I was unable to keep it when I ported an existing project from ASP.NET to ASP.NET Core 1.1. It'll be nice to have it back again. |
Also, this works.
but this does not
|
@nickalbrecht Thanks! I fixed both of those just now, they'll appear on NuGet shortly. Please keep sending any issues you hit! |
On a more feedback/suggestion note, versus bug - is it possible to use a I know the MiniProfiler uses The use case for me is that I have razor views for certain tasks stored in a database that an end user can alter (email templates for example), and it'd be nice to let them see their own errors when the error is due to the dynamic compilation, but not let them see exceptions core to the application we provide to them. |
It would be fantastic if v2 offered a better display on mobile devices. So
often I'm trying to troubleshoot production issues while away from my
computer and it would be nice to not have to pinch zoom and be able to get
to the information I need faster. A thought here is to embed bootstrap and
use the grid layout. A more in depth option would be to leverage jQuery
Data Tables (which has a bootstrap version), but provides some additional
functionality such as sorting and searching, but would come at the cost of
more dependencies.
…On Tue, Aug 15, 2017 at 1:55 PM, Nick Albrecht ***@***.***> wrote:
On a more feedback/suggestion note, versus bug - is it possible to use a Func<HttpRequest,
bool> for a UseExceptionalPageOnThrow check (or some variation of it), to
allow the possibility of letting certain users view the Exceptional error
page, otherwise it falls through to the standard friendly error handler (or
whatever else is setup)? It'd be awesome to be able to call
IAuthorizationService.AuthorizeAsync() and pass the Exception as the
Resource parameter to see if they are are allowed to view details for that
type of error. Though that might require the signature to be something like Func<HttpRequest,
Exception, bool> unless Exceptional could just take the Policy name as
the parameter and it calls AuthorizeAsync() (which would add a dependency
of Microsoft.AspNetCore.Authorization.
I know the MiniProfiler uses Func<HttpRequest, bool> for checking things
like ResultsListAuthorize, ShouldProfile, ResultsAuthorize, etc.
The use case for me is that I have razor views for certain tasks stored in
a database that an end user can alter (email templates for example), and
it'd be nice to let them see their own errors when the error is due to the
dynamic compilation, but not let them see exceptions core to the
application we provide to them.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#85 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAU-504X_iVS1oHtLnJqBPPriIeeYjwvks5sYgWygaJpZM4OZe7z>
.
|
I've got the Exceptions list working but nothing else. Any of the child 'KnownRoutes', I was getting a 404 error on. Here's what I have, since it contributed to my problem, and might help others. I have an ErrorsController that handles feedback for BadRequest (400), Forbidden (403), and NotFound (404), etc. I've added a new action for Exceptional as suggested
And this is wired up in the Startup.cs under
I can view the exceptions at
Using the word 'resource' in the route is not important, it just needs to have something there. It's the |
@ctorx The problem is that introduces a dependency on Bootstrap Since the logic for rendering the page's content is in UPDATE: It is not PERFECT. I didn't place a version number at the bottom of the page, and the ellipse character is messed up. It also ONLY replaces the main List page, Info would be a separate page, and all of the other known routes for JS are still handled internally to Exceptional. But still. https://gist.github.com/nickalbrecht/8f0c8f8cf82906049eb1e266f81e6f59 Note: This is not a new list. I basically translated @NickCraver's StringBuilder implementation out to a cshtml file |
It's also worth pointing out that while this is denoted as 2.0, a lot of the changes are only refactorings to get a reusable This is not to criticize, more to point out to others who have been using AspNet Core for a while, and might expect these aspects to be available.
|
I'm trying all of this on AspNetCore, not sure if its a contributing factor. Wanted to mention a few other bugs before I head home. Clicking on the lock icon to protect an exception and clicking the delete icon both work, but the response results in a Javascript error. Error details are from Chrome. IE gives an error too though it complains about expecting ';'
Clicking on the "Clear all non-protected errors" link at the bottom of the page results in an
Lastly, once you protect an exception, you can't un-protect it. |
@nickalbrecht Thanks for all the feedback, lots of good stuff here. Would you mind filing the "things I'd expect in ASP.NET Core" as an issue? Let's enumerate those, I agree things like I'll think about the |
Ok, so I'm using the package from MyGet. It all seems to be working fine at first glance. Will continue to play with it and see how it goes. I stepped away from my desk for a bit however, and when I came back I noticed around 3 errors spread out in the output window of VS (all with the same details) System.ArgumentNullException: Value cannot be null.
Parameter name: provider
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
at StackExchange.Exceptional.Extensions.Log(Exception ex, HttpContext context, String category, Boolean rollupPerServer, Dictionary`2 customData, String applicationName) I can't get them to happen again however, so I'm not confident it's an issue. All of the links for Protecting/Clearing an error, and clearing all unprotected errors - seem to work fine. Though they do still give JS errors in the browser console, it doesn't seem to interfere with their primary function. It's worth noting that in most cases the default for adding json files to the Configuration is to specify Or just use Singletons registered to I had to reword that a few times, hopefully my final explanation still makes sense :-P |
On a brainstorming note regarding my comment of expected overloads of Option 1 services.AddExceptional(Configuration.GetSection("Exceptional"), settings => { /* Manipulate settings here */ }); //pull from app settings first, then apply code settings Option 2 services.Configure<ExceptionalSettings>(Configuration.GetSection("Exceptional")); //start with configuration sources first
services.Configure<ExceptionalSettings>(settings => { /* Manipulate settings here */ }); //then apply code changes Option 3 services.AddExceptional()
.FromConfig(Configuration.GetSection("Exceptional"))
.FromFunc(settings => { /* Manipulate settings here */ }); Option 4 services.AddExceptional(svcs =>{
svcs.Configure<ExceptionalSettings>(Configuration.GetSection("Exceptional"));
svcs.Configure<ExceptionalSettings>(settings => { /* Manipulate settings here */ });
}); I realize part of the reason it's done inside of the encapsulated method forgive any typos I'm just typing it and copy/pasting parts, but you get the idea services.ConfigureAll<ExceptionalSettings>(Configuration.GetSection("Exceptional"));
services.Configure<ExceptionalSettings>("AdminPortal", Configuration.GetSection("AdminExceptional"));
services.Configure<ExceptionalSettings>("PublicPortal", Configuration.GetSection("PublicExceptional"));
services.PostConfigureAll<ExceptionalSettings>(settings => {settings.Email.ToAddress += "IT@example.com";});
//Edit: On second though, I'm not sure the sequential bindings to Configuration.GetSection works as expected in my head, but the idea is still there :-P //Usage
var options = HttpContext.RequestServices.GetService<IOptionsSnapshot<ExceptionalSettings>>();
var adminOptions = options.Get("AdminPortal"); Might be better for 2.1, but it's worth thinking about to get an API surface/accessibility modifiers that doesn't need to change between versions At some point I'm sure you're going to be thinking "Damnit Nick, stop poking holes already!" :-P |
@nickalbrecht I appreciate the thoughts - all of that makes sense as stated, thanks! The tradeoffs there are complexity and in the snapshot case, allocations - both of which we want to avoid. I think I'm most comfortable with the current setup and let's see id we need to add additional options later. I'm currently considering registering the HttpContextAccessor for users if they use the I'm thinking |
Sounds good. Just wanted to make sure you were aware of the potential hurdles so that it's easier to triage should something come up.
The Logger method sounds good. The only catch would be making sure the implementation would ignore calls to log errors from within the StackExchange.Exceptional namespace to prevent recursive calls. It might also be worth seeing if using |
Were you using Identity? That's the big often-used item that adds the context accessor. Some or most will have it, but not all. Those after extreme perf (e.g. here at Stack Overflow) won't have it registered, so can't assume :) |
You're right, it's not resolving when I do it in a new project with no authentication. So it'd have to be either added in the setup of Exceptional using the How much of an impact does it have performance wise? I think there's a bug on the list of Exceptions in the current alpha version. The URL column is always empty, even though in the details the value is there? |
@nickalbrecht The weight is from registering it, regardless of usage. The weight is all in the I'm going to assume you're using SQL if hitting the |
Really? Wow, ok. Well that means I get to start refactoring some code to not rely on it so much since it was so easy to use. Yup I am. Sweet! I'll grab the update :-) |
Finally had a chance to upgrade our projects to ASPNET Core 2.0 and installed Exceptional in one of them so far. Using the In Memory store, there don't seem to be any errors. Thanks @NickCraver! |
Using the latest version on Nuget ( Here's my config: appsettings.json "Exceptional": {
"Store": {
"ApplicationName": "APP NAME",
"Type": "SQL",
"ConnectionString": "Server=***;Database=***;User ID=***;Password=***;"
},
}, Startup.ConfigureServices() services.AddExceptional(Configuration.GetSection("Exceptional"), settings =>
{
settings.UseExceptionalPageOnThrow = HostingEnvironment.IsDevelopment();
}); |
That's exactly how mine is configured, and it's working correctly. I'm not sure what else you might be missing? It's worth noting you might want to upgrade to |
Alpha 3 is now on NuGet. @mmillican the latest libs should resolve the loading error that @nickalbrecht is referencing above. |
I noticed today that emails aren't working with Exceptional in my ASP Net Core project, and haven't for a while by the looks of it? I'm not sure. To try and debug this, I'm using the most recent version of the source from here <658888e>. |
@nickalbrecht ...oops? Yeah this never worked in ASP.NET Core but does in ASP.NET non-Core. I've just pushed fixes up for that and new packages should land on MyGet shortly. Thanks for the heads up! |
Awesome. I updated to build 00185 from MyGet and it looks like it's works as expected now. Thanks :-) |
Also, did something change with the recorded server variables? It used to record the Update: I was able to get the same functionality working with ASP.NET Core however using an public class UserNameExceptionFilterAttribute : IExceptionFilter
{
public void OnException(ExceptionContext context)
{
context.Exception.Data.Add("UserName", context.HttpContext.User.Identity.Name);
}
} public void ConfigureServices(IServiceCollection services)
{
services.AddExceptional(Configuration.GetSection("Exceptional"), x =>
{
x.GetCustomData = (exception, dictionary) =>
{
foreach (var key in exception.Data.Keys)
{
dictionary.Add(key.ToString(), exception.Data[key].ToString());
}
};
});
} |
You can also include any keys easily in the regex, for example to log any key containing "UserName", it'd be: public void ConfigureServices(IServiceCollection services)
{
services.AddExceptional(Configuration.GetSection("Exceptional"), x =>
{
x.DataIncludeRegex = new RegEx("UserName", RegexOptions.Compiled);
});
} |
Would this work to simply specify a regex that should match any Data item on the exception? x.DataIncludeRegex = new RegEx(".*", RegexOptions.Compiled); |
@nickalbrecht yep, it would indeed. We've been running with no problems at Stack Overflow now for a few weeks, so I'm promoting the packages to beta at this point. I want to revisit the ILogger stuff as soon as time allows, but since that's not a breaking change, we can just add at any time which is nice. I'm trying to catch up on so many OSS projects, so just doing what I can as time allows. |
Haha, yeah I can only imagine. Some brainstorming on ILogger However I can see if being useful to introduce a way to call Exceptional as part of the middleware pipeline, and create a logging scope for the request. Or maybe have it implicitly done with a LoggingProvider that logs to a backing collection on the HttpContext? Not sure which is better, LoggingProvider backed by the HttpContext, or a LoggingScope, or both? Then when an exception is written into the store, the logged messages up until that point could be retrieved from the scope/context, and included. Maybe a new section along side Server Variables, Cookies, Form, etc. Sort of like recording the old trace messages from classic asp.net? Be nice way to include extra debugging info that would be recorded in Exceptional, as well as any other logging providers that might also be connected, like console, or Azure. |
OK, I'm missing something here I think. I just upgraded to the latest Nuget version ( However, if my environment is Production, nothing gets logged. Is it because of this?
Note, when in production, I'm still seeing the standard .NET error page, which I would expect, but not sure if that's somehow swallowing the exception before it gets logged? |
@NickCraver Any timeframe for RC2? No commits for a while now, so I assume the changes after RC1 are stable? Edit: whoops, only just found out about the MyGet feed. |
@NickCraver,
|
The first one I can see being useful, but while I think you can override it to use your own page if you want to customize it, It'll take a bit of work to do. |
@NickCraver When i configure Exceptional with StackExchange.Exceptional/src/StackExchange.Exceptional.AspNetCore/ExceptionalServiceExtensions.cs Line 29 in 0529002
values from action is not setted in Statics.Settings and LogNoContext using InMemoryStorage :( |
@nickalbrecht @NickCraver Although the second suggestion is more user-friendly. The first one is very useful. That's awesome to rapidly and easily add a customisable link in top of Errors listing and Error Detail pages , for example add a Two options must be add to Exceptional options with overridable values, for example |
@DaniilSokolyuk do you have an example? (A separate issue will be best) - I missed it in this thread but happy to help and want to be sure that's working correctly. |
@NickCraver Add support for capturing logs of ASP.NET Core logging infrastructure with configurable log levels. |
Since v2.0 is live and #129 is captured separately, I'm going to close out this issue. Thanks again to everyone who gave feedback and help to make this release happen. It's very much appreciated! |
This is a planning post for V2, which I've already started into in the v2 branch (needs VS 2017 15.3 Preview to work with). I've made pretty sweeping changes to settings locations to centralize them on the code side (old
web.config
sections still work!), and will continue to do so as I find a good solution for ASP.NET and ASP.NET Core both. I've also been working on the UI, code sharing etc. Let's break it down.Here's a list of what I'm planning for V2:
netstandard2.0
buildStackExchange.Exceptional.Shared
StackExchange.Exceptional.AspNetCore
.UseExceptional()
setup call).Shared
library thatStackExchange.Exceptional
andStackExchange.Exceptional.AspNetCore
depend onStackExchange.Exceptional.Shared
GetCustomData
and where it livesIPNet
(a utility class I created for Opserver).less
files with simpler, more consistent, and purpose-based styles).js
(via Web Essentials) toBundle.js
.Shared
for consistency and DRY between libs<T1,T2>
instead of`2
async
stack tracesUtils.StackTrace.PrettyHtml(string stackTrace)
method)System.Web
dependencyGet
/GetAll
IHttpModule
Log()
/LogAsync()
equivalents.AddLogData()
extension for easily adding custom data*List
suffixes from Delete/Protect methods. Just make them overloads that takeIEnumerable<Guid>
.Data["SQL"]
will be recognized by default) - for example Redis commands, Elastic queries and responses, etc.SQL
still acts as a deserializer set for viewing old exceptionsSQL
column for SQL store scriptsLastLoggedDate
column/property to fix rollup issues, e.g. roll up every 10 minutes (as before) but show the actual last date it occured in the UI, rather than the start of the rollupCategory
(string) field to the exception for a variety of use cases (categorization, severity, etc.)/docs
(GitHub pages, like miniprofiler.com/dotnet (source)That's just an overview of the stuff I already have in queue, for example here are what the new stack traces look like to a user (you can confiure it to looke like C#, VB, or F# syntax):
![screen shot 2017-07-16 at 19 24 12](https://user-images.githubusercontent.com/454813/28253090-2dc4084a-6a6d-11e7-8744-1b9d844ca533.png)
![screen shot 2017-07-16 at 21 26 15](https://user-images.githubusercontent.com/454813/28253128-7d4ddeae-6a6d-11e7-8db5-0fc655c1538e.png)
...and here's what the list page currently looks like:
...so that's what I've been working on. Now...
What functionality would you like to see in V2 for exceptional?
I'm not promising to implement all ideas, but I want to at least (hopefully) make all breaking changes necessary in a major release where they belong. I'm sure there are a lot of other great ideas (I know some co-workers have some that I'm forgetting right now) and I wanted to collect them all here so that none are forgotten. Please, share your ideas!
Also, as always, feedback on any of the above is welcome. I'm just trying to get a release out as personal time allows!
The text was updated successfully, but these errors were encountered: