Cloudwatch: Fix Unexpected error

[sarahzinger]

What is this feature?

Fixes issue where an unparsed error is thrown in Cloudwatch when a datasource is created:

In order to address fixing this, I ended up refactoring the /regions endpoint to follow the same architecture we now have in most of cloudwatch resource routes. This helps to ensure we return a well formatted error response like so:

rather than this string which our global error handling does not parse well:

Arguably there are a few reasons this error is happening, and many of them are worth tackling. Some of them I've covered in this pr and some of which I did not, figuring we could address it later:

1. (not addressed) when regions doesn't have auth session yet it returns an error rather than our fallback list of regions. It is debatable to me what the expected behavior should be in this instance. Perhaps this is as desired...but then what's the point of fallback regions if we're not going to use them? We end up having fallback regions in our frontend as well as in our backend which seems wrong. I think probably we should someday make this return the fallback rather than an error, but that felt like something we could work on in the future.
2. (not addressed) our UI is making a request for /regions before we have an auth saved, even though the frontend already has a default list of regions it can use (located in the grafana-aws-sdk). Again debatable to me what the behavior here should be. If we remove the error entirely and always use the fallback list, then it seems reasonable to me that we make this request right away. But if we're going to throw an error, then it seems strange, like we should wait until there is a saved configuration. But again I think probably it's fine as is once we stop returning errors.
3. (addressed) we were trying to create a session with a default region of "", rather than throwing an error. This led to a confusing error message, now we return an error that clearly states we're trying to create an auth session with a default region.
4. (addressed) we were return a string rather than an error object, which our frontend could not parse. We still throw the error but it is no longer noticable unless you have the network tab open. In my opinion this is a good reminder that we need to revisit error handling in Cloudwatch in general.

For such a tiny bug, I ended up re-writing a lot of code, because it felt like a good opportunity to clean up some tech debt and get better familiar with our backend architecture. But it also made me a bit concerned that I might have unexpected changes here, so I went ahead and put the new stuff behind a feature flag. I feel reasonably confident with it so I was going to have it enabled by default, but if it seems to be causing issues we'll have a quick way to turn it off without reverting the commit.

An interesting note here is that I saw we have a regionsCache which I've removed. I realized while trying to add it back, that we're caching regions under a key of Profile which is often undefined, which seems wrong. I think it's not unreasonable for us to add in a cache and create a proper cache key (maybe out of a hash of the auth settings the way we do with settings?) but for now since it seems it was often unused, I've just removed it, but let me know if I'm missing anything!

Which issue(s) does this PR fix?:

Fixes #73294

Please check that:

• It works as expected from a user's perspective.
• If this is a pre-GA feature, it is behind a feature toggle.
• The docs are updated, and if this is a notable improvement, it's added to our What's New doc.

[fridgepoet]

Just some thoughts and totally defer to your judgement:
Pretty large number of changes and I feel a bit surprised that we feel a need for a feature flag. I wonder if we would have been able to break this up and felt more comfortable about delivering a smaller set of changes a bit at a time?