feat: add catalog ingestion processor for indexing AWS accounts #3874
Conversation
Signed-off-by: Jonah Back <jback@legalzoom.com>
🦋 Changeset detectedLatest commit: 205638b The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
| kind: 'Component', | ||
| metadata: { | ||
| annotations: { | ||
| 'amazonaws.com/arn': |
There was a problem hiding this comment.
this redirects to aws.amazon.com, would that be a better domain to use here?
There was a problem hiding this comment.
I chose amazonaws.com based on how AWS decided to use annotations for their Kubernetes components - i.e. https://docs.aws.amazon.com/eks/latest/userguide/specify-service-account-role.html - I'd prefer to be consistent with the domain name that they're already using for some of their K8S components
There was a problem hiding this comment.
Nice! Well that's a great argument to keep this as is then
plugins/catalog-backend/src/ingestion/processors/AwsOrganizationProcessor.ts
Outdated
Show resolved
Hide resolved
| location, | ||
| entity: { | ||
| apiVersion: 'backstage.io/v1alpha1', | ||
| kind: 'Component', |
There was a problem hiding this comment.
A cloud accounts sounds a lot like a Resource, not like a Component which is more like a software entity. Resource is not yet added though, which is a bit unfortunate ... Hm.
Maybe we could continue using Component here just to unblock you, but with a clear plan to move over ASAP to using a Resource kind instead when one is available.
There was a problem hiding this comment.
Yeah, agreed - I think eventually it'd make sense for AWS accounts, GCP projects, and Azure Subscriptions to all end up as Resource objects with similar traits
plugins/catalog-backend/src/ingestion/processors/AwsOrganizationProcessor.ts
Outdated
Show resolved
Hide resolved
plugins/catalog-backend/src/ingestion/processors/AwsOrganizationProcessor.ts
Outdated
Show resolved
Hide resolved
plugins/catalog-backend/src/ingestion/processors/AwsOrganizationProcessor.ts
Outdated
Show resolved
Hide resolved
plugins/catalog-backend/src/ingestion/processors/AwsOrganizationProcessor.ts
Outdated
Show resolved
Hide resolved
plugins/catalog-backend/src/ingestion/processors/AwsOrganizationProcessor.ts
Outdated
Show resolved
Hide resolved
plugins/catalog-backend/src/ingestion/processors/AwsOrganizationProcessor.ts
Outdated
Show resolved
Hide resolved
|
This is aws-e-some (ughh, sorry)! A number of review comments but don't let that put you off; a lot of them are discussion pieces / questions :) |
There was a problem hiding this comment.
If I may say: this contrib is super cool! 🤩 Very excited to see where this takes things.
My company is heavily invested in AWS and exposing this information in a way that better maps accountability to various components, resources, and squads could be hugely valuable. Can't wait to help out and try this!
plugins/catalog-backend/src/ingestion/processors/AwsOrganizationCloudAccountProcessor.ts
Show resolved
Hide resolved
| }, | ||
| spec: { | ||
| type: 'cloud-account', | ||
| lifecycle: 'unknown', |
There was a problem hiding this comment.
Possible future enhancement, but this could potentially be populated via the account status flag with some sort of a map. Is it an active account, then mark production, versus a suspended one, etc.
https://docs.aws.amazon.com/organizations/latest/APIReference/API_Account.html
Signed-off-by: Jonah Back jback@legalzoom.com
Hey, I just made a Pull Request!
This PR adds a processor for ingesting AWS account data from AWS Organizations. It is a very basic processor, but it ingests the minimal amount of data (account name / accound id) for it to be useful for other applications (we are using it to power cost insights for our aws accounts - hopefully that's coming later in another PR :) )
✔️ Checklist