feat(export): export artifacts from clusters #1071
Conversation
| @@ -413,7 +413,10 @@ object ActiveServerGroupTest : ModelParsingTestSupport<CloudDriverService, Activ | |||
| image = ActiveServerGroupImage( | |||
| imageId = ami, | |||
| appVersion = "$app-3.16.0-h205.121d4ac", | |||
There was a problem hiding this comment.
Here's an example of what the data looks like for debs. To find the baseOS, I look for either "xenial" or "bionic" as strings in the name, description, and imageLocation fields. It's excellent garbage hackery. For netflix, these files are autogenerated by the bakery so it's reliable.
There was a problem hiding this comment.
Won't that mean we need to make code updates to keel to support new base OS names? Doesn't seem great.
There was a problem hiding this comment.
Yes we definitely will. I agree, not great.
Do you have any ideas if the operating system is present in any field in any clouddriver response? I checked the server group and image response. I don't see the data surfaced anywhere in either of those or the spinnaker UI, or the internal edda link for the image. So, I'm thinking we just don't have the data.
If we don't have the data, do you have any ideas on how to do this better?
| @@ -491,7 +509,7 @@ class TitusClusterHandler( | |||
| capacity = capacity, | |||
| capacityGroup = capacityGroup, | |||
| constraints = constraints, | |||
| container = generateContainer(container, location.account), | |||
| container = ReferenceProvider(container.repository()), | |||
There was a problem hiding this comment.
Now when we export clusters we return them with a reference provider.
| @@ -45,7 +46,7 @@ class ExportController( | |||
| "securitygroup" to "security-group", | |||
| "securitygroups" to "security-group", | |||
| "cluster" to "cluster", | |||
There was a problem hiding this comment.
Pretty sure this was a typo.
This is something we expect Newt to do internally? |
You should be able to reverse engineer it from the |
I guess the presence of |
|
@robfletcher |
|
Have you considered looking into pipeline bake stages instead of clouddriver? The info you’re missing is available there, but I wasn’t sure the use case is only for new migrations or if Cory wants export to work with already-managed clusters. If you think it’s worth considering, there’s code you can reuse from that PoC I did: Also, #1055 introduced some code to find pipelines based on entity tags from clouddriver. Just a thought. |
|
@luispollo thanks for the suggestion. Right now I don't think that it is worth it to try and find the pipeline bake stage to discover OS - users typically understand what OS their package needs to run on, and I think we can ask them if we don't have the information. If it turns out that this is a major migration pain point I think we can add more logic to how we find these things. |
| return DebianArtifact( | ||
| name = artifactName, | ||
| vmOptions = VirtualMachineOptions(regions = serverGroups.keys, baseOs = guessBaseOsFrom(base.image))) | ||
| } |
There was a problem hiding this comment.
lol that method name :)
|
@emjburns sorry for coming late to the party... was wondering few things:
|
|
answered in private, thank you! |
This PR adds an endpoint to export artifacts given a cluster. The existing cluster export will now return a reference provider with a reference to the name of the artifact instead of the old style full artifact (which is no longer supported)
These endpoints should be used together to produce the artifact and cluster that are both needed to put a cluster correctly in a delivery config.
A
GET /export/artifact/{provider}/{account}/{cluster}will return something like:{ "name": "deb-sample-app-server", "reference": "deb-sample-app-server", "vmOptions": { "baseLabel": "RELEASE", "baseOs": "xenial", "regions": [ "us-east-1", "us-west-2" ], "storeType": "EBS" }, "statuses": [], "type": "deb" }Notes:
deb) I'm guessing the baseOs from netflix naming conventions. Awesome Hackery. Turns out the image details from clouddriver do not tell us what OS an image is.deb) No idea how to calculate the store type. This is defaulted.deb) No idea how to calculate the statuses. This might need to be a user prompt.docker) I'm trying to determine the type of tag versioning strategy to use based on the running tag. This won't work if the cluster is a one-off. I throw an exception in the case I can't determine the strategy, because I don't know how else to signal that the output wouldn't be correct.