Ability to overwrite/update vSphere templates in Content Libraries

[rhjensen79]

Description

Today when using the export to Content Libraries, on vSphere, it requires a unique name, for each upload.
If the same name already exist, the export will fail.

Use Case(s)

When using VRA or other tools, that uses the ID of the Content Librarie item, and not the name, it's breaks the integration, when a new item appears. If we just update the item, then the integration continues to work.
I only know of VRA that uses this integration, but I would suspect, other products, that uses Content Librarie items, to have the same issue.

Potential configuration

Allow an option for "Update existing template", just as you can do in the vSphere gui.

Potential References

[mpoore]

I second the request for this feature / enhancement, it would of great benefit.

[virtualhobbit]

+1 for this!

[vNerdyNate]

This would be an amazing feature to have.

[sylviamoss]

Hey there, thanks for reaching out! I guess the content catalog is the same as the content library, right? I'll take a look when we get a chance, and see the possibilities of doing this with their library. I thinks is a great enhancement.

[rhjensen79]

@sylviamoss You are right. Sorry for that. I have corrected the issue, so it refers to the correct naming, to avoid confusion. And thanks.

[cmbits]

Thanks rhjensen79 for posting this request. We are using vRA's Code Stream to automate the template build process and have eagerly been waiting for the content library destination option. The update content library option will allow our vRA blueprints to automatically get the latest template build which will be awesome.

[rhjensen79]

@cmbits Exactly my use case as well :-)

[sammcgeown]

+1 for me too

To add a some context, like cmbits we're also using vRealize Code Stream to automate packer builds for several customers, at the moment we're moving templates to Content Libraries using API calls, but it would be much nicer to go directly to the Content Library.

[tigerhawk6]

+1 for me

[grantorchard]

Just a quick note - add a thumbs up to the original post rather than a "+1" as that's where the metrics are coming from.

[sylviamoss]

Hello there! I implemented the function to update an existing item with the new template. Could some of you test the solution and let me know if it works as expected?
Here the binaries you can try out: https://circleci.com/gh/hashicorp/packer/73959#artifacts/containers/0

Important note: this operation was added in vSphere API 6.9.1 so for lower versions this is not going to work.

[rhjensen79]

Cool. Thanks. I will test this out on friday. that’s the first time i have avaliable, and report back. /Robert Send from mobile device

…

Den 12. aug. 2020 kl. 12.00 skrev Sylvia Moss ***@***.***>:  Hello there! I implemented the function to update an existing item with the new template. Could some of you test the solution and let me know if it works as expected? Here the binaries you can try out: https://circleci.com/gh/hashicorp/packer/73959#artifacts/containers/0 Important note: this operation was added in vSphere API 6.9.1 so for lower versions this is not going to work. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[rhjensen79]

@sylviamoss is there any properties, we need to set, to make it overwrite, or is the force flag enough ?

[sylviamoss]

@rhjensen79 you just need to give it a name and if an item exists with the same name Packer will then update the content with the new template.
You can preview the doc here

basically it would be something like:

content_library_destination {
        library = "Packer Library Test"
        name = "test-item-name"
}

If test-item-name doesn't exist it will be created and the second time you run the build it will be updated.

[rhjensen79]

Damn. I'm running vCenter 7 and vSphere 6.7. And I get an error saying "failed to check in virtual machine (name) into the library.
And another one saying VM is not checked out.
So I'm guessing it due to my vSphere version. I had hoped vCenter 7 was enough.

My env don't support vSphere 7, so I'm hoping one of the other interested here, can test it win their env.

[rhjensen79]

[sylviamoss]

I think you have a compatible version but is failing for some reason and I need to figure out why 😐 . If it wasn't compatible it would be just a common 404 Not Found error.
Thanks for running the test! I don't have the compatible version so I couldn't test on my own. I'll come back to you if I need more information on this.

[rhjensen79]

Great to hear. Let me know, what you need for me.
I will be happy to do an online session, if you want to see for yourself. And I can also give you access to the env, if that helps.

[sylviamoss]

Thanks! I'll let you know.

For now, I figured out some stuff. It turns out that is not possible to update an Item with any VM template, it must be from a VM that it was checked out from the library item. Their docs explain better: chekout VM and checkin VM
Basically, an item must have always a template for the same VM. When the VM needs to be updated then you should check out the VM from the template you have in the content library, update it, and later check-in back to the library to have the updated version of the template.
The vsphere-iso will create a new VM every time, with a new id, so it would never be possible to update a template item from it.

One solution I can think right now is to use the vsphere-iso to create the library item with the template, and use vsphere-clone to checkout the VM, update it, and check-in back. For that, I'd need to make changes to the vsphere-clone, or maybe another solution could be to create a new vsphere builder type that would work as a content library item checkout-checkin flow. I need to study more to understand what is possible and to give more details about this, but can you tell me if this would be a solution for your case?

[rhjensen79]

The way I do it manually today, which works, is that I run packer, and end up creating a vSphere template.
I then select "Clone to Template in Library", and select update existing template.
That keeps the id, from the template in the Content Catalog, and I works, with me, creating a brand new VM, with a new id, using the sphere-iso.

I don't know if that helps you in any way ?

I have attached a screenshot of the UI way to do it.

[sylviamoss]

I noticed in your screenshot that you're updating items with an OVF template. I implemented the code to allow also importing OVF templates to the content library, not only VM templates. If you choose to import OVF templates then you will be able to update existing library items with it, like you're doing through the UI.
To do that, just add the ovf = true to your config. For example in HCL2:

    content_library_destination {
        library = "Packer Library Test"
        name = "packer-test-ovf-template"
        ovf = true
    }

Here the binaries you can try out: https://app.circleci.com/pipelines/github/hashicorp/packer/6715/workflows/ca4acf17-a1ba-4126-987d-28f3259fc94a/jobs/74981/artifacts

I was able to test this locally and it worked 👍 but I want to know if it works for you too as expected.

[rhjensen79]

@sylviamoss this is working perfectly.
It does exactly what I was doing manually.
Thank you so much.

It actually works so well, that I already have updated my docker image, in which I run packer from, to your new version :-)

one thing thou. Just a minor thing, but something I can't se the reason for.
I get an error, when my vm name, is the same as my Content library name.
Is there any reason for that ?

[sylviamoss]

@rhjensen79 Nice!!
So I did the last changes to make it pretty and you can try the new binaries here: https://app.circleci.com/pipelines/github/hashicorp/packer/6733/workflows/b1b4f14c-182a-4fed-8bf1-b079df8a9eaf/jobs/75220/artifacts

With the below example Packer will create or update a library item with an OVF template and will have the same name as your VM name 👍

content_library_destination {
      library = "Packer Library Test"
      ovf = true
}

You can run your build any times you want and the library item will be updated with the new template :D

Thanks for your patience in giving me all the necessary information and making tests to get to this solution. I don't know all of the vsphere features and it was super useful to know this one!

[mpoore]

Hi Sylvia, that’s awesome! I’ll give it a try shortly too. Thank you so much!

[rhjensen79]

@sylviamoss I have now testet it, with the latest binaries, and it works perfectly.
And I also added the Destroy parameter, so it deletes the build VM, after the export.

Note my formatting, is a bit diffrent than yours. I don't know why yours don't work. But have have pasted my below, so you have a copy, of what's working from my side.

"content_library_destination": {
"library": "Content library name",
"ovf": "true",
"destroy": "true"
}

Thank you so much for your help. I am truly impressed, with the speed and involvement you have shown.
Thanks :-)

If you want to close the issue, then just do so, or let me know, and I will do it.

[sylviamoss]

@rhjensen79 That's really great! Thanks again 😄
I copied the snippet from my HCL2 template, that's why is different from yours.
The issue will close when the pull request is merged, so you don't need to worry about closing it!

If you need anything else, feel free to reach us again as you did!

[jdotsmith]

I know this is closed, but it would be cool if you could use the same name when converting it to a vm-template as well. Currently you can only use the same name when converting it to an OVF.

I have a packer process that runs every month so Ill be getting a bunch of vm_name + timestampvm-templates and would love to overwrite them.

[rhjensen79]

@jdotsmith why not just create a new feature request on it ?

[jdotsmith]

@jdotsmith why not just create a new feature request on it ?

Thanks for the motivation -

#9967

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.