Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Saltcloud virtualbox provider #27089

Closed
LoveIsGrief opened this issue Sep 12, 2015 · 16 comments
Closed

Saltcloud virtualbox provider #27089

LoveIsGrief opened this issue Sep 12, 2015 · 16 comments
Labels
Execution-Module Feature new functionality including changes to functionality and code refactors, etc. Platform Relates to OS, containers, platform-based utilities like FS, system based apps State-Module
Milestone

Comments

@LoveIsGrief
Copy link
Contributor

Hello everybody,

For those of us that don't have a cloud provider available, it would be nice to provision VirtualBox machines. This would come in handy to e.g test saltcloud before going live (my case) or simply having an environment on one machine.

As the guidelines says I'm asking for this before starting implementation as I haven't seen this implemented yet.

I'm looking forward to comments, suggestions and whatnot e.g if this fits in the philosophy of saltcloud.

Cheers

@p-hennessy
Copy link

I usually use Vagrant spin up virtual machines, perhaps that would work for you?

@LoveIsGrief
Copy link
Contributor Author

As far as I understand that would be if saltcloud were't being used. If I'd want to use Saltcloud locally, it wouldn't be possible. It doesn't seem to have a vagrant or virtualbox provider in order to manage VMs.

@jfindlay jfindlay added Feature new functionality including changes to functionality and code refactors, etc. Salt-Cloud RIoT Relates to integration with cloud providers, hypervisors, API-based services, etc. labels Sep 14, 2015
@jfindlay jfindlay modified the milestones: Approved, Blocked Sep 14, 2015
@jfindlay jfindlay added info-needed waiting for more info Pending-Discussion The issue or pull request needs more discussion before it can be closed or merged and removed Feature new functionality including changes to functionality and code refactors, etc. RIoT Relates to integration with cloud providers, hypervisors, API-based services, etc. Salt-Cloud labels Sep 14, 2015
@jfindlay jfindlay modified the milestones: Approved, Blocked Sep 14, 2015
@jfindlay jfindlay added Feature new functionality including changes to functionality and code refactors, etc. Execution-Module State-Module Platform Relates to OS, containers, platform-based utilities like FS, system based apps and removed info-needed waiting for more info Pending-Discussion The issue or pull request needs more discussion before it can be closed or merged labels Sep 14, 2015
@jfindlay
Copy link
Contributor

@LoveIsGrief, thanks for the report. I think the typical salt approach with VM management software like virtualbox or qemu is to handle them with execution and state modules. You may be able to get at least some of what you need by using the vbox_guest modules. It doesn't look like they yet support full virtualbox functionality.

@LoveIsGrief
Copy link
Contributor Author

Hi @jfindlay

I was looking for the ability to call salt-cloud --list-providers and see virtualbox in the output. Of course with the same functionality as other providers e.g. define a provider config for virtualbox (for a local virtualbox hypervisor as well as a remote one) and profiles to later call create VMs according to these profiles.

The goal behind all this is be able to cheaply test a salt and salt-cloud configuration locally before pushing everything to different environments without the need for AWS, Azure, linode or any of the other providers.

Don't know if anybody else has started implementing the salt-cloud plugin but I have in LoveIsGrief/saltcloud-virtualbox-provider

@jfindlay
Copy link
Contributor

@LoveIsGrief, from what I understand, that means you have virtualbox configured somewhere under /etc/salt/cloud.providers.d/ even though salt-cloud does not support it yet. I am looking forward to your pull request though. :-)

@DanyC97
Copy link

DanyC97 commented Oct 15, 2015

@jfindlay i'm curious - how you guys test salt-cloud ? i find it very hard to test it w/o having a virtualbox provider (every other provider adds additional dependencies/ costs)

@LoveIsGrief
Copy link
Contributor Author

That's exactly what I'm struggling with right now. I have only been able to do manual tests of salt-cloud. Still trying to find a way to write proper unit tests.

@rallytime
Copy link
Contributor

@DanyC97 and @LoveIsGrief We need to write more unit tests for salt cloud. Those should be easy to add to the test suite as they should be using mock data to test against.

We have some integration tests that run nightly on our Jenkins installation. These run on a nightly basis because of the cost factors you're describing. The code for these tests is located in salt/tests/integration/cloud/providers/ if you want to see how we're doing it.

@DanyC97
Copy link

DanyC97 commented Oct 15, 2015

@rallytime thanks for feedback, appreciated!
However i hope you guys won't give up on vbox driver :)

@rallytime
Copy link
Contributor

Nope! Still a great feature request. We just haven't had time to tackle too many features since we're working on getting the bug count down first.

@DanyC97
Copy link

DanyC97 commented Oct 16, 2015

@rallytime fair point

LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
The debug log exposed private keys

Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
It wouldn't find the correct modules and would crash

Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
When called in subprocesses it would have a tendency of forgetting the sys.path, which was modified by the Virtualbox SDK to find its internal modules
Reloading does that for the subprocesses

Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
We will wait for the session state to unlock before letting exiting the functions (with a timeout of course)

If we don't wait they might still be in a bad session state before the next call. Sadly, that might still be the case as virtualbox decided to introduce "internal" sessions to which we have no access, yet still depend on... yay

Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
This should help us stop copy-pasting the calculations to let us timeout functions

Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
Simplifies the code

Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
Simplifies the code and tries to wait for the correct session state when starting machines

Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
Outdated values may be retrieved from machines that aren't running

Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 10, 2016
It might leave the test env in a dirty state, maybe further tests might fail, but we have the log to trace it back to that

Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 11, 2016
Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 11, 2016
Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 12, 2016
Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 12, 2016
Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 12, 2016
Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 12, 2016
Related to saltstack#27089 Saltcloud virtualbox provider
LoveIsGrief added a commit to LoveIsGrief/salt that referenced this issue Feb 12, 2016
@rallytime
Copy link
Contributor

The virtualbox cloud driver was added a couple of months ago:
https://github.com/saltstack/salt/blob/develop/salt/cloud/clouds/virtualbox.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Execution-Module Feature new functionality including changes to functionality and code refactors, etc. Platform Relates to OS, containers, platform-based utilities like FS, system based apps State-Module
Projects
None yet
Development

No branches or pull requests

7 participants