Eliminate need to run composer.install before vagrant up but still have a working initial site

[mikeschinkel]

Some of the goals for WPLib box are:

1. It should provision as fast as possible
2. Failure of provisioning should not be possible, or at least greatly eliminated.

Running anything complicated for provisioning, or running anything that could potentially fail during provision then goes counter to these goals. Examples of things that could fail:

1. Any script that needs to download software from the internet to be able to install it.
2. Any script that might download newer versions of software that has not been testing to work in the box.

That means we cannot run anything more complex than a simple Bash script that downloads nothing for provisioning.

Thus to eliminate the need to run composer install -- inside the box or out -- we should copy a ZIP file containing the WordPress website for wplib.box from an /opt/provision directory inside the box to the box's /var/www directory (which maps to www/ in our host's directory) and then unzip it.

This will have the same effect as running composer install as if composer install was run at the time of box image creation instead of at time of first vagrant up. That will give us a known working wplib.box website and on that website we can provide instructions for the user that would include one of:

1. Run composer update after running vagrant ssh, or
2. Run composer update on the host machine after ensuring that composer is installed.

[franz-josef-kaiser]

My 2cents after all the discussions we had:

This project is already at a point where it slightly defies the purpose of Vagrant: Provisioning. At the point where you start adding zipped packages and as there's already close to no provisioning going on, you can as well just provide a fully pre-provisioned VM. But that's like back in 2000 and obviously not this projects goal.

Ad Composer and the ZIP: IIRC Composer can't switch resources, so you will have to either provide local packages that you can use, or you just drop Composer completely. There's no fallback from some remote location possible. Using Composer for a local install imho is somehow pointless. Also you move yourself somewhere where you have to use Gemnasium and other tools just to monitor dependencies to keep this project up to date.

The only reason to still use Vagrant now is the hostsupdater or a similar plugin. That could as well be replaced by a simple shell script to update the vhosts file.

Summed up, I personally think that the goal is set a bit too high and slowly but surely moves your project into a dead end and backwards instead of forwards. Still hope it helps that I have written my opinion here for you.

[mikeschinkel]

My 2cents after all the discussions we had:

Thanks for taking the effort.

This project is already at a point where it slightly defies the purpose of Vagrant: Provisioning.

Interesting. That is not what I have understood the point of Vagrant to be:

1. To provide a command line interface for VirtualBox (or VMware or Parallels)...
2. ...that loads an operating system and optionally provisioned image...
3. ...that mounts the host's project directory as the VM's web root directory...
4. ...that allows setting the IP address and local domain name...
5. ...that updates the host machine's hosts file...
6. ...and that optionally runs a(n additional) provisioning script (Bash/Ansible/Puppet/Chef/etc.)

But rather than assume my understanding was right I decided to google a bit. Here is what I found:

• On Why Vagrant? (emphasis mine) which seems to support my understanding that use of provisioning tools is optional:

Vagrant provides easy to configure, reproducible, and portable work environments built on top of industry-standard technology and controlled by a single consistent workflow to help maximize the productivity and flexibility of you and your team.

To achieve its magic, Vagrant stands on the shoulders of giants. Machines are provisioned on top of VirtualBox, VMware, AWS, or any other provider. Then, industry-standard provisioning tools such as shell scripts, Chef, or Puppet, can be used to automatically install and configure software on the machine.

Further I found the most up-voted answer to the question "What's the purpose of Vagrant?" on Hacker News (with 37 votes) where none of the four (4) points presented even mentions provisioning or implies that provisioning is a primary purpose of Vagrant:

1. Because you want to provide a packaged development environment that's exactly the same across all of your developers. A new developer can just check out the code and run vagrant up and begin developing without needing to care about all of the resources used by your application (redis? postgresql? mysql? sphinx? solr?)
2. Because you want to create a VM that closely resembles your production environment. For testing purposes, or pure developing, things work better the closer your development and production environments are (helps hammer out hard to find bugs in configuration/OS)
3. Because your OPs team will benefit from it. Now that everyone's on the same page having a production-replica environment, your operations or deployment team (even if that's just you) can now test changing the infrastructure (your set of chef recipes, etc) and can test your deployment procedure.
4. You can very easily spin up a new box with a different configuration of resources and test against it. Want to see what'll happen when upgrade your ruby version, your redis version, and your java version/solr version?

I even remember being told that Mitchel Hashimoto said that doing significant provisioning in a Vagrantfile was crazy and that a better approach was to use his Packer software to create an initial box that is pre-provisioned as much as possible. And while I can't find a reference to his comment the fact his company offers Atlas and bills it as a place to "Discover Vagrant Boxes" would tend to support the idea of using pre-provisioned boxes instead of provisioning inside the Vagrantfile is a best practice.

So while I do absolutely respect your opinion and appreciate that you see Vagrant's primary value as its ability to support Provisioning for your use-cases I will also say that others -- including myself -- see provisioning within Vagrant as just being a necessary evil if you do not have a pre-provisioned box image.

Ad Composer and the ZIP

You've used "Ad" several times when communicating with me, but I do not know what it means. Does it mean "In Reference To?"

IIRC Composer can't switch resources, so you will have to either provide local packages that you can use, or you just drop Composer completely. There's no fallback from some remote location possible. Using Composer for a local install imho is somehow pointless.

I must not have explained what I meant well enough. Let me try a different approach.

Let's assume we had Composer run inside the box as part of a provisioning process, like your pull request. That would produce a directory of files that had been "composed."

Now assume that you ZIP and then delete the directory, and then move the ZIP file to /opt/provision.

In practice that is would you are doing but using Packer to call composer install when building our base box image.

When the developer runs vagrant up initially or vagrant reload --provision we have a tiny bash script that copies the ZIP file from /opt/provision to /var/www and unzips it. Boom, we are done. No need for switching resources, no local packages, no fallback to remote locations; just a ZIP file contain the results of a previous composer install.

Now you may ask "What if the ZIP file is out of date?" No problem, it will still work when someone new to WPLib Box clones the box for the first time -- which is one of our goals -- and yet to ensure it is up-to-date the developer only need run composer update; best of both worlds!

Maybe the best way to fully understand it is to see it in action after we get 0.10.6 out the door?

Summed up, I personally think that the goal is set a bit too high and slowly but surely moves your project into a dead end and backwards instead of forwards.

I do respect your opinion, and I thank you again for taking the time to verbalize it. And you may be right, and time will tell.

But I can see a really clear vision about where we are headed and I see it as moving forwards significantly, not backwards. Maybe what we offer will not provide value today for people with your skill levels, but we think it will benefit the 80% who do not have your skills and really have no interest in gaining them.

And having people using this box will give us calendar time to develop a much more powerful box later this year -- based on CoreOS and Docker, with an admin console -- or maybe early next year. We do need some time for that anyway because some of the software we will be using needs to mature first.

[mikeschinkel]

@clubdeuce (and @franz-josef-kaiser):

After leaving that long reply above, another approach occurred to me that I had not previously considered.

Rather than use a ZIP file on the box image that would need to be updated every time any of the software contained within would need to be updated, why not run composer update on from a local copy of the repo at github.com/wplib/wplib-box that would include WordPress, plugins, theme, etc. and then instead of unzipping a zip file from the box itself change the provision process to simply add WordPress core and plugins and the vendor directory to .gitignore?

This would achieve the same goals I want to achieve because cloning the Git repo is something people will have to do and clone is typically super fast and I've never seen a clone fail as long as Git access has been set up to work correctly on the host/developer's machine. And I think this would eliminate the need for issue #13 too.

@clubdeuce If you agree with this I can take this ticket and make the changes.

[clubdeuce]

@mike What is the status of this? Should we include in 0.10.7 or punt?

[mikeschinkel]

@clubdeuce Given I am heavily focused on WPLib 0.13.0 let's push this to 0.10.8?

[mikeschinkel]

@clubdeuce I have basically resolved this. We should discuss how I did it so that you are aware are you are adding features to the box.