Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VVV 3 Roadmap #1469

Closed
tomjn opened this issue Apr 26, 2018 · 26 comments

Comments

@tomjn
Copy link
Member

commented Apr 26, 2018

Hello!

After some discussion with @LoreleiAurora, our consensus is that this is the approach to take for VVV 3 and the primary changes that need to happen.

The general gist being:

  • Build our own Vagrant box and do the provisioning in advance, users won't have to provision the OS anymore
  • Move mysql back to a shared/synced folder so that box updates are easier
  • Replace Vagrant triggers with our own internal thing
  • Have the vagrant file auto-install the hosts updater plugin
  • Improve logging, with per site log files
  • Move database creation into vvv-custom.yml on a per site basis so that backups can be turned on and off

This will likely involve a lot of rewriting of the vagrant file, and provisioner work.

Thoughts? Suggestions? Offers of assistance?

@tomjn

This comment has been minimized.

Copy link
Member Author

commented Apr 26, 2018

Notes on infrastructure needed:

OVH

  • 1 Jenkins BlueOcean/owncloud?/box storage - persistent
  • 1 per build, ephemeral, Ubuntu, build runner, tester

Build pipeline

  • git repos
    • 1 is a base box, ubuntu with all its sec updates, configured to work as a vagrantbox
      this process takes a long fucking time, runs once a week or something
    • 1 contains all our VVV specific provisioning things, but will run every build
    • 1 repo with the terraform and packer stuff, builds and destroys build servers
    • 1 repo with the jenkins stuff so that we can restore and keep our jenkins server versioned

install all php versions during the build process

  • config turns them on and off, rather than installing and uninstalling
@aaronware

This comment has been minimized.

Copy link

commented Apr 29, 2018

I know it's relatively easy to remove an provisioned site; remove the host entry, kill the entry in the custom.yml kill the db but it would be cool to have some sort of uninstall process for a provisioned site.

@tomjn

This comment has been minimized.

Copy link
Member Author

commented Apr 29, 2018

The DB change might help with that, but it would still need to be an additional process, and one that's explicitly triggered. There's too many things that could go wrong

I added a new doc in a PR here Varying-Vagrant-Vagrants/varyingvagrantvagrants.org#83

But automatic removal would mean typos in vvv-custom.yml lead to data loss, or even commenting a site out. Nevermind the problem with sites that use a shared database. It would need to be an explicit command added in.

@Mte90

This comment has been minimized.

Copy link
Contributor

commented May 4, 2018

Maybe plan also to improve the dashboard, like a button to do the provision of a specific site?
Or a command line way simplified, because maybe I du provision every day because I am contributing on core so I cannot wait for updates from the top for the box.

@tomjn

This comment has been minimized.

Copy link
Member Author

commented May 4, 2018

There is a way to do it via CLI but it's not documented as it could open a can of worms :) Sadly without moving the dashboard to an electron app there's no way to provision from inside the VM

Fortunately, the dashboard was decoupled, so updates to the dashboard can be done without a VVV release

@tomjn

This comment has been minimized.

Copy link
Member Author

commented May 4, 2018

Something like --provision-with="site-mysite" where mysite is the site to be provisioned

@Mte90

This comment has been minimized.

Copy link
Contributor

commented May 4, 2018

Maybe we can write this command inside the dashboard so is easy a c&p and avoid interaction from dashboard.

@Mte90

This comment has been minimized.

Copy link
Contributor

commented May 4, 2018

As I can see the new vagrant version integrate Vagrant triggers as core https://github.com/hashicorp/vagrant/blob/master/CHANGELOG.md so maybe is something not required anymore?

@tomjn

This comment has been minimized.

Copy link
Member Author

commented May 4, 2018

That sounds like a good idea, can you raise it as a new issue?

@Mte90

This comment has been minimized.

Copy link
Contributor

commented May 4, 2018

I am trying vagrant 2.1 with hosts-updater and see if works.

mte90:/var/www/VVV/www  ⑂ develop +  $  vagrant plugin uninstall vagrant-triggers
WARNING: Vagrant has detected the `vagrant-triggers` plugin. This plugin conflicts
with the internal triggers implementation. Please uninstall the `vagrant-triggers`
plugin and run the command again if you wish to use the core trigger feature. To
uninstall the plugin, run the command shown below:

  vagrant plugin uninstall vagrant-triggers

Note that the community plugin `vagrant-triggers` and the core trigger feature
in Vagrant do not have compatible syntax.

To disable this warning, set the environment variable `VAGRANT_USE_VAGRANT_TRIGGERS`.
Uninstalling the 'vagrant-triggers' plugin...
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually                                                                                                                                               
caused by misconfigured plugin installations or transient network                                                                                                                                                   
issues. The error from Bundler is:                                                                                                                                                                                  

Unable to resolve dependency: user requested 'vagrant-hostsupdater (= 1.0.2)' 
@tomjn

This comment has been minimized.

Copy link
Member Author

commented May 4, 2018

was that intended on #1475 ? ^^

@Mte90

This comment has been minimized.

Copy link
Contributor

commented May 5, 2018

One of the common problems that we have on contributor days is the people on windows that is using the wrong version of Powershell that is not supported completely from Vagrant, but vagrant alert about that in a complicated way (write an error that require to search on internet).

I am wondering if we can speed up the provision process with parallel download and script running.

@tomjn

This comment has been minimized.

Copy link
Member Author

commented May 5, 2018

@Mte90

This comment has been minimized.

Copy link
Contributor

commented May 6, 2018

Yes but I mean instead to process every single site as cascade maybe using python (instead of bash) can useful to provision the sites locally in parallel so the time for do that task can be more fast.

@tomjn

This comment has been minimized.

Copy link
Member Author

commented May 6, 2018

@Mte90

This comment has been minimized.

Copy link
Contributor

commented May 6, 2018

There are solution built in linux like https://www.gnu.org/software/parallel/
I think that we have only to define the log, because with parallel execution of provisioning a log can be a mess.

I think that I can open a new ticket about the parallel execution with my ideas for this.

@tomjn

This comment has been minimized.

Copy link
Member Author

commented May 6, 2018

See #1479 and #1478

We'll track feature requests and enhancements under the 3.x.x milestone as new issues

@tomjn

This comment has been minimized.

Copy link
Member Author

commented May 6, 2018

As an aside, I've been speaking with @LoreleiAurora and she thinks it would be a good idea to ask for help going forward with regards to Jenkins so we don't have a single point of failure should things break and she's not around

@Mte90

This comment has been minimized.

Copy link
Contributor

commented May 6, 2018

Cool!
I am not sure if I can help but I think also #1417, #1277, #710 seems interested for release 3.

@Mte90

This comment has been minimized.

Copy link
Contributor

commented May 16, 2018

https://core.trac.wordpress.org/ticket/43711#ticket reading this about the new dev step https://make.wordpress.org/core/2018/05/16/preparing-wordpress-for-a-javascript-future-part-1-build-step-and-folder-reorganization/ about grunt maybe we can add a script for that.

I mean when you need to work on wordpress-develop you can run something like vagrant wp-dev and automatically is executed grunt-watch inside the machine avoding to enter with ssh like we are doing right now for xdebug on/off.

@tomjn

This comment has been minimized.

Copy link
Member Author

commented May 16, 2018

@Mte90 the problem is that file watching inside the VM in vanilla vagrant isn't reliable, which can be fixed at the cost of disk performance. There's definitely things that can be done to improve it, but it will never be as great as a watch on the host. But I feel that should be another ticket #1447

For requests, lets keep things in separate issues, @LoreleiAurora is working off of a project board so it'd be easier for her to focus if everything appeared itemised

@tomjn

This comment has been minimized.

Copy link
Member Author

commented May 16, 2018

A general update though, there's a CI server running a packer build that generates a box and runs a script to upload it to vagrant cloud. Currently there are going to be 2 boxes, vvv and vvv-base, with vvv-base being the Ubuntu box with the packages installed to speed up CI so that VVV provisioning doesn't have to wait for MariaDB etc to be installed.

The vvv-base box is what has been done so far. I'm told cleanup of boxes needs implementing, then there's the second box, which is what Vagrant would be grabbing on users machines, then finally the modified VVV itself with a rewritten provisioner/vagrant file

@benlumia007

This comment has been minimized.

Copy link
Contributor

commented May 19, 2018

here is a thought, if we are going to build our own box, why not use .vhd instead of .vdi, that way .vhd can be use under hyper-v and virtualbox as well.

@Mte90

This comment has been minimized.

Copy link
Contributor

commented Aug 22, 2018

I was thinking that can be helpful to have in the dashboard a button to execute scripts like xdebug_on so can be a real dashboard with actions and not only links.

@Mte90

This comment has been minimized.

Copy link
Contributor

commented May 10, 2019

we have #1792 so I think that we can close this one

@tomjn

This comment has been minimized.

Copy link
Member Author

commented May 10, 2019

Agreed, lets close this out

@tomjn tomjn closed this May 10, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.