Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Named Volume Performance Feature #1460

Open
dustinleblanc opened this issue Feb 13, 2019 · 61 comments

Comments

@dustinleblanc
Copy link
Member

commented Feb 13, 2019

As a Mac OS / Windows Lando user,
I want to selectively mount paths within my application in named volumes,
So that I can increase my development speed by bypassing a significant portion of the speed penalty imposed by Docker's underlying filesharing mechanisms.

Notes from today's discussion:

  • Feature should be activated in each app specifically, within it's .lando.yml
  • 'Easy button' inside recipe config that would activate for common paths (D8 - vendor, core, etc).
  • Exclude key in root of lando.yml takes an array of paths that are relative to app root
  • On rebuild we spin up a util container, that copies the exclude paths into their respective volumes, then stops, starts the app, and mounts those volumes in the needed locations.
  • Upon disabling of feature, don't do anything with mounted code, notify user of change, and give them steps to remediate.
  • start/stop/restart should make no changes, rebuild required for all ops
  • potential future feature is two-way syncing, or even, container-authorative one way syncing.

@dustinleblanc dustinleblanc added task feature and removed task labels Feb 13, 2019

@pirog pirog added this to the 3.0.0-rc.11 milestone Feb 19, 2019

pirog added a commit that referenced this issue Feb 21, 2019

pirog added a commit that referenced this issue Feb 21, 2019

pirog added a commit that referenced this issue Feb 21, 2019

@pirog

This comment has been minimized.

Copy link
Member

commented Feb 25, 2019

@dustinleblanc so have some initial stuff over in https://github.com/lando/lando/tree/hyperdrive-motivator

the syntax is as follows

name: my-lando-app
excludes:
  - vendor
  - modules/contrib

And requires a lando rebuild.

We also could consider adding support for this sort of syntax as well, eg exclude all of X except Y, but that might be better as a separate issue.

name: my-lando-app
excludes:
  - modules/contrib
  - '!modules/contrib/search_api'
@pirog

This comment has been minimized.

Copy link
Member

commented Feb 25, 2019

@dustinleblanc dropping a note here:

  1. Let's make sure we create any exclude directories that don't exist
@pirog

This comment has been minimized.

Copy link
Member

commented Feb 25, 2019

@serundeputy... adding you to the party here. Dustin and I need a third opinion, can you...

  1. Get lando running on macOS from source with https://github.com/lando/lando/tree/hyperdrive-motivator checked out
  2. Get the dgreat and pe sites spin up with database and files and all needed things pulled
  3. Try out a few time lando drush cr and time lando drush status and log the average times somewhere

Then let's meet and see if we can get the hyperdrive-motivator working. Here are my stats for dgreat

without excluding

lando drush status: 20s
lando drush cr: 1m30s

excluding vendor, web/core and web/*/contrib

lando drush status: 1.3s
lando drush cr: 19s

So one thing i did notice @dustinleblanc (and @serundeputy), was restarting docker seemed to be necessary to get these gains to show up. No idea why but until i did that i saw "good" performance improvements but not "great"(eg the above) ones

@serundeputy

This comment has been minimized.

Copy link
Member

commented Feb 27, 2019

In order to get the performance increase I had to restart docker as @pirog also indicated.

dgreat

sans exclude

lando drush st
  real	1m56.403s
  user	0m0.907s
  sys	0m0.287s

lando drush cr:
  real	2m39.110s
  user	0m0.816s
  sys	0m0.280s

excluding vendor, web/core, and web/*/contrib

lando drush st:
  real	0m3.855s
  user	0m0.657s
  sys	0m0.318s

lando drush cr:
  real	0m15.643s
  user	0m0.724s
  sys	0m0.175s

pe

sans exclude

lando drush st: 
  real 0m1.328s
  user 0m0.457s
  sys 0m0.507s

lando drush cc all:
  real 3m0.532s
  user 0m0.808s
  sys 0m0.183s

excluding vendor, sites/all/modules/contrib

lando drush st: 
  real	0m3.120s
  user	0m0.628s
  sys	0m0.163s

lando drush cc all:
  real	1m16.717s
  user	0m0.699s
  sys	0m0.313s

pirog added a commit that referenced this issue Feb 27, 2019

pirog added a commit that referenced this issue Feb 27, 2019

pirog added a commit that referenced this issue Feb 27, 2019

@pirog

This comment has been minimized.

Copy link
Member

commented Feb 27, 2019

Alright gents i've injected the coaxium into the motivator
https://github.com/lando/lando/releases/tag/v3.0.0-rc.13

This is currently an "unsupported/undocumented" feature but i've got a PR queued up for when we want to put a ring on it
#1500
http://pr-1500-v4h3hcq-iaimrn624qfk6.us.platform.sh/config/performance.html

I'm going to ping people over at #763 to see if we can find any pioneers who want to try this out

@wss-chadical

This comment has been minimized.

Copy link

commented Feb 27, 2019

@dustinleblanc @pirog @serundeputy
So far so good. Sorry, no actual benchmarks other then I would say it's BEYOND faster then vanilla lando, and significantly faster then the functioning NFS based setup we have been using for the last week.

.lando.yml for my pantheon wordpress recipe....

excludes:
  - vendor
  - web/wp-content/uploads
  - web/wp-content/themes/wholesalesolar/node_modules
  - web/wp-content/themes/wholesalesolar/vendor

Other observations:

  • Before rebuilding I rmrf'ed the above directories on the host.
  • Our lando build which consists of 2 seperate composer installs, and 1 yarn install was definitely faster.
  • A terminus rsync of a WP uploads folder with 2000+ files was MUCH faster (Excluded).
  • The yarn install and node builds were faster.
  • Browsersync is faster.
  • SASS builds are a "little" faster - maybe 5%?
  • Wordpress page loads much faster
  • Wordpress admin performance much faster

Overall I'd say this is a huge win.

We could probably exclude even more files since we are using a composer based wordpress install. Assuming we don't need our code editors to search wp core from the host, we could exclude web/wp as well.

EXCELLENT PROGRESS! Thank You.

I'll roll this out to some other devs and see how it goes.

@danielnitsche

This comment has been minimized.

Copy link
Contributor

commented Feb 28, 2019

On RC13 comparing no excludes -> excludes I get:
lando start 69 seconds -> 155 seconds
drush site-install is 110 seconds -> 73 seconds

So pretty good on that front. Interestingly, site install was taking more like 300 seconds on RC1, so I feel like there have been other improvements along the way too.

Thanks!

🚀 🚀 🚀 👍

These were my excludes:

excludes:
  - vendor
  - web/themes/custom/my_theme/node_modules
  - web/themes/custom/my_theme/pattern-lab
  - modules/contrib
@danielnitsche

This comment has been minimized.

Copy link
Contributor

commented Feb 28, 2019

About 10 minutes in, things are going really fast, but I'm wishing I didn't exclude modules/contrib. Considering the number of contrib modules and files in each of those modules, I not sure it's worth including this path in the example documentation.

@pirog

This comment has been minimized.

Copy link
Member

commented Feb 28, 2019

@tormi

This comment has been minimized.

Copy link
Contributor

commented Feb 28, 2019

For me, lando composer update drupal/mymodule --with-dependencies doesn't work anymore after excluding vendor folder:

screenshot 2019-02-28 at 06 35 24

@vasikgreif

This comment has been minimized.

Copy link

commented Feb 28, 2019

Does excluding the directories mean the autocompletition in IDE also won't work?

@pirog

This comment has been minimized.

Copy link
Member

commented Feb 28, 2019

@tormi i also ran into this earlier this morning. what do you have for lando ssh -c "ls -lsa /app"?

my excluded directories are showing 1000:1000 ownership. Resetting these to www-data:www-data ownership seemed to be a workaround for now. I've also pushed up some stuff to master so we can enforce the ownership of excluded directories.

@pirog

This comment has been minimized.

Copy link
Member

commented Feb 28, 2019

@vasikgreif possibly, i think it would depend on whether the code you exclude exists on your host before you exclude it or not. There are definitely consequences to excluding code and we aren't 100% sure of what they all are at this point hence why we've only pinged this thread about the feature and haven't released it generally yet.

@wss-chadical

This comment has been minimized.

Copy link

commented Feb 28, 2019

Also seeing permissions related stuff going on...

➜  pl-wss-dot-com (master) ✗ lando ssh --service=sage
node@faa5180fbdd4:/app$ cd $LANDO_MOUNT/web/wp-content/themes/wholesalesolar
node@faa5180fbdd4:/app/web/wp-content/themes/wholesalesolar$ yarn install
yarn install v1.13.0
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info fsevents@1.2.4: The platform "linux" is incompatible with this module.
info "fsevents@1.2.4" is an optional dependency and failed compatibility check. Excluding it from installation.
[4/5] Linking dependencies...
warning " > stylelint-webpack-plugin@0.10.5" has incorrect peer dependency "webpack@^1.13.2 || ^2.7.0 || ^3.11.0 || ^4.4.0".
error An unexpected error occurred: "EACCES: permission denied, mkdir '/app/web/wp-content/themes/wholesalesolar/node_modules/abbrev'".
info If you think this is a bug, please open a bug report with the information provided in "/app/web/wp-content/themes/wholesalesolar/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
node@faa5180fbdd4:/app/web/wp-content/themes/wholesalesolar$

Seems like we don't have proper perms in the node_modules folder for some reason:

node@faa5180fbdd4:/app/web/wp-content/themes/wholesalesolar$ cd node_modules/
node@faa5180fbdd4:/app/web/wp-content/themes/wholesalesolar/node_modules$ ls -al
total 4
drwxr-xr-x  2 www-data www-data 4096 Feb 27 23:01 .
drwxr-xr-x 23 www-data www-data  736 Feb 28 16:42 ..
node@faa5180fbdd4:/app/web/wp-content/themes/wholesalesolar/node_modules$ touch t
touch: cannot touch 't': Permission denied

May have something to do with the directory already being there (or not) on rebuild?

@pirog

This comment has been minimized.

Copy link
Member

commented Feb 28, 2019

@tormi

This comment has been minimized.

Copy link
Contributor

commented Feb 28, 2019

@pirog , I have

4 drwxr-xr-x 50 www-data www-data 4096 Feb 28 16:44 vendor.

Fresh site install (git clone repo && lando start) with excludes: - vendor gives

0 drwxr-xr-x 2 root root 64 Feb 28 16:33 vendor

initially and lando composer update drupal/mymodule works.

The ownership of vendor folder has changed to

0 drwxr-xr-x 50 www-data www-data 1600 Feb 28 16:58 vendor

after using lando composer update .. stuff and I can use Composer from here on as usual.

@wss-chadical

This comment has been minimized.

Copy link

commented Mar 21, 2019

TURBO MODE.

@richardbporter

This comment has been minimized.

Copy link

commented Mar 26, 2019

Actually, with rc14 I seem to have permission issues when the exclude directory lives within a composer package (which I know is an odd use case). Lando seems to initially create that directory and then Composer cannot overwrite it. For example, if I exclude docroot/themes/custom/mytheme/node_modules then I get the following error when running lando composer install:

[RuntimeException]                                                    
  Could not delete docroot/themes/custom/mytheme/node_modules: 

I have to delete that directory manually and then lando composer install again. Works after that.

@stale

This comment has been minimized.

Copy link

commented Apr 25, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions and please check out this if you are wondering why we auto close issues.

@stale stale bot added the wontfix label Apr 25, 2019

@dustinleblanc

This comment has been minimized.

Copy link
Member Author

commented Apr 25, 2019

Stalebot, refreshen this issue please!

@stale stale bot removed the wontfix label Apr 25, 2019

@pirog pirog modified the milestones: 3.0.0-rc.16, 3.0.0-rc.17 May 7, 2019

@GMSteuart

This comment has been minimized.

Copy link

commented May 12, 2019

@pirog do you have a solution to resolving package dependencies through the guest vm so that an editor/ide does not throw errors for unfound modules since the files are not installed on the host?

Example:

import axios from 'axios'
// cannot find module 'axios'

I started looking into this today, so I will begin going down the rabbit hole of looking at ssh-agents and other 'remote' dev tool integrations, but at least wanted to ask here and maybe get some documentation going.

@GMSteuart

This comment has been minimized.

Copy link

commented May 12, 2019

@pirog follow up:

Steps to Resolve:

# install visual studio code insiders to allow module installation
brew cask install homebrew/cask-versions/visual-studio-code-insiders

# install remote extension pack on insiders (though only (remote-containers is necessary)
code-insiders --install-extension ms-vscode-remote.vscode-remote-extensionpack

# run code-insiders
code-insiders

# spin up your lando app if it is not already running
cd /to/lando/app && lando start

In code-insiders open command palette and run Remote-Containers: Attach to Running Container and select your lando container

@dustinleblanc

This comment has been minimized.

Copy link
Member Author

commented May 13, 2019

@GMSteuart sounds like you're using the new VSCODE features that were announced recently to access code running inside Docker. I was very curious when I heard the announcement about that. One of my long term ideas has been figuring out how to mount ZERO code to the host and still be able to develop as this would probably bring performance up even further. If you are feeling dangerous, you could try an experiment of telling lando to ignore your entire app directory and report back what performance is like and if you run into any issues. I'd be quite curious to hear the results.

@GMSteuart

This comment has been minimized.

Copy link

commented May 13, 2019

@dustinleblanc I have no problem running some tests to see what the performance difference is. If there is a preferred method for testing, please let me know (a prebuilt script would be helpful), otherwise I can put something together for the environment I have.

@dustinleblanc

This comment has been minimized.

Copy link
Member Author

commented May 13, 2019

Our current very unscientific way of doing it is to take the biggest worst site we can, then do

time lando drush cr

and see how long it takes. Insert various ops like drush cim etc to get a good idea about where things are at. We've always noticed the biggest difference when doing these kinds of 'development' tasks on Drupal sites.

This feature ends up cutting the time in half on some sites, and I'd love to see that get really, really close to a 1-to-1 of doing the same op on a native php/nginx/mysql setup (Laravel valet, etc). The biggest bottleneck is file sharing, and being able to stop the entire tree from being shared seems like potential for huge gains on non-Linux systems.

@GMSteuart

This comment has been minimized.

Copy link

commented May 13, 2019

@dustinleblanc i'll test it out with some fresh site installations this evening and see what happens. the cool part about using VS Code's container extension is that you no longer have to pipe commands through lando as the terminal ends up being ssh'd already. Ill test out no excludes, excludes of core/contrib/vendor directories, and exclude everything.

Do I need to set delegated or anything like that on the volumes?
example lando recipe

@GMSteuart

This comment has been minimized.

Copy link

commented May 14, 2019

@dustinleblanc not sure on the accuracy of this, and i would probably need to do a test with a for loop of 100 to 1000 rather than 10, but here are the results:

#!/bin/bash
for i in {1..10}; do
  lando drush cr
done
config:
  webroot: web
# excludes:
#  ...
./speed.sh  3.86s user 0.66s system 10% cpu 44.645 total
excludes:
  - vendor
  - web/core
  - web/modules/contrib
  - web/themes/contrib
./speed.sh  3.32s user 0.58s system 23% cpu 16.574 total
excludes:
  - vendor
  - web

^ pretty much everything. Could not get lando service to startup when I did
literally everything but I should probably try by putting recipe one directory
above and exclude that way...with that said, the results:

./speed.sh  3.39s user 0.58s system 24% cpu 16.275 total
# inside container running same script without lando
real    0m12.642s user    0m6.860s sys     0m1.400s

I'll look into testing more thoroughly another day, but for the time being. Will definitely look into testing front end rebuilds though as with headless development hot module reloading is what takes up the most time.

@dustinleblanc

This comment has been minimized.

Copy link
Member Author

commented May 14, 2019

Thanks @GMSteuart! The difference between the normal excludes and the 'pretty much everything' was less than I'd hoped, but I guess it makes sense. The majority of files are going to be in core and contrib modules, so you'd expect that is where the meat of the issue would be too.

The speed up from 44 seconds to 16 is the kind of thing the feature was designed to accomplish, so good to see it is doing it's work there, but I was hopeful we could get even further by being more zealous :)

At this point, I'd be interested in how much of the speed is now wrapped up in database volume/mount read/write, but for now, thanks for checking on the 'big exclude' performance difference.

@GMSteuart

This comment has been minimized.

Copy link

commented May 14, 2019

@dustinleblanc If there is already built in functionality for testing that let me know and I will run it. Otherwise, I will look into getting something into place on my lando.dev instance to try it out. I would like to get down to the nitty gritty on speeding up anything I can.

@nullvariable

This comment has been minimized.

Copy link
Contributor

commented May 28, 2019

This may be an obvious question, but Lando seems to map my $HOME folder to /user (which documentation suggests is the intended behavior: https://docs.devwithlando.io/config/files.html)

Wouldn't having a lot of files in this folder also cause slowness? Is there an easy way to disable that feature to test if it makes a difference? I mean my Dropbox files sync to this folder (and I don't need them in the container), so this mount would be the bulk of my local files on this machine.

On bare metal linux I never have performance issues, but definitely feel it on my windows laptop. Using excludes on node_modules and vendor did make a big difference.

@pirog

This comment has been minimized.

Copy link
Member

commented May 28, 2019

@q0rban

This comment has been minimized.

Copy link
Contributor

commented Jun 20, 2019

Thank you for the work on this. Trying this out for the first time.

From http://pr-1500-v4h3hcq-iaimrn624qfk6.us.platform.sh/config/performance.html:

After this happens you should expect to have an empty vendor directory on your host but a fully populated one inside each container.

I have added the following to my .lando.local.yml:

excludes:
  - docroot/themes/custom/ga_forest/node_modules
  - docroot/themes/custom/ga_forest/vendor
  - docroot/themes/custom/ga_forest/arboretum

On my local, I then ran the following to clear everything out and start from scratch:

rm -rf docroot/themes/custom/ga_forest/node_modules \
       docroot/themes/custom/ga_forest/vendor \
       docroot/themes/custom/ga_forest/arboretum
lando destroy -y

I then restarted Docker. And ran a docker system prune for good measure. After running lando start:

________             ______       ______  ___     _________
___  __/___  ___________  /__________   |/  /___________  /____
__  /  _  / / /_  ___/_  __ \  __ \_  /|_/ /_  __ \  __  /_  _ \
_  /   / /_/ /_  /   _  /_/ / /_/ /  /  / / / /_/ / /_/ / /  __/
/_/    \__,_/ /_/    /_.___/\____//_/  /_/  \____/\__,_/  \___/


Lando has detected that you wish to exclude certain directories from Docker file sharing!
Please allow a few moments for us to sync your host into the container

Syncing docroot/themes/custom/ga_forest/node_modules...
              0 100%    0.00kB/s    0:00:00 (xfr#0, to-chk=0/1)
Syncing docroot/themes/custom/ga_forest/vendor...
              0 100%    0.00kB/s    0:00:00 (xfr#0, to-chk=0/1)
Syncing docroot/themes/custom/ga_forest/arboretum...
              0 100%    0.00kB/s    0:00:00 (xfr#0, to-chk=0/1)

Strange… how is it syncing things that no longer exist? No worries, I see it created those directories, and they are currently empty. Then I run a lando npm --prefix docroot/themes/custom/ga_forest install and now I see a bunch of stuff in my docroot/themes/custom/ga_forest/node_modules on my local.

Am I doing this right? Is this expected behavior? If so, the documentation on this is misleading to me. Thank you!

@pirog

This comment has been minimized.

Copy link
Member

commented Jun 20, 2019

@q0rban

This comment has been minimized.

Copy link
Contributor

commented Jun 20, 2019

Thanks @pirog . So is it expected that after running npm install inside the container, that outside on my local I'd be seeing the npm packages in node_modules?

@pirog

This comment has been minimized.

Copy link
Member

commented Jun 20, 2019

@q0rban

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2019

Thanks for the quick and detailed responses @pirog !

So, I rinsed and repeated, and now everything seems to be as I expect:

$ rm -rf docroot/themes/custom/ga_forest/node_modules \
       docroot/themes/custom/ga_forest/vendor \
       docroot/themes/custom/ga_forest/arboretum
$ lando destroy -y
# For good measure, clean up all the Docker volumes.
$ docker volume prune -f

# Restarted Docker. Waited until it was running again, then:
$ lando start
$ lando npm install --prefix docroot/themes/custom/ga_forest

# Checking to see if node_modules got pupulated on my local.
$ ls -la  docroot/themes/custom/ga_forest/node_modules
total 0
drwxr-xr-x   2 jsansbury  staff   64 Jun 20 15:55 .
drwxr-xr-x@ 30 jsansbury  staff  960 Jun 20 15:55 ..
# YAAAYY no files on my local.

# Checking to see if the files are there inside the docker container
$ lando ssh -c 'ls -la  docroot/themes/custom/ga_forest/node_modules'
total 1776
drwxr-xr-x 436 www-data dialout 20480 Jun 20 20:02 .
drwxr-xr-x  30 www-data dialout   960 Jun 20 19:55 ..
drwxr-xr-x   2 www-data dialout  4096 Jun 20 20:02 .bin
drwxr-xr-x   3 www-data dialout  4096 Jun 20 20:01 @sailshq
drwxr-xr-x   2 www-data dialout  4096 Jun 20 20:02 abbrev
drwxr-xr-x   4 www-data dialout  4096 Jun 20 20:02 acorn
drwxr-xr-x   5 www-data dialout  4096 Jun 20 20:02 ajv
drwxr-xr-x   2 www-data dialout  4096 Jun 20 20:02 amdefine
...
# YAYAYAY files are there inside the container!

So I guess it just took a few times of doing this before it "stuck".

@dustinleblanc

This comment has been minimized.

Copy link
Member Author

commented Jun 21, 2019

Yep, be careful with node installs on your host and in the container, they often compile different C extension/binary thingies and if you have the OSX version on your host and the Linux version in the container, it will RESYNC every time you rebuild because it sees the files as different

@nathandentzau

This comment has been minimized.

Copy link
Contributor

commented Jun 23, 2019

Just added the excludes directive on a very large Drupal enterprise application. I can say Drush commands are almost a minute faster and page load times are 20 - 30 seconds faster with page cache turned off running Docker Desktop on macOS. Nice work on this Lando team!

Edit: added host os

@pirog pirog added this to the 3.0.0 milestone Jun 24, 2019

@digitole

This comment has been minimized.

Copy link

commented Jul 3, 2019

Just reporting that I have had similar experiences – drush cr is about a minute faster, and page loads are faster as well.

It needed a lot of rebuilds and restarts and sometimes it stops being faster, but once you have it setup it works very well.

Will try this for a while and report back if there are any other problems, thank you!

@eelkeblok

This comment has been minimized.

Copy link

commented Jul 11, 2019

I've seen someone was using a feature from VSCode to sync the excluded locations back to the host (or maybe get access to the files in the container itself, not sure), but is there someone who has rolled something like a tooling command to sync back changes made within the container (to solve the problem that the IDE does not know what is going on, e.g. for debugging and autocompletion purposes)? This is very much keeping me from giving this a serious spin, ATM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.