Adding VMware AppCatalyst Driver and Photon provisioner.

[frapposelli]

... and a small fix to the bats test.

VMware AppCatalyst can be downloaded here: https://communities.vmware.com/community/vmtn/devops/vmware-appcatalyst

[zchee]

@frapposelli Wow! Great!

[frapposelli]

/cc: @ehazlett @sthulb @nathanleclaire

[sthulb]

@frapposelli Nice! I'll check this out.

[sthulb]

@frapposelli can we set --vmwareappcatalyst-vm-location to be the storage dir? So all VMs for machine are stored in the same place, as per VMware Fusion & Virtualbox?

[silverbp]

when you create the shared folder, what are the flag options? I saw your passing 4, I'm getting some permission issues. Specifically if you try running npm install in a shared folder.

[frapposelli]

@sthulb yes, but you'll also have to set it in ~/.appcatalyst.conf and restart the REST daemon.

@silverbp flag 4 means that the shared folder is R/W, what kind of errors are you getting?

[silverbp]

untar errors along with this:

npm ERR! Error: EPERM, chown '/usr/src/app/node_modules/cookie-parser/package.json'
npm ERR! { [Error: EPERM, chown '/usr/src/app/node_modules/cookie-parser/package.json']
npm ERR! errno: 50,
npm ERR! code: 'EPERM',
npm ERR! path: '/usr/src/app/node_modules/cookie-parser/package.json',
npm ERR! fstream_finish_call: 'chown',
npm ERR! fstream_type: 'File',
npm ERR! fstream_path: '/usr/src/app/node_modules/cookie-parser/package.json',
npm ERR! fstream_class: 'FileWriter',
npm ERR! fstream_stack:
npm ERR! [ '/usr/local/lib/node_modules/npm/node_modules/fstream/lib/writer.js:306:21',
npm ERR! '/usr/local/lib/node_modules/npm/node_modules/graceful-fs/polyfills.js:143:7',
npm ERR! 'Object.oncomplete (evalmachine.:108:15)' ] }
npm ERR!
npm ERR! Please try running this command again as root/Administrator.

[silverbp]

these are the tar errors that occur first

npm ERR! tar.unpack untar error /root/.npm/cookie-parser/1.3.5/package.tgz
npm ERR! tar.unpack untar error /root/.npm/lodash/3.9.3/package.tgz
npm ERR! tar.unpack untar error /root/.npm/compression/1.5.0/package.tgz
npm ERR! tar.unpack untar error /root/.npm/express-handlebars/1.2.2/package.tgz
npm ERR! tar.unpack untar error /root/.npm/express-jwt/3.0.1/package.tgz
npm ERR! tar.unpack untar error /root/.npm/pensco-common/1.0.2/package.tgz
npm ERR! tar.unpack untar error /root/.npm/root-path/0.2.0/package.tgz
npm ERR! tar.unpack untar error /root/.npm/request-promise/0.4.2/package.tgz
npm ERR! tar.unpack untar error /root/.npm/pen-logger/1.1.3/package.tgz
npm ERR! tar.unpack untar error /root/.npm/jsonwebtoken/5.0.2/package.tgz
npm ERR! tar.unpack untar error /root/.npm/q/1.4.1/package.tgz
npm ERR! tar.unpack untar error /root/.npm/error/6.4.3/package.tgz
npm ERR! tar.unpack untar error /root/.npm/gulp/3.9.0/package.tgz
npm ERR! tar.unpack untar error /root/.npm/moment/2.10.3/package.tgz
npm ERR! tar.unpack untar error /root/.npm/express/4.13.0/package.tgz
npm ERR! tar.unpack untar error /root/.npm/pen-build/2.0.0/package.tgz
npm ERR! tar.unpack untar error /root/.npm/handlebars/3.0.3/package.tgz

[silverbp]

replicating this is just pulling down the official node image.. mapping in your root directory (remove your node_modules folder if it's there) and run npm install

[silverbp]

docker run --rm -it -v $(pwd):/usr/src/app node npm install

[silverbp]

I guess the working dir isn't set.. so you'll have to execute a change directory before running npm install or..

docker run --rm -it -v $(pwd):/usr/src/app node /bin/bash
then cd /usr/src/app
npm install

[frapposelli]

@silverbp apparently it's npm that doesn't play nice with shared folders both on Virtualbox and VMware, can't help with that but seems like a fix is being worked on and there are workarounds available: npm/npm#3565

[silverbp]

thanks! I'll probably end up using unison to sync..

[silverbp]

last comment, is there a way to change the size of the hard drive? or is that a limitation on AppCatalyst?

[frapposelli]

@silverbp it's currently a limitation of the first tech preview of AppCatalyst, it will be removed with the next release and will update the driver accordingly.

[silverbp]

btw, the virtualbox driver for docker-machine (boot2docker image) doesn't have this issue. It also maps my user folder into the vm as docker:staff where as AppCatalyst maps my folder in as 501:20 (the user and group). My username is not docker, but I assume they are just mapping it that way. We were trying out AppCatalyst because of the poor disk performance in virtualbox.

[frapposelli]

@silverbp I'll take a second look at that tomorrow, thanks for the tip.

[frapposelli]

@silverbp I did a deep analysis on that specific problem, apparently the difference between the two shared folders implementations (vbox vs. vmw) is that vbox will do noop on any chown command sent to files in a shared folder, while our implementation returns an error as the operation is (correctly) not permitted.

I understand this can cause some troubles when working with tools such as npm but right now there is no fix I can add to this driver to make it work, I raised an internal ticket to see if we can modify this behavior in the next release of the vmhgfs driver.

Thanks for bringing this up.

[ehazlett]

I'm assuming that AppCatalyst takes care of creating this VM upon installation? Will this work with a non-Photon based OS?

[frapposelli]

Right now this only works with the embedded Photon VM that ships with AppCatalyst but there's nothing preventing support for other OS.

The embedded Photon VM is a template and docker-machine creates a clone of it every time it spins up a new machine so the instances are completely independent.

[ehazlett]

Hi , thanks for your efforts and persistence in submitting this driver. We are extremely excited that there is so much interest in Docker Machine and we really appreciate your interest. However, at this time it is proving to be extremely difficult for us to keep up with reviewing and testing each of these drivers for inclusion in the Machine core. We really want to switch to a more pluggable model, as well as polish up a few things about the driver model which need to be changed to ensure a smooth and sustainable future.

Therefore, we will be moving to a plugin model for 0.5 and would love to have you involved in the design and development process. We are closing the outstanding driver PRs at this time, but please keep the code. We will stick closely to the current driver interface and you should be able to re-use a lot (if not all) of the existing driver along with the new plugin model. We will be moving all of the drivers which are merged directly into Machine today to the plugin model when it is available, so there will be no special treatment of those, and there will be documentation outlining the process of developing and using a Docker Machine driver plugin.

With all of that being said, we want to apologize for the lack of feedback on your pull request. As contributors ourselves, we understand that being left in limbo is no fun. We would have liked to address this sooner, and in the future we will be more responsive around these kinds of issues.

Once again, we thank you for the contribution and the tremendous support. Keep hacking strong!

If you want to contribute to the design of the plugin model, we'd love to get your input on this issue where we will be planning it:

#1626

@ehazlett & @nathanleclaire

[dancompton]

@frapposelli This driver should have been merged into master and matured. This setup appears to have magicked away my volume issues on OSX. Think of the hours lost on things like https://github.com/brikis98/docker-osx-dev I appreciate the time saved.

[nathanleclaire]

@dan-compton It will not be merged into master, but the good news is that with the next release it will be possible to distribute support for this as its own driver plugin binary.

[dancompton]

Ah! That makes sense. Sorry!

[GalGavu]

Did anyone create the driver actually that can be used with new docker-machine version?

[frapposelli]

@GalGavu I'm working on the plugin right now, it will be available later this week.

[jeanlaurent]

👍

[joshuajabbour]

@frapposelli is this plugin being worked on somewhere else? or is it on hold? would love to try appcatalyst with docker machine.