Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

no automatic restart when using boot2docker docker volume #419

Closed
chrishiestand opened this issue Nov 20, 2014 · 66 comments
Closed

no automatic restart when using boot2docker docker volume #419

chrishiestand opened this issue Nov 20, 2014 · 66 comments
Labels

Comments

@chrishiestand
Copy link

@chrishiestand chrishiestand commented Nov 20, 2014

nodemon doesn't seem to restart node when using a docker volume, or in this case a boot2docker ( os x -> virtualbox -> docker) volume. nodemon starts okay, but then fails to restart node when a watched file has been modified in the shared OS X volume.

root@760d5d2ff8cf:/src# nodemon --watch index.js 
20 Nov 21:06:58 - [nodemon] v1.2.1
20 Nov 21:06:58 - [nodemon] to restart at any time, enter `rs`
20 Nov 21:06:58 - [nodemon] watching: index.js
20 Nov 21:06:58 - [nodemon] starting `nodejs index.js`
Running on http://localhost:8080
request received
#index.js changed and saved here but output does not change as it should
request received
request received

nodemon -L causes restarts at least once a second even when no files have changed - way too frequently.

forever -w seems to work just fine. That seems like the best workaround for now.

output of dump:

root@b36ab2778133:/src# nodemon --dump
20 Nov 22:08:27 - [nodemon] v1.2.1
20 Nov 22:08:27 - [nodemon] to restart at any time, enter `rs`
20 Nov 22:08:27 - [nodemon] watching: *.*
--------------
node: v0.10.25
nodemon: v1.2.1
command: node /usr/local/bin/nodemon --dump
cwd: /src
OS: linux x64
--------------
{ run: false,
  system: 
   { cwd: '/src',
     useFind: false,
     useWatch: true,
     useWatchFile: false },
  required: false,
  dirs: [ '/src' ],
  timeout: 1000,
  options: 
   { scriptPosition: 0,
     dump: true,
     script: 'index.js',
     args: [],
     ignore: 
      [ '.git',
        'node_modules',
        'bower_components',
        '.sass-cache',
        re: /\.git|node_modules|bower_components|\.sass\-cache/ ],
     watch: [ '*.*', re: /.*\..*/ ],
     restartable: 'rs',
     execMap: { py: 'python', rb: 'ruby' },
     stdin: true,
     verbose: false,
     stdout: true,
     execOptions: 
      { script: 'index.js',
        exec: 'node',
        args: [],
        nodeArgs: undefined,
        ext: 'js',
        env: {},
        execArgs: [] },
     ext: 'js',
     monitor: 
      [ '*.*',
        '!.git',
        '!/src/node_modules/**/*',
        '!bower_components',
        '!.sass-cache' ] },
  load: [Function],
  reset: [Function: reset],
  lastStarted: 0,
  loaded: [],
  command: 
   { raw: { executable: 'node', args: [ 'index.js' ] },
     string: 'node index.js' } }
@chisleu
Copy link

@chisleu chisleu commented Nov 23, 2014

When I use boot2docker with my branch off of library/node, when I try to use nodemon /path/to/app --watch, the cpu goes nuts like it is constantly restarting, although I can't verify.

I tried -L, but it also caused the CPU to go nuts.

I changed to pm2 for now, but it has 2-3 seconds of delay before it restarts. Still, better than nothing.

@tanis2000
Copy link

@tanis2000 tanis2000 commented Nov 25, 2014

+1 this happens to me as well with a mounted volume on boot2docker (OSX + VirtualBox).

@remy remy added the help wanted label Dec 4, 2014
@remy
Copy link
Owner

@remy remy commented Dec 4, 2014

I'm marking as "help wanted" - I don't have the infrastructure to even begin testing and debugging this.

I'm sure it's linked to clocks being out of whack somehow, but can't replicate myself with simple tests.

@tanis2000
Copy link

@tanis2000 tanis2000 commented Dec 4, 2014

@remy if I can help you in any way, just let me know what I should do.

@remy
Copy link
Owner

@remy remy commented Dec 4, 2014

@tanis2000 if you can pull the project inside of your virtual machine. I'd start by trying to discover why -L sets the CPU insane. The watching logic is all in watch.js - it's a little complicated because there's multiple methods to actually watch.

Really you want to get a handle on what's causing the issue in the first place. I usually console log the crap out of the code when I'm spelunking for errors.

@mattapperson
Copy link

@mattapperson mattapperson commented Dec 16, 2014

still not seeing an automatic restart with a volume mounted nodejs project in b2d

@mrmurphy
Copy link

@mrmurphy mrmurphy commented Jan 2, 2015

Remy, thanks for nodemon. Did you close this issue because you resolved it? Or did you not have enough helpful feedback to fix?

We've been experiencing the same problems as mentioned above.

@remy
Copy link
Owner

@remy remy commented Jan 3, 2015

No, it looks like it was closed by mistake in the associated PR. I'll reopen and ask for help (from you lot!).

@remy remy reopened this Jan 3, 2015
@remy
Copy link
Owner

@remy remy commented Jan 4, 2015

If anyone wants to push a Docker image that I can test and debug this with – I'd be grateful.

@olalonde
Copy link

@olalonde olalonde commented Jan 5, 2015

👍 same problem here

@remy
Copy link
Owner

@remy remy commented Jan 5, 2015

@olalonde then same question to you: "[do you have] a Docker image that I can test and debug this with – I'd be grateful.".

@chisleu
Copy link

@chisleu chisleu commented Jan 5, 2015

I will make one if someone else doesn't. I can't get to it until tonight though, which is why I had not yet replied.

Assuming Ubuntu 12.04 is ok unless I hear otherwise.

A faster solution would be to use vagrant. If you have vagrant and virtualbox, you can install a VM with

vagrant up ubuntu/precise64 --provider virtualbox
vagrant ssh

Then install node/npm and nodemon. Place a node file in a shared resource (such as /vagrant) and you should find these problems to be easy to reproduce.

I'm not sure about docker as I no longer use docker for development (because of this issue, and because I use MacOSX and the boot to docker image causes CPU and sleep problems for my little Macbook Air.)

It still affects my work in Vagrant and seems to have the same results.

BTW, PM2 suffers a similar fate now in Vagrant. It simply fails to restart at all.

@olalonde
Copy link

@olalonde olalonde commented Jan 5, 2015

Ok, I just made a test but can't push it to Github at the moment.

Install boot2docker. Run $(boot2docker shellinit).

Put all the following files in the same directory (e.g. nodemontest).

./Dockerfile

FROM node:0.10-slim
RUN mkdir /app
WORKDIR /app

CMD ["./node_modules/nodemon/bin/nodemon.js", "index.js"]

./index.js

console.log(new Date());

./package.json

{
  "name": "testnodemon",
  "devDependencies": {
    "nodemon": "^1.2.1"
  }
}

Once, you have those files, run the following commands.

npm install;
docker build -t testnodemon .;
docker run -v $(pwd):/app testnodemon;

Edit index.js, nodemon doesn't restart the script.

You can use the following to get a shell in the container:

docker run -i -t -v $(pwd):/app testnodemon /bin/bash;
root@762c18df76ee:/app# mount | grep app
none on /app type vboxsf (rw,nodev,relatime)

Related? https://www.virtualbox.org/ticket/10660

@remy
Copy link
Owner

@remy remy commented Jan 5, 2015

@olalonde thanks for this. I'll go through the steps this week to try to replicate. I'm sure it's linked the latest information in this issue #11 but I need to see it first hand.

@olalonde
Copy link

@olalonde olalonde commented Jan 5, 2015

The problem I believe is that vboxsf doesn't support inotify notifications. The alternative would be to poll the directory but that could be end up eating all the CPU on large code bases. Another solution which I believe was adopted by Ruby's guard is to have a client on the host machine which sends restart messages to a another process running on the guest machine.

Anyways, I hope you can come up with something better :)

@chisleu
Copy link

@chisleu chisleu commented Jan 5, 2015

It is such a problem to resolve that we started running node locally with a remote Cassandra DB (since C* would be the only service in docker.)

@olalonde
Copy link

@olalonde olalonde commented Jan 5, 2015

Another solution would be to watch files normally on the host and whenever a file changes, write to .file-changed. The nodemon running in the container could just tail .file-changed and reload the process whenever it receives new data.

@olalonde
Copy link

@olalonde olalonde commented Jan 5, 2015

I just submitted a PR with the solution I posted above. It's not elegant, but it's simple and it works.

Basically, nodemon needs to run on the host with the --local flag:

nodemon --local

and the VM/container can listen for file change events by using the --remote flag:

nodemon --remote index.js

The local nodemon communicates with the remote one by toggling a file called .nodemon-ping

remy added a commit that referenced this issue Jan 5, 2015
Though the command is slightly different, so fork the code logic. This will still fall back to alternative method if find isn't available.

Ref #419
@remy
Copy link
Owner

@remy remy commented Jan 5, 2015

Can you try the development version of nodemon that I've pushed? It contains a lot of other tweaks, but the main thing I've changed is it'll use find instead of fs.watchFile (which I think uses the broken IO notify...).

To install:

npm install -g nodemon@dev

Can someone quickly give me an idea of whether that works or not? I tried it myself, but I'm still pretty new to docker so wasn't quite sure of what I was doing...

@remy
Copy link
Owner

@remy remy commented Jan 5, 2015

(it should be nodemon@1.3.0-7)

@remy
Copy link
Owner

@remy remy commented Jan 5, 2015

Nope, okay, I'm pretty sure nodemon@1.3.0-7 farts and makes a mess in docker. Anyone else want to confirm that's cool, otherwise I'll take this as a fail.

@aisipos
Copy link

@aisipos aisipos commented Sep 24, 2015

I can confirm for myself at least using -L in 1.7.0 in a docker container is working for me. Without it, it doesn't work for me. (I'll note also, with some sadness, that while polling does work it does use a bit of CPU).

@chrishiestand
Copy link
Author

@chrishiestand chrishiestand commented Sep 25, 2015

@remy Thanks for your care on this issue. I've changed my workflow and haven't used this feature in some time. If other people have reported success with -L that is good enough for me. Thanks!

@pencilcheck
Copy link

@pencilcheck pencilcheck commented Sep 26, 2015

-L works but the CPU will soar when run in boot2docker vm.

@remy
Copy link
Owner

@remy remy commented Sep 26, 2015

@pencilcheck you should also be wary of the number of files its watching.

@d6u
Copy link

@d6u d6u commented Sep 30, 2015

@remy -L works for me with docker-compose (previously boot2docker) version 1.4.2 on Node 0.10 with Nodemon 1.7.1

@pencilcheck
Copy link

@pencilcheck pencilcheck commented Sep 30, 2015

In a VM -L is still needed it seems but I have switched to a different van that setup NFS instead of vboxfs, now it runs much much quieter but unfortunately writing speed is not as responsive as native

@oncletom
Copy link

@oncletom oncletom commented Oct 7, 2015

-L works perfectly, thanks!

@akselnordal
Copy link

@akselnordal akselnordal commented Oct 15, 2015

Nothing of the above worked for me, so I ended up creating a small bash-script that starts my servers, restarts the pm2 instance inside docker from the host OS and tails docker logs. To avoid zombie processes, I'vce included a trap that kills the docker processes started by docker-compose

#!/bin/bash

function clean_up {
  docker-compose kill
  exit
}

trap clean_up SIGHUP SIGINT SIGTERM

docker-compose build
docker-compose start

# tail all logs 
docker-compose logs&

# fswatch is available for linux and osx.
fswatch -0 src | while read -d "" event
  do
    echo "file changed: ${event}"
    image=$(docker ps | grep <docker app-name> | cut -d " " -f1 )
    docker exec -t $image ./node_modules/.bin/pm2 restart <pm2 app-name>
  done
@Omnomios
Copy link

@Omnomios Omnomios commented Dec 17, 2015

I was chasing this issue for a while. Nodemon was catching the change no problem but node never received the signal to terminate. I was looking through the nodemon source and saw that there was an issue with child processes and as a result it was using psTree to get to the children.

The docker image I was using (fedora) didn't come with PS pre installed and so nodemon was using a fallback to pull the process that wasn't working for me.

Installing the 'procps' package in the dockerbuild got it working for me.

Hope this helps someone.

@thalesfsp
Copy link

@thalesfsp thalesfsp commented Dec 29, 2015

So, what's the final solution and how to, to get nodemon watching for changes in Mac OS X -> Boot2Docker -> Docker Container? It's possible or not? There's some documentation/README.md/Wiki? @remy

@pyper
Copy link

@pyper pyper commented Dec 29, 2015

@thalesfsp It looks as though there is a few issues possibly causing this. @Omnomios has done some next level bug chasing and found something. For me, I simply had to explicitly set --watch.

@altieres
Copy link

@altieres altieres commented Jan 3, 2016

The best way I have found to have filesystem notifications visible within the container was as follows:
1- Create two folders, one to map the project and a "mirror"
2- Map the project in the first folder
3- Keep a background script running in the container, rsyncing the project folder to "mirror"
4- Run the project from "mirror"

It may not be the most efficient or most elegant way, but this way was transparent to users of the container. No additional script needs to be run.
Not tested in a larger project, but in my case I did not realize performance issues.

https://github.com/altieres/docker-jekyll-s3

@nettofarah
Copy link

@nettofarah nettofarah commented Jan 5, 2016

I have similar solution to @Omnomios.
I wrote this bash script

#!/bin/bash

image=$(docker ps | grep <container_name> | cut -d " " -f1 )
docker exec -t $image pm2 restart server

and hooked it up to webpack in my gulpfile

gulp.task('watch-backend', function() {
  webpack(backendConfig).watch(100, function(err, stats) {
    onBuild(err, stats)
    restartServer()
  })
})

var exec = require('child_process').exec

function restartServer() {
  exec('./restart-container.sh', function(err, stdout, stderr) {
    if (stdout) {
      console.log('stdout: ' + stdout)
    }

    if (stderr) {
      console.log('stderr: ' + stderr)
    }

    if (err !== null) {
      console.log('exec error: ' + err)
    }
  })
}

It has been working pretty well for me.
You have to run webpack (or whatever other watcher) in the host machine though.

@darrinholst
Copy link

@darrinholst darrinholst commented Jan 30, 2016

I cannot figure out how to get this working either. Shelling into the container and touching a file triggers nodemon...doing the same thing from osx doesn't do anything.

@darrinholst
Copy link

@darrinholst darrinholst commented Jan 30, 2016

whoops...I missed the need for -L. Working once I added that.

@kirkstrobeck
Copy link

@kirkstrobeck kirkstrobeck commented May 2, 2016

-L ftw ! 👍

@yordis
Copy link

@yordis yordis commented Aug 7, 2016

I am using Docker for Mac and it doesn't work either but -L flag fixed it

@mshick
Copy link
Contributor

@mshick mshick commented Mar 10, 2017

Using a barebones security distro and was having this issue. @Omnomios solution installing procps worked for me.

@bradcosine
Copy link

@bradcosine bradcosine commented Mar 30, 2017

Only -L fixed the issue for me

@IonelLupu
Copy link

@IonelLupu IonelLupu commented May 22, 2018

Has anyone found a better solution, more official fix (maybe without any parameter)t o this -L parameter?

@mshick
Copy link
Contributor

@mshick mshick commented May 22, 2018

@jcwolf have you tried the solution that worked for me? procps? I haven't been using legacy mode for over a year now and get automatic restarts.

Here's a snippet of my Dockerfile

FROM node:8.9.4-wheezy
RUN apt-get update
RUN apt-get install -y procps make gcc g++ python libpq-dev
@remy
Copy link
Owner

@remy remy commented May 23, 2018

@mshick would you be able to add this to the FAQ? Sounds like you've solved a problem that many people run in to! Thanks in advance

@mshick
Copy link
Contributor

@mshick mshick commented May 23, 2018

@remy Added a note. Credit goes to @Omnomios. Hope it helps!

remy added a commit that referenced this issue May 23, 2018
[skip ci]

Addresses #419
@ms20hj
Copy link

@ms20hj ms20hj commented Sep 20, 2019

a simple patch would be to change the defaultConnectionProperties.properties inside the jar file.

oracle.jdbc.timezoneAsRegion=false

defaultConnectionProperties.properties file path: oracle.jdbc,you can find it from ojdbc6.jar.
My oracle6.jar version 11.2.0.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

You can’t perform that action at this time.