Meteor is restarted if file is saved into public directory #894

Closed
mitar opened this Issue Mar 31, 2013 · 24 comments

Comments

Projects
None yet
6 participants
@mitar
Collaborator

mitar commented Mar 31, 2013

I was creating an application which for the user downloads some files from the Internet, saves them into the public directory, so that user can then locally access them. The issue I am having is that every time I save file into public directory, Meteor is restarted, connections to clients restarted. Is there a way to prevent this?

Here is example app which is restarted again and again.

@MarcusRiemer

This comment has been minimized.

Show comment Hide comment
@MarcusRiemer

MarcusRiemer Mar 31, 2013

Are you aware of production mode?

Are you aware of production mode?

@mitar

This comment has been minimized.

Show comment Hide comment
@mitar

mitar Mar 31, 2013

Collaborator

Yes. But even while developing this is problematic.

Collaborator

mitar commented Mar 31, 2013

Yes. But even while developing this is problematic.

@mitar

This comment has been minimized.

Show comment Hide comment
@mitar

mitar Mar 31, 2013

Collaborator

Is there a reason why public directory would be watched?

Collaborator

mitar commented Mar 31, 2013

Is there a reason why public directory would be watched?

@raix

This comment has been minimized.

Show comment Hide comment
@raix

raix Mar 31, 2013

Contributor

Have a look at collectionFS - filesystem_server.js - I've spent a couple of hours making a general solution.

Contributor

raix commented Mar 31, 2013

Have a look at collectionFS - filesystem_server.js - I've spent a couple of hours making a general solution.

@mitar

This comment has been minimized.

Show comment Hide comment
@mitar

mitar Mar 31, 2013

Collaborator

I didn't want to use GridFS, but (distributed) filesystem which can I then later on simply publish directly with nginx or something.

Collaborator

mitar commented Mar 31, 2013

I didn't want to use GridFS, but (distributed) filesystem which can I then later on simply publish directly with nginx or something.

@bbbmmmlll

This comment has been minimized.

Show comment Hide comment
@bbbmmmlll

bbbmmmlll Mar 31, 2013

Having public watched is sometimes useful in development. It saves me a manual restart when I change web page images. I have a love hate relationship with the the restart code in general so some ability to configure it would be useful. Sublime Text 2 w/remote SFTP editing of a large (3401 lines) LESS file has about a 50% of leaving Meteor broken for me.

My applications does a lot of user image uploads and my personal preference is to keep my application isolated from user content as much as possible so I setup a separate Node.js upload service using https://github.com/aguidrevitch/jquery-file-upload-middleware - which works well for my purposes, but isn't tightly integrated with Meteor.

Having public watched is sometimes useful in development. It saves me a manual restart when I change web page images. I have a love hate relationship with the the restart code in general so some ability to configure it would be useful. Sublime Text 2 w/remote SFTP editing of a large (3401 lines) LESS file has about a 50% of leaving Meteor broken for me.

My applications does a lot of user image uploads and my personal preference is to keep my application isolated from user content as much as possible so I setup a separate Node.js upload service using https://github.com/aguidrevitch/jquery-file-upload-middleware - which works well for my purposes, but isn't tightly integrated with Meteor.

@mitar

This comment has been minimized.

Show comment Hide comment
@mitar

mitar Mar 31, 2013

Collaborator

Hm, why would you need a restart in the first place when you change web page image? Doesn't Meteor server them anyway?

Collaborator

mitar commented Mar 31, 2013

Hm, why would you need a restart in the first place when you change web page image? Doesn't Meteor server them anyway?

@raix

This comment has been minimized.

Show comment Hide comment
@raix

raix Mar 31, 2013

Contributor

@mitar : It's not the gridFS part - I'm referering to the symlink scheme for having a public folder not restarting server - and not deleted when bundle is deployed again... The scheme works in general.
https://github.com/raix/Meteor-CollectionFS/blob/master/collectionFS_filesystem.js

Contributor

raix commented Mar 31, 2013

@mitar : It's not the gridFS part - I'm referering to the symlink scheme for having a public folder not restarting server - and not deleted when bundle is deployed again... The scheme works in general.
https://github.com/raix/Meteor-CollectionFS/blob/master/collectionFS_filesystem.js

@mitar

This comment has been minimized.

Show comment Hide comment
@mitar

mitar Mar 31, 2013

Collaborator

Uh, I see some issues there. :-) You should use path.sep, not just hardcode /, no? Otherwise this does not work on Windows.

So if I understand correctly, you store files under .meteor and do a symlink there? I find this hackish, but probably is the best approach without changing the Meteor.

I still wonder why it has to monitor public directory in the first place? Things there do not go into any bundle.

Collaborator

mitar commented Mar 31, 2013

Uh, I see some issues there. :-) You should use path.sep, not just hardcode /, no? Otherwise this does not work on Windows.

So if I understand correctly, you store files under .meteor and do a symlink there? I find this hackish, but probably is the best approach without changing the Meteor.

I still wonder why it has to monitor public directory in the first place? Things there do not go into any bundle.

@bbbmmmlll

This comment has been minimized.

Show comment Hide comment
@bbbmmmlll

bbbmmmlll Mar 31, 2013

@mitar For development: it's same reason why you would restart for any change which is to reduce your development cycle time. I make a change and all three of my test browsers update which saves me from a manual stop/start of Meteor and reloading three browsers. I don't change my application images often, but when I was working on a reactive navigation bar and vector graphics it was useful.

@mitar For development: it's same reason why you would restart for any change which is to reduce your development cycle time. I make a change and all three of my test browsers update which saves me from a manual stop/start of Meteor and reloading three browsers. I don't change my application images often, but when I was working on a reactive navigation bar and vector graphics it was useful.

@mitar

This comment has been minimized.

Show comment Hide comment
@mitar

mitar Mar 31, 2013

Collaborator

Why would you have to stop/start the Meteor? Wouldn't you just have to reload the browsers?

Collaborator

mitar commented Mar 31, 2013

Why would you have to stop/start the Meteor? Wouldn't you just have to reload the browsers?

@raix

This comment has been minimized.

Show comment Hide comment
@raix

raix Mar 31, 2013

Contributor
  • If you use public for saving user files they would all be deleted when delpoying a new bundle
  • Files that goes into public is used eg. when doing appcache? so it kinda relates to the bundling process
  • The symlink, actually allows placing an upload folder a "safe" place inline with the db folder - the bundle symlinks to it from its static folder, the symbolic link is re/ created at server startup

@mitar thanks for pointing out path.sep - I should prop. use that - I dont regard the symlink as a hack - I did at first: old ref to same issue #595 (the explained symlink in #595 is not working when deploying a bundle - why I share my code from collectionFS)

Contributor

raix commented Mar 31, 2013

  • If you use public for saving user files they would all be deleted when delpoying a new bundle
  • Files that goes into public is used eg. when doing appcache? so it kinda relates to the bundling process
  • The symlink, actually allows placing an upload folder a "safe" place inline with the db folder - the bundle symlinks to it from its static folder, the symbolic link is re/ created at server startup

@mitar thanks for pointing out path.sep - I should prop. use that - I dont regard the symlink as a hack - I did at first: old ref to same issue #595 (the explained symlink in #595 is not working when deploying a bundle - why I share my code from collectionFS)

@mitar

This comment has been minimized.

Show comment Hide comment
@mitar

mitar Mar 31, 2013

Collaborator

I am using saving to public directory only while developing. In production, I will probably server static files directly through nginx. I plan to host Meteor app on my own server, so I do not have issues of not being able to write locally or that anything would be overridden (too) automatically.

Hm, things from public go into appcache? I didn't know that.

See here my code. As I see you split path into segments and join from root up? Why?

Collaborator

mitar commented Mar 31, 2013

I am using saving to public directory only while developing. In production, I will probably server static files directly through nginx. I plan to host Meteor app on my own server, so I do not have issues of not being able to write locally or that anything would be overridden (too) automatically.

Hm, things from public go into appcache? I didn't know that.

See here my code. As I see you split path into segments and join from root up? Why?

@mitar

This comment has been minimized.

Show comment Hide comment
@mitar

mitar Apr 12, 2013

Collaborator

OK. I now see that was a bad idea to store things into public, because it tries to bundle it. So for now I moved things under .meteor and I do:

__meteor_bootstrap__.app.use('/pdf', connect.static(Storage._storageDirectory, {maxAge: 24 * 60 * 60 * 1000, redirect: false}))

I didn't really get the symlink trick to work. If I made symlink from /public/pdf/ to /.meteor/pdf/, it still tried to bundle everything.

Collaborator

mitar commented Apr 12, 2013

OK. I now see that was a bad idea to store things into public, because it tries to bundle it. So for now I moved things under .meteor and I do:

__meteor_bootstrap__.app.use('/pdf', connect.static(Storage._storageDirectory, {maxAge: 24 * 60 * 60 * 1000, redirect: false}))

I didn't really get the symlink trick to work. If I made symlink from /public/pdf/ to /.meteor/pdf/, it still tried to bundle everything.

@mitar

This comment has been minimized.

Show comment Hide comment
@mitar

mitar Apr 12, 2013

Collaborator

Aaa, you are creating from /.meteor/local/build/static/. OK.

Collaborator

mitar commented Apr 12, 2013

Aaa, you are creating from /.meteor/local/build/static/. OK.

@mitar

This comment has been minimized.

Show comment Hide comment
@mitar

mitar Jun 27, 2013

Collaborator

So I ended adding my own middleware to server files from under /.meteor/. I am closing this ticket because I think that restarting is correct and public directory should not be misused like that. But maybe there should be some feature request in the future to allow some easy way to server static files from some directory, without them being bundled or touched in any way.

Collaborator

mitar commented Jun 27, 2013

So I ended adding my own middleware to server files from under /.meteor/. I am closing this ticket because I think that restarting is correct and public directory should not be misused like that. But maybe there should be some feature request in the future to allow some easy way to server static files from some directory, without them being bundled or touched in any way.

@mitar mitar closed this Jun 27, 2013

@raix

This comment has been minimized.

Show comment Hide comment
@raix

raix Jun 27, 2013

Contributor

Agree, from a security point there should be kind of a rest point or seperated as a standalone fileserver

Contributor

raix commented Jun 27, 2013

Agree, from a security point there should be kind of a rest point or seperated as a standalone fileserver

@littke

This comment has been minimized.

Show comment Hide comment
@littke

littke Nov 18, 2013

@mitar, could you expand a little on the solution you came up with here? Trying to prevent public from reloading (and the Meteor server to restart) because of CSS changes.

littke commented Nov 18, 2013

@mitar, could you expand a little on the solution you came up with here? Trying to prevent public from reloading (and the Meteor server to restart) because of CSS changes.

@mitar

This comment has been minimized.

Show comment Hide comment
@mitar

mitar Nov 18, 2013

Collaborator

Restarting when a CSS changes is probably a feature. In my case I was using public directory for user file uploads. There is no reason why Meteor would restart when a user uploads a file.

Collaborator

mitar commented Nov 18, 2013

Restarting when a CSS changes is probably a feature. In my case I was using public directory for user file uploads. There is no reason why Meteor would restart when a user uploads a file.

@mitar

This comment has been minimized.

Show comment Hide comment
@mitar

mitar Nov 18, 2013

Collaborator

My solution is to not store uploaded files into public directory. :-)

Collaborator

mitar commented Nov 18, 2013

My solution is to not store uploaded files into public directory. :-)

@littke

This comment has been minimized.

Show comment Hide comment
@littke

littke Nov 18, 2013

Right, and I'd like a similar solution because I want to disable auto reloading and restarting when a CSS file changes, which is the default today :)

littke commented Nov 18, 2013

Right, and I'd like a similar solution because I want to disable auto reloading and restarting when a CSS file changes, which is the default today :)

@innomonster

This comment has been minimized.

Show comment Hide comment
@innomonster

innomonster Aug 31, 2014

HI. @mitar may I know your solution to serve the file after symlink /.meteor/local/build/static/ ? I am able to write the file into ./meteor/local/XXX, and symlink it to /.meteor/local/build/static/XXX. but how can I serve it to client?

Sorry I am a bit slow here.

HI. @mitar may I know your solution to serve the file after symlink /.meteor/local/build/static/ ? I am able to write the file into ./meteor/local/XXX, and symlink it to /.meteor/local/build/static/XXX. but how can I serve it to client?

Sorry I am a bit slow here.

@mitar

This comment has been minimized.

Show comment Hide comment
@mitar

mitar Aug 31, 2014

Collaborator

I don't do symlinking. Not moving anything into build. I use this: https://github.com/peerlibrary/peerlibrary/blob/development/server/storage.coffee

Collaborator

mitar commented Aug 31, 2014

I don't do symlinking. Not moving anything into build. I use this: https://github.com/peerlibrary/peerlibrary/blob/development/server/storage.coffee

@innomonster

This comment has been minimized.

Show comment Hide comment
@innomonster

innomonster Aug 31, 2014

oo.. ok thanks. :)

oo.. ok thanks. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment