Assets return 404 after ~5 minutes on dev

[botandrose]

Assets come through at first, but ~5 minutes later they return 404s. A rails server restart fixes this for another 5 minutes.

[botandrose]

Interesting, looks like the directory gets deleted when the server is up for five minutes:

$ ls -la tmp/ | grep ember-cli-
drwxr-xr-x   4 micah micah 4096 Nov 23 16:14 ember-cli-c49aa0b4-61e6-4dbc-b7e4-6a38669eeafb
$ sleep 300
$ ls -la tmp/ | grep ember-cli-
$

Struggling to come up with a hypothesis for this behavior...

[botandrose]

The plot thickens. Directory does not disappear when running rails s, but does with apache 2.4 + latest passenger (5.0.21)! Closing for now, as this appears to be Passenger's fault.

[botandrose]

Okay, turns out this is fallout from using EmberCLI-Rails in development with Passenger's default configuration. Specifically, "smart spawning" mode, and the preloader's timeout of five minutes.

Passenger's smart spawning boots the app in a separate process (aka preloader), then subsequently forks a new worker processes off of the preloader. The benefit of this is that spinning up new worker processes is very fast, and all the worker processes get to share memory via copy-on-write. These worker processes are the ones that actually serve responses, while the preloader never does. If the preloader goes five minutes without spawning a new process, it will shut down to conserve memory.

So, when a Rails app with EmberCLI-Rails boots in this default Passenger environment, the at_exit hook¹ is registered in the preloader process. Then, when that process dies after the timeout, the at_exit hook is run, which eventually deletes the tmp/ember-cli-rails-[uuid] directory², and we get our 404s from the still-running worker processes.

There are very easy workarounds one could employ in the Passenger configuration, like disabling smart spawning, or setting the preloader timeout to zero. However, this took me a fair bit of effort to even figure out what was going on, much less strategies for working around it, so I think it'd be nice to try to address this in EmberCLI-Rails itself, if possible. The first idea that comes to mind is deleting old apps on boot, rather than deleting the current app on shutdown. What do you think? I'd be happy to put together a PR for this, if this strategy sounds acceptable.

---

Footnotes:

1. https://github.com/thoughtbot/ember-cli-rails/blob/7e8cd93/lib/ember-cli-rails.rb#L35
2. https://github.com/thoughtbot/ember-cli-rails/blob/7e8cd93/lib/ember-cli-rails.rb#L72

[seanpdoyle]

@botandrose thanks for putting in the (what sounds like immense) effort to track this down.

I don't use Passenger, so I don't have much context on what a fix would take.

It sounds like you've pair in blood sweat and tears to build up the necessary context, so please open your proposed PR.