Disable EJS View Cache & New Relic [Sails 0.9.16]

[crh3675]

I know this is an older version and we will upgrade soon however the issue I am trying to address is how or where to supply a configuration to disable the EJS caching.

We use NewRelic for monitoring and it isn't playing nice because it won't render it's tracking code most likely due to the templates being cached in production

[mikermcneil]

Hey @crh3675 -- hm not sure offhand, but I'll take a look at https://github.com/balderdashy/sails/tree/v0.9.16 ASAP and let you know what I come up with.

[mikermcneil]

Hmm.. ok so looking back at this, I'm not sure I understand about the new relic thing. Can you tell me more about that?

Re view template caching-- it's going to be caching the contents of your raw EJS templates (specifically the EJS strings themselves, just like you see them in your text editor). It sounds like this might actually have more to do w/ ETag caching; not sure without knowing more.

Thanks!

[Salakar]

@crh3675 was this to do with new relic timing headers / real time user monitoring?

Could you show me how you're passing the new relic tracking into your template? Is it via:

<%- newrelic.getBrowserTimingHeader() %>

?

[crh3675]

Yes that is how it is added and it should execute for every single page load for any/all routes. I am grasping at straws here so bear with me. But it seems that the in-memory caching of the EJS templates may be causing New Relic to send NREUM (1) errors. https://docs.newrelic.com/docs/agents/nodejs-agent/troubleshooting/troubleshooting-page-load-timing-nodejs

New Relic injects Javascript into the layout via a "layout.ejs" file. Since the layout is global, I presume EJS is somehow caching the actual content up to an including the call for

<%- newrelic.getBrowserTimingHeader() %>

I have newrelic loaded in app.js via global.newrelic = require('newrelic');

I can't create this problem in development mode, only production which leads me to believe it is some sort of template cache issue.

Note I have also tried degrading their NodeJS module at least 10 versions as well as using node app vs sails lift vs using forever and also tries pm2. They all yield the same results in production.

[mikermcneil]

@crh3675 ah so the production view cache in Express should only be caching the template, not the actual rendered view (it's definitely confusing because these two different types of caching are referred to almost interchangeably). There's nothing in Sails that should be caching rendered views on the server (in my experience the only time you'd ever want that sort of caching is when a page is dynamic, but also exactly the same for everyone-- and when showing the latest content doesn't matter. Even then I've found it's easier/better to use something like cloudflare). In any case, I've never seen that behavior before, so my guess is that this might have more to do with the rendered page being cached in the browser.

First thing to check if you're testing in a production/staging deployment is whether the domain is hooked up to any kind of cache (e.g. cloudflare). If you're sure that's not it, then I'd try loading the page without any sort of caching-- i.e. with the Chrome inspector open to the network tab, ensuring that the "Disable cache" option is checked. If you can verify that the New Relic plugin works in those isolated conditions, then I think it's safe to say this is probably a caching/etag issue: https://github.com/balderdashy/sails/blob/v0.9.16/lib/configuration/defaults.js#L193

One other thing to check on is if you're using a top-level local named cache anywhere in your app (see http://stackoverflow.com/questions/28294429/sailsjs-view-caching-bug). A while back (if I'm remembering correctly, it was some time during Express 2), @tj set up the view engine to accept options and locals interchangeably (i.e. so the provided locals actually get parsed as options first). This solved some problems, but it also created some confusion.

[crh3675]

Thanks @mikermcneil. I found something totally crazy interesting. I have Redis in the backend of the application to do some low-level caching of external queries. When I disable it, the problem from NewRelic goes away.

I am not using the adapter method within Sails, just loading in through a service I created. So it seems more related to a conflict with Redis and the NewRelic code.

[mikermcneil]

Ah interesting... Glad it's nothing in Sails, but it sucks it's still not resolved. We noticed issues in early 2015 with leaking Redis connections in Treeline that sent us on a similar wild goose chase. Turns out nodetime (the monitoring tool which told us about the leaking Redis connections) was actually the source of the leaking Redis connections. The solution was to disable nodetime x_x

One other thing to keep in the back of your mind is that if you're using the default recommended production setup for Sails, you're probably using Redis for the session store and potentially also for delivering Socket.io broadcasts, so be sure and keep an eye on those too.

[crh3675]

I think you can close this. I traced enough to find SailsJS cache had no impact. Thanks!

[sgress454]

Thanks @crh3675!