Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for UNIX sockets (#7392) #8702

Merged
merged 11 commits into from Sep 6, 2017
Merged

Support for UNIX sockets (#7392) #8702

merged 11 commits into from Sep 6, 2017

Conversation

vlasky
Copy link
Contributor

@vlasky vlasky commented May 17, 2017

@abernix et al, here's my enhancement to implement Support for UNIX sockets (#7392) in meteor.

If the PORT environment variable contains a UNIX file/path, this will be treated as the UNIX domain socket file to listen for incoming connections.

To summarise what I've done:

  1. I added an additional regex check to the existing code in WebAppInternals.parsePort() to test if the PORT variable contains a UNIX file/path and if so, return it as a string.

  2. In function main, we now pass an object called listenOptions to httpServer.listen() and this object contains the required options to configure httpServer to either listen on a TCP port or UNIX socket.

  3. There is a sanity check that handles the situation where there is an existing an existing socket file. If there is, it will try to connect to it to confirm that it does not belong to a running server instance. If it does not, it presumes that the socket file is stale and deletes it.

I note that this support only works in production mode and not when using the meteor tool. I would have liked to add support to the meteor tool, however I see that the package node-http-proxy that it relies on does not appear to support forwarding connections to UNIX sockets, so that would have required a lot more work.

…treated as the UNIX domain socket file to listen for connections on
@vlasky vlasky mentioned this pull request May 17, 2017
Copy link
Contributor

@hwillson hwillson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for submitting this PR @vlasky! I've looked things over and added a few comments, but this will be reviewed more thoroughly very shortly. Thanks again!

WebAppInternals.parsePort = function (port) {
if( /\\\\?.+\\pipe\\?.+/.test(port) ) {
if( /\\\\?.+\\pipe\\?.+/.test(port) || /^(.+)\/([^/]+)$/.test(port) ) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you update the existing parsePort tests to verify this additional parsing works properly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done :-)

httpServer.listen(localPort, localIp, Meteor.bindEnvironment(function() {

if (typeof localPort == "number")
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bit of a nit pick, but can you follow the existing opening/closing brace style that's used in the rest of the file (open brace on same line, etc.)?

console.log("Deleting stale socket file");
fs.unlinkSync(socketPath);

httpServer.listen(listenOptions, Meteor.bindEnvironment(function() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this httpServer.listen block of code is repeated a few lines down, and is the exact same. Could this be extracted into a common function to clean the code up a bit?

…cketPath and made cosmetic fixes to UNIX socket code to conform better with meteor's coding style.
Copy link
Contributor

@abernix abernix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for submitting this PR! It would certainly be great to support Unix sockets.

WebAppInternals.parsePort = function (port) {
if( /\\\\?.+\\pipe\\?.+/.test(port) ) {
if( /\\\\?.+\\pipe\\?.+/.test(port) || /^(.+)\/([^/]+)$/.test(port) ) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not convinced this new RegExp is much better than just checking to see if the path has a / in it at all (i.e. /\//, which is still not ideal. As it stands, this wouldn't allow the socket file to be in the current directory, something we should probably try to support via path.resolve, for example.

In addition to that, can we also could we consider another option for detecting if it's a TCP socket file? This should also probably check process.platform (to ensure it's win32) for the first part.

var socketPath = localPort;
var listenOptions = { path: socketPath };

httpServer.on('error', Meteor.bindEnvironment(function(e) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The addition of all this additional logic suggests that these should probably be come more modular functions instead of this rather-large else block.

var listenOptions = { path: socketPath };

httpServer.on('error', Meteor.bindEnvironment(function(e) {
if (e.code == 'EADDRINUSE') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this EADDRINUSE ➡️ ECONNREFUSED logic based on? I'm hesitant to have all this extra logic in here which would normally produce valid error messages anyhow. Can we really safely assume this is stale?

if (e.code == 'EADDRINUSE') {
var clientSocket = new net.Socket();
clientSocket.on('error', Meteor.bindEnvironment(function(e) {
if (e.code == 'ECONNREFUSED') {
Copy link
Contributor

@abernix abernix May 17, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does a new socket'sECONNREFUSED error make it safe to cleanup the previous socket file?

if (e.code == 'ECONNREFUSED') {
console.log("Deleting stale socket file");
fs.unlinkSync(socketPath);
startHttpServer(listenOptions);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we completely okay with the startHttpServer getting called again here as it will have already been called on line 859 below?

@benjamn benjamn added this to the Release 1.5.1 milestone May 18, 2017
@vlasky
Copy link
Contributor Author

vlasky commented May 19, 2017

@abernix In commit d9e7f86 I have rewritten the code to address the majority of your review comments.

  1. The second regexp has been removed. The code now checks to see if the PORT value is a number or a string containing a windows named pipe (via the original regex), otherwise the value is presumed to be of the desired UNIX domain socketPath (and socket file).

  2. The code no longer calls startHttpServer() twice. It now performs fs.existsSync(socketPath) to determine if there is an existing file with the same name as the desired socket file specified in PORT and then perform fs.statSync(socketPath).isSocket() to determine if that existing file is in fact a socket file (which is a special file) as opposed to an ordinary file (or any other type).

If that file is a socket file, it will then try to perform a dummy connection to it to determine if it's live - i.e. belonging to another running process which is listening for connections on it.

If the dummy connection attempt is successful, we know the socket file is live and being used by a running process, so we will not delete the file. Instead, we will log an error and exit.

If the connection attempt results in the error ECONNREFUSED, we know that the socket file is stale and we can safely delete it using fs.unlinkSync() and call startHttpServer() which will create a fresh socket file.

  1. There is now cleanup code that deletes the socket file created by this process when the node process exits or is ended using SIGINT (Control-C). For those unfamiliar with UNIX socket files, it is the duty of the programmer to delete them up once the program no longer needs them - the OS does not remove them itself.

  2. I removed parsePort() as it is no longer needed by the new logic. Also, it no will no longer convert invalid port values like "8080abc" to the numeric port value 8080 - they will instead be treated as a string.

I really think that old behaviour was wrong and served only to conceal mistakes made by those writing launcher scripts.

@vlasky vlasky mentioned this pull request Jun 13, 2017
@abernix
Copy link
Contributor

abernix commented Jun 13, 2017

@vlasky Sorry I didn't get to review this today but it's first on my plate tomorrow! (And rest assured, Meteor 1.5.1 is not coming out tomorrow. 😉)

Copy link
Contributor

@abernix abernix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping in mind that this is a major entry point into every application, and in consideration of my comments below, I think this still needs work before it's ready to merge.

Keep in mind that .listen creates the socket if it doesn't exist, and it also fails if the file does exist already with an error that can be captured. Also, calling .close on a socket does remove it if the sock file was created.

Aside from the deep nesting (of if > if > if > if > if), which could be broken out into more maintainable functions, I think there is still a safer approach here of just trying to startHttpServer and reacting to the error appropriately, possibly by fixing or checking something and calling back to startHttpServer again (with a couple retries max).

Thanks for taking a look at this. It won't be ready for 1.5.1, unfortunately. I do still think it's a very valuable change if you're willing to work through tidying it up.

@@ -439,15 +440,6 @@ var getUrlPrefixForArch = function (arch) {
'' : '/' + '__' + arch.replace(/^web\./, '');
};

// parse port to see if its a Windows Server style named pipe. If so, return as-is (String), otherwise return as Int
WebAppInternals.parsePort = function (port) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize it's called WebAppInternals, and I think the impact is very low given the general uselessness of this parsePort function, but this is currently a public API and we should consider the repercussions of removing it or if it's worth keeping in place. (I suspect it was originally only to facilitate testing, but it could have been adopted.)

@@ -156,23 +156,6 @@ Tinytest.add("webapp - generating boilerplate should not change runtime config",
test.isFalse(__meteor_runtime_config__.WEBAPP_TEST_KEY);
});

// Support 'named pipes' (strings) as ports for support of Windows Server / Azure deployments
Tinytest.add("webapp - port should be parsed as int unless it is a named pipe", function(test){
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I realize this isn't an exhaustive test by itself, it was testing something and I'm not comfortable removing the only test for this, changing this very core logic and not adding new tests (probably in tools/tests/). Preferably a few, especially since file-system operations and sockets can behave differently on different operating systems.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abernix, in my previous commit 693ac8a, I preserved the legacy parsePort() behaviour and tests, but you weren't happy because socket path would require an absolute path starting with a forward slash, i.e. you could not create a socket file in the current directory.

I note that I have seen other frameworks that use the presence of a forward slash to distinguish between listening on a UNIX socket and listening on a TCP port.

In order to satisfy your concern, I changed this behaviour in commit d9e7f86 so that a number would be treated as a TCP port and anything else would be treated as a UNIX socket, and this required removing the existing parsePort() functionality.

if ( /\\\\?.+\\pipe\\?.+/.test(socketPath)) {
startHttpServer(listenOptions);
} else {
if (fs.existsSync(socketPath)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While it still works, fs.existsSync is technically deprecated (even in Node.js 4) so we should avoid introducing this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, running fs.statSync on the next line is running the same check again but with a race-condition in between. Reacting to the error from statSync would be better than having two separate calls.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abernix existsSync is not deprecated, exists is however:
https://nodejs.org/api/fs.html#fs_fs_existssync_path

Afaik, exists was deprecated because it didn't follow the (err, result) callback signature.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, bizarre, but true! But still not necessary to use it here. :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's right, existsSync() is not deprecated, however I take @abernix's point about replacing the separate calls to existsSync() and statSync() with a single call to statSync() to avoid a race condition.

startHttpServer(listenOptions);
}
}));
clientSocket.connect({ path: socketPath }, function() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's typically discouraged to use the net.Socket.prototype.connect method directly, but rather to rely on net.createConnection. Something to consider?

Copy link
Contributor Author

@vlasky vlasky Jul 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

net.createConnection() is not suitable here because it is necessary to set the error handler in advance for this very short-lived client socket object whose sole purpose is to distinguish between a live UNIX socket and a stale UNIX socket file.

The documentation does not discourage its use, but says "use this only when implementing a custom socket" or in my view "use it when you have to".

startHttpServer(localPort);
} else {
console.error("Invalid PORT specified");
process.exit();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should be calling process.exit anywhere in this code. That will be handled by the caller to main() here in boot.js. I think you can just return ERROR or something (though you should leave the console.error.

process.on('exit', Meteor.bindEnvironment(function(e) {
fs.unlinkSync(socketPath);
}));
process.on('SIGINT', Meteor.bindEnvironment(function(e) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem relevant to this change at all, what's the reason for this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without this cleanup code, a stale UNIX socket file (.sock) file is left on disk if you terminate the node process with Control-C (SIGINT).

@hwillson
Copy link
Contributor

Hi @vlasky - have you had a chance to look over the recent review comments?

@hwillson
Copy link
Contributor

hwillson commented Jul 8, 2017

Hi @vlasky - are you still interested in working on this PR? We'd love to get these changes in place, after the recent review questions have been addressed. If you're not available to work on this let me know, and I'll jump in to help out. Thanks!

@vlasky
Copy link
Contributor Author

vlasky commented Jul 12, 2017

@hwillson I have been away dealing with other business commitments recently.

My company has been using commit d9e7f86 in production without any issues.

The only change I plan to make to this PR for time being is the one mentioned in #8702 (comment) to resolve a race condition.

After that you (or anyone else) can jump in if you see a need.

@hwillson
Copy link
Contributor

No problem at all @vlasky - thanks again for putting this together. After you add in changes to help with the race condition, I'll jump in to help to get this wrapped up / merged.

…l for race condition when checking for existing UNIX socket file
@vlasky
Copy link
Contributor Author

vlasky commented Jul 12, 2017

@hwillson - the race condition fix has been implemented in 76808e8. Feel free to jump in.

@hwillson hwillson self-assigned this Jul 19, 2017
@skirunman
Copy link
Contributor

Just wondering if this is moving forward in 1.5.2 or 1.6 releases as currently tagged as 1.5.1.

@hwillson hwillson removed this from the Release 1.5.1 milestone Aug 10, 2017
@hwillson
Copy link
Contributor

Good question @skirunman - I'm holding this one up unfortunately; hopefully I'll have the discussed changes wrapped up this week. We'll then be able to see which release it makes sense to add this to. Thanks!

@hwillson
Copy link
Contributor

Hi all - I have a new version of this just about ready to go, but I noticed a rather large issue while testing.

First off, regarding net.Socket.prototype.connect versus net.createConnection, as long as the error handler callback is set immediately after the createConnection call, we should be okay. createConnection handles this by deferring the error event to the nextTick. That being said, unfortunately I don't think either approach will work as attempted.

Let's assume we're using createConnection (but this issue stands with net.Socket.prototype.connect). Using createConnection to check to see if we can connect to an existing socket file, then if we connect flag that socket file as already being used by another server, isn't a perfect check. If you SIGKILL the running node process, the socket file is left in place (not much we can do about this). Then if you use createConnection to connect to that file again, it will work. When that happens, we're flagging the socket file as being used by another server, when in reality it isn't.

Unfortunately, there isn't a perfectly reliable way to tell if a socket file is already being used by another process. That means we need to handle this check in another way. A few options:

  1. (Least safe but automatic) Use net.createConnection as outlined, but if we can connect to the socket file, throw an exception, remove the existing socket file, and continue on. With this approach we don't care if the existing socket file is being used by another process or if it was left over from a SIGKILL. In either case we remove it, and continue to start the http server. This approach will work, but if another process really is using the socket file, and that socket file is removed while it's running, well, needless to say that wouldn't be good ...

  2. (Most safe but manual) We don't try to connect to an existing socket file, ever. Instead, when firing up the webapp we first check to see if an existing socket file exists (like we're doing now), but if found we throw an exception describing the problem and the location of the existing socket file, and exit. This approach is nice and safe (as we're not killing a socket file that could be used by another process), but we're leaving the responsibility of cleaning up a potentially stale socket file to the developer deploying/starting the app. Requiring manual intervention like this could be risky, especially when the application is deployed to an environment that tries to automatically re-start crashed apps. It could be waiting for the stale socket file to be removed manually, and it could take some time before people notice.

As you can see, neither approach is perfect. I'm leaning towards the first approach and making sure people know that they need to define their socket paths very carefully, to make sure they aren't clobbering another process. This approach falls more in-line with other areas of Meteor, that make certain assumptions to keep the development/deployment process more automatic. I'd love to hear what others think, and if anyone has other suggestions? Thanks!

@hwillson
Copy link
Contributor

hwillson commented Aug 11, 2017

Just to add - the existing code in this PR is already achieving #1 above, since the socketPath file is being removed during the specified process.on('exit', ... callback. Firing up a second server, getting the socket file already being used message, then watching my first server crash really threw me off during testing however. If we agree this is the way to go though, I'll adjust the error handling / messaging a bit to be more explicit about what's happening.

console.error("Error listening:", e);
console.error(e && e.stack);
}));
if (typeof localPort == "string") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

=== ?

@hwillson
Copy link
Contributor

Thanks @zimme - yes, I have seen that approach and have tried several variations. None of them are perfect unfortunately. If I SIGKILL the running node instance, the sock file is left in place, and I can still connect to it on retry. That means we never actually get a ECONNREFUSED when retrying a connection with a stale socket file, or when connecting using a socket file that's already in use - in both cases I can connect using that socket file, so there isn't an easy way to differentiate between the two scenarios.

@zimme
Copy link
Contributor

zimme commented Aug 13, 2017

I guess using ref/unref won't help either when SIGKILL is used?

@hwillson
Copy link
Contributor

Hi all - based on my comments in #8702 (comment), I've adjusted this PR a bit. If an existing socket file is found when the application starts up, it is first removed before a new socket file is created when the HTTP server starts. As mentioned previously, this could be a bit dangerous if another application is configured to use the same socket file, on the same server. As long as people are configuring their applications properly, and making sure to use unique socket paths for different applications, this really shouldn't be an issue. The good news is by making this assumption up-front, we're able to simplify the codebase (and possible error paths).

With this new approach, some of the previous (outstanding) review comments are no longer valid. A quick synopsis:

  • I've re-arranged things to help reduce the nested if levels.

  • Regarding WebAppInternals.parsePort, since it was available as part of the previous public API, I've restored that functionality. It will now return a parsed numeric port if it can, otherwise it will return the full port string. This should provide the same functionality that was in place (I've restored the removed tests to verify this), and allow unix socket path's to be returned as well. That being said, since this function is no longer used internally, I've added a DEPRECATED label, and mentioned that using this function directly is not recommended.

  • net.Socket.prototype.connect vs. net.createConnection no longer matters (we're not attempting to connect to an existing socket file - we're just removing it if it exists).

  • process.on('exit', ... and process.on('SIGINT', ... are no longer being used. Since we're removing any previous socket file on startup no matter what, we don't need to worry about cleaning the socket file at the various possible program exit states.

  • I've removed the use of process.exit and am instead throwing exceptions where necessary (since as mentioned process.exit is already being handled upstream in https://github.com/meteor/meteor/blob/devel/tools/static-assets/server/boot.js#L353).

I believe this addresses all outstanding comments. After these new changes have been reviewed, I'll put a History.md entry together. I'll also (shortly) post a link to the test application I've been using to verify these changes (Nginx + Meteor), along with testing steps.

Thanks!

@hwillson
Copy link
Contributor

Adding a test app: https://github.com/hwillson/meteor-pr-8702

@zimme
Copy link
Contributor

zimme commented Aug 14, 2017

process.on('exit', ... and process.on('SIGINT', ... are no longer being used. Since we're removing any previous socket file on startup no matter what, we don't need to worry about cleaning the socket file at the various possible program exit states.

Shouldn't we try and do cleanup anyway? As we're creating the socket file we should be removing it when we do a graceful shutdown, even though we always remove it before starting again. But what if you don't start it again or change the name of the socket, and all of a sudden you have an unnecessary socket file.

@hwillson
Copy link
Contributor

Good question @zimme - I had exit cleanup code in place just before I committed, but decided to take it out in the end to simplify things. My logic was that we're removing it again at startup, and if a socket file using version of the app is never started again, we've ended up leaving one 0 byte file on the filesystem. I also wasn't crazy about the idea of having additional exit handlers defined in the webapp package (when the process is managed via tools/static-assets/server/boot.js). Again though, I was on the fence on this one as well, so maybe I'll add it back in. Thanks for the feedback!

@zimme
Copy link
Contributor

zimme commented Aug 14, 2017

we've ended up leaving one 0 byte file on the filesystem

Still littering the filesystem, what if someone setup meteor to always generate a new socket file on boot with a script? Then you might end up with a lot of unused socket files.

@hwillson
Copy link
Contributor

hwillson commented Aug 14, 2017

Haha - that would be horrible, but valid point. I'm re-visiting the cleanup code now. Thanks!

@hwillson
Copy link
Contributor

Oh Circle - 4 failures when everything passes locally 🙁. Restarting, but the reported failures shouldn't be related to these changes. Also re-moving the WIP label from the title; this should be ready for a final review. I still have to add a History.md entry, but I'd like to make sure we're all on board with this new direction before finalizing. Thanks!

@hwillson hwillson changed the title [Work in Progress] Support for UNIX sockets (#7392) Support for UNIX sockets (#7392) Aug 16, 2017
Copy link
Contributor

@abernix abernix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few comments!

I think the logic chosen here is great and there doesn't seem to be any de-facto standard on it anyhow.

If anything, this new code (and the code that was there before) is more clean now than it was before, easy to iterate on and looks like a clean execution path in this (relatively critical) entry point. This was a relatively popular feature request, so hopefully we'll get helpful feedback quickly from those who use it and can iterate further if necessary.

// DEPRECATED: Direct use of this function is not recommended; it is no
// longer used internally, and will be removed in a future release.
WebAppInternals.parsePort = port => {
let parsedPort = parseInt(port);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we actually print a deprecation warning to the console, once per session? Or just add this to the History.md? Or do nothing? (It is called Internals, after all.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does appear to still be used from webapp_tests.js. Should we export this function with another internal name for that purpose?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's only used in webapp_tests.js to test itself 🙂. Regarding how to deprecate it, I think just mentioning it in the History.md should be enough (because as you said, Internals).

import {
removeExistingSocketFile,
registerSocketFileCleanup,
} from './socket_file';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we explicitly state the .js extension? At least for consistency with other code?

_.each(callbacks, function (x) { x(); });
var startHttpServer = function(listenOptions) {
httpServer.listen(listenOptions, Meteor.bindEnvironment(function() {
if (process.env.METEOR_PRINT_ON_LISTEN)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit-⛏: Braces.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing - I didn't really review that part of the code; I'll update that entire function a bit.

} from './socket_file';
import { EventEmitter } from 'events';

const testSocketFile = '/tmp/socket_file_tests';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that we test on Windows, but this will fail on Windows. Could we use files.mkdtemp from tools/fs/files.js to create a directory for this file to live in?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ha - oops. Good point - changing, thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right - I can't reference tools/fs/files.js directly from webapp; I'll use Node's os.tmpdir instead.

@hwillson
Copy link
Contributor

hwillson commented Aug 18, 2017

Well, I was just prepping a History.md entry for this and was about to say we're good to go here, but then remembered we needed to add a mention that this was production only. The thing is, this doesn't need to be production only if we don't use the PORT environment variable to set the UNIX socket file path. If we use another unique environment variable like UNIX_SOCKET_PATH, then socket files can be used with the Meteor Tool and in production Node apps. Since hijacking PORT to store a pipe or socket path was a hack anyways, I've gone ahead and changed this around. UNIX domain sockets can now be used by setting something like UNIX_SOCKET_PATH=/tmp/somesock.sock. Windows named pipes are still set using PORT, but that's a change for another day/PR. When starting the Meteor Tool with the UNIX_SOCKET_PATH set, it will now show that a socket file is being used (instead of displaying the host:port combo). I've also finalized the History.md entry. Standing by for feedback - thanks!

@zimme
Copy link
Contributor

zimme commented Aug 18, 2017

Windows named pipes are still set using PORT, but that's a change for another day/PR.

Are we sure we don't want to add WINDOWS_NAMED_PIPE which would work the exact same way as UNIX_SOCKET_FILE?

@hwillson
Copy link
Contributor

We could - I left those changes out since this PR should be good to go as is and meets the requirements of the original issue (#7392). Adding adjustments for Windows seemed to be a bit out of scope, and the current Windows named pipe hack has been in place for a while, so it's being used in production. Wiring up something like WINDOWS_NAMED_PIPE didn't seem to be as pressing since we'll still have to support the PORT hack for Windows anyways. Unless of course we deprecate the Windows PORT approach, but we'll definitely have to handle that situation carefully.

@kentonv
Copy link
Contributor

kentonv commented Oct 22, 2017

Hi all. This is largely our fault, not yours, but I wanted to note for informational purposes that this change played a part in a global outage of self-hosted Sandstorm.io servers. It turns out our startup code would often set a PORT variable with multiple ports, comma-separated, like "80,443". In the past, Meteor would use the first port, and other code in our server would parse out the remaining ports and do other things with them. This change implemented more strict parsing of ports, so broke that configuration. Unfortunately, our usual testing configurations did not happen to have multiple ports set, so the problem was not caught until real servers started dying after auto-updating.

Again, this is our fault for questionable code (we should never have relied on lenient PORT parsing) and insufficient testing, but I thought having visibility into this kind of issue would be informative.

Sandstorm issue and postmortem here: sandstorm-io/sandstorm#2993

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants