2 different instance to the same server

[rafahoro]

Hi,
I'm trying to deploy 2 instances of meteor to the same server IP, but different name.
That means, I have set my DNS so that the server with IP 1.2.3.4, can be accessed by 2 different names: either test.example.com or dev.example.com.

Now I would like to use mup, to deploy both 'test' and 'dev' to that server. For that I've created 2 mup.js files:

.deploy/test/mup.js

module.exports = {
  servers: {
    one: {
      host: 'test.example.com',
      username: 'myname',
      pem: 'mypemfile.pem'
    }
  },

  meteor: {
    name: 'test',
    path: '../..', // as we run it from .deploy/xxxx, relative path to app
    servers: {
      one: {},
    },

    buildOptions: {
      serverOnly: true,
      server: 'https://test.example.com'
    },

    env: {
      ROOT_URL: 'https://test.example.com',
      MONGO_URL: 'mongodb://USER:PASS@db.example.com:11111/db'
    },

    proxy: {
      domains: 'test.example.com'
    },

    ssl: {
      autogenerate: {
        email: 'ssl@example.com',
        domains: 'test.example.com'
      }
    },
    docker: {
      image: 'abernix/meteord:base',   // Meteor 1.4
    },
    deployCheckWaitTime: 60,
    enableUploadProgressBar: true
  },
};

.deploy/dev/mup.js

module.exports = {
  servers: {
    one: {
      host: 'dev.example.com',
      username: 'myname',
      pem: 'mypemfile.pem'
    }
  },

  meteor: {
    name: 'dev',
    path: '../..', // as we run it from .deploy/xxxx, relative path to app
    servers: {
      one: {},
    },

    buildOptions: {
      serverOnly: true,
      server: 'https://dev.example.com'
    },

    env: {
      ROOT_URL: 'https://dev.example.com',
      MONGO_URL: 'mongodb://USER:PASS@db.example.com:22222/db2'
    },

    proxy: {
      domains: 'dev.example.com'
    },

    ssl: {
      autogenerate: {
        email: 'ssl@example.com',
        domains: 'dev.example.com'
      }
    },
    docker: {
      image: 'abernix/meteord:base',       // change to 'kadirahq/meteord' if your app is not using Meteor 1.4
    },
    deployCheckWaitTime: 60,
    enableUploadProgressBar: true
  },
};

Any of them works by itself, but when I try to run both, the second one fails:

[dev.example.com] - Sending Environment Variables
[dev.example.com] - Sending Environment Variables: SUCCESS

Started TaskList: Start Meteor
[dev.example.com] - Start Meteor
[dev.example.com] x Start Meteor: FAILED

        -----------------------------------STDERR-----------------------------------
        emon: endpoint dev-nginx-letsencrypt not found
        Error response from daemon: endpoint dev-nginx-proxy not found
        Finished removing docker containers
        --2017-11-15 12:06:19--  https://raw.githubusercontent.com/jwilder/nginx-proxy/master/nginx.tmpl
        Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.32.133
        Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.32.133|:443... connected.
        HTTP request sent, awaiting response... 200 OK
        Length: 11745 (11K) [text/plain]
        Saving to: ‘/opt/dev/config/nginx.tmpl’

             0K .......... .                                          100% 57.6M=0s

        2017-11-15 12:06:19 (57.6 MB/s) - ‘/opt/dev/config/nginx.tmpl’ saved [11745/11745]

        docker: Error response from daemon: driver failed programming external connectivity on endpoint dev-nginx-proxy (d807ac6088c4f7feca2e56656148228ad6c50e2c8067c96f6a65ffddc6fbaed5): Bind for 0.0.0.0:443 failed: port is already allocated.
        -----------------------------------STDOUT-----------------------------------
        using image
        removing last
        Image mup-dev:latest
        Volume
        dev
        dev-nginx-proxy
        3e94bbf622eae03c1a4934d35ea5c825f8d9af1012f4d626fc28f9b5cb591541
        Ran abernix/meteord:base
        Running autogenerate
        latest: Pulling from jrcs/letsencrypt-nginx-proxy-companion
        Digest: sha256:5bb2e91c0503baf9357c3b5eee659a8b259cf65c49cbf86717fb1a4f2307c0ba
        Status: Image is up to date for jrcs/letsencrypt-nginx-proxy-companion:latest
        latest: Pulling from jwilder/nginx-proxy
        Digest: sha256:da8d4b6ba27f24cead2e368a063634b54c3dd6e2a426d05d134dfde770ac0103
        Status: Image is up to date for jwilder/nginx-proxy:latest
        Pulled autogenerate images
        a629c5945f2cd62999ace38aa2f16806b1e899ca5b6cdd3713f81d1793824d23
        ----------------------------------------------------------------------------

Notes:

• Of course I've changed my domain by example.com
• mup --version ==> 1.3.4 (in my local)
• meteor --version ==> 1.4.4.1 (in my local)

Reading other posts, I understand it's possible to run multiple instances on the same machine on different ports, however i'm not sure if it's possible to access them on the same port (443) but with different hostname (and let nginx do the right port forwarding based on hostname).
Is that possible? How?

[zodern]

It is possible to do this. Try following the instructions at http://meteor-up.com/docs#reverse-proxy. Also, you can run mup validate to make sure your config is valid.

The proxy object should be beside instead of inside meteor. Also, you can configure ssl in the proxy object, which will use a different implementation that supports multiple apps on the server.

[enzolutions]

I am trying to do the same.

I did what the documentation recommend about two create two folders with separate configurations files as you can see in the following image.

Here the first configuration file

module.exports = {
    servers: {
        one: {
            // TODO: set host address, username, and authentication method
            host: '123.456.789.01',
            username: 'root',
            pem: '~/.ssh/myapp'
            //password: '***'
            // or neither for authenticate from ssh-agent
        }
    },

    app: {
        // TODO: change app name and path
        name: 'myapp',
        path: '../',

        servers: {
            one: {},
        },

        buildOptions: {
            serverOnly: true,
        },

        env: {
            // TODO: Change to your app's url
            // If you are using ssl, it needs to start with https://
            ROOT_URL: 'https://myapp.example.com',
            MONGO_URL: 'mongodb://administrator:XXXXX@SERVER.mlab.com:15563/myapp?replicaSet=rs-SERVER',
        },

        ssl: { // (optional)
            // Enables let's encrypt (optional)
            autogenerate: {
                email: 'yes@example.com',
                // comma separated list of domains
                domains: 'myapp.example.com'
            }
        },

        docker: {
            image: 'abernix/meteord:node-8.4.0-base',
        },

        // Show progress bar while uploading bundle to server
        // You might need to disable it on CI servers
        enableUploadProgressBar: true
    },
    proxy: {
        domains: 'myapp.example.com'
    }
}

And my second file is almost the same

module.exports = {
    servers: {
        one: {
            // TODO: set host address, username, and authentication method
            host: '123.456.789.01',
            username: 'root',
            pem: '~/.ssh/myappdev'
            //password: '***'
            // or neither for authenticate from ssh-agent
        }
    },

    app: {
        // TODO: change app name and path
        name: 'myappdev',
        path: '../',

        servers: {
            one: {},
        },

        buildOptions: {
            serverOnly: true,
        },

        env: {
            // TODO: Change to your app's url
            // If you are using ssl, it needs to start with https://
            ROOT_URL: 'https://myappdev.example.com',
            MONGO_URL: 'mongodb://administrator:XXXXX@SERVER.mlab.com:15563/myappdev?replicaSet=rs-SERVER',
        },

        ssl: { // (optional)
            // Enables let's encrypt (optional)
            autogenerate: {
                email: 'yes@example.com',
                // comma separated list of domains
                domains: 'myappdev.example.com'
            }
        },

        docker: {
            image: 'abernix/meteord:node-8.4.0-base',
        },

        // Show progress bar while uploading bundle to server
        // You might need to disable it on CI servers
        enableUploadProgressBar: true
    },
    proxy: {
        domains: 'myappdev.example.com'
    }
}

As you can see I am using the proxy recommendation.

My application was running alone, and no problem, after change the config with proxy approach stills works.

But when I tried to deploy the second instance, I always got the following error, so the app is never deployed on the server.

Retrying in 3 seconds
Failed to copy file  { Error: No such file
    at SFTPStream._transform (/Users/enzo/.npm-packages/lib/node_modules/mup/node_modules/nodemiral/node_modules/ssh2-streams/lib/sftp.js:384:27)
    at SFTPStream.Transform._read (_stream_transform.js:186:10)
    at SFTPStream._read (/Users/enzo/.npm-packages/lib/node_modules/mup/node_modules/nodemiral/node_modules/ssh2-streams/lib/sftp.js:170:15)
    at SFTPStream.Transform._write (_stream_transform.js:174:12)
    at doWrite (_stream_writable.js:387:12)
    at writeOrBuffer (_stream_writable.js:373:5)
    at SFTPStream.Writable.write (_stream_writable.js:290:11)
    at Channel.ondata (_stream_readable.js:639:20)
    at emitOne (events.js:116:13)
    at Channel.emit (events.js:211:7)
    at addChunk (_stream_readable.js:263:12)
    at readableAddChunk (_stream_readable.js:250:11)
    at Channel.Readable.push (_stream_readable.js:208:10)
    at SSH2Stream.<anonymous> (/Users/enzo/.npm-packages/lib/node_modules/mup/node_modules/nodemiral/node_modules/ssh2/lib/Channel.js:140:15)
    at emitOne (events.js:116:13)
    at SSH2Stream.emit (events.js:211:7) code: 2, lang: '' }
Retrying in 3 seconds

A lot of issues mentioned something about memory, but I get on the server and using htop the memory consumption is not the issue.

I think there is something in mup app that I am missing out 😃

[zodern]

@enzolutions you need to run mup setup for both apps.

[yanickrochon]

If the docker image names reflected their name in the configuration, there shouldn't be a need to have multiple confugrations... For example, such configuration :

{
  servers: {
    app1: {
      host: '10.2.0.21',
      username: 'user',
      password: 'pass',
    },
    app2: {
      host: '10.2.0.21',
      username: 'user',
      password: 'pass',
    }
  },

  meteor: {
    name: 'demo',
    path: '../',
    servers: {
      app1: {
        env: {
          PORT: 4001
        }
      },
      app2: {
        env: {
          PORT: 4002
        }
      }
    },
    buildOptions: {
      serverOnly: true,
    },
    env: {
      ROOT_URL: 'http://10.2.0.21',
      // we use our own MongoDB installation....
      MONGO_URL: 'mongodb://user:pass@10.2.0.10:27017/demo'
    },

    docker: {
      image: 'abernix/meteord:node-8.4.0-base',
    },
    deployCheckWaitTime: 120
  },
}

Should get this result from Docker (i.e. sudo docker ps) :

CONTAINER ID    IMAGE                  COMMAND                  CREATED      STATUS      PORTS                  NAMES
7da6bba90ce5    mup-demo-app1:latest   "/bin/sh -c 'bash ..."   7 days ago   Up 7 days   0.0.0.0:4001->4001/tcp   demo-app1
540bd0809a38    mup-demo-app2:latest   "/bin/sh -c 'bash ..."   7 days ago   Up 7 days   0.0.0.0:4002->4002/tcp   demo-app2

Then setup nginx as a load balancer and route to either app on demand. And a single mup deploy to update the app instances.

[elie222]

@yanickrochon he has prod and staging he wants to deploy

[giordanocardillo]

@yanickrochon I've tried your solution but it didn't work out... I wanted to setup load balancing using nginx, but simply it falls in name conflict with the containter names... it doesn't append the "servers" object key after container name.