New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Container IP always change when deploying an app on no-domain server #1123

Closed
ingro opened this Issue Apr 18, 2015 · 22 comments

Comments

Projects
None yet
4 participants
@ingro

ingro commented Apr 18, 2015

Hello! I've setup a Dokku app on Digital Ocean and deployed a Node.js app on it. Everything seems working fine but every time I push a new version to the server it changes the running port, for example if the app was running on http://123.123.123.123:49156 after the restart it will run on http://123.123.123.123:49157.

Is there a way to make an app always run on a given port? Thank you.

@michaelshobbs

This comment has been minimized.

Show comment
Hide comment
@michaelshobbs

michaelshobbs Apr 18, 2015

Member

If you use the vhost config and associated plugin is will always run on port 80.

Check out the docs for more details

Member

michaelshobbs commented Apr 18, 2015

If you use the vhost config and associated plugin is will always run on port 80.

Check out the docs for more details

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Apr 19, 2015

Member

What @michaelshobbs said. Restarting on the same port isn't possible due to how port usage works (we start the new container and run checks against it before killing the old one, and don't reuse ports).

As there isn't anything we can do here - and the fix is to use a vhost config - I'm closing the issue.

Member

josegonzalez commented Apr 19, 2015

What @michaelshobbs said. Restarting on the same port isn't possible due to how port usage works (we start the new container and run checks against it before killing the old one, and don't reuse ports).

As there isn't anything we can do here - and the fix is to use a vhost config - I'm closing the issue.

@ingro

This comment has been minimized.

Show comment
Hide comment
@ingro

ingro Apr 20, 2015

Thanks @michaelshobbs and @josegonzalez for the clarification.
If I understand well I should use this plugin?

Thanks again and sorry for the disturb.

ingro commented Apr 20, 2015

Thanks @michaelshobbs and @josegonzalez for the clarification.
If I understand well I should use this plugin?

Thanks again and sorry for the disturb.

@michaelshobbs

This comment has been minimized.

Show comment
Hide comment
@michaelshobbs

michaelshobbs Apr 20, 2015

Member

@ingro no nginx is built directly into dokku.

See this:
http://progrium.viewdocs.io/dokku/nginx

Basically everything below the Customizing the nginx config section is pertinent.

Member

michaelshobbs commented Apr 20, 2015

@ingro no nginx is built directly into dokku.

See this:
http://progrium.viewdocs.io/dokku/nginx

Basically everything below the Customizing the nginx config section is pertinent.

@ingro

This comment has been minimized.

Show comment
Hide comment
@ingro

ingro Apr 20, 2015

Thanks a lot @michaelshobbs , I will check it ou! :)

ingro commented Apr 20, 2015

Thanks a lot @michaelshobbs , I will check it ou! :)

@codeHatcher

This comment has been minimized.

Show comment
Hide comment
@codeHatcher

codeHatcher Sep 16, 2015

@michaelshobbs looking at that section of the site didn't help me. I need to run the app on a consistent port without using VHOST.

codeHatcher commented Sep 16, 2015

@michaelshobbs looking at that section of the site didn't help me. I need to run the app on a consistent port without using VHOST.

@michaelshobbs

This comment has been minimized.

Show comment
Hide comment
@michaelshobbs

michaelshobbs Sep 16, 2015

Member

That is not possible as the docker container's port changes every deployment.

Member

michaelshobbs commented Sep 16, 2015

That is not possible as the docker container's port changes every deployment.

@codeHatcher

This comment has been minimized.

Show comment
Hide comment
@codeHatcher

codeHatcher Sep 16, 2015

Can I have nginx route an external port to an internal port? Will that be the closest way to accomplish my goal of having a consistent external port without using VHOST?

codeHatcher commented Sep 16, 2015

Can I have nginx route an external port to an internal port? Will that be the closest way to accomplish my goal of having a consistent external port without using VHOST?

@michaelshobbs

This comment has been minimized.

Show comment
Hide comment
@michaelshobbs

michaelshobbs Sep 16, 2015

Member

Maybe we're crossed up on what using vhost means. In this context, using vhost means using nginx to proxy requests to your container.

In order to do what you want, you would enable vhosts and ship a custom nginx config that listened on your specified port. That config would get updated on every deploy to proxy to the newly running app container.

Member

michaelshobbs commented Sep 16, 2015

Maybe we're crossed up on what using vhost means. In this context, using vhost means using nginx to proxy requests to your container.

In order to do what you want, you would enable vhosts and ship a custom nginx config that listened on your specified port. That config would get updated on every deploy to proxy to the newly running app container.

@codeHatcher

This comment has been minimized.

Show comment
Hide comment
@codeHatcher

codeHatcher Sep 16, 2015

Yeah, i'm not getting it. I'm idling in #dokku as well. I changed my app NO_VHOST = 0, added a VHOST file with my ip address in /home/dokku and added this nginx.conf to my /home/dokku/appname/ folder.

upstream flask-domains { server 127.0.0.1:5000; }
  server {
    listen        [::]:8081;
    listen        8081;
    server_name   external_ip;

    location / {
      proxy_pass http://flask-domains;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection upgrade;
      proxy_set_header Host $host;
      proxy_cache_bypass $http_upgrade;
    }
}

where 5000 seems to be the internal port of myapp.

I also just realized that I have spent enough time trying to solve this I should have either bought a random domain or paid for Heroku :(

Any chance you will be on #dokku in freenode?

codeHatcher commented Sep 16, 2015

Yeah, i'm not getting it. I'm idling in #dokku as well. I changed my app NO_VHOST = 0, added a VHOST file with my ip address in /home/dokku and added this nginx.conf to my /home/dokku/appname/ folder.

upstream flask-domains { server 127.0.0.1:5000; }
  server {
    listen        [::]:8081;
    listen        8081;
    server_name   external_ip;

    location / {
      proxy_pass http://flask-domains;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection upgrade;
      proxy_set_header Host $host;
      proxy_cache_bypass $http_upgrade;
    }
}

where 5000 seems to be the internal port of myapp.

I also just realized that I have spent enough time trying to solve this I should have either bought a random domain or paid for Heroku :(

Any chance you will be on #dokku in freenode?

@codeHatcher

This comment has been minimized.

Show comment
Hide comment
@codeHatcher

codeHatcher Sep 16, 2015

I figured this out. I think my use case is a common use case and should be supported better, even if just with simple documentation.

codeHatcher commented Sep 16, 2015

I figured this out. I think my use case is a common use case and should be supported better, even if just with simple documentation.

@michaelshobbs

This comment has been minimized.

Show comment
Hide comment
@michaelshobbs

michaelshobbs Sep 16, 2015

Member

Please feel free to open a PR to add documentation of your use case.

Member

michaelshobbs commented Sep 16, 2015

Please feel free to open a PR to add documentation of your use case.

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Sep 16, 2015

Member

@codeHatcher Note: we've optimized for the "heroku" case, which is that everyone has a domain name. We can definitely have better docs around this, though honestly it hasn't come up as often as you'd think. Maybe people with this issue are just quieter? shrug

In terms of online support, sorry we didn't have a great answer for you on the #dokku irc channel.

Member

josegonzalez commented Sep 16, 2015

@codeHatcher Note: we've optimized for the "heroku" case, which is that everyone has a domain name. We can definitely have better docs around this, though honestly it hasn't come up as often as you'd think. Maybe people with this issue are just quieter? shrug

In terms of online support, sorry we didn't have a great answer for you on the #dokku irc channel.

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Sep 16, 2015

Member

@michaelshobbs as discussed online, maybe we can:

  • on app creation, we can call something like get_random_ports and output the single random port to $DOKKU_ROOT/$APP/NGINX_PORT (the PORT file is the container's port, where this would be a static port)
  • on app deploy - during the nginx phase - we would check for the NGINX_PORT and use that in a listen call within an nginx conf. This line would be removed if there is no VHOST. These lines would use the contents of NGINX_PORT (and default to port 80 when there is a VHOST. Note that you'll want to take care of the ssl and ssl_terminated templates as well.

That would allow us to continue having zero-downtime deploys while enabling us to have a static port for applications. We could also potentially configure that port using dokku nginx:port <APP> <PORT>.

Makes sense?

Member

josegonzalez commented Sep 16, 2015

@michaelshobbs as discussed online, maybe we can:

  • on app creation, we can call something like get_random_ports and output the single random port to $DOKKU_ROOT/$APP/NGINX_PORT (the PORT file is the container's port, where this would be a static port)
  • on app deploy - during the nginx phase - we would check for the NGINX_PORT and use that in a listen call within an nginx conf. This line would be removed if there is no VHOST. These lines would use the contents of NGINX_PORT (and default to port 80 when there is a VHOST. Note that you'll want to take care of the ssl and ssl_terminated templates as well.

That would allow us to continue having zero-downtime deploys while enabling us to have a static port for applications. We could also potentially configure that port using dokku nginx:port <APP> <PORT>.

Makes sense?

@josegonzalez josegonzalez reopened this Sep 16, 2015

@josegonzalez josegonzalez added the 0.4.0 label Sep 16, 2015

@michaelshobbs

This comment has been minimized.

Show comment
Hide comment
@michaelshobbs

michaelshobbs Sep 17, 2015

Member

So let me make sure we're on the same page wrt what we want to to do here.

  • always configure nginx whether we have a VHOST or not
  • in the case we do have a VHOST, no change in functionality
  • if we do not have a VHOST or NO_VHOST is set
    • we will use nginx to listen on either a user provided port (if available) or find a random port to listen on (i.e. listen <NEW_PORT>; and remove server_name)
    • we will still bind to the instance/hardware interface (i.e. BIND_EXTERNAL=true)

anything else?

Member

michaelshobbs commented Sep 17, 2015

So let me make sure we're on the same page wrt what we want to to do here.

  • always configure nginx whether we have a VHOST or not
  • in the case we do have a VHOST, no change in functionality
  • if we do not have a VHOST or NO_VHOST is set
    • we will use nginx to listen on either a user provided port (if available) or find a random port to listen on (i.e. listen <NEW_PORT>; and remove server_name)
    • we will still bind to the instance/hardware interface (i.e. BIND_EXTERNAL=true)

anything else?

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Sep 17, 2015

Member

That makes sense to me. Does it make sense to provide a NO_NGINX or similar config if people also want to disable this?

Member

josegonzalez commented Sep 17, 2015

That makes sense to me. Does it make sense to provide a NO_NGINX or similar config if people also want to disable this?

@michaelshobbs

This comment has been minimized.

Show comment
Hide comment
@michaelshobbs

michaelshobbs Sep 17, 2015

Member

I'm fine with that.

Member

michaelshobbs commented Sep 17, 2015

I'm fine with that.

@codeHatcher

This comment has been minimized.

Show comment
Hide comment
@codeHatcher

codeHatcher Sep 17, 2015

I think this is a good way to go. I can tell you that I almost just stopped using dokku and just put up a normal server to bypass it until I figured out what was going on. I definitely appreciate you guys taking a look at this issue seriously and not just writing it off. Here are the basic steps of what I did in order to get it working the way I was describing.

  1. add a VHOST file in your ~dokku (not to be confused with ~/dokku which is nothing)
  2. add a fake domain to your HOSTNAME (fakedomain.com) file in your ~dokku directory
  3. deploy your app git remote add dokku dokku@ipaddress:myApp git push dokku master
  4. (optional) restart and relink your mongodb if applicable dokku mongodb:link myApp dokku ps:restart myApp
  5. (optional) set extra env variables on your app dokku config:set myApp NODE_ENV=dev
  6. change the ~dokku/myApp/nginx.conf file to
server {
  listen      [::]:80;
  listen      80;
  server_name myApp.fakedomain.com ;
  access_log  /var/log/nginx/myApp-access.log;
  error_log   /var/log/nginx/myApp-error.log;

  location    / {

    gzip on;
    gzip_min_length  1100;
    gzip_buffers  4 32k;
    gzip_types    text/css text/javascript text/xml text/plain text/x-component application/javascript application/x-javascript application/json application/xml  application/rss+xml font/truetype application/x-font-ttf font/opentype application/vnd.ms-fontobject image/svg+xml;
    gzip_vary on;
    gzip_comp_level  6;

    proxy_pass  http://myApp;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection upgrade;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header X-Forwarded-Port $server_port;
    proxy_set_header X-Request-Start $msec;
  }
  include /home/dokku/myApp/nginx.conf.d/*.conf;
}
upstream myApp {
  server 172.17.0.55:5000;
}

codeHatcher commented Sep 17, 2015

I think this is a good way to go. I can tell you that I almost just stopped using dokku and just put up a normal server to bypass it until I figured out what was going on. I definitely appreciate you guys taking a look at this issue seriously and not just writing it off. Here are the basic steps of what I did in order to get it working the way I was describing.

  1. add a VHOST file in your ~dokku (not to be confused with ~/dokku which is nothing)
  2. add a fake domain to your HOSTNAME (fakedomain.com) file in your ~dokku directory
  3. deploy your app git remote add dokku dokku@ipaddress:myApp git push dokku master
  4. (optional) restart and relink your mongodb if applicable dokku mongodb:link myApp dokku ps:restart myApp
  5. (optional) set extra env variables on your app dokku config:set myApp NODE_ENV=dev
  6. change the ~dokku/myApp/nginx.conf file to
server {
  listen      [::]:80;
  listen      80;
  server_name myApp.fakedomain.com ;
  access_log  /var/log/nginx/myApp-access.log;
  error_log   /var/log/nginx/myApp-error.log;

  location    / {

    gzip on;
    gzip_min_length  1100;
    gzip_buffers  4 32k;
    gzip_types    text/css text/javascript text/xml text/plain text/x-component application/javascript application/x-javascript application/json application/xml  application/rss+xml font/truetype application/x-font-ttf font/opentype application/vnd.ms-fontobject image/svg+xml;
    gzip_vary on;
    gzip_comp_level  6;

    proxy_pass  http://myApp;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection upgrade;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header X-Forwarded-Port $server_port;
    proxy_set_header X-Request-Start $msec;
  }
  include /home/dokku/myApp/nginx.conf.d/*.conf;
}
upstream myApp {
  server 172.17.0.55:5000;
}
@michaelshobbs

This comment has been minimized.

Show comment
Hide comment
@michaelshobbs

michaelshobbs Sep 17, 2015

Member

Thoughts on supporting the ssl config here? What do we set the listener(s) to? (There are two of them currently)

EDIT: or do we just not support SSL on random ports?

Member

michaelshobbs commented Sep 17, 2015

Thoughts on supporting the ssl config here? What do we set the listener(s) to? (There are two of them currently)

EDIT: or do we just not support SSL on random ports?

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Sep 17, 2015

Member

I'd say for a first pass, you don't get ssl. I think it's a fair tradeoff to make, given that I would expect ssl traffic to come from a vhost, not some random ip address.

I'd also be okay with giving an app two ports, one for ssl and one for non-ssl. Maybe find the first two adjacent ports that are free and use those?

Member

josegonzalez commented Sep 17, 2015

I'd say for a first pass, you don't get ssl. I think it's a fair tradeoff to make, given that I would expect ssl traffic to come from a vhost, not some random ip address.

I'd also be okay with giving an app two ports, one for ssl and one for non-ssl. Maybe find the first two adjacent ports that are free and use those?

@codeHatcher

This comment has been minimized.

Show comment
Hide comment
@codeHatcher

codeHatcher Sep 18, 2015

I think people would be ok without ssl but I would definitely use the SSL
option. Only because it's so easy to self sign a cert it's a no brainier to
do
it especially when the internal system is talking to external systems and
there
is medical data being exchanged. You wouldn't need a VHOST because it's a
backend system but it's still publicly facing.

Sent using CloudMagic
[https://cloudmagic.com/k/d/mailapp?ct=pi&cv=6.0.64&pv=8.1.3]

codeHatcher commented Sep 18, 2015

I think people would be ok without ssl but I would definitely use the SSL
option. Only because it's so easy to self sign a cert it's a no brainier to
do
it especially when the internal system is talking to external systems and
there
is medical data being exchanged. You wouldn't need a VHOST because it's a
backend system but it's still publicly facing.

Sent using CloudMagic
[https://cloudmagic.com/k/d/mailapp?ct=pi&cv=6.0.64&pv=8.1.3]

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Sep 18, 2015

Member

I think in this case if it's easy to do ssl, we'll do it, otherwise I think we should err on the side of less complexity. The nginx stuff is already quite hairy :(

Member

josegonzalez commented Sep 18, 2015

I think in this case if it's easy to do ssl, we'll do it, otherwise I think we should err on the side of less complexity. The nginx stuff is already quite hairy :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment