New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dokku should have a command to keep its base system up to date #1089

Closed
egerlach opened this Issue Apr 3, 2015 · 28 comments

Comments

Projects
None yet
7 participants
@egerlach

egerlach commented Apr 3, 2015

Currently, it's a bit of a pain to keep Dokku's base images up to date (so that your apps have all the security updates). I've written a script that I'm using as a way to keep my installation up to date.

It would be cool if there was some sort of systemupdate command that did some/all of this.

If progrium/buildstep#146 and progrium/cedarish#19 are resolved, this script will simplify significantly.

@michaelshobbs

This comment has been minimized.

Show comment
Hide comment
@michaelshobbs

michaelshobbs Apr 5, 2015

Member

👍 on the concept. @josegonzalez what are your thoughts here given the deb package situation?

Member

michaelshobbs commented Apr 5, 2015

👍 on the concept. @josegonzalez what are your thoughts here given the deb package situation?

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Apr 18, 2015

Member

Installation is now performed - by default - from debian packages. You can simply apt-get install dokku to upgrade. Source-based installations will continue to be a manual process. Closed by #1058.

Member

josegonzalez commented Apr 18, 2015

Installation is now performed - by default - from debian packages. You can simply apt-get install dokku to upgrade. Source-based installations will continue to be a manual process. Closed by #1058.

@egerlach

This comment has been minimized.

Show comment
Hide comment
@egerlach

egerlach Apr 18, 2015

@josegonzalez I'm not sure that this addresses the reported issue. This isn't about keeping dokku up to date, it's about keeping the stack that underlies the apps up to date (i.e. the Ubuntu debootstrap container image, etc)

egerlach commented Apr 18, 2015

@josegonzalez I'm not sure that this addresses the reported issue. This isn't about keeping dokku up to date, it's about keeping the stack that underlies the apps up to date (i.e. the Ubuntu debootstrap container image, etc)

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Apr 19, 2015

Member

Gotcha. Part of this is taken care of by the dokku install - plugin-install is re-run every time you upgrade. The deb package for buildstep should pull latest ubuntu stuff, so I'm re-opening so we can fix that.

We can probably make this something like dokku upgrade which updates all deb packages and plugins? I'd def be okay with a PR for that :D

Member

josegonzalez commented Apr 19, 2015

Gotcha. Part of this is taken care of by the dokku install - plugin-install is re-run every time you upgrade. The deb package for buildstep should pull latest ubuntu stuff, so I'm re-opening so we can fix that.

We can probably make this something like dokku upgrade which updates all deb packages and plugins? I'd def be okay with a PR for that :D

@bchr02

This comment has been minimized.

Show comment
Hide comment
@bchr02

bchr02 commented Oct 26, 2015

+1

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Oct 26, 2015

Member

@bchr02 thanks for the +1, do you have a potential pr that will implement such a feature?

Member

josegonzalez commented Oct 26, 2015

@bchr02 thanks for the +1, do you have a potential pr that will implement such a feature?

@michaelshobbs

This comment has been minimized.

Show comment
Hide comment
@michaelshobbs

michaelshobbs Oct 26, 2015

Member

What happened to using the deb package method of upgrade?

Member

michaelshobbs commented Oct 26, 2015

What happened to using the deb package method of upgrade?

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Oct 26, 2015

Member

I think he means rebuilding the herokuish container?

Member

josegonzalez commented Oct 26, 2015

I think he means rebuilding the herokuish container?

@bchr02

This comment has been minimized.

Show comment
Hide comment
@bchr02

bchr02 Oct 26, 2015

@josegonzalez Nope sorry. Dokku is still very new to me. I find the concept intriguing and full of potential.

Over the weekend I started testing some Node.js apps on Cloud Foundry within IBM BlueMix PaaS infrastructure. I found it really convenient not having to install, configure, and maintain anything (OS, NGINX, ) except for my Node.js app themselves. The only problem was I didn't care for the monthly fees 😄 especially because I already have my own servers.

I first learned about Dokku reading a Digital Ocean tutorial... I then started reading the the Dokku documentation but I didn't find anything about how to maintain the whole thing. It's great that Dokku abstracts all the component of setting up and configuring the underlying system which runs/contains the apps that one is looking to deploy but I feel like that promotes a false sense of security. This solution will appease most to those that don't know how to maintain a system especially one that they don't understand. Which is why I feel it is important that Dokku should document how to best keep everything up-to-date and secure and perhaps provide a mechanism for doing so. Things like:

There are tons of articles online on how easy it is to use Dokku but not one of them mentions how to keep it secure. It's awesome that I'm not alone in this concern. Thanks @egerlach for being first to bring this issue to light because I feel like it's an important one.

bchr02 commented Oct 26, 2015

@josegonzalez Nope sorry. Dokku is still very new to me. I find the concept intriguing and full of potential.

Over the weekend I started testing some Node.js apps on Cloud Foundry within IBM BlueMix PaaS infrastructure. I found it really convenient not having to install, configure, and maintain anything (OS, NGINX, ) except for my Node.js app themselves. The only problem was I didn't care for the monthly fees 😄 especially because I already have my own servers.

I first learned about Dokku reading a Digital Ocean tutorial... I then started reading the the Dokku documentation but I didn't find anything about how to maintain the whole thing. It's great that Dokku abstracts all the component of setting up and configuring the underlying system which runs/contains the apps that one is looking to deploy but I feel like that promotes a false sense of security. This solution will appease most to those that don't know how to maintain a system especially one that they don't understand. Which is why I feel it is important that Dokku should document how to best keep everything up-to-date and secure and perhaps provide a mechanism for doing so. Things like:

There are tons of articles online on how easy it is to use Dokku but not one of them mentions how to keep it secure. It's awesome that I'm not alone in this concern. Thanks @egerlach for being first to bring this issue to light because I feel like it's an important one.

@u2mejc

This comment has been minimized.

Show comment
Hide comment
@u2mejc

u2mejc Oct 26, 2015

Member

It's my impression that the dokku environment is always rebuildable? Simply having apt-get update && apt-get upgrade will update your docker version, nginx (proxy) and dokku client which would pull down new target image references for docker. @josegonzalez Does the herokish image run apt-get when it's ran? Pushing an empty commit that documents your upgrading the env seems like it would satisfy this need. Also keeps constancy in the project of features supported directly vs indirectly (example, uninstalling is not supported if you don't use apt-get).

Member

u2mejc commented Oct 26, 2015

It's my impression that the dokku environment is always rebuildable? Simply having apt-get update && apt-get upgrade will update your docker version, nginx (proxy) and dokku client which would pull down new target image references for docker. @josegonzalez Does the herokish image run apt-get when it's ran? Pushing an empty commit that documents your upgrading the env seems like it would satisfy this need. Also keeps constancy in the project of features supported directly vs indirectly (example, uninstalling is not supported if you don't use apt-get).

@egerlach

This comment has been minimized.

Show comment
Hide comment
@egerlach

egerlach Oct 28, 2015

@u2mejc You are correct, when herokuish updates, it does update the base image. However, it does not appear that the herokuish package is updated every time the herokuish image is updated (which is updated everytime the base image is updated, now).

Plus, it's not any work here that needs to be captured in the image, but any security updates to underlying packages that the image installs when it's built.

egerlach commented Oct 28, 2015

@u2mejc You are correct, when herokuish updates, it does update the base image. However, it does not appear that the herokuish package is updated every time the herokuish image is updated (which is updated everytime the base image is updated, now).

Plus, it's not any work here that needs to be captured in the image, but any security updates to underlying packages that the image installs when it's built.

@michaelshobbs

This comment has been minimized.

Show comment
Hide comment
@michaelshobbs

michaelshobbs Oct 28, 2015

Member

So what is this feature going to look like?

apt-get update
apt-get install dokku
docker pull gliderlabs/herokuish:latest

??

Member

michaelshobbs commented Oct 28, 2015

So what is this feature going to look like?

apt-get update
apt-get install dokku
docker pull gliderlabs/herokuish:latest

??

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Oct 28, 2015

Member

@egerlach We definitely release a new version of the herokuish package every time we release a new herokuish image. Since those images are built "from source", then they have whatever Ubuntu 14.04 deems is the most secure/up to date for it's system.

@bchr02 The doc would go something like:

  • install unattended security updates
  • call apt-get update && apt-get upgrade on a regular basis.

Not sure what else you are looking for. Anything else - configuring firewalls, locking down remote ssh using fail2ban, etc. - is pretty far out there in terms of what dokku aims to do imo.

Member

josegonzalez commented Oct 28, 2015

@egerlach We definitely release a new version of the herokuish package every time we release a new herokuish image. Since those images are built "from source", then they have whatever Ubuntu 14.04 deems is the most secure/up to date for it's system.

@bchr02 The doc would go something like:

  • install unattended security updates
  • call apt-get update && apt-get upgrade on a regular basis.

Not sure what else you are looking for. Anything else - configuring firewalls, locking down remote ssh using fail2ban, etc. - is pretty far out there in terms of what dokku aims to do imo.

@bchr02

This comment has been minimized.

Show comment
Hide comment
@bchr02

bchr02 Oct 28, 2015

@josegonzalez thanks for the reply.

Okay, so what you are saying is:

  1. Install the unattended-upgrades package: sudo apt-get install unattended-upgrades
  2. Set it with: sudo dpkg-reconfigure --priority=low unattended-upgrades

Or manually run them by periodically calling apt-get update && apt-get upgrade

And this should keep everything (Ubuntu, Dokku, etc.) updated. Is my understanding correct? Thank you.

bchr02 commented Oct 28, 2015

@josegonzalez thanks for the reply.

Okay, so what you are saying is:

  1. Install the unattended-upgrades package: sudo apt-get install unattended-upgrades
  2. Set it with: sudo dpkg-reconfigure --priority=low unattended-upgrades

Or manually run them by periodically calling apt-get update && apt-get upgrade

And this should keep everything (Ubuntu, Dokku, etc.) updated. Is my understanding correct? Thank you.

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Oct 28, 2015

Member

The unattended-upgrades are only for security upgrades, not every package. Hence the other apt-get calls (which will upgrade dokku/herokuish/etc., as we don't make "security" releases).

https://help.ubuntu.com/community/AutomaticSecurityUpdates

Member

josegonzalez commented Oct 28, 2015

The unattended-upgrades are only for security upgrades, not every package. Hence the other apt-get calls (which will upgrade dokku/herokuish/etc., as we don't make "security" releases).

https://help.ubuntu.com/community/AutomaticSecurityUpdates

@bchr02

This comment has been minimized.

Show comment
Hide comment
@bchr02

bchr02 Oct 28, 2015

@josegonzalez thank you. I understand now.
by the way, I found an interesting article on automating the apt-get commands here:
http://www.techrepublic.com/article/automatically-update-your-ubuntu-system-with-cron-apt/

It's too bad we couldn't default new installs to automate both these tasks. It seems like it would make things more secure since most people are likely running Dokku headless.

bchr02 commented Oct 28, 2015

@josegonzalez thank you. I understand now.
by the way, I found an interesting article on automating the apt-get commands here:
http://www.techrepublic.com/article/automatically-update-your-ubuntu-system-with-cron-apt/

It's too bad we couldn't default new installs to automate both these tasks. It seems like it would make things more secure since most people are likely running Dokku headless.

@egerlach

This comment has been minimized.

Show comment
Hide comment
@egerlach

egerlach Oct 29, 2015

@josegonzalez Both herokuish and heroku/cedar:14 need to be updated on a regular basis if apps running on dokku are to remain secure (though ubuntu-debootstrap is exempt because heroku/cedar:14 runs apt-get update && apt-get upgrade). Unfortunately, it appears that heroku/cedar:14 is not updated when there's an Ubuntu Security Notice, rendering it, herokuish, and all Dokku-deployed apps potentially insecure as well.

I've written a blog post about the issue if you want more detail.

For @bchr02, updating the herokuish package in your host OS is insufficient, because there's a container in the chain of containers that doesn't get updated regularly.

egerlach commented Oct 29, 2015

@josegonzalez Both herokuish and heroku/cedar:14 need to be updated on a regular basis if apps running on dokku are to remain secure (though ubuntu-debootstrap is exempt because heroku/cedar:14 runs apt-get update && apt-get upgrade). Unfortunately, it appears that heroku/cedar:14 is not updated when there's an Ubuntu Security Notice, rendering it, herokuish, and all Dokku-deployed apps potentially insecure as well.

I've written a blog post about the issue if you want more detail.

For @bchr02, updating the herokuish package in your host OS is insufficient, because there's a container in the chain of containers that doesn't get updated regularly.

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Oct 29, 2015

Member

@egerlach so you're saying a full-rebuild would be good enough?

Member

josegonzalez commented Oct 29, 2015

@egerlach so you're saying a full-rebuild would be good enough?

@egerlach

This comment has been minimized.

Show comment
Hide comment
@egerlach

egerlach Oct 29, 2015

@josegonzalez You need to rebuild the heroku/cedar:14 image, and everything on top of it, on a regular basis, so that packages installed at that layer are up to date. Or do an update at the gliderlabs/herokuish level, or at the app container level, but that seems a bit insane.

As I'm thinking about the right way to do this, I'm thinking Dokku should maintain a local herokuish container derived from the one from Docker Hub that has had apt-get update && apt-get upgrade run on it, and then have a detector that examines that container for out-of-date packages with some frequency and then rebuilds that local container and all apps whenever an out-of-date package is detected.

The other option is to have some sort of process that is centrally run that looks for those out of date packages and builds a new global herokuish package whenever there's an update, and the local machines look for that update and rebuild when they find one.

egerlach commented Oct 29, 2015

@josegonzalez You need to rebuild the heroku/cedar:14 image, and everything on top of it, on a regular basis, so that packages installed at that layer are up to date. Or do an update at the gliderlabs/herokuish level, or at the app container level, but that seems a bit insane.

As I'm thinking about the right way to do this, I'm thinking Dokku should maintain a local herokuish container derived from the one from Docker Hub that has had apt-get update && apt-get upgrade run on it, and then have a detector that examines that container for out-of-date packages with some frequency and then rebuilds that local container and all apps whenever an out-of-date package is detected.

The other option is to have some sort of process that is centrally run that looks for those out of date packages and builds a new global herokuish package whenever there's an update, and the local machines look for that update and rebuild when they find one.

@vincentfretin

This comment has been minimized.

Show comment
Hide comment
@vincentfretin

vincentfretin Oct 31, 2015

Contributor

Maybe a bit off topic, but I wanted to share this.
https://github.com/mafr/docker-update-check is a nice script to have a reporting of security updates of ubuntu containers. Maybe you have non Dokku containers on your host and you need to see if there is security updates to do.
I have a cron that executes the following script:

cd /home/vincentfretin/docker-update-check                                      
./update-check run                                                              
./update-check update                                                           
for container in `docker ps -q`; do                                             
    ./update-check check $container;                                            
    docker inspect --format '{{.Name}}' $container                              
done

that produces the following output:

--- caa59d090ffa ---
0 packages can be updated.
0 updates are security updates.
/myapp.dokku.me
--- 43f8a14035fd ---
13 packages can be updated.
5 updates are security updates.
/elk_kibana_1
Contributor

vincentfretin commented Oct 31, 2015

Maybe a bit off topic, but I wanted to share this.
https://github.com/mafr/docker-update-check is a nice script to have a reporting of security updates of ubuntu containers. Maybe you have non Dokku containers on your host and you need to see if there is security updates to do.
I have a cron that executes the following script:

cd /home/vincentfretin/docker-update-check                                      
./update-check run                                                              
./update-check update                                                           
for container in `docker ps -q`; do                                             
    ./update-check check $container;                                            
    docker inspect --format '{{.Name}}' $container                              
done

that produces the following output:

--- caa59d090ffa ---
0 packages can be updated.
0 updates are security updates.
/myapp.dokku.me
--- 43f8a14035fd ---
13 packages can be updated.
5 updates are security updates.
/elk_kibana_1
@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Dec 7, 2015

Member

@egerlach If we copied the script in your blog post, what would we call the command? Ideally we have a Security page in the dokku docs and push people towards updating their stuff.

@vincentfretin how does that script compare with what's in @egerlach's post?

Member

josegonzalez commented Dec 7, 2015

@egerlach If we copied the script in your blog post, what would we call the command? Ideally we have a Security page in the dokku docs and push people towards updating their stuff.

@vincentfretin how does that script compare with what's in @egerlach's post?

@vincentfretin

This comment has been minimized.

Show comment
Hide comment
@vincentfretin

vincentfretin Dec 7, 2015

Contributor

@josegonzalez the script doesn't update anything, it just show you if there are updates.

Contributor

vincentfretin commented Dec 7, 2015

@josegonzalez the script doesn't update anything, it just show you if there are updates.

@ElRoberto538

This comment has been minimized.

Show comment
Hide comment
@ElRoberto538

ElRoberto538 Apr 25, 2016

Hi all, any update on this? So far I am using the info from the blog post by @egerlach, and checking the package versions using docker-update-check linked by @vincentfretin. However it is a very overkill way to maintain things...

ElRoberto538 commented Apr 25, 2016

Hi all, any update on this? So far I am using the info from the blog post by @egerlach, and checking the package versions using docker-update-check linked by @vincentfretin. However it is a very overkill way to maintain things...

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Apr 26, 2016

Member

@ElRoberto538 would you be willing to contribute a patch for this functionality?

Member

josegonzalez commented Apr 26, 2016

@ElRoberto538 would you be willing to contribute a patch for this functionality?

@ElRoberto538

This comment has been minimized.

Show comment
Hide comment
@ElRoberto538

ElRoberto538 May 2, 2016

@josegonzalez I'm not sure the solution I'm using is particularly ideal - it kind of hammers the server during the rebuild. I could look into making a patch for this, it won't be anytime soon though as I don't have the time available for a while.

ElRoberto538 commented May 2, 2016

@josegonzalez I'm not sure the solution I'm using is particularly ideal - it kind of hammers the server during the rebuild. I could look into making a patch for this, it won't be anytime soon though as I don't have the time available for a while.

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez May 2, 2016

Member

it won't be anytime soon though as I don't have the time available for a while.

You and me both ;)

Member

josegonzalez commented May 2, 2016

it won't be anytime soon though as I don't have the time available for a while.

You and me both ;)

@josegonzalez josegonzalez added this to the v1.0.0 milestone Jan 26, 2017

josegonzalez added a commit to gliderlabs/herokuish that referenced this issue Jan 14, 2018

@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Jan 14, 2018

Member

Okay so tasks here:

  • Ensure we build against an updated base os when releasing herokuish
  • Setup monthly releases of herokuish
  • Write docs on updating dokku dependencies
    • Link to upstream-related update documentation versus our own for updating stuff like docker or ubuntu...
  • Have a command that will output all the commands that need to run in order for a user to update their system. Potentially have this as an extra binary/package users can run that doesn't depend upon any of the dokku stuff.
Member

josegonzalez commented Jan 14, 2018

Okay so tasks here:

  • Ensure we build against an updated base os when releasing herokuish
  • Setup monthly releases of herokuish
  • Write docs on updating dokku dependencies
    • Link to upstream-related update documentation versus our own for updating stuff like docker or ubuntu...
  • Have a command that will output all the commands that need to run in order for a user to update their system. Potentially have this as an extra binary/package users can run that doesn't depend upon any of the dokku stuff.

josegonzalez added a commit that referenced this issue Jan 18, 2018

feat: add dokku-update binary
This binary will be packaged separately from dokku and can be used to keep the system up to date.

Refs #1089
@josegonzalez

This comment has been minimized.

Show comment
Hide comment
@josegonzalez

josegonzalez Feb 19, 2018

Member

Please see the main comment in #3039 for more information as to why I think this can be marked as closed. Thanks everyone for your input, and hopefully this first pass brings us closer to the path everyone is seeking :)

Member

josegonzalez commented Feb 19, 2018

Please see the main comment in #3039 for more information as to why I think this can be marked as closed. Thanks everyone for your input, and hopefully this first pass brings us closer to the path everyone is seeking :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment