New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need an update mechanism #98

Closed
jacobsalmela opened this Issue Nov 27, 2015 · 47 comments

Comments

Projects
None yet
8 participants
@jacobsalmela
Member

jacobsalmela commented Nov 27, 2015

There is no easy way for existing Pi-hole users to update their gravity.sh script and Pi-hole support files or the Web interface. I would like to get some sort of mechanism installed that can query the repos and install updates.

@jacobsalmela jacobsalmela added this to the Updater milestone Nov 27, 2015

@dschaper

This comment has been minimized.

Member

dschaper commented Nov 27, 2015

That's part of what I'm working on with the .deb package. With a pihole repository, then an install would just be another apt-get and when updates are ready the usual apt-get update/upgrade should pull in the updates. Debian packages are just a little more involved in creating, plus we'd need to make a repository (repo.pihole.net?) to host. Any help from other DD's or people with packaging experience would be great, I'm used to supporting Arch. But it's on the list! lol...

@dschaper

This comment has been minimized.

Member

dschaper commented Nov 27, 2015

That being said, I don't remember if I asked yesterday, so forgive me if I did, but can you cut a 2.1 release and put up a tarball? It would help with a baseline for the package and reduce the patch count.

@jacobsalmela

This comment has been minimized.

Member

jacobsalmela commented Nov 27, 2015

You did ask, but I haven't actually made a tarball before; I mean I know what tools to use, but I'm not quite sure how to package it up. Do I just archive all the necessary files and then use a script to put them in the right place? I'm a bit of n00b in this department.

@dschaper

This comment has been minimized.

Member

dschaper commented Nov 27, 2015

No problem, when you made your 2.0 release it created the zip and the tar.gz for you. You just need to make a new release and tag it. https://help.github.com/articles/creating-releases/

I just tested it, and the tarballs are auto-created. Just set the version to 'v2.1' or something like that, the release number prepended with a v.

@jacobsalmela

This comment has been minimized.

Member

jacobsalmela commented Nov 27, 2015

Done.

@dschaper

This comment has been minimized.

Member

dschaper commented Nov 27, 2015

Awesome, thanks!

@DonLexos

This comment has been minimized.

DonLexos commented Dec 7, 2015

Following this, what would be the best way to update from say a 1 month old installation? Complete reinstall or is there a smart method (since functions have moved, and maybe even helper files have been added)

@dschaper

This comment has been minimized.

Member

dschaper commented Dec 7, 2015

Well, you wouldn't lose anything by doing a fresh install, may hold off a few days though, there are some changes coming. I'm working on a .deb that would eliminate the upgrade issues, but we need to set a good baseline to get a good .deb that isn't going to change locations with each new release.

@DonLexos

This comment has been minimized.

DonLexos commented Dec 7, 2015

No rush, it all works fine, but I've seen many new commits and wondered what the best path forward would be for "an early adopter" (although that is a bit overstating it). There have been many changes recently so I figured I'd give it a shot. Not sure f.e. if the list selection is still in the script or in a configuration file (which would eliminate any issues on an update in the future, provided there is a system like apt-get update).

@dschaper

This comment has been minimized.

Member

dschaper commented Dec 7, 2015

The list that comes as a default with gravity.sh hasn't changed much, so that's why I said you wouldn't lose much doing a fresh install, you'd basically get what you already have. There have been a lot of bug fixes and expansions to support other platforms and a lot of skeleton framework to set up some pretty cool changes coming up, so it's kind of a transition period. Functionally there isn't much different right now, but there will be some changes soon.

Jacob is doing a lot with the web interface that will allow you to customize your blocking in a very user friendly way. Gravity.sh is still the workhorse and all that has changed there is internal structure. There may be a little speed up, but I'm not sure.

So if it's working, hold on a bit and see what's in store...

@DonLexos

This comment has been minimized.

DonLexos commented Jan 16, 2016

HI,

I jumped and did a fresh installation earlier today (with same IP setup etc) and all works fine, however it it a bit unclear to me. Assume there is a new version, can I simply update gravity.sh or are the more files that need to be changed? I am not very familiar with the github system but am able to SSH into the pi and change it on that level (and restart services if required).

Thanks!

@jacobsalmela

This comment has been minimized.

Member

jacobsalmela commented Jan 16, 2016

It's possible new files could be added or modified (see #174, which was just merged). So in order to get these files you can basically reinstall using curl -L install.pi-hole.net | bash but it's not a true "update" as there isn't logic for checking what files exist or how they compare to the new ones. This would basically just re-install everything.

@PromoFaux

This comment has been minimized.

Member

PromoFaux commented Jan 20, 2016

I have a cunning plan...

How about adding a version number txt file to the repo?

when gravity.sh runs (which it does every night) It checks the currently downloaded version file against the one in the repo. If the one in the repo is newer then run install.sh (which can then be expanded to run silently if pi-hole is already installed)

@jacobsalmela

This comment has been minimized.

Member

jacobsalmela commented Jan 21, 2016

But doesn't it seem a bit excessive to try to apt-get update and to try and install everything again?

I was hoping for something more along the lines of curl -L update.pi-hole.net | bash. I have been thinking about how updateDashboard.sh use git to keep things synchronized. Something like that would be cool. Then it could just be a cron job.

@PromoFaux

This comment has been minimized.

Member

PromoFaux commented Jan 21, 2016

Could updatedashboard be integrated into the existing install.sh I wonder?

apt-get update could easily be skipped if install.sh detects it's an existing installation. I don't mind tackling it tomorrow, if you like? I know my way around the install script now!

Then it could just be a cron job.

No need if you defer the work to gravity.sh, as that is already on a cron job...

@jacobsalmela

This comment has been minimized.

Member

jacobsalmela commented Jan 21, 2016

But the latest Web interface is always downloaded with the installer; updateDashboard.sh just keeps it up to date with git. I'm not sure I understand what you mean by integrating it into the installer.

@PromoFaux

This comment has been minimized.

Member

PromoFaux commented Jan 21, 2016

Basically, the part in the installer that downloads master.zip can be replaced with the same logic from updateDashboard.sh

if ! is_repo; then
    make_repo
fi
update_repo

If you look over the changes I've suggested for the install script, you'll see I've been working on making it a bit more intelligent. (which will essentially turn it into a dual purpose installer/updater)

@jacobsalmela

This comment has been minimized.

Member

jacobsalmela commented Jan 21, 2016

So would you use git or some other method to get the updated files from master?

@PromoFaux

This comment has been minimized.

Member

PromoFaux commented Jan 21, 2016

For the Admin Dashboard? git. Same as updateDashboard.sh

For the rest of the files, no reason why they too could not be checked out to a local repo and distributed to where they need to be, rather than using curl as it is right now.

@jacobsalmela

This comment has been minimized.

Member

jacobsalmela commented Jan 21, 2016

So wouldn't we need to restructure the repo to reflect the file structure of the OS so the files end up in the right place?

@dschaper

This comment has been minimized.

Member

dschaper commented Jan 21, 2016

It would add git to the dependencies, but I don't think it's all that big of a package. There's a nice StackExchange post with a function on evaluating the status of the local repo and deciding on the need to fetch/pull or stay.

@PromoFaux

This comment has been minimized.

Member

PromoFaux commented Jan 21, 2016

@jacobsalmela - I shouldn't have thought so, the script can handle copying file out to their relevant directories!

@dschaper - Apparently It's already a dependency

@dschaper

This comment has been minimized.

Member

dschaper commented Jan 21, 2016

@PromoFaux Well, I learn something new everyday! Lol...

@jacobsalmela

This comment has been minimized.

Member

jacobsalmela commented Jan 21, 2016

Yeah, I added that back when I merged the updateDashboard.sh script.

@PromoFaux PromoFaux self-assigned this Jan 23, 2016

@kmr2319

This comment has been minimized.

kmr2319 commented Feb 2, 2016

Fantastic! I had updated to 2.5 just after posting my comment, saw that the dnsmasq.conf was overwritten and the new 01-pihole.conf was created. Dropped my few config items into their own conf file in /etc/dnsmasq.d. Saw that 2.5.1 was out there and updated again and my changes are still available!!! Awesome.

However, it would be nice if there was a separate upgrade script, so existing users don't have to go through the install script and be prompted for any input. Just grab the latest code and recycle....

Thanks again!

@DonLexos

This comment has been minimized.

DonLexos commented Feb 2, 2016

Agree with @kmr2319 on the update part. Ofcourse the script checks for all kinds of files in existance and makes sure settings are resolved, but dedicated IP check, ip4/6 selection and more can be skipped in general on an upgrade imho aswell. If you want to change it you could always just run the installer instead of the upgrade. :-)

@PromoFaux

This comment has been minimized.

Member

PromoFaux commented Feb 2, 2016

The general idea was that if they wanted to change things, they could.. but it's a good point actually, just an "update.sh" that ships with it, so you don't even need to curl it, just run it! I'll get on that later!

@DonLexos

This comment has been minimized.

DonLexos commented Feb 2, 2016

if that would exist it could even be CRON'd to auto update weekly, provided it only gets stuff from master

@PromoFaux

This comment has been minimized.

Member

PromoFaux commented Feb 2, 2016

Autoupdate COULD be set up, but not by default. We could provide an option to run it automatically, or provide instructions on how it can be run through cron, but if we start updating automatically out of the box, we're gonna have unhappy users somewhere along the line...

@DonLexos

This comment has been minimized.

DonLexos commented Feb 2, 2016

Yes I am aware of that, not by default, but same as earlier to purge the pihole.log daily (which is now default I believe regardless if any statistics are saved) to keep the system quick and lean.

@jacobsalmela

This comment has been minimized.

Member

jacobsalmela commented Feb 2, 2016

I have often thought of a

curl -L update.pi-hole.net | bash

I wouldn't want to do this until we do a revamp of some code, which would prevent us from repeating code or missing out on any hardcoded variables.

As as example:

  1. /etc/phole/pihole.vars would be a bash script that would export variables.
  2. /usr/local/bin/updatePihole would source /etc/phole/pihole.vars to get all the folder name, IP's, etc.

Then /usr/local/bin/updatePihole could just be added as a cron job without the need to be piping stuff to bash over the Internet, which I know doesn't sit well with everyone.

Also, we could make a separate update script now, but we would probably end up using a lot of the same code, which doesn't make sense to have it in two places.

The git method helps a lot right now, and as @dschaper said, the .deb would be ideal and make it more portable for updating via apt-get. So there are a couple of different approaches that will need some thought.

@dschaper

This comment has been minimized.

Member

dschaper commented Feb 3, 2016

Thanks ;)

@jacobsalmela

This comment has been minimized.

Member

jacobsalmela commented Feb 3, 2016

Oops. I though I could assign more than one person...I was trying to assign myself and @PromoFaux

@dschaper

This comment has been minimized.

Member

dschaper commented Feb 3, 2016

You should be able, it should just be a toggle...

@dschaper dschaper assigned jacobsalmela and unassigned PromoFaux Feb 3, 2016

@dschaper

This comment has been minimized.

Member

dschaper commented Feb 3, 2016

Huh, I guess you cant? That's odd?

@jacobsalmela

This comment has been minimized.

Member

jacobsalmela commented Feb 3, 2016

Yeah, I thought it would be like Labels where you can select multiple ones...

@jacobsalmela jacobsalmela referenced this issue Mar 18, 2016

Closed

Update mechanism #393

0 of 2 tasks complete
@Mcat12

This comment has been minimized.

Member

Mcat12 commented Apr 12, 2016

The hardest part with update.pi-hole.net is that we would have to handle every version case, especially if we change locations of files or add new ones. @PromoFaux suggested bundling an update script with pi-hole, which would do some update prep and then download a new updater that would finish the job. I'm thinking that the first one could get it to a stable state (all files are either deleted or in a specific location) and then the second just does a headless install over it.

@nate-ubiquisoft

This comment has been minimized.

Contributor

nate-ubiquisoft commented Apr 12, 2016

I think we also need a purge script to remove any Pi-Hole specific files, even from old installations. Something using mlocate would work well but I think it would require a lot of user prompts (one for each deletion) to ensure that we dont "purge" anything that we shouldnt

@Mcat12

This comment has been minimized.

Member

Mcat12 commented Apr 12, 2016

As long as we keep track of what files we create, we shouldn't have a problem of not deleting our files.

@diginc

This comment has been minimized.

Member

diginc commented Jun 18, 2016

Hi, I would endorse the Debian approach since #77 solves lots of problems being homegrown right now.

https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html

removing orphaned files new debian versions don't have

6.6 Details of unpack phase of installation or upgrade
6. Any files which were in the old version of the package but not in the new are removed.
11. The backup files made during installation, above, are deleted.

Uninstall + cleanup

6.8 Details of removal and/or configuration purging
7. The package's file list is removed.

I can't find where it says it prompts to replace/preserve/merge user modified files in the package file list but I know I've experienced debians stopping mid install to ask that to me too.

@dschaper dschaper removed the in progress label Aug 21, 2016

@dschaper dschaper removed this from the Updater milestone Aug 21, 2016

@PromoFaux

This comment has been minimized.

Member

PromoFaux commented Sep 2, 2016

No longer needed!

@PromoFaux PromoFaux closed this Sep 2, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment