Skip to content

Using the RSYNC skin as a backup solution

Glenn edited this page Feb 14, 2017 · 6 revisions

The intent of this page is to show 'HowTo' use weewx's RSYNC report skin to back up the database. It can obviously be extended via further skin 'renaming' to do other file transfers, but it would be best to keep it simple and not overload weewx (see the bottom of the page if that hint interests you).

Rsyncing your sqlite database falls under simple, that action can be done often without overloading weewx.

Table of Contents

Riding along with StdReport

The report cycle in weewx only starts once all writes to the database have finished, this provides some assurance that any copy of the database taken at that time will be a good one.

This method relies on the ability to use multiple instances of a skin by simply giving them their own skin [[section]] name in the weewx.conf file, or skin.conf file

The simplest - using weewx.conf

[[RSYNCSQL]]
  skin = Rsync
we then redefine HTML_ROOT (this only affects this skin, and only when it's run) to point to the local database
[[RSYNCSQL]]
        skin = Rsync
        HTML_ROOT = /var/lib/weewx
then add the other requirements to get it to start and run
[[RSYNCSQL]]
        skin = Rsync
        HTML_ROOT = /var/lib/weewx
        server = 192.168.0.100
        user = graybeard
        path = /home/graybeard/rsynsql-weewx1
Finally, add a report_time stanza of your choosing (or not if you want a rolling copy) It would pay to check your archive_interval in weewx.conf, and make this interval longer than that one.

For me, besides catching the database between archive intervals, this report_timing config is the real sweetener.

[[RSYNCSQL]]
        skin = Rsync
        HTML_ROOT = /var/lib/weewx
        # If it's local, an ip address is fine for the server
        server = 192.168.0.100
        # account that has password less access to the server
        user = graybeard
        # where to put it all, don't nest too deeply unless you create the tree first
        path = /home/graybeard/rsynscql-weewx
        # every 20 minutes, run this report
        report_timing = '*/20 * * * *'
        # add the default
        delete = 0

An alternative - using skin.conf

Alternatively, transfer everything ( except the HTML_ROOT variable, ) to a skin.conf file and that way you can make changes to your setup while weewx is running. The weewx.conf file takes precedence over the skin.conf files, so in weewx.conf comment out the values you don't want to be permanent (as in require a restart of weewx to action) and place them into the skin.conf file.

Putting it another way: Any argument that is unique to skin.conf will be read and actioned each report cycle. If it also exists in weewx.conf that value takes over and can only be read (actioned) by restarting weewx. And HTML_ROOT is exclusive to the weewx.conf file

[[RSYNCSQL]]
        skin = Rsyncsql
        HTML_ROOT = /var/lib/weewx
        # If it's local, an ip address is fine for the server
        #server = 192.168.0.100
        # account that has password less access to the server
        #user = graybeard
        # where to put it all, don't nest too deeply unless you create the tree first
        #path = /home/graybeard/rsyncsql-weewx1
        # every 20 minutes, run this report
        # report_timing = '*/20 * * * *'
        # add the default
        delete = 0
Note the change to the first line skin = Rsync, we now point to skin = Rsyncsql.

So we need to create it...

cd skins
cp -r Rsync Rsyncsql/
and then edit it, skins/Rsyncsql/skin.conf to add the following after the comments at the top...
        HTML_ROOT = /var/lib/weewx
        # If it's local, an ip address is fine for the server
        server = 192.168.0.100
        # account that has password less access to the server
        user = graybeard
        # where to put it all, don't nest too deeply unless you create the tree first
        path = /home/graybeard/rsyncsql-weewx-skin
        # every 20 minutes, run this report
        report_timing = '*/20 * * * *'
        # add the default
        delete = 0

Create the password-less access

Finally create the password-less access for the remote user, from the local root (the account weewx is running as and for most users that will be root)

When asked for the passphrase, just hit enter. If you're worried about security then investigate running weewx with a less privileged user - see Run as a non root user

Do this (use your own account!)...

sudo -i
ssh-keygen
ssh-copy-id -i ~/.ssh/id_rsa.pub graybeard@192.168.0.100
ssh 'graybeard@192.168.0.100'

and you'll see something like this (names have been changed to protect the innocent)...

READ WRITE : root@raspberrypi:~# ssh-keygen 
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
e6:54:4e:43:b1:72:fb:8d:5f:3f:8c:d6:b4:2a:8b:fa root@raspberrypi
The key's randomart image is:
+---[RSA 2048]----+
|          o.     |
|         . .     |
|        . *      |
|         * o     |
|        S o      |
|       +   . o . |
|        .   o * o|
|          .. + *.|
|       .oE .+.o o|
+-----------------+
READ WRITE : root@raspberrypi:~#  ssh-copy-id -i ~/.ssh/id_rsa.pub graybeard@192.168.0.100
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
graybeard@192.168.0.100's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'graybeard@192.168.0.100'"
and check to make sure that only the key(s) you wanted were added.

READ WRITE : root@raspberrypi:~# ssh 'graybeard@192.168.0.100'

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
You have new mail.
Last login: Tue Feb 14 08:56:07 2017 from 192.168.0.215
xterm graybeard@whitebeard:~
09:31 AM $ logout
Connection to 192.168.0.100 closed.

Try it out

Restart weewx and watch your logs

/etc/init.d/weewx restart && tail -f /var/log/syslog

# (use Ctrl-C to exit tail when you're done and you'll drop back to your shell, weewx will still be running!)

Expanding on the concept

Visit the weewx docs, create and populate a new [[section]], set a sensible report_timing period then keep an eye on your logs. What we are particularly wanting to note, is the rsync timings that are displayed there. Decisions are made on the basis of those - you want time available for your standard reports to run.

Clone this wiki locally
You can’t perform that action at this time.