Skip to content

Commit

Permalink
Updated documentation
Browse files Browse the repository at this point in the history
Signed-off-by: Jim Easterbrook <jim@jim-easterbrook.me.uk>
  • Loading branch information
jim-easterbrook committed May 1, 2018
1 parent 68486c4 commit 8488377
Show file tree
Hide file tree
Showing 3 changed files with 44 additions and 50 deletions.
39 changes: 21 additions & 18 deletions src/doc/guides/hourlylogging.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.. pywws - Python software for USB Wireless Weather Stations
http://github.com/jim-easterbrook/pywws
Copyright (C) 2008-14 Jim Easterbrook jim@jim-easterbrook.me.uk
Copyright (C) 2008-18 pywws contributors
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
Expand All @@ -23,24 +23,25 @@ Introduction
------------

There are two quite different modes of operation with pywws.
Traditionally :py:mod:`pywws.Hourly` would be run at regular intervals (usually an hour) from cron.
Traditionally :py:mod:`pywws.hourly` would be run at regular intervals (usually an hour) from cron.
This is suitable for fairly static websites, but more frequent updates can be useful for sites such as Weather Underground (http://www.wunderground.com/).
The newer :py:mod:`pywws.LiveLog` program runs continuously and can upload data every 48 seconds.
The newer :py:mod:`pywws.livelog` program runs continuously and can upload data every 48 seconds.

Note that although this document (and the program name) refers to 'hourly' logging, you can run :py:mod:`pywws.Hourly` as often or as infrequently as you like, but don't try to run it more often than double your logging interval.
For example, if your logging interval is 10 minutes, don't run :py:mod:`pywws.Hourly` more often than every 20 minutes.
Note that although this document (and the program name) refers to 'hourly' logging, you can run :py:mod:`pywws.hourly` as often or as infrequently as you like, but don't try to run it more often than double your logging interval.
For example, if your logging interval is 10 minutes, don't run :py:mod:`pywws.hourly` more often than every 20 minutes.

Getting started
---------------

First of all, you need to install pywws and make sure it can get data from your weather station.
See :doc:`getstarted` for details.

Try running :py:mod:`pywws.Hourly` from the command line, with a high level of verbosity so you can see what's happening.
Use the ``pywws-hourly`` command to run :py:mod:`pywws.Hourly`::
Try running :py:mod:`pywws.hourly` from the command line, with a high level of verbosity so you can see what's happening.
Use the ``pywws-hourly`` command to run :py:mod:`pywws.hourly`::

pywws-hourly -vvv ~/weather/data

(As usual, replace ``~/weather/data`` with your weather data directory.)
Within five minutes (assuming you have set a 5 minute logging interval) you should see a 'live_data new ptr' message, followed by fetching any new data from the weather station and processing it.

.. versionchanged:: 14.04.dev1194
Expand All @@ -53,13 +54,13 @@ Open your weather.ini file with a text editor.
You should have a ``[paths]`` section similar to the following (where ``xxx`` is your user name)::

[paths]
work = /tmp/weather
work = /tmp/pywws
templates = /home/xxx/weather/templates/
graph_templates = /home/xxx/weather/graph_templates/
local_files = /home/xxx/weather/results/

Edit these to suit your installation and preferences.
``work`` is an existing temporary directory used to store intermediate files, ``templates`` is the directory where you keep your text template files, ``graph_templates`` is the directory where you keep your graph template files and ``local_files`` is a directory where template output that is not uploaded to your web site is put.
``work`` is a temporary directory used to store intermediate files, ``templates`` is the directory where you keep your text template files, ``graph_templates`` is the directory where you keep your graph template files, and ``local_files`` is a directory where template output that is not uploaded to your web site is put.
Don't use the pywws example directories for your templates, as they will get over-written when you upgrade pywws.

Copy your text and graph templates to the appropriate directories.
Expand All @@ -72,18 +73,18 @@ The ``pywws-version -v`` command should show you where the examples are on your
Configuring periodic tasks
--------------------------

In weather.ini you should have ``[logged]``, ``[hourly]``, ``[12 hourly]`` and ``[daily]`` sections similar to the following::
In weather.ini you should have ``[live]``, ``[logged]``, ``[hourly]``, ``[12 hourly]``, and ``[daily]`` sections similar to the following::

[logged]
services = []
plot = []
text = []
plot = []

[hourly]
...

These specify what :py:mod:`pywws.Hourly` should do when it is run.
Tasks in the ``[logged]`` section are done every time there is new logged data, tasks in the ``[hourly]`` section are done every hour, tasks in the ``[12 hourly]`` section are done twice daily and tasks in the ``[daily]`` section are done once per day.
These specify what :py:mod:`pywws.hourly` should do when it is run.
Tasks in the ``[live]`` and ``[logged]`` sections are done when there is new logged data, tasks in the ``[hourly]`` section are done every hour, tasks in the ``[12 hourly]`` section are done twice daily and tasks in the ``[daily]`` section are done once per day.

The ``services`` entry is a list of online weather services to upload data to.
The ``plot`` and ``text`` entries are lists of template files for plots and text files to be processed and, optionally, uploaded to your web site.
Expand Down Expand Up @@ -111,7 +112,10 @@ Add the names of your template files and weather services to the appropriate ent

Note the use of the ``'T'`` flag -- this tells pywws to send the template result to Twitter instead of uploading it to your ftp site.

You can test that all these are working by removing the ``[last update]`` section from status.ini, then running :py:mod:`pywws.Hourly` again::
It's probably best not to add all of these at once.
You could start by uploading one file to your web site, then when that's working add the remaining web site files.
You can add Twitter and other services later on.
You can test that things are working by removing the ``[last update]`` section from status.ini, then running :py:mod:`pywws.hourly` again::

pywws-hourly -v ~/weather/data

Expand All @@ -123,7 +127,6 @@ You can test that all these are working by removing the ``[last update]`` sectio
.. versionchanged:: 13.06_r1015
added the ``'T'`` flag.
Previously Twitter templates were listed separately in ``twitter`` entries in the ``[hourly]`` and other sections.
The older syntax still works, but is deprecated.

.. versionchanged:: 13.05_r1009
the last update information was previously stored in weather.ini, with ``last update`` entries in several sections.
Expand All @@ -133,12 +136,12 @@ Run as a cron job

Most UNIX/Linux systems have a 'cron' daemon that can run programs at certain times, even if you are not logged in to the computer.
You edit a 'crontab' file to specify what to run and when to run it.
For example, to run :py:mod:`pywws.Hourly` every hour, at zero minutes past the hour::
For example, to run :py:mod:`pywws.hourly` every hour, at zero minutes past the hour::

0 * * * * pywws-hourly /home/xxx/weather/data

This might work, but if it didn't you probably won't get any error messages to tell you what went wrong.
It's much better to run a script that runs :py:mod:`pywws.Hourly` and then emails you any output it produces.
It's much better to run a script that runs :py:mod:`pywws.hourly` and then emails you any output it produces.
Here's the script I use::

#!/bin/sh
Expand All @@ -158,4 +161,4 @@ Here's the script I use::
# mail the log file
/home/jim/scripts/email-log.sh $log "weather log"

You’ll need to edit this quite a lot to suit your file locations and so on, but it gives some idea of what to do.
You’ll need to edit this quite a lot to suit your file locations and so on, but it gives some idea of what to do.
49 changes: 20 additions & 29 deletions src/doc/guides/livelogging.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.. pywws - Python software for USB Wireless Weather Stations
http://github.com/jim-easterbrook/pywws
Copyright (C) 2008-14 Jim Easterbrook jim@jim-easterbrook.me.uk
Copyright (C) 2008-18 pywws contributors
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
Expand All @@ -23,26 +23,26 @@ Introduction
------------

There are two quite different modes of operation with pywws.
Traditionally :py:mod:`pywws.Hourly` would be run at regular intervals (usually an hour) from cron.
Traditionally :py:mod:`pywws.hourly` would be run at regular intervals (usually an hour) from cron.
This is suitable for fairly static websites, but more frequent updates can be useful for sites such as Weather Underground (http://www.wunderground.com/).
The newer :py:mod:`pywws.LiveLog` program runs continuously and can upload data every 48 seconds.
The newer :py:mod:`pywws.livelog` program runs continuously and can upload data every 48 seconds.

Getting started
---------------

First of all, you need to install pywws and make sure it can get data from your weather station.
See :doc:`getstarted` for details.

If you have previously been using :py:mod:`pywws.Hourly` then disable your 'cron' job (or whatever else you use to run it) so it no longer runs.
You should not run :py:mod:`pywws.Hourly` and :py:mod:`pywws.LiveLog` at the same time.
If you have previously been using :py:mod:`pywws.hourly` then disable your 'cron' job (or whatever else you use to run it) so it no longer runs.
You should never run :py:mod:`pywws.hourly` and :py:mod:`pywws.livelog` at the same time.

Try running :py:mod:`pywws.LiveLog` from the command line, with a high level of verbosity so you can see what's happening.
Use the ``pywws-livelog`` command to run :py:mod:`pywws.LiveLog`::
Try running :py:mod:`pywws.livelog` from the command line, with a high level of verbosity so you can see what's happening.
Use the ``pywws-livelog`` command to run :py:mod:`pywws.livelog`::

pywws-livelog -vvv ~/weather/data

Within five minutes (assuming you have set a 5 minute logging interval) you should see a 'live_data new ptr' message, followed by fetching any new data from the weather station and processing it.
Let :py:mod:`pywws.LiveLog` run for a minute or two longer, then kill the process by typing '<Ctrl>C'.
Let :py:mod:`pywws.livelog` run for a minute or two longer, then kill the process by typing '<Ctrl>C'.

.. versionchanged:: 14.04.dev1194
the ``pywws-livelog`` command replaced ``scripts/pywws-livelog.py``.
Expand All @@ -54,13 +54,13 @@ Open your weather.ini file with a text editor.
You should have a ``[paths]`` section similar to the following (where ``xxx`` is your user name)::

[paths]
work = /tmp/weather
work = /tmp/pywws
templates = /home/xxx/weather/templates/
graph_templates = /home/xxx/weather/graph_templates/
local_files = /home/xxx/weather/results/

Edit these to suit your installation and preferences.
``work`` is an existing temporary directory used to store intermediate files, ``templates`` is the directory where you keep your text template files, ``graph_templates`` is the directory where you keep your graph template files and ``local_files`` is a directory where template output that is not uploaded to your web site is put.
``work`` is an existing temporary directory used to store intermediate files, ``templates`` is the directory where you keep your text template files, ``graph_templates`` is the directory where you keep your graph template files, and ``local_files`` is a directory where template output that is not uploaded to your web site is put.
Don't use the pywws example directories for your templates, as they will get over-written when you upgrade pywws.

Copy your text and graph templates to the appropriate directories.
Expand All @@ -77,20 +77,20 @@ In weather.ini you should have a ``[live]`` section similar to the following::

[live]
services = []
plot = []
text = []
plot = []

This section specifies what pywws should do every time it gets a new reading from the weather station, i.e. every 48 seconds.
The ``services`` entry is a list of online weather services to upload data to, e.g. ``['underground_rf']``.
The ``services`` entry is a list of online weather services to upload data to, e.g. ``['underground']``.
The ``plot`` and ``text`` entries are lists of template files for plots and text files to be processed and, optionally, uploaded to your web site.
You should probably leave all of these blank except for ``services``.

If you use YoWindow (http://yowindow.com/) you can add an entry to the ``[live]`` section to specify your YoWindow file, e.g.::

[live]
services = ['underground_rf']
services = ['underground']
text = [('yowindow.xml', 'L')]
...
plot = []

Note the use of the ``'L'`` flag -- this tells pywws to copy the template result to your "local files" directory instead of uploading it to your ftp site.

Expand Down Expand Up @@ -120,6 +120,10 @@ Add the names of your template files to the appropriate entries, for example::

Note the use of the ``'T'`` flag -- this tells pywws to send the template result to Twitter instead of uploading it to your ftp site.

It's probably best not to add all of these at once.
You could start by uploading one file to your web site, then when that's working add the remaining web site files.
You can add Twitter and other services later on.

.. versionadded:: 14.05.dev1211
``[cron name]`` sections.
If you need more flexibility in when tasks are done you can use ``[cron name]`` sections.
Expand All @@ -128,30 +132,17 @@ Note the use of the ``'T'`` flag -- this tells pywws to send the template result
.. versionchanged:: 13.06_r1015
added the ``'T'`` flag.
Previously Twitter templates were listed separately in ``twitter`` entries in the ``[hourly]`` and other sections.
The older syntax still works, but is deprecated.

.. versionchanged:: 13.05_r1013
added a ``'yowindow.xml'`` template.
Previously yowindow files were generated by a separate module, invoked by a ``yowindow`` entry in the ``[live]`` section.
This older syntax still works, but is deprecated.

Asynchronous uploads
--------------------

.. versionadded:: 13.09_r1057

Uploading data to web sites or 'services' can sometimes take a long time, particularly if a site has gone off line and the upload times out.
In normal operation pywws waits until all uploads have been processed before fetching any more data from the weather station.
This can lead to data sometimes being missed.

The ``asynchronous`` item in the ``[config]`` section of weather.ini can be set to ``True`` to tell :py:mod:`pywws.LiveLog` to do these uploads in a separate thread.

Run in the background
---------------------

.. versionadded:: 13.12.dev1118

In order to have :py:mod:`pywws.LiveLog` carry on running after you finish using your computer it needs to be run as a "background job".
In order to have :py:mod:`pywws.livelog` carry on running after you finish using your computer it needs to be run as a "background job".
On most Linux / UNIX systems you can do this by putting an ampersand ('&') at the end of the command line.
Running a job in the background like this doesn't always work as expected: the job may suspend when you log out.
It's much better to run as a proper UNIX 'daemon' process.
Expand Down Expand Up @@ -198,4 +189,4 @@ My solution to both problems is to run the following script from cron, several t
The process id of the daemon is stored in ``pidfile``.
If the process is running, the script does nothing.
If the process has crashed, it emails the last 40 lines of the log file to me (using a script that creates a message and passes it to sendmail) and then restarts :py:mod:`pywws.livelogdaemon`.
You'll need to edit this quite a lot to suit your file locations and so on, but it gives some idea of what to do.
You'll need to edit this quite a lot to suit your file locations and so on, but it gives some idea of what to do.
6 changes: 3 additions & 3 deletions src/pywws/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
__version__ = '18.4.2'
_release = '1525'
_commit = '08f5d34'
__version__ = '18.5.0'
_release = '1526'
_commit = '68486c4'

0 comments on commit 8488377

Please sign in to comment.