Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

LifeDB server to manage personal data

branch: master
README
NOTE: these docs are not complete yet, so do please get in touch
with me (anil@recoil.org) if you want to be an early adopter!

Build
-----

The build process is slightly complicated by needing a patched
ocamlnet with some extra functions, and also some packages which
dont seem to be included by distributions.

MacOS X

To build this repository on MacOS X, you will need an up-to-date
dports repo and the additional local repository available at:
http://github.com/avsm/darwinports/

You will need at least:

 * ocaml 3.11.0
 * caml-json-wheel
 * caml-json-static
 * caml-sqlite3
 * caml-ocamlnet
 * caml-ansicolor
 * cryptokit

Just do "./configure && make" in the current directory to build the sources.

Linux

On Debian Sid, you can clone:
http://github.com/avsm/lifedb-depends

and then run:

# ./debian.sh
# make all install

...which will install the standard Debian packages and then some additional
ones from the local repository with the relevant patches.

Once the patches in the lifedb-depends repository are integrated upstream
or worked around, the build process should become a lot simpler...

Configuration
-------------

The server gets its base configuration from "config.json" in the build
directory, and there is a template in 'config.json.in' in the src tree.

 * lifedb_directory: Root of the user's LifeDB files
 * plugins_directory: List of directories to search for plugins
 * log_directory: Directory to drop task logfiles and database to
 * cache_directory: temporary space to hold cache files from plugins
 * config_directory: directory where passwords and task information is stored.
 * static_directory: static HTML files to serve up via /static

The server accepts some command-line arguments:
 * -conf: Location of the configuration file (default current dir)
 * -test: if true, will not actually run external commands but do everything else.

Password Management
-------------------

The server has a global passphrase (stored in the Keychain on MacOS X) which
must be set before it will start.  This can be done with the ./set_passphrase.sh
script in this repository.  On Linux and other platforms, there is an insecure
flat file which contains a list of "username password" lines in plaintext.

Once that is done, you can do  "make scripts && . ./export-var.sh" to build
a development copy of the Python console scripts.  Run ./scripts/lifedb-passwd
to add encrypted service/username/password entries for the various tasks.
The service string should match the service/username defined in the task
configuration file (see below).

Plugins
-------

Each plugin is in a directory with a LIFEDB_PLUGIN file in the directory
indicating it should be registered.  Here is an example file:

{
   "name" : "Twitter",
   "cmd" : "make run",
   "declares" : [
       {
         "pltype" : "com.twitter",
         "description" : "Twitter",
         "implements" : "public.message"
       }
   ]
}

The name is the plugin UUID, the cmd is how to run it (it executes with
the current working directory being the plugin directory).  The declares
registers a media type in the LifeDB SQL database.

For every plugin, the user may create 0 or more task configurations which
declares how to execute the plugin.  These are files ending with '.conf' in
the configuration_directory.  Here is an example (twitter again):

{
  "name": "twitter-avsm",
  "plugin": "Twitter",
  "mode": "periodic",
  "silo", "Twitter/avsm",
  "period" : 3600,
  "secret" : {
      "service" : "twitter-avsm",
      "username" : "avsm"
  },
  "args" : {
      "FULL_SYNC" : "0"
  }

The 'name' is how to identify the running task for debugging and querying
it over the HTTP interface.  The 'plugin' is the previously registered plugin
name.  'mode' can be:
  * periodic : run it regularly every 'period' seconds.
  * constant : make sure this is always running, and restart if it dies
  * single : run it once off, and dont repeat it.

'secret' is optional and used for looking up passwords to send to the script
when it executes.  The service/username combination are looked up from the
database and passed to the script via the LIFEDB_USERNAME and LIFEDB_PASSWORD
environment variables if this key is present.

'silo' defines the subdirectory of the LifeDB this task should put its data
into.

'args' are for additional environment variables to pass to the script, so the
user can toggle knobs for different syncing behaviour.

As a convention, there is a 'default.conf' file in each plugin directory which
can be copied to the configuration directory for a reasonable sensible default.

Running
-------

To run the server in the foreground:

# ./lifedb_server 

This is also the "make run" Makefile target.

Databases
---------

Log messages and progress are output to stdout and also to <logging_dir>/log.db

The main LifeDB cache is held in <lifedb_dir>/life.db

Useful URLS
-----------

You must authenticate successfully using HTTP auth before any other actions
are permitted.

The various JSON return types are defined in client/ocaml/lifedb.ml

 * /config  : returns the global server configuration as JSON with all 
   variables fully substituted in.

 * /date/<year>/<month> : returns Rpc.Query.month_list with message counts
   for the days in the selected month.  Useful for populating calendar widget.

 * /date/<year>/<month>/<day> : return Rpc.Query.day_list with message
   info for the entries for the particular day. 

 * /doc/<id> : return Rpc.Entry.doc with the LifeDB entry and additional
   info (e.g. contacts) for the particular doc ID.

 * /pltype/<plugin type> : returns Rpc.Plugin.decl with information about
   the media type.  Includes a filesystem location for icon if it exists.

 * /pltype/<plugin type>/icon : returns the icon via HTTP if it exists.

 * /att/<attuid> : returns the attachment binary

 * /q/contact/date/<year>/<month>/<day> : Returns all the contacts involved
   in all the conversations in the time period.  <day> can be omitted.

 * All files in the "./htdocs" directory (the static_directory setting in the
   config file) are served up via the /static URL.
Something went wrong with that request. Please try again.