diff --git a/docs/Bots.md b/docs/Bots.md index 6f13b9ae5f..6f03009606 100644 --- a/docs/Bots.md +++ b/docs/Bots.md @@ -486,58 +486,30 @@ pip3 install pymongo>=2.7.1 #### Configuration Parameters: -* `autocommit`: FIXME +The parameters marked with 'PostgreSQL' will be send +to libpq via psycopg2. Check the +[libpq parameter documentation] (https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-PARAMKEYWORDS) +for the versions you are using. + +* `autocommit`: [psycopg's autocommit mode](http://initd.org/psycopg/docs/connection.html?#connection.autocommit), optional, default True +* `connect_timeout`: PostgreSQL connect_timeout, optional, default 5 seconds * `database`: PostgreSQL database * `host`: PostgreSQL host * `port`: PostgreSQL port * `user`: PostgreSQL user * `password`: PostgreSQL password -* `sslmode`: FIXME -* `autocommit`: FIXME -* `table`: FIXME +* `sslmode`: PostgreSQL sslmode +* `table`: name of the database table into which events are to be inserted #### Installation Requirements -``` -pip3 install psycopg2>=2.5.5 -``` +See [REQUIREMENTS.txt](../intelmq/bots/outputs/postgresql/REQUIREMENTS.txt) +from your installation. #### PostgreSQL Installation -* Install PostgreSQL, at least version 9.4 is recommended. - -```bash -> apt-get install postgresql-9.4 python-psycopg2 postgresql-server-dev-9.4 -``` - -* Create a User and Database: - -```shell -> su - postgres -> createuser intelmq -W - Shall the new role be a superuser? (y/n) n - Shall the new role be allowed to create databases? (y/n) y - Shall the new role be allowed to create more new roles? (y/n) n - Password: - -> createdb -O intelmq --encoding='utf-8' intelmq-events -``` - -* Please note the --encoding='utf-8' in the line above! Without it, the output but will not be able to insert utf-8 data into the table. - -* Depending on your setup adjust `/etc/postgresql/9.4/main/pg_hba.conf` to allow network connections for the intelmq user. - -* Restart PostgreSQL. - -* Generate `initdb.sql` by using the [psql_initdb_generator.py](https://github.com/certtools/intelmq/blob/master/intelmq/bin/intelmq_psql_initdb.py) tool which extracts all field names and data types from `Data-Harmonization.md`. - -* Create the `events` table: - -```bash -> psql intelmq-events < /tmp/initdb.sql # as intelmq user -> psql -U intelmq intelmq-events -W < /tmp/initdb.sql # as other user -``` - +See [outputs/postgresql/README.md](../intelmq/bots/outputs/postgresql/README.md) +from your installation. * * * diff --git a/intelmq/bots/outputs/postgresql/README.md b/intelmq/bots/outputs/postgresql/README.md index 427d0f9d05..5204af95aa 100644 --- a/intelmq/bots/outputs/postgresql/README.md +++ b/intelmq/bots/outputs/postgresql/README.md @@ -1,3 +1,20 @@ +You have two basic choices to run PostgreSQL: +1. on the same machine as intelmq, then you could use unix-sockets if available on your platform +2. on a different machine. In which case you would need to use a TCP connection and make sure you give the right connection parameters to each psql or client call. + +Make sure to consult your PostgreSQL documentation +about how to allow network connections and authentification in case 2. + + +# PostgreSQL Version +Any supported version of PostgreSQL should work +(v>=9.2 as of Oct 2016)[[1](https://www.postgresql.org/support/versioning/)]. + +If you use PostgreSQL server v >= 9.4, it gives you the possibility +to use the time-zone [formatting string](https://www.postgresql.org/docs/9.4/static/functions-formatting.html) "OF" for date-times +and the [GiST index for the cidr type](https://www.postgresql.org/docs/9.4/static/release-9-4.html#AEN120769). This may be useful depending on how +you plan to use the events that this bot writes into the database. + # How to install: Use `intelmq_psql_initdb` to create initial sql-statements @@ -13,15 +30,20 @@ the expert/certbund_contact bot.) Therefore if still necessary: create the database-user as postgresql superuser, which usually is done via the system user `postgres`: ``` -createuser --encrypted --pwprompt intelmq +createuser --no-superuser --no-createrole --no-createdb --encrypted --pwprompt intelmq ``` Create the new database: ``` -createdb --owner=intelmq intelmq-events +createdb --encoding='utf-8' --owner=intelmq intelmq-events ``` -Now initialise it as database-user `intelmq` (should ask for the password): +(The encoding parameter should ensure the right encoding on platform +where this is not the default.) + +Now initialise it as database-user `intelmq` (in this example +a network connection to localhost is used, so you would get to test +if the user `intelmq` can authenticate): ``` psql -h localhost intelmq-events intelmq