Permalink
Browse files

node-dbmon initial release

  • Loading branch information...
0 parents commit f3ec11076dc828a350709d49006f4cd29a577407 @straps committed Oct 25, 2011
@@ -0,0 +1,17 @@
+1.0.2 / 2011-10-25
+==================
+
+ * Added the possibility to notify not only if something changes, but also what have changed, see channelDefaults.keyfld
+ * Added the truncate monitoring via TRIGGER for postgresql driver
+
+1.0.1 / 2011-10-24
+==================
+
+ * Initial release
+ * Added PostgreSQL driver with TRIGGER and LISTEN/NOTIFY support added
+ * Added Console and EventEmitter transports
+
+1.0.0 / 2011-10-22
+==================
+
+ * Idea
196 Readme.md
@@ -0,0 +1,196 @@
+# Database monitor utilities for nodejs
+If you are trying to update a GUI when a database table changes (_insert_, _update_, _delete_), this library is for you.
+
+This is a node.js module supporting a growing number of database drivers and notification transports
+you can extend and improve.
+
+It is designed to be easily extended with simple sintax by anyone and, where possibile,
+to notify of changes without classic polling, but with real-time notification mechanism
+
+## Usage sample
+As of today, there is only one driver developed, the PostgreSQL one because I'm a PostgreSQL fan and heavy user.
+This is a short example; you can find more on `test/test-postgresql.js`
+
+Install a local postgresql database server; grant temporary trust access to the postgresql
+user editing the pg_hba.conf file and create a test table like this `create table testtable(id integer primary key, val varchar(10));`
+then run the following
+
+ var pg=require('pg'), cli=new pg.Client('tcp://postgres@localhost/template1'), dbmon=require('dbmon');
+
+ cli.connect();
+ //uncomment if you want node to create the temporary table for you
+ //cli.query('drop table if exists testtable; create table testtable(id integer primary key, val varchar(10));');
+
+ var channel=dbmon.channel({
+ driver:'postgresql',
+ driverOpts:{
+ postgresql:{
+ cli:cli
+ }
+ },
+ table:'testtable',
+ monitor:'all',
+ keyfld:{
+ name:'id',type:'integer'
+ }
+ });
+
+Now monitor the console and execute some insert/update/delete and see what happens...
+
+You should see come console messages saying you are modifiyng `testtable` like this
+
+ Console Transport Notification: insert, row={"op":"i","k":2,"oldk":2,"id":1}
+
+In this case I've executed a simple insert like `insert into testtable values(2,'TWO');`.
+Console says that the type of notification is an insert and the row modified
+from last notification is `{"op":"i","k":2,"oldk":2,"id":1}` where fields means:
+
+ - *op* is the operation type; can be *i* for insert, *u* for update, *d* for delete and *t* for truncate
+ - *k* is the key inserted/updated/deleted based on what specified in `keyfld.name`
+ - *oldk* is the old key value, see what happens executing `update testtable set id=20, val='twenty' where id=2`
+ - *id* is an internal change sequence id, an ordered number useful to keep track of modifications
+
+It is very interesting to know that if you update 2 or more rows in the same transaction, there will
+arrive 2 ore more notifications based on the number or rows being modified
+
+Another good thing is that for PostgreSQL, *dbmon* is powered by the NOTIFY/LISTEN constructs. It means
+that, when something changes, the server that contacts node and node notify listeners via the transports specified, making
+it really real-time, not like other polling-based alternatives.
+
+To see the complete list of options see [lib/channelDefaults.js](https://github.com/straps/node-dbmon/blob/master/lib/channelDefaults.js)
+
+
+## Structure and Naming Conventions
+
+Dbmon is designed to be dynamic and easily extensible; there are 3 main actors to extend it
+
+ - **transports**, in [lib/providers](https://github.com/straps/node-dbmon/tree/master/lib/transports) are the way dbmon notify events. You can use how many tranports you want separating them by comma. The name specified in the options object have to match the name of the file followed by `-tranport.js` in the `transports` foler, like the `console` transport in the example above.
+ - **providers**, in [lib/providers](https://github.com/straps/node-dbmon/tree/master/lib/providers), have to initialize their method to fetch data and notify transports whene something happen; in most cases (surely for postgresql case) they should only require the `generic-driver` that dynamiccaly instantiate the method and notify transports
+ - **methods**, in [lib/providers](https://github.com/straps/node-dbmon/tree/master/lib/methods), are the core of the system; their implementation depends upon the driver and the method specified in the configuration object and their name should respect `DRIVER-METHOD-method.js` convention (ie: postgresql-trigger-method.js). Methods init function return an `EventEmitter` inherited object that notify listeners where data changes firing the event notification chain
+
+
+### How Create a new Transport
+
+Creating a new transport is very easy; the node module have to export a single function `init` that `dbmon` will call passing the global options object.
+
+The `init` function have to return an object with a `notify` method, magically called from drivers, when something server side changed.
+
+Say we want a generic TCP Socket transport to communicate with another application, transmitting db update notification.
+
+Create the file `lib/transports/tcpsocket.js` and insert the following lines:
+
+ //TCP Socket Tranport
+ var init=function init(opts){
+ console.log('TCP Socket Transport init');
+ var me={
+ notify:function(type, row){
+ opts.transportsOpts.tcpsocket.client.write(JSON.stringify(row));
+ return me;
+ }
+ };
+ return me;
+ };
+ module.exports={init:init};
+
+Now use it from your node.js server socket app:
+
+ var net = require('net');
+ var server = net.createServer(function (c) {
+ c.on('data', function(data){
+ console.log('DATA FROM SOCKET HURRAAA --> '+data);
+ });
+ });
+ server.listen(8124, 'localhost', function(){
+ var client=new net.Socket();
+ client.connect(8124, 'localhost', function(){
+ console.log('connected');
+
+ var pg=require('pg'), cli=new pg.Client('tcp://postgres@localhost/template1'), dbmon=require('dbmon');
+ cli.connect();
+
+ dbmon.channel({
+ driver:'postgresql',
+ driverOpts:{
+ postgresql:{
+ cli:cli
+ }
+ },
+ table:'testtable',
+ method:'trigger',
+ transports:'tcpsocket',
+ transportsOpts:{
+ tcpsocket:{
+ client:client
+ }
+ },
+ keyfld:{ name:'id', type:'integer'}
+ });
+ });
+ });
+
+In 10 lines of code you can create and use a new tranport, contribute to the library and make others happy (me too :)
+
+Creating a new driver and a new driver method, could be some more complicated, but I thing, in next releases will be a generic
+mixed trigger/polloging based driver I'm thinking on.
+
+
+## Testing
+
+Test cases are home-made and could not be complete or well done, so feel free to fork and improve tests too.
+
+In any case, you can test the library doing a `node test/test-*` from main directory
+
+
+## Installation
+
+Using npm: `npm install dbmon`
+
+Or `npm install dbmon -g` and `npm link dbmon` if you prefer linking a global installation
+
+Or you can download/fork and copy on a local folder inside your project
+
+
+### External Dependencies, automatically installed if you use npm
+
+ - [Underscore.js](http://documentcloud.github.com/underscore/) (`npm install underscore`)
+ - [Step](https://github.com/creationix/step) (`npm install step`)
+
+Database drivers, depends on the driver you use, including
+
+ - [Pg](https://github.com/brianc/node-postgres) (`npm install pg`)
+
+Only for test purposes
+
+ - [Colors](https://github.com/Marak/colors.js) (`npm install colors`)
+
+
+
+## ToDo
+ - Develop other drivers (MySQL, Oracle, MsSQL, etc...)
+ - Develop other transports (Faye, Hook.io, etc..)
+
+
+## License
+
+(The MIT License)
+
+Copyright (c) 2011 Francesco Strappini <f@strx.it>
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1 @@
+module.exports = require('./lib/dbmon');
@@ -0,0 +1,49 @@
+/** Dbmon channel defaults */
+var d={
+ /** Database driver to use; postgresql8, postgresql9, mysql, oracle, etc...
+ REQUIRED */
+ driver: null,
+ /** Driver dedicated opts */
+ driverOpts:{
+ postgresql:{
+ /** connected client, REQUIRED */
+ cli:null
+ }
+ },
+
+ /** Table to monitor for changes,
+ REQUIRED */
+ table: null,
+ /** Informazioni sul campo chiave da restituire nelle notifiche con il valore inserito/modificato/eliminato */
+ keyfld:{
+ /** Key Field Name, ie: 'id' */
+ name:null,
+ /** Key Field Type, ie: 'integer', or 'varchar(100)', etc.. */
+ type:null
+ },
+
+ /** What to monitor, comma separated list of insert,update,delete or all */
+ monitor: 'all',
+
+ /** Type of monitor, trigger or polling */
+ method: 'trigger',
+ /** Options dedicated to method types */
+ methodOpts:{
+ trigger:{},
+ polling:{}
+ },
+
+ /** Comma separated list of transports, console, eventEmitter, tcp, faye, hookio, redis, etc.. */
+ transports: 'console',
+ transportsOpts:{
+ eventEmitter:{
+ /** if transports contains eventEmitter, this is REQUIRED */
+ eventEmitter:null
+ }
+ },
+
+ /** Debounce notification support, avoid server and listeners overload on frequest updates */
+ debouncedNotifications:100
+};
+
+module.exports={channelDefaults:d};
@@ -0,0 +1,32 @@
+// DbMon - Copyright Francesco Strappini <f@strx.it> (MIT Licensed)
+var _=require('underscore')._,
+ channelDefaults=require('./channelDefaults').channelDefaults;
+
+var dbmon={
+ version : '1.0.0'
+};
+
+dbmon.channel = function(opts){
+ opts=_.extend({}, channelDefaults, opts);
+ if (opts.monitor==='all') {
+ opts.monitor='insert,update,delete,truncate';
+ }
+
+ //Dynamic Transports Init
+ var transports=[];
+ _.each(opts.transports.split(','), function(t){
+ t=t.trim();
+ transports.push(require('./transports/'+t+'-transport').init(opts));
+ });
+
+ //Main Object to Return
+ var me={
+ //Dynamic driver initialization
+ driver:require('./drivers/'+opts.driver+'-driver').init(opts, transports),
+
+ start:function(){}
+ };
+ return me;
+};
+
+module.exports=dbmon;
@@ -0,0 +1,29 @@
+var _=require('underscore')._;
+
+var init=function init(opts, transports){
+ console.log('Postgresql Driver Init');
+
+ var shorts={'i':'insert','u':'update','d':'delete','t':'truncate'};
+
+ //Dynamic Method Init
+ var method=require('../methods/'+opts.driver+'-'+opts.method+'-method').init(opts);
+ _.each(['insert', 'update', 'delete', 'truncate'], function(op){
+ method.on(op, function(rows){
+ if (rows && rows.length){
+ _.each(rows, function(row){
+ _.each(transports, function(t){
+ t.notify(shorts[row.op], row);
+ });
+ });
+ }else{
+ t.notify(op);
+ }
+ });
+ });
+
+ var me={
+ };
+ return me;
+};
+
+module.exports={init:init};
@@ -0,0 +1 @@
+module.exports=require('./generic-driver');
Oops, something went wrong.

0 comments on commit f3ec110

Please sign in to comment.