eodtime bugfix

code/common/eodtime.q

@@ -1,23 +1,34 @@
-/ - system eodtime configuartion
-/ - loaded into and used in the tp and pdb processes
-\d .eodtime
-rolltime:@[value;`rolltime;0D00:00:00.000];
-datatimezone:@[value;`datatimezone;`$"GMT"];
-rolltimezone:@[value;`rolltimezone;`$"GMT"];
-dayoffset:@[value;`dayoffset;0];
-getdailyadjustment:{first exec adjustment from aj[`timezoneID`gmtDateTime;([]timezoneID:enlist .eodtime.datatimezone;gmtDateTime:enlist .z.p); .tz.t]};
-dailyadj:getdailyadjustment[];
-adjrolltime:{[p] 
-     z:rolltime - first exec adjustment from aj[`timezoneID`gmtDateTime;([]timezoneID:enlist .eodtime.rolltimezone;gmtDateTime:enlist p); .tz.t];  
-     z:$[z >= 1D;z - 1D;z]			// don't let it go past one day
-     }
-getroll:{[p] 
-     z:adjrolltime[p];
-     ("d"$p) + $[z <= p;z+1D;z]			// if past time already today, make it tomorrow
-     }
-getday:{[p] 
-     z:adjrolltime[p];
-     ("d"$p) + dayoffset+$[z > p;-1;0]		// if not past roll time, subtract one day
-     }
-d:getday[.z.p];
-nextroll:getroll[.z.p];
+/ - system eodtime configuraion
+/ - loaded into and used in the tp and pdb processes
+\d .eodtime
+
+// default settings
+rolltimeoffset:@[value;`rolltimeoffset;0D00:00:00.000];     // offset from standard midnight rollover
+datatimezone:@[value;`datatimezone;`$"GMT"];                // timezone for stamping data
+rolltimezone:@[value;`rolltimezone;`$"GMT"];                // timezone for EOD roll
+
+// function to determine offset from UTC for timestamping data
+getdailyadjustment:{exec adjustment from .tz.t asof `timezoneID`gmtDateTime!(.eodtime.datatimezone;.z.p)};
+
+dailyadj:getdailyadjustment[];                              // get offset when loading process and store it in dailyadj
+
+// function to determine offset from UTC for EOD roll
+adjtime:{[p]
+     :exec adjustment from .tz.t asof `timezoneID`gmtDateTime!(.eodtime.rolltimezone;.z.p);
+     };
+
+// function to get time (in UTC) of next roll after UTC timestamp, p
+getroll:{[p]
+     z:rolltimeoffset-adjtime[p];                           // convert rolltimeoffset from rolltimezone to UTC
+     z:`timespan$(mod) . "j"$z, 1D;                         // force time adjust to be between 0D and 1D
+     ("d"$p) + $[z <= p;z+1D;z]                             // if past time already today, make it tomorrow
+     };
+
+// function to determine the date (in rolltimezone) from UTC timestamp, p
+getday:{[p]
+     p+:adjtime[p];                                         // convert date from UTC to rolltimezone
+     "d"$p-rolltimeoffset                                   // adjust day according to rolltimeoffset
+     };
+
+d:getday[.z.p];                                             // get current date when loading process, store in d
+nextroll:getroll[.z.p];                                     // get next roll when loading process, store in nextroll

code/processes/tickerplant.q

@@ -38,7 +38,7 @@ icounts:()!()  / set up dictionary for per table counts
 ld:{if[not type key L::`$(-10_string L),string x;.[L;();:;()]];i::j::@[-11!;L;i::-11!(-2;L)];jcounts::icounts;if[0 < type i;-2 (string L)," is a corrupt log. Truncate to length ",(string last i)," and restart";exit 1];hopen L};
 tick:{init[];if[not min(`time`sym~2#key flip value@)each t;'`timesym];@[;`sym;`g#]each t;d::.eodtime.d;if[l::count y;L::`$":",y,"/",x,10#".";l::ld d]};
 
-endofday:{end d;d+:1;icounts::()!();.eodtime.nextroll:.eodtime.getroll[.z.p];.eodtime.dailyadj:.eodtime.getdailyadjustment[];if[l;hclose l;l::0(`.u.ld;d)]};
+endofday:{end d;d+:1;icounts::()!();if[.z.p>.eodtime.nextroll:.eodtime.getroll[.z.p];system"t 0";'"next roll is in the past"];.eodtime.dailyadj:.eodtime.getdailyadjustment[];if[l;hclose l;l::0(`.u.ld;d)]};
 ts:{if[.eodtime.nextroll < x;if[d<("d"$x)-1;system"t 0";'"more than one day?"];endofday[]]};
 
 

config/settings/default.q

@@ -126,7 +126,6 @@ broadcast:1b;                   // broadcast publishing is on by default. Availb
 
 // timezone
 \d .eodtime
-rolltime:0D00:00:00.000000000;		// time to roll in rolltimezone
+rolltimeoffset:0D00:00:00.000000000;	// offset from default midnight roll
 datatimezone:`$"GMT";			// timezone for TP to timestamp data in
 rolltimezone:`$"GMT";			// timezone to perform rollover in
-dayoffset:0;				// 0 = on rollover, day becomes today; 1 = on rollover, day becomes tomorrow

docs/Processes.md

@@ -455,6 +455,24 @@ Viewer](http://code.kx.com/wiki/Cookbook/InterfacingWithJava):
 Some of the unofficially supported APIs may only allow synchronous calls
 to be made.
 
+<a name="tickerplant"></a>
+
+Tickerplant
+-----------
+
+The tickerplant is a modified version of the standard kdb+tick tickerplant.
+The modifications from the standard tick.q include:
+
+-   Applies timestamps as timestamp rather than timespan;
+
+-   Tracks per table record counts in .u.icounts dictionary for faster recovery
+    of real time subscribers;
+
+-   Allows configuration of timezones for timestamping data and performing
+    end of day rollover (see [eodtime.q](utilities/#eodtimeq));
+
+The tickerplant log file will be written to hdb/database.
+
 <a name="rdb"></a>
 
 Real Time Database (RDB) 
@@ -506,7 +524,9 @@ number of modifications and enhancements over w.q:
 5.  Informs other RDB, HDB and GW processes that end of day save and
      sort has completed;
 
-6.  More log information supplied.
+6.  More log information supplied;
+
+7.  End of day timezone can be configured (see [eodtime.q](utilities/#eodtimeq)).
 
 The WDB process can broken down into two main functions:
 

docs/utilities.md

@@ -772,6 +772,41 @@ Monitoring process. It includes functions to format dates, tables to csv
 to configure the HTML file to work on the correct process. It is
 accessible from the `.html` namespace.
 
+<a name="eodtime"></a>
+
+eodtime.q
+---------
+
+This script provides functionality for managing timezones. TorQ can be 
+configured to timestamp data in a specific timezone, while also being
+configured to perform the end of day rollover in another timezone, at a
+configurable time.
+
+These options are handled by three settings:
+
+| Setting | Req  |  Type   |               Description                |
+| :-----: | :--: | :-----: | :--------------------------------------: |
+| .eodtime.rolltimeoffset |  Y   | timespan  | Offset from default midnight roll time |
+| .eodtime.rolltimezone |  Y   | symbol  | Time zone in which to rollover |
+| .eodtime.datatimezone |  Y   | symbol  | Time zone in which to timestamp data in TP |
+
+The default configuration sets both timezones to GMT and has the rollover
+performed at midnight.
+
+A table containing the valid timezones is loaded into TorQ processes as .tz.t
+
+An example configuration where data is stamped in GMT, but the rollover
+occurs at 5PM New York time would be:
+
+    .eodtime.rolltimeoffset:-0D07:00:00.000000000; // 5 PM i.e. 7 hours before midnight
+    .eodtime.rolltimezone:`$"America/New_YorK";    // roll in NYC time
+    .eodtime.datatimezone:`$"GMT";                 // timestamp in GMT
+
+Note that the rolltimeoffset can be negative - this will cause the rollover to happen 
+"yesterday", meaning that at the rolltime, the trading date will become the day *after*
+the calendar date. Where this is positive, the rollover occurs "today" and so the trading
+date will become the current calendar date.
+
 Modified u.q
 ------------