Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse code

migrate from 'when' to 'start/end'

  • Loading branch information...
commit 1ad78ffc359270ef057a9f1ac37ef52096007b66 1 parent 407cece
Alex Chaffee authored July 04, 2011
2  Gemfile
... ...
@@ -0,0 +1,2 @@
  1
+gem "rake"
  2
+gem "crxmake"
13  Gemfile.lock
... ...
@@ -0,0 +1,13 @@
  1
+GEM
  2
+  specs:
  3
+    crxmake (2.0.3)
  4
+      zipruby (>= 0.3.2)
  5
+    rake (0.9.2)
  6
+    zipruby (0.3.6)
  7
+
  8
+PLATFORMS
  9
+  ruby
  10
+
  11
+DEPENDENCIES
  12
+  crxmake
  13
+  rake
52  README.markdown
Source Rendered
@@ -10,23 +10,30 @@ Complaints to [@alexch](http://twitter.com/alexch) / <mailto:alex@stinky.com>
10 10
 
11 11
 1. Install [CouchBase](http://www.couchbase.com/downloads/couchbase-server/community)
12 12
 2. Select "Automatically Start At Login" from CouchBase menu ![CouchBase menu](automatically-start-couchbase.png)
13  
-3. Open [builds/whence.crx](builds/whence.crx) in Google Chrome
  13
+3. Open [builds/whence.crx](builds/whence.crx) in Google Chrome (double clicking might work here)
  14
+4. Click "Continue" and "Install" and "X" and whatever other security theater hoops come up
  15
+
  16
+Does it work on Windows or Linux? I don't know yet! Please tell me your experiences.
14 17
 
15 18
 ## Usage
16 19
 
17 20
 Open <http://localhost:5984/_utils/database.html?whence> and see your data aggregating
18 21
 
19  
- * http://localhost:5984/whence/_design/sample/_view/by_when
  22
+ * http://localhost:5984/whence/_design/sample/_view/by_start
  23
+
  24
+TODO: UI :-)
20 25
 
21 26
 ## Chromium vs Google Chrome
22 27
 
23 28
 Multiple Chromes installed? Drag [builds/whence.crx](builds/whence.crx) to whichever is your favorite.
24  
-    
  29
+
  30
+# Developers
  31
+
25 32
 ## Rebuild
26 33
 
27 34
     gem install crxmake
28 35
     rake install:chrome
29  
-    
  36
+
30 37
 ## Debug
31 38
 
32 39
 From Chrome's "Extensions" window, select "Developer mode", scroll down to Whence, then click the "background.html" link to look at the console. You can also look at the console in any page.
@@ -45,43 +52,52 @@ We could gather all sorts of other events as well, like tab opens and closes. Bu
45 52
 
46 53
 * Better error if CouchDB is not running/installed
47 54
   * use Chrome Desktop Notifications http://code.google.com/chrome/extensions/notifications.html
48  
-* Smoosh successive hits to same host together
49  
-  *   possibly using map/reduce
50 55
 * Detect idle time
51 56
   * either stop recording, or record "time since activity" per sample
52 57
   * http://paulirish.com/2009/jquery-idletimer-plugin/
53 58
   * http://stackoverflow.com/questions/667555/detecting-idle-time-in-javascript-elegantly
54 59
   * http://www.frebsite.nl/werk/scripts/jquery_nap_plugin/index_en.php#voorbeelden
  60
+* Reports (aka "a user interface of some sort")
  61
+  * by day
  62
+  * by date range
  63
+  * by host
55 64
 * Store path as well as host
  65
+* Store client hostname
  66
+* "Destination" rules (potentially several destinations per host, based on path)
  67
+  * need better name, but "location" implies geolocation, so...
  68
+* Context menu item: add this path as a destination (gets tracked separately)
  69
+  * http://code.google.com/chrome/extensions/contextMenus.html
  70
+* Export to Locker Project
  71
+  * http://gigaom.com/2011/02/04/the-locker-project-why-leave-data-tracking-to-others-do-it-yourself/
  72
+  * https://github.com/LockerProject/Locker
56 73
 * Change icon
57  
-* Include crxmake library
58 74
 * Visualization
59 75
 * Replicate/synchronize DBs from multiple computers
60 76
 * Gather info on which app (other than browser) is active
61  
-* Integrate with MacLogger
62 77
 * Settings page
63 78
   *    http://code.google.com/chrome/extensions/options.html
  79
+* Integrate with MacLogger
64 80
 * Firefox plugin
65 81
 * Safari plugin
66 82
 * Clone RescueTime (Everything version - track active app, not just active tab)
67  
-* Include CouchDB executable
68  
-* Use IndexedDB for local storage (in case Couch is missing)  
69  
-* Write server in node.js
70  
-* Stick server *inside* CouchDB
71  
-  *   see http://www.livelycouch.org/
72 83
 * Store tab.favIconUrl (tab is "sender" param of onRequest)
73 84
 * Limits (e.g. warn if you're on Slashdot too long)
74 85
 * Categories a la RescueTime
75 86
 * log per-page info e.g. GMail subject and from (with content scripts)
76 87
 * Geolocation (possibly with GeoCouch)
77 88
   *  https://github.com/couchbase/geocouch
78  
-* Store times in GMT?
79  
-* Context menu item: add this path as a destination (gets tracked separately)
80  
-  * http://code.google.com/chrome/extensions/contextMenus.html
81 89
 * Option: Respect incognito mode (don't track incognito tabs)
82 90
   * http://code.google.com/chrome/extensions/overview.html#incognito
83  
-* cache samples; flush cache every minute (to avoid annoying disk whir)
84  
-  
  91
+
  92
+## Technical chores TODO
  93
+
  94
+* Compact DB on startup
  95
+* Include CouchDB executable(s) in distro?
  96
+* Use IndexedDB for local storage (in case Couch is missing)
  97
+* Include crxmake library inside project
  98
+* Stick server *inside* CouchDB?
  99
+  *   see http://www.livelycouch.org/
  100
+
85 101
 # Credits
86 102
 
87 103
 - Written by [Alex Chaffee](http://alexch.github.com)
BIN  builds/whence.crx
Binary file not shown
79  ext/whence.js
@@ -147,30 +147,39 @@ function setupDesignDocs() {
147 147
   });
148 148
 }
149 149
 
  150
+
150 151
 function migrateWhens() {
151 152
   couch('get', {
152 153
     docId: '_design/sample/_view/by_when',
153 154
     success: function(response) {
  155
+      console.log("Migrating " + response.rows.length + " whens")
154 156
       var tick = null;
155  
-      for (var i=0; i<response.rows.length; ++i) {
156  
-        var doc = response.rows[i].value;
157  
-        if (tick == null) {
158  
-          tick = {host: doc.host, start: doc.when, end: doc.when};
159  
-        } else {
160  
-          if (doc.host == tick.host) {
161  
-            tick.end = doc.when;
162  
-          } else {
163  
-            console.log(tick);
164  
-            couch('post', {data: tick});
  157
+      var rows = response.rows;
  158
+      function migrateNextRow() {
  159
+        row = rows.shift();
  160
+        if (row) {
  161
+          var doc = row.value;
  162
+          if (tick == null) {
165 163
             tick = {host: doc.host, start: doc.when, end: doc.when};
  164
+          } else {
  165
+            if (doc.host == tick.host) {
  166
+              tick.end = doc.when;
  167
+            } else {
  168
+              console.log("saving " + JSON.stringify(tick));
  169
+              couch('post', {data: tick});
  170
+              tick = {host: doc.host, start: doc.when, end: doc.when};
  171
+            }
166 172
           }
  173
+          couch('delete', {docId: doc['_id'], params: 'rev=' + doc['_rev']});
  174
+          setTimeout(migrateNextRow, 0);
  175
+        } else {
  176
+          if (tick != null) {
  177
+            console.log("saving " + JSON.stringify(tick));
  178
+            couch('post', {data: tick});
  179
+          }            
167 180
         }
168  
-        couch('delete', {docId: doc['_id'], params: 'rev=' + doc['_rev']});
169  
-      }
170  
-      if (tick != null) {
171  
-        console.log(tick);
172  
-        couch('post', {data: tick});
173 181
       }
  182
+      migrateNextRow();
174 183
     }
175 184
   })    
176 185
 }
@@ -203,31 +212,37 @@ function startTicker() {
203 212
         ) {
204 213
         chrome.windows.get(tab.windowId, function(window) {
205 214
           if (window.focused) {     // we haven't lost focus
206  
-            var host = parseUrl(tab.url).host;
207  
-            var now = new Date();
208  
-            if (lastTick &&
209  
-              lastTick.host == host &&
210  
-              lastTick.end >= (now - period - slop)
211  
-              ) {
  215
+            if (tab.url == null || tab.url == '') {
  216
+              console.log("Blank URL for tab:")
  217
+              console.log(tab)
  218
+            } else {
  219
+              var host = parseUrl(tab.url).host;
  220
+              var now = new Date();
  221
+              if (lastTick &&
  222
+                lastTick.host == host &&
  223
+                lastTick.end >= (now - period - slop))
  224
+              {
212 225
                 lastTick.end = now;
213 226
                 couch('put', {
214 227
                   docId: lastTick['_id'],
215 228
                   data: lastTick,
216 229
                   success: function(response) {
  230
+                    console.log("Logged " + lastTick.host + " " + (lastTick.end - lastTick.start)/1000 + " sec");                    
217 231
                     lastTick['_rev'] = response.rev;
218 232
                   }
219 233
                 });
220  
-            } else {
221  
-              var newTick = {host: host, start: now, end: now};
222  
-              couch('post', {
223  
-                data: newTick,
224  
-                success: function(response) {
225  
-                  console.log(response);
226  
-                  lastTick = newTick;  // todo: dup
227  
-                  lastTick['_id'] = response.id;
228  
-                  lastTick['_rev'] = response.rev;
229  
-                }
230  
-              });
  234
+              } else {
  235
+                var newTick = {host: host, start: now, end: now};
  236
+                couch('post', {
  237
+                  data: newTick,
  238
+                  success: function(response) {
  239
+                    console.log("Logging " + newTick.host);
  240
+                    lastTick = newTick;  // todo: dup
  241
+                    lastTick['_id'] = response.id;
  242
+                    lastTick['_rev'] = response.rev;
  243
+                  }
  244
+                });
  245
+              }
231 246
             }
232 247
           }
233 248
         });

0 notes on commit 1ad78ff

Please sign in to comment.
Something went wrong with that request. Please try again.