Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Comparing changes

Choose two branches to see what's changed or to start a new pull request. If you need to, you can also compare across forks.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also compare across forks.
  • 13 commits
  • 5 files changed
  • 0 commit comments
  • 3 contributors
Showing with 185 additions and 206 deletions.
  1. +0 −3  .gitignore
  2. +41 −97 README.md
  3. +59 −62 app.js
  4. +72 −33 index.html
  5. +13 −11 package.json
View
3  .gitignore
@@ -1,3 +0,0 @@
-node_modules
-venv
-.env
View
138 README.md
@@ -1,117 +1,61 @@
-# Tractor Push: A Real-time Demo with Ruby, Node.js, Socket.io and MongoDB on Heroku Cedar
+This application is the message-consuming and web component of the system described in [this Heroku Dev Center article](https://devcenter.heroku.com/articles/build-realtime-polyglot-node-ruby-mongodb-socketio-app) and deployed to http://tractorpush.herokuapp.com/.
+The Ruby version of the message queuing app can be found at: https://github.com/mongolab/tractorpush-writer-ruby
-Learn how to use three rising infrastructure technologies that put
-real-time interactivity into Web apps: `socket.io`, `Node.js` and
-`MongoDB`. Real-time interactivity examples include chat, large-scale
-games, editing collaboration, and low-latency notification. In a
-market with dynamic, mobile, and social applications, a strong Web
-interface is still an important component to the entire experience.
-To demonstrate cross-app functionality with an existing Ruby
-application, the article first walks through using a separate Heroku
-app to write to a shared MongoDB acting as a simple queue.
-Fortuitously, Heroku’s new Celadon Cedar stack offers several features
-that support these techniques: a polyglot stack and flexible worker
-dynos that connect to arbitrary queuing systems.
+The Node version can be found at: https://github.com/mongolab/tractorpush-writer-node
-<p class="callout" markdown="1"> Part I code:
-<a href="https://github.com/mongolab/tractorpush-writer-ruby">https://github.com/mongolab/tractorpush-writer-ruby</a></p>
-<p class="callout" markdown="1"> Part II code:
-<a href="https://github.com/mongolab/tractorpush-server">https://github.com/mongolab/tractorpush-server</a></p>
+### Heroku deploy instructions
-## Overview
+#### Writer app in Ruby
-<img src="http://blog.mongolab.com/wp-content/uploads/2012/03/TailableCursorDiagram.png" width="822" height="401" alt="Overview of TractorPush components."/>
-
-In the rest of the article, you
+Pull from github
-* create the database,
-* connect a writer in Ruby,
-* connect a reader in Node.js,
-* instruct a browser to connect over socket.io.
+```term
+$ git clone https://github.com/mongolab/tractorpush-writer-ruby.git
+$ cd tractorpush-writer-ruby
+```
-First, create a MongoDB collection on MongoLab that serves as a simple
-but flexible message queue. The Ruby application writes documents to
-the collection, and a Node.js application reads the documents on
-demand. The read query in Node.js is initiated by a browser request.
-The query returns a tailable cursor to read documents as they arrive.
-Additionally, the browser negotiates a XHR-long polling process to
-simulate a push of documents to the browser. In effect, the entire
-stack, from database to browser presentation works in a push
-notification manner.
+Create the app on Heroku and add the MongoLab add-on.
-<p class="callout" markdown="1">As of this writing, Heroku does not
-support the newer true push WebSocket protocol. XHR-long polling works acceptably. </p>
+```term
+$ heroku create tp-writer
+$ heroku addons:add mongolab
+```
-<img src="http://blog.mongolab.com/wp-content/uploads/2012/03/TractorPushScreenshot.png" width="841" height="563" alt="End user screenshot of TractorPush demo."/>
+(deprecated) Next, [configure the required MongoDB capped collection](https://devcenter.heroku.com/articles/build-realtime-polyglot-node-ruby-mongodb-socketio-app#configure_mongodb_capped_collection).
-From an end user perspective, a browser window updates as existing
-messages are sent. Once the existing messages are exhausted, the
-browser window is static until new messages arrive at the server. As
-they arrive, the browser window continues to update. For the
-impatient: http://tractorpush.herokuapp.com shows the running
-application.
+Then deploy to Heroku and scale the `worker` process.
-Given MongoDB's schema-free design, our messages can be arbitrarily
-complex JSON. In this demonstration to show the flexibility of the
-object marshalling and unmarshalling, there are three types of
-document-based messages that are pushed through the system:
+```term
+$ git push heroku master
+$ heroku ps:scale worker=1
+$ cd ..
+```
+#### Web server app in Node.js
-* simple (name-value),
-* array,
-* complex (nested documents).
+Pull from github
-To demonstrate filtering of the different types of messages on the
-queue, one area of the screen shows all types of messages: with their
-sequence number and time of creation. In a second side of the screen,
-only complex-type messages are shown.
+```term
+$ git clone https://github.com/mongolab/tractorpush-server.git
+$ cd tractorpush-server
+```
-# Running instance
-A running version of the demo is at: http://tractorpush.herokuapp.com
+Create the app on Heroku.
-The server is node.js running socket.io. It reads from a capped
-MongoDB database with a tailable cursor. A secondary set of servers
-(it happens to be in Ruby) writes to the database.
+```term
+$ heroku create tp-web
+```
-Note: Sequence numbers may not be unique.
+Copy the `MONGOLAB_URI` config var from the message writer app to this one.
-In the online demo, the node.js and Ruby components are running on
-Heroku's Celedon Cedar stack. The MongoDB database is hosted at
-MongoLab. See https://github.com/mongolab/tractorpush-server and https://github.com/mongolab/tractorpush-writer-ruby for more information.
+```term
+$ heroku config:add -a tp-web `heroku config -a tp-writer -s | grep MONGOLAB_URI`
+```
-# HOWTO
+Then deploy to Heroku and open in your browser
-See: (TODO: URL) for detailed instructions on how to run the demo, but succintly:
+```term
+$ git push heroku master
+$ heroku open
+```
-* Create a mongodb database 'testdatabase' with a capped collection 'messages' on localhost OR use Heroku and create a MONGOLAB database.
-* Run tractorpush-writer-ruby to insert files
-* Run 'npm install' Run tractorpush-server (this project): 'node app.js'
-* Point your browser to http://localhost:2000 and enjoy
-
-# Server REQUIRES
-
-* Node.js 0.6.9 +
-* npm 1.1.0-3
-* see package.json for more dependencies (socket.io and mongodb)
-
-
-Original Demo: 2012 March by ObjectLabs Corporation, makers of MongoLab.com hosted MongoDB.
-
-If you have comment or questions please contact us at @mongolab on Twitter or support AT mongolab.com. Thanks for looking.
-
-## Legal stuff
-Copyright 2012 ObjectLabs Corporation.
-
-Code licensed under the Apache License, Version 2.0 (the "Apache
-License"); you may not use this file except in compliance with
-the Apache License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the Apache License is distributed on an "AS IS"
-BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
-implied. See the Apache License for the specific language governing
-permissions and limitations under the Apache License.
-
-Non-code content licensed under Creative Commons
-Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0) at
-http://creativecommons.org/licenses/by-sa/3.0/
View
121 app.js
@@ -6,16 +6,44 @@
//
//
//
-// Copyright 2012 ObjectLabs Corp.
-// isCapped() & intervalEach() is also Copyright 2009-2010 Christian Amor Kvalheim
-// see package.json for attributions
-// ObjectLabs is the maker of MongoLab.com, a cloud, hosted MongoDB service
+
//
+// ObjectLabs is the maker of MongoLab.com a cloud, hosted MongoDb
+// service
+
+// Copyright 2014 ObjectLabs Corp.
+
+// MIT License, except intervalEach()
+
+// Permission is hereby granted, free of charge, to any person
+// obtaining a copy of this software and associated documentation files
+// (the "Software"), to deal in the Software without restriction,
+// including without limitation the rights to use, copy, modify, merge,
+// publish, distribute, sublicense, and/or sell copies of the Software,
+// and to permit persons to whom the Software is furnished to do so,
+// subject to the following conditions:
+
+// The above copyright notice and this permission notice shall be
+// included in all copies or substantial portions of the Software.
+// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+// EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+// BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+// ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+// CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+// SOFTWARE.
-// Licensed under the Apache License, Version 2.0 (the "Apache License");
-// you may not use this file except in compliance with the Apache License.
-// You may obtain a copy of the License at
+//
+// intervalEach() is also Copyright 2009-2010 Christian Amor Kvalheim
+// see package.json for attributions.
+//
+//
+// intervalEach() is licensed under the Apache License, Version 2.0
+// (the "Apache License"); you may not use this file except in
+// compliance with the Apache License. You may obtain a copy of the
+// License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
@@ -24,42 +52,42 @@
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the Apache License for the specific language governing permissions and
// limitations under the Apache License.
+//
//
// NB: I don't provide a durable connection to mongodb that retries on
-// failures. Instead I'm relying on a PaaS's durable restarts.
+// failures. Instead I'm relying on a PaaS's durable restarts, ok
+// really it's a TODO.
//
-var fs = require('fs'),
-url = require('url'),
-emitter = require('events').EventEmitter,
-assert = require('assert'),
+var fs = require("fs"),
+url = require("url"),
+emitter = require("events").EventEmitter,
+assert = require("assert"),
-mongo = require('mongodb'),
-QueryCommand = mongo.QueryCommand,
-Cursor = mongo.Cursor,
-Collection = mongo.Collection;
+mongo = require("mongodb"),
+Cursor = mongo.Cursor;
// Heroku-style environment variables
-var uristring = process.env.MONGOLAB_URI || "mongodb://localhost/testdatabase"
+var uristring = process.env.MONGOLAB_URI || "mongodb://localhost/testdatabase";
var mongoUrl = url.parse (uristring);
//
// Start http server and bind the socket.io service
//
-var app = require('http').createServer(handler), // handler defined below
-io = require('socket.io').listen(app);
+var app = require("http").createServer(handler), // handler defined below
+io = require("socket.io").listen(app);
theport = process.env.PORT || 2000;
app.listen(theport);
console.log ("http server on port: " + theport);
function handler (req, res) {
- fs.readFile(__dirname + '/index.html',
+ fs.readFile(__dirname + "/index.html",
function (err, data) {
if (err) {
res.writeHead(500);
- return res.end('Error loading index.html');
+ return res.end("Error loading index.html");
}
res.writeHead(200);
res.end(data);
@@ -70,7 +98,7 @@ function handler (req, res) {
// Open mongo database connection
// A capped collection is needed to use tailable cursors
//
-mongo.Db.connect (uristring, function (err, db) {
+mongo.MongoClient.connect (uristring, function (err, db) {
console.log ("Attempting connection to " + mongoUrl.protocol + "//" + mongoUrl.hostname + " (complete URL supressed).");
db.collection ("messages", function (err, collection) {
collection.isCapped(function (err, capped) {
@@ -79,7 +107,7 @@ mongo.Db.connect (uristring, function (err, db) {
process.exit(1);
}
if (!capped) {
- console.log (collection + " is not a capped collection. Aborting. Please use a capped collection for tailable cursors.");
+ console.log (collection.collectionName + " is not a capped collection. Aborting. Please use a capped collection for tailable cursors.");
process.exit(2);
}
console.log ("Success connecting to " + mongoUrl.protocol + "//" + mongoUrl.hostname + ".");
@@ -89,21 +117,12 @@ mongo.Db.connect (uristring, function (err, db) {
});
//
-// Bind send action to 'connection' event
+// Bind send action to "connection" event
//
function startIOServer (collection) {
console.log("Starting ...");
- // Many hosted environments do not support all transport forms currently, (specifically WebSockets).
- // So we force a relatively safe xhr-polling transport.
- // Modify io.configure call to allow other transports.
-
- io.configure(function () {
- io.set("transports", ["xhr-polling"]);
- io.set("polling duration", 10);
- io.set("log level", 2);
- });
- io.sockets.on('connection', function (socket) {
+ io.sockets.on("connection", function (socket) {
readAndSend(socket, collection);
});
};
@@ -115,51 +134,29 @@ function startIOServer (collection) {
// (known bug: if there are no documents in the collection, it doesn't work.)
//
function readAndSend (socket, collection) {
- collection.find({}, {'tailable': 1, 'sort': [['$natural', 1]]}, function(err, cursor) {
+ collection.find({}, {"tailable": 1, "sort": [["$natural", 1]]}, function(err, cursor) {
cursor.intervalEach(300, function(err, item) { // intervalEach() is a duck-punched version of each() that waits N milliseconds between each iteration.
if(item != null) {
- socket.emit('all', item); // sends to clients subscribed to type 'all'
+ socket.emit("all", item); // sends to clients subscribed to type "all"
}
});
});
- collection.find({'messagetype':'complex'}, {'tailable': 1, 'sort': [['$natural', 1]]}, function(err, cursor) {
+ collection.find({"messagetype":"complex"}, {"tailable": 1, "sort": [["$natural", 1]]}, function(err, cursor) {
cursor.intervalEach(900, function(err, item) {
if(item != null) {
- socket.emit('complex', item); // sends to clients subscribe to type 'complex'
+ socket.emit("complex", item); // sends to clients subscribe to type "complex"
}
});
});
};
-//
-// Monkey patching mongodb driver 0.9.9-3
-//
-/**
- * Returns if the collection is a capped collection
- *
- * @param {Function} callback returns if collection is capped.
- * @return {null}
- * @api public
- */
-Collection.prototype.isCapped = function isCapped(callback) {
- this.options(function(err, document) {
- if(err != null) {
- callback(err);
- } else if (document == null) { // SEEMS to be a bug? Just hacke it: by testing document==null and punting back an error.
- callback ("Collection.isCapped options document is null.");
- } else {
- callback(null, document.capped);
- }
- });
-};
-
// Duck-punching mongodb driver Cursor.each. This now takes an interval that waits
-// 'interval' milliseconds before it makes the next object request...
+// "interval" milliseconds before it makes the next object request...
Cursor.prototype.intervalEach = function(interval, callback) {
var self = this;
if (!callback) {
- throw new Error('callback is mandatory');
+ throw new Error("callback is mandatory");
}
if(this.state != Cursor.CLOSED) {
View
105 index.html
@@ -1,23 +1,29 @@
<!doctype html>
<!--
- Copyright 2012 ObjectLabs Corp.
- LICENSE
- Code licensed under the Apache License, Version 2.0 (the "Apache
- License"); you may not use this file except in compliance with
- the Apache License. You may obtain a copy of the License at
+ Copyright 2014 ObjectLabs Corp.
- http://www.apache.org/licenses/LICENSE-2.0
+ MIT License
- Unless required by applicable law or agreed to in writing, software
- distributed under the Apache License is distributed on an "AS IS"
- BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied. See the Apache License for the specific language governing
- permissions and limitations under the Apache License.
+ Permission is hereby granted, free of charge, to any person
+ obtaining a copy of this software and associated documentation files
+ (the "Software"), to deal in the Software without restriction,
+ including without limitation the rights to use, copy, modify, merge,
+ publish, distribute, sublicense, and/or sell copies of the Software,
+ and to permit persons to whom the Software is furnished to do so,
+ subject to the following conditions:
- Non-code content licensed under Creative Commons
- Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0) at
- http://creativecommons.org/licenses/by-sa/3.0/
+ The above copyright notice and this permission notice shall be
+ included in all copies or substantial portions of the Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ SOFTWARE.
TITLE
Tractor Push: Node.js, socket.io, Ruby, MongoDB tailed cursor
@@ -26,6 +32,10 @@
to server and displays returned data dynamically.
by Ben Wen, ObjectLabs
+
+ ObjectLabs is the maker of MongoLab.com a cloud MongoDb-as-a-Service
+ provider.
+
-->
<html xmlns='http://www.w3.org/1999/xhtml'>
<head>
@@ -51,10 +61,10 @@
</style>
<title>Tractor Push: MongoDB Tailable Cursor + Socket.IO demo</title>
- <script src='http://ajax.googleapis.com/ajax/libs/dojo/1.7.1/dojo/dojo.js'
+ <script src='/socket.io/socket.io.js'></script>
+ <script src="//ajax.googleapis.com/ajax/libs/dojo/1.8.3/dojo/dojo.js"
data-dojo-config='async: true'>
</script>
- <script src='/socket.io/socket.io.js'></script>
<script>
// require the dom resource
@@ -91,6 +101,12 @@
</script>
</head>
<body>
+ <!-- Background -->
+ <div class="backimg">
+ <img class="fillscreen" alt='Tractor Engine, Copyright 2011 Ben Wen CC
+ BY-SA 3.0' src="http://blog.mongolab.com/wp-content/uploads/2012/03/tractor_engine_1600_Web.jpg"/>
+ </div>
+
<div class="titlebox">
<h1>Tractor Push</h1>
<h2>
@@ -100,7 +116,7 @@
Socket.IO Demo <br/>
from MongoLab
</h2>
- </div>
+ </div>
<div class='container'>
<div class='content' style='top: 0; left: 0'>
<div class='innercontent'>
@@ -127,22 +143,45 @@
</div>
<div class='descript'>
- This is a demonstration of a Socket.IO client that accepts two message streams from a server asynchronously. The left box updates on the arrival of the 'all messages' stream: simple, array, or complex. The right box updates from a message stream of only 'complex' type. The streams are independent but draw from the same source.
-<p> The server is node.js running Socket.IO. It reads from a capped MongoDB database with a tailable cursor. A secondary set of servers (it happens to be in Ruby) originally wrote to the database.
-<p> Note: Sequence numbers may not be unique.
-<p> The node.js and Ruby components are running on Heroku's Celedon Cedar stack. The MongoDB database is hosted at MongoLab. https://github.com/mongolab/tractorpush-server for more information.
-<p> Original Demo: 2012 March
-<p> LICENSE Code licensed under the Apache License, Version 2.0 (the "Apache
- License"); you may not use this file except in compliance with
- the Apache License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
-
-<p> Unless required by applicable law or agreed to in writing, software
- distributed under the Apache License is distributed on an "AS IS"
- BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied. See the Apache License for the specific language governing
- permissions and limitations under the Apache License.
-
-<p> Non-code content licensed under Creative Commons
+ This demonstration of a Socket.IO client accepts two message streams from a server asynchronously. The left box is updated on the arrival of the 'all messages' stream. Messages can be one of three types: simple, array, or complex. The right box is updated from a message stream of only the 'complex' type. The streams are independent but draw from the same source.
+
+<p> The server is running Node.js and uses Socket.IO. It reads from a MongoDB capped collection using a tailable cursor. The tailable cursor allows the server to pick up changes to the database dynamically. A separate writer process writes to the database.
+
+<p> The server components have been tested on Heroku's Celedon Cedar stack and Joyent's Cloud. The MongoDB database is hosted at MongoLab. Code at<br/>
+ <a href="https://github.com/mongolab/tractorpush-server">https://github.com/mongolab/tractorpush-server</a>,<br/>
+ <a href="https://github.com/mongolab/tractorpush-writer-ruby">https://github.com/mongolab/tractorpush-writer-ruby</a>, and<br/>
+ <a href="https://github.com/mongolab/tractorpush-writer-node">https://github.com/mongolab/tractorpush-writer-node</a>.<br/>
+
+
+
+<p> Original Demo: 2012 March. Updated 2013 March. Updated 2014 September.
+<p>
+ Copyright 2012, 2013, 2014 ObjectLabs Corp.
+
+<p> MIT License
+
+
+<p> Permission is hereby granted, free of charge, to any person
+ obtaining a copy of this software and associated documentation files
+ (the "Software"), to deal in the Software without restriction,
+ including without limitation the rights to use, copy, modify, merge,
+ publish, distribute, sublicense, and/or sell copies of the Software,
+ and to permit persons to whom the Software is furnished to do so,
+ subject to the following conditions:
+
+<p> The above copyright notice and this permission notice shall be
+ included in all copies or substantial portions of the Software.
+
+<p> THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ SOFTWARE.
+
+<p> Non-code content licensed under Creative Commons
Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0) at
http://creativecommons.org/licenses/by-sa/3.0/
View
24 package.json
@@ -1,13 +1,15 @@
{
- "name": "TractorPush",
- "description" : "A Node.js, socket.io, and MongoDB tailed cursor push demo.",
- "dependencies": {
- "mongodb": ">=0.9.9",
- "socket.io" : "" },
- "engines" : {
- "node" : ">=0.6.10"},
- "keywords" : ["mongodb", "mongo", "mongolab", "node.js", "node", "socket.io", "WebSockets"],
- "version": "0.9.0",
- "author" : "Ben Wen",
- "contributors" : {"mongodb - node.js driver" : "Christian Amor Kvalheim <christkv@gmail.com>, et. al."}
+ "name": "TractorPush",
+ "description" : "A Node.js, socket.io, and MongoDB tailed cursor push demo.",
+ "dependencies": {
+ "mongodb": "1.4.10",
+ "socket.io" : "1.1.0" },
+ "engines" : {
+ "node" : "0.10.31",
+ "npm" : "1.4.25" },
+ "keywords" : ["mongodb", "mongo", "mongolab", "node.js", "node", "socket.io", "WebSockets"],
+ "version": "0.10.1",
+ "author" : "Ben Wen",
+ "contributors" : {"mongodb - node.js driver" : "Christian Amor Kvalheim <christkv@gmail.com>, et. al.",
+ "numerous valuable additions" : "Ryan Daigle, Heroku"}
}

No commit comments for this range

Something went wrong with that request. Please try again.