Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Tweak message publishing process to be continuous. Update README. #1

merged 2 commits into from Mar 24, 2012
Jump to file or symbol
Failed to load files and symbols.
+26 −125
1 .rvmrc
@@ -0,0 +1 @@
+rvm use ruby-1.9.2@demo.tractorpush-writer --create
124 README.md
@@ -1,117 +1,21 @@
-# Tractor Push: A Real-time Demo with Ruby, Node.js, Socket.io and MongoDB on Heroku Cedar
+This application is the message-publishing portion of the system described in [this Heroku Dev Center article](https://devcenter.heroku.com/articles/build-realtime-polyglot-node-ruby-mongodb-socketio-app).
+The Node.js web-component can be found at: https://github.com/mongolab/tractorpush-server
-Learn how to use three rising infrastructure technologies that put
-real-time interactivity into Web apps: `socket.io`, `Node.js` and
-`MongoDB`. Real-time interactivity examples include chat, large-scale
-games, editing collaboration, and low-latency notification. In a
-market with dynamic, mobile, and social applications, a strong Web
-interface is still an important component to the entire experience.
-To demonstrate cross-app functionality with an existing Ruby
-application, the article first walks through using a separate Heroku
-app to write to a shared MongoDB acting as a simple queue.
-Fortuitously, Heroku’s new Celadon Cedar stack offers several features
-that support these techniques: a polyglot stack and flexible worker
-dynos that connect to arbitrary queuing systems.
+#### Deploy
-<p class="callout" markdown="1"> Part I code:
-<a href="https://github.com/mongolab/tractorpush-writer-ruby">https://github.com/mongolab/tractorpush-writer-ruby</a></p>
-<p class="callout" markdown="1"> Part II code:
-<a href="https://github.com/mongolab/tractorpush-server">https://github.com/mongolab/tractorpush-server</a></p>
+Create the app on Heroku and add the MongoLab add-on.
-## Overview
+$ heroku create -s cedar tp-writer
+$ heroku addons:add mongolab
-<img src="http://blog.mongolab.com/wp-content/uploads/2012/03/TailableCursorDiagram.png" width="822" height="401" alt="Overview of TractorPush components."/>
-In the rest of the article, you
+Next, [configure the required MongoDB capped collection](https://devcenter.heroku.com/articles/build-realtime-polyglot-node-ruby-mongodb-socketio-app#configure_mongodb_capped_collection).
-* create the database,
-* connect a writer in Ruby,
-* connect a reader in Node.js,
-* instruct a browser to connect over socket.io.
+Then deploy to Heroku and scale the `worker` process.
-First, create a MongoDB collection on MongoLab that serves as a simple
-but flexible message queue. The Ruby application writes documents to
-the collection, and a Node.js application reads the documents on
-demand. The read query in Node.js is initiated by a browser request.
-The query returns a tailable cursor to read documents as they arrive.
-Additionally, the browser negotiates a XHR-long polling process to
-simulate a push of documents to the browser. In effect, the entire
-stack, from database to browser presentation works in a push
-notification manner.
-<p class="callout" markdown="1">As of this writing, Heroku does not
-support the newer true push WebSocket protocol. XHR-long polling works acceptably. </p>
-<img src="http://blog.mongolab.com/wp-content/uploads/2012/03/TractorPushScreenshot.png" width="841" height="563" alt="End user screenshot of TractorPush demo."/>
-From an end user perspective, a browser window updates as existing
-messages are sent. Once the existing messages are exhausted, the
-browser window is static until new messages arrive at the server. As
-they arrive, the browser window continues to update. For the
-impatient: http://tractorpush.herokuapp.com shows the running
-Given MongoDB's schema-free design, our messages can be arbitrarily
-complex JSON. In this demonstration to show the flexibility of the
-object marshalling and unmarshalling, there are three types of
-document-based messages that are pushed through the system:
-* simple (name-value),
-* array,
-* complex (nested documents).
-To demonstrate filtering of the different types of messages on the
-queue, one area of the screen shows all types of messages: with their
-sequence number and time of creation. In a second side of the screen,
-only complex-type messages are shown.
-# Running instance
-A running version of the demo is at: http://tractorpush.herokuapp.com
-The server is node.js running socket.io. It reads from a capped
-MongoDB database with a tailable cursor. A secondary set of servers
-(it happens to be in Ruby) writes to the database.
-Note: Sequence numbers may not be unique.
-In the online demo, the node.js and Ruby components are running on
-Heroku's Celedon Cedar stack. The MongoDB database is hosted at
-MongoLab. See https://github.com/mongolab/tractorpush-server and https://github.com/mongolab/tractorpush-writer-ruby for more information.
-See: (TODO: URL) for detailed instructions on how to run the demo, but succintly:
-* Create a mongodb database 'testdatabase' with a capped collection 'messages' on localhost OR use Heroku and create a MONGOLAB database.
-* Run tractorpush-writer-ruby to insert files
-* Run 'npm install' Run tractorpush-server (this project): 'node app.js'
-* Point your browser to http://localhost:2000 and enjoy
-# Server REQUIRES
-* Node.js 0.6.9 +
-* npm 1.1.0-3
-* see package.json for more dependencies (socket.io and mongodb)
-Original Demo: 2012 March by ObjectLabs Corporation, makers of MongoLab.com hosted MongoDB.
-If you have comment or questions please contact us at @mongolab on Twitter or support AT mongolab.com. Thanks for looking.
-## Legal stuff
-Copyright 2012 ObjectLabs Corporation.
-Code licensed under the Apache License, Version 2.0 (the "Apache
-License"); you may not use this file except in compliance with
-the Apache License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
-Unless required by applicable law or agreed to in writing, software
-distributed under the Apache License is distributed on an "AS IS"
-implied. See the Apache License for the specific language governing
-permissions and limitations under the Apache License.
-Non-code content licensed under Creative Commons
-Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0) at
+$ git push heroku master
+$ heroku ps:scale worker=1
@@ -37,11 +37,11 @@
require 'mongo'
require 'pp'
-uristring = ENV['MONGOLAB_URI'] || 'mongodb://localhost/testdatabase'
-debug = ENV['WRITER_DEBUG'] || "false"
-rate = ENV['WRITER_RATE'].to_f
-if (rate = 0.0) then rate = 10.0 end
+STDOUT.sync = true # Write in real-time
+uristring = ENV['MONGOLAB_URI'] || 'mongodb://localhost/testdatabase'
+debug = ENV['WRITER_DEBUG'] ? ENV['WRITER_DEBUG'] == "true" : false
+rate = ENV['WRITER_RATE'] ? ENV['WRITER_RATE'].to_i : 1 # Every second, write a message
uri = URI.parse(uristring)
conn = Mongo::Connection.from_uri(uristring)
@@ -53,17 +53,13 @@
{'messagetype' => 'array', 'ordinal' => 0, 'somearray' => ['a', 'b', 'c', 'd']},
{'messagetype' => 'complex', 'ordinal' =>0, 'subdocument' => {'fname' => 'George', 'lname' => 'Washington', 'subproperty' => 'US-president'}}]
-count = 500
-print("starting insertion of ", count, " documents into ", uri.scheme, "://", uri.host, uri.path, "\n")
-for i in 1..count
+# Run until killed
+i = 0
doc = docs[rand(3)].dup # MongoDB collection.insert mutates document, editing the _id key; we need a deep dup (copy).
doc['time'] = Time.now().to_f * 1000 # Switch to Javascript's convention
- doc['ordinal'] = i
+ doc['ordinal'] = i + 1
coll.insert(doc, :safe => true)
- if (debug == "true") then pp (doc) end
- sleep(1.0/rate)
-print("Finished. Inserted ", count, " messages.\n")
+ debug ? pp(doc) : puts("Inserting #{doc['messagetype']} message")
+ sleep(rate)