more rewriting and updates

source/arch/admin-console.md

@@ -20,8 +20,8 @@ A brief overview of the different sections:
 - **Experiments**: shows experiment instances, their participants, and timing information.
 - **Manage**: controls batch and treatment data which are used to configure experiments.
 
-We aim to provide more information about the interface as its design is
-finalized. For now, the admin interface is mostly self-explanatory.
+The admin interface is mostly self-explanatory. We aim to provide more
+information about the details of using the interface as its design is finalized.
 
 <!--
 Batches and treatments can be viewed and edited from the

source/arch/treatments.md

@@ -1,7 +1,23 @@
 # Treatments 
 
-Treatments can have structured data which are made
-available to the front-end app under `TurkServer.treatment()`, making
-them a useful way to control the display of different parts of the
-user interface or app behavior. They can be defined for batches,
-users, or worlds.
+Being able to show users different instructions, interfaces, or information is
+an important part of conducting controlled experiments. 
+
+TurkServer accomplishes this through **treatments**, containing structured data
+that are made available to the front-end app under `TurkServer.treatment()` as a
+[reactive variable][rv] in Meteor.
+
+[rv]: http://guide.meteor.com/data-loading.html#stores 
+
+Treatments contain arbitrary key-value pairs, and can be assigned to both users
+and worlds. The client code for a user in a particular world will have access to
+all of the treatment data for both the user and the world, creating a useful way
+to programming control the display of different parts of the user interface
+(such as instructions) and mechanisms of interaction. Since treatment data is
+reactive, it is easy to use with [Blaze], Meteor's UI rendering system.
+
+[blaze]: http://guide.meteor.com/blaze.html
+
+Treatments can also be viewed in the [admin interface](admin-console.md).
+
+Additional details and examples of treatments to be added.

source/arch/world-assignment.md

@@ -1,7 +1,37 @@
 # Worlds and Assignment 
 
-TurkServer uses the concept of **batches** to logically group
-instances of experiments together. Each batch limits repeat
-participation.
+TurkServer uses Meteor's [powerful publication-based data model][meteor-pubsub]
+to create multiple *worlds* or *instances* within which data can be synchronized
+among a group of users in real time. This makes it easy to build and study
+mechanisms of real-time interaction. However, the real utility of this
+abstraction is the ability to create new worlds and assign or re-assign users
+among them, allowing for flexibility of many experiment designs while keeping
+programming simple.     
 
-Batch controls the assignment of incoming users to an [assigner](../design/assigning.md). 
+[meteor-pubsub]: http://guide.meteor.com/data-loading.html
+
+TurkServer uses the concept of **batches** to logically group instances of
+experiments together. Each batch allows for some high-level parameters to be
+set, such as a limit on repeat participation.
+
+Each batch has a **lobby** or waiting area, a state that users are in when they
+are not part of any world. An **[assigner](../design/assigning.md)** controls
+the state of the waiting area and therefore can arrange arriving or returning
+users to different worlds.
+
+A world can return users to the lobby once a task is complete, allowing for
+users to be assigned to different groups, matched with different partners, and
+so on. The assigner can also send users to an **exit survey** to complete the
+task and debrief.
+
+## Examples
+
+To be completed.
+
+## Additional References
+
+TurkServer's multiple worlds abstraction is implemented by the [partitioner]
+package in Meteor, which was written specifically for TurkServer.
+
+[partitioner]: https://github.com/mizzao/meteor-partitioner 
+

source/design/assigning.md

@@ -1,11 +1,41 @@
 # Assignment and Matching
 
-### Matching Issues:
+Experiments with individual users are simple and each participant can proceed
+independently. Use the [`SimpleAssigner`][sa] to create this common form of
+experiment in TurkServer, but gain all the benefits of the live experimenter
+view.
+
+[sa]: http://turkserver.meteorapp.com/#Assigners-SimpleAssigner
 
-- It's important to make sure Turkers aren't waiting an inordinate amount of time for another user to arrive before the experiment starts
-- While this doesn't happen too often, it **will** happen at least 10% of the time **especially** if your HIT has been live for a while 
-- One solution is to monitor the amount of the time a user has spent in the lobby and if it exceeds some threshold, push them to the TS Exit survey and pay them for their time (e.g. half the HIT amount)
-    - This sucks because we have to pay for 0 data, but we can write it off as the cost of saving lab reputation 
-* Another solution for users that do wait an exceedingly long time and eventually complete your experiment is to pay them a bonus for being patient
+When matching users together, however, the precise details of the matching
+mechanism become an important part of the experiment design. This is because
+users who are waiting an inordinate amount of time for partner(s) to arrive to
+start an experiment can either get frustrated, or go on to another task and
+become inactive. This can aggravate problems of [disconnection and dropout](disconnection.md)
+ and result in worse quality data.
 
-- When running really large groups, schedule people in advance.
+Depending on how you post your HITs and how attractive they are, this may happen
+rarely. However, it's still possible, especially if your HIT has been posted
+for a while and the supply of workers is beginning to dwindle. Here are a few
+ways to work around this issue:
+
+- Configure your [assigner](assigning.md) to to monitor the amount of the time
+users are spending in the lobby. If it exceeds some threshold, allow them to
+submit the task and pay them for their time (e.g. half the HIT amount). Although
+this can be viewed as paying something for nothing, it's important to [preserve the reputation](../launching/forums-reputation.md)
+ of your requester account in order to recruit workers.
+
+- For users that do wait an exceedingly long time and eventually complete your
+experiment, consider paying them a bonus for their patience.
+
+- When running really large groups, pre-recruit participants with an
+informational task and schedule a participation session in advance. TurkServer
+provides functions to e-mail users for this purpose. Then, you can expect all
+users to arrive at once and be ready to participate.
+
+## Assigner Examples
+
+To be added. For now, take a look at some [source code][src] for inspiration.
+
+[src]: https://github.com/TurkServer/turkserver-meteor/tree/master/server
+

source/design/attrition.md

@@ -1,5 +1,13 @@
 # Minimizing Attrition
 
+- How to [minimize attrition](attrition.md) in experiments that run over a long 
+period of time or over multiple days.
+
+Attrition in your experiments will appear as disconnected users in TurkServer
+who have not completed their task. This will result in incomplete data and
+likely unhappy participants. Attrition can further cause problems when users are
+matched with partners. 
+
 - Set expectations for involvement
 - Financial incentives
 - Soft incentives 

source/design/debugging.md

@@ -1,10 +1,15 @@
 # Debugging and Testing
 
-Debug your app thoroughly in local mode before launching on MTurk.
-Grab some friends or use multiple browsers. Test things like dropouts, 
-reconnections, and so on.
+Debug and test your app thoroughly before launching with real participants. We
+recommend going through the following steps:
 
-Use the MTurk Sandbox for end-to-end testing.
+1. Grab some friends and have them play with your app. Are the instructions
+clear? Is it obvious what a participant should do?
+2. Test your app using multiple browsers in a local machine. Try dropping out of
+your experiment and seeing how your code handles it. Reconnect to your
+experiment (within a short amount of time) and ensure that data isn't lost.
+3. Use the MTurk Sandbox for end-to-end testing, and make sure that the HIT can
+be submitted correctly and is collecting all the data that you need.
 
 Think everything's ready? The ultimate test of your app is to conduct a 
 [pilot](../launching/pilot.md). 

source/design/design.md

@@ -64,9 +64,7 @@ See also the following topics:
 
 - Constructing a [one-way mirror](mirror.md) to qualitatively observe activity 
 and diagnose potential issues.
-- How to [debug and test your app](debugging.md) before running it live. 
-- How to [minimize attrition](attrition.md) in experiments that run over a long 
-period of time or over multiple days.
+- How to [debug and test your app](debugging.md) before running it live.
 - Helpful [packages and software](software.md) that you may want to use in 
 your app.
 - Other [frequently asked questions](faq.md) about building experiments. 

source/design/disconnection.md

@@ -2,9 +2,11 @@
 
 Users may disconnect and reconnect to your experiment, or leave without 
 coming back. A good experiment design accounts for this and minimizes the 
-disruption to other users and the quality of the data.
+disruption to other users and preserves the quality of the data.
 
-### Connection Issues:
+## Examples
+
+### Handling disconnections
 
 - Your project code should keep tabs on the connection state of each user and should have some kind of connection timer that prevents one user from getting "stuck" if their partner(s) disconnect
 - Likewise if rematching users is important, make sure to build that into your project code and critically set it up in a way that your Assigner knows what to do, for example:
@@ -14,7 +16,7 @@ disruption to other users and the quality of the data.
     - If it doesn't see that flag it assumes they have completed their experiment instance and pushes them to the TS Exit survey 
     - If it doesn't see any experiment instances it assumes this is their first time in the lobby and leaves them there to get matched 
 
-### Idling Issues:
+### Handling idle users
 
 - Your project should probably deal with users that take exceedingly long to respond
 - One solution is to setup the Assignment duration so that that a user must complete it within a certain time otherwise risk being unable to submit

source/design/good-instructions.md

@@ -1,9 +1,20 @@
 # Providing Good Instructions
 
-### Social Issues:
+When running experiments with crowdsourcing systems, users benefit greatly from
+good instructions and will be more attentive to the task. Poorly worded or
+incomprehensible instructions will result in bad quality data. The [tutorials][tutorials]
+package for Meteor was written alongside TurkServer to provide interactive instructions for
+how to use a Meteor app, with this use case in mind.
 
-* It's obviously important that Turkers understand your experiment and don't simple "click-through" to get their payment
-* Instituting comprehension questions that prevent a user from advancing or moving past a certain point in your meteor app is a good way to filter out this kind of behavior
-* [tutorials][1]
+[![tutorials example](https://j.gifs.com/W6QO44.gif)](https://www.youtube.com/watch?v=smax46TNPPk)
+
+When giving instructions, ensure that participants will understand your
+experiment and can't simply "click through" to get paid. A good way to filter
+out this kind of behavior is to provide comprehension questions that prevent a
+user from advancing or moving past a certain point in your meteor app without
+understanding the task. The tutorials package makes it easier to implement these
+ideas.
 
-[1]: https://github.com/mizzao/meteor-tutorials 
+For more information, see the documentation for the [tutorials] package.
+
+[tutorials]: https://github.com/mizzao/meteor-tutorials 

source/design/software.md

@@ -1,6 +1,16 @@
 # Helpful Software
 
-Meteor already makes it pretty easy to design a reactive and responsive user interface, but you may find some of the following packages useful.
+Meteor already makes it pretty easy to design a reactive and responsive user
+interface, but you may find some of the following packages useful. For the most
+part, Meteor makes the process of [installing packages][install-packages] in an
+app a one-line command. TurkServer itself is a Meteor package.
+
+[install-packages]: http://guide.meteor.com/using-atmosphere-packages.html
 
 - [Bootstrap](http://getbootstrap.com/), a CSS framework for front-end development.
-- [Tutorials](https://github.com/mizzao/meteor-tutorials), a Meteor-specific tutorials package that I wrote for providing interactive and concise instructions for web apps. Very useful for delivering easily digestible experiment instructions.
+- [Tutorials](https://github.com/mizzao/meteor-tutorials), a Meteor-specific
+tutorials package that for providing interactive and concise instructions for
+web apps. It is very useful for delivering easily digestible experiment
+instructions.
+
+Add your links to helpful packages here!

source/index.rst

@@ -1,7 +1,11 @@
 Welcome to Turkserver's documentation!
 ======================================
 
-The main documentation for the site is organized into a couple sections:
+This documentation is being updated continually and is subject to change. If you
+see anything missing, use the **Edit on GitHub** button at the top right of
+every page to suggest improvements.
+
+This site is organized into a few sections:
 
 * :ref:`intro-docs`
 * :ref:`example-docs`
@@ -47,15 +51,14 @@ The main documentation for the site is organized into a couple sections:
 
 .. toctree::
    :maxdepth: 2
-   :caption: Designing Experiments
+   :caption: Building Experiments
 
    design/design
    design/assigning
    design/good-instructions
    design/disconnection
    design/mirror
    design/debugging
-   design/attrition
    design/software
    design/faq
 

source/launching/forums-reputation.md

@@ -1,16 +1,13 @@
 # Working with Workers
 
-Best crowdsourcing practices: 
-http://wiki.wearedynamo.org/index.php/Guidelines_for_Academic_Requesters
-
 - Treat workers like people. They have a community, and talk to each other.
-- Active worker forums
+- Workers are [active on many forums][ygsv16]. They communicate there, and you can communicate with them!
 - Monitor your reputation on [TurkOpticon][to]. A good reputation means that 
 you will be able to recruit more workers and they will be more diligent.
+- [WeAreDynamo] is a community of workers aiming to improve practices on MTurk.  
+- Consider [best practices][bp] for crowdsourcing. 
 
+[ygsv16]: http://people.seas.harvard.edu/~myin/WWW-16/turker_network1.pdf 
 [to]: https://turkopticon.ucsd.edu 
-
-- http://www.wearedynamo.org/
-- [best practices][bp] 
-
+[WeAreDynamo]: http://www.wearedynamo.org/
 [bp]: http://wiki.wearedynamo.org/index.php/Guidelines_for_Academic_Requesters

source/launching/launching.md

@@ -1,5 +1,7 @@
 # Launching Experiments
 
+This section of the guide is rough, and we could use your help to clean it up.
+
 - When you finally launch your meteor experiment rejoice for you have earned a long-rest and a vacation on the beach
 - Actually don't because things aren't as "automatic" as they may seem
 - You still need to [manage your active experiment](live-management.md) to