Skip to content

Commit

Permalink
fix remaining links
Browse files Browse the repository at this point in the history
  • Loading branch information
mizzao committed Jun 19, 2016
1 parent 0ba1601 commit 998fae9
Show file tree
Hide file tree
Showing 19 changed files with 60 additions and 57 deletions.
4 changes: 2 additions & 2 deletions README.md
Expand Up @@ -7,8 +7,8 @@ please post an [issue][is]. This will help us improve documentation and general
information for prospective future experimenters. Better yet, suggest edits
and send a [pull request][pr] (see below).

[is]: https://github.com/TurkServer/turkserver.github.io/issues
[pr]: https://github.com/TurkServer/turkserver.github.io/pulls
[is]: https://github.com/TurkServer/docs/issues
[pr]: https://github.com/TurkServer/docs/pulls

# Getting Started

Expand Down
3 changes: 3 additions & 0 deletions source/api.md
@@ -0,0 +1,3 @@
# API Documentation
The TurkServer API is built automatically from the code and can be accessed [here](https://turkserver.meteorapp.com/).
3 changes: 3 additions & 0 deletions source/arch/admin-console.md
Expand Up @@ -20,6 +20,9 @@ A brief overview of the different sections:
- **Experiments**: shows experiment instances, their participants, and timing information.
- **Manage**: controls batch and treatment data which are used to configure experiments.

We aim to provide more information about the interface as its design is
finalized. For now, the admin interface is mostly self-explanatory.

<!--
Batches and treatments can be viewed and edited from the
administration interface.
Expand Down
11 changes: 2 additions & 9 deletions source/arch/alternatives.md
Expand Up @@ -6,8 +6,8 @@ experiments. Here are a list of the ones we know that are currently active.
The main differentiation of TurkServer is relative to the platforms and
frameworks below is its focus on facilitating and instrumenting real-time and
synchronous interaction. But we've kept the software simple, and you'll find
features such as the [live experimenter console](admin-console) and [one-way
mirror](mirror) useful even for single-user experiments.
features such as the [live experimenter console](admin-console.md) and [one-way
mirror](mirror.md) useful even for single-user experiments.

- [Breadboard][breadboard] focuses on network and cooperation experiments.
- [CogniLab][cognilab] is a paid service for launching online experiments, no
Expand Down Expand Up @@ -42,10 +42,3 @@ that may supplement or replace Mechanical Turk in the future.
[prolific]: https://prolific.ac/
[crowdsourcing]: https://www.quora.com/Are-there-any-similar-services-to-Amazon-Mechanical-Turk

<!--
Other references, stuff that is not well supported
[SoPHIE][sophie]
[sophie]: http://www.sophie.uni-osnabrueck.de/
http://cogsci.stackexchange.com/questions/109
-->
10 changes: 5 additions & 5 deletions source/arch/architecture.md
Expand Up @@ -12,14 +12,14 @@ Why is TurkServer designed this way, and what are the core concepts?

- Read [why we chose Meteor](why-meteor.md) and Javascript as the
underlying framework, and why this facilitates social experiments research.
- Understand the [multiple worlds abstraction](worlds-assignment), its relation
- Understand the [multiple worlds abstraction](world-assignment.md), its relation
to Meteor's real-time data framework, and how this makes it easier to design
apps for group interaction.
- Check out the [live experimenter view](admin-console) in TurkServer, which
- Check out the [live experimenter view](admin-console.md) in TurkServer, which
allows you to see the status of all connected users and ongoing experiments.
- Take a look at how [treatments](treatments) can be assigned to users or
- Take a look at how [treatments](treatments.md) can be assigned to users or
worlds in TurkServer.
- Look at [research methods](research-methods) that guided the development of
- Look at [research methods](research-methods.md) that guided the development of
TurkServer.
- See references to [alternative platforms](alternatives) and concepts for
- See references to [alternative platforms](alternatives.md) and concepts for
doing software-based, crowdsourcing-driven experiments.
5 changes: 2 additions & 3 deletions source/arch/why-meteor.md
Expand Up @@ -26,14 +26,13 @@ synchronizing data across multiple clients, making social, collaborative apps
public is often a lot of work for those unfamiliar with devops, and ease of
deployment means you can get on with your research.

[deploy]: http://guide.meteor.com/deployment.html#galaxy

- **Active and growing community of developers**. Meteor is one of the top 10
frameworks on GitHub, and is built on the widely popular NodeJS. Building an
app with TurkServer leverages these communities as much as possible.
Thus, you can find answers to many common issues on sites like
[StackOverflow][so] and [Meteor's forums][forums].


[deploy]: http://guide.meteor.com/deployment.html#galaxy
[so]: http://stackoverflow.com/
[forums]: https://forums.meteor.com/

Expand Down
2 changes: 1 addition & 1 deletion source/arch/world-assignment.md
Expand Up @@ -4,4 +4,4 @@ TurkServer uses the concept of **batches** to logically group
instances of experiments together. Each batch limits repeat
participation.

Batch controls the assignment of incoming users to an [assigner](assignment-matching).
Batch controls the assignment of incoming users to an [assigner](../design/assigning.md).
4 changes: 2 additions & 2 deletions source/design/debugging.md
Expand Up @@ -7,9 +7,9 @@ reconnections, and so on.
Use the MTurk Sandbox for end-to-end testing.

Think everything's ready? The ultimate test of your app is to conduct a
[pilot](pilot).
[pilot](../launching/pilot.md).

# Notes and Troubleshooting
## Code Troubleshooting

Because TurkServer runs alongside your app on both the server and client,
strange behavior can occur when writing code without thoughtfulness. While we've
Expand Down
24 changes: 12 additions & 12 deletions source/design/design.md
Expand Up @@ -19,18 +19,18 @@ investigate whether your experiment makes sense. Here are some examples:
hundred people, one unit was two players in a repeated game.

2. If the design is promising, add TurkServer to your app, and integrate with
its [assignment mechanism](worlds-assignment), which will allow you to make
its [assignment mechanism](../arch/world-assignment.md), which will allow you to make
different instances of your prototype world and control how participants
enter and leave them.

3. Test, debug, and pilot your experiment extensively, making sure that your
[code works as intended](debugging) and your [instructions are easy to
understand](good-instructions). Start with your research team, then others in
[code works as intended](debugging.md) and your [instructions are easy to
understand](good-instructions.md). Start with your research team, then others in
your lab/department, and finally run a pilot with real crowdsourcing workers.
Experiments inevitably involve many redesigns and modifications, and it's
better to do this in testing than to shoot from the hip.

4. [Run your experiment](launching)!
4. [Run your experiment](../launching/launching.md)!

5. Share the source code of your app so others can look at your experiment
protocol, and replicate or build on it.
Expand All @@ -52,23 +52,23 @@ conditions a user passes through as they participate in a group task:

Among other things, the diagram illustrates:

- A [matching mechanism](assignment-matching) to group people together as
- A [matching mechanism](assigning.md) to group people together as
they are waiting in the lobby.
- Providing [good instructions](good-instructions) and using a comprehension
- Providing [good instructions](good-instructions.md) and using a comprehension
task to check that they are understood.
- Accounting for the possibility of [disconnection and dropout](disconnection)
- Accounting for the possibility of [disconnection and dropout](disconnection.md)
in your design, both to minimize the amount of data that might be discarded
and ensure a good experience for all remaining participants.

See also the following topics:

- Constructing a [one-way mirror](mirror) to qualitatively observe activity
- Constructing a [one-way mirror](mirror.md) to qualitatively observe activity
and diagnose potential issues.
- How to [debug and test your app](debugging) before running it live.
- How to [minimize attrition](attrition) in experiments that run over a long
- How to [debug and test your app](debugging.md) before running it live.
- How to [minimize attrition](attrition.md) in experiments that run over a long
period of time or over multiple days.
- Helpful [packages and software](software) that you may want to use in
- Helpful [packages and software](software.md) that you may want to use in
your app.
- Other [frequently asked questions](faq) about building experiments.
- Other [frequently asked questions](faq.md) about building experiments.


2 changes: 1 addition & 1 deletion source/design/faq.md
Expand Up @@ -31,7 +31,7 @@ interaction with the participants. There is much less variation than in
traditional brick-and-mortar experiments, as much of this is encapsulated in the
software. However, there are still exceptions, a notable one being the
importance of [engaging with workers and maintaining your
reputation](forums-reputation). As a result, it's very difficult to run a
reputation](../launching/forums-reputation.md). As a result, it's very difficult to run a
software-based experiment without understanding how the user interface and the
rest of the app was built. Either you'll need to be familiar with it anyway, or
your developer will also need to be a "lab manager" and run the experiment.
Expand Down
4 changes: 2 additions & 2 deletions source/examples/examples.md
Expand Up @@ -2,7 +2,7 @@

## Basic Examples

The basic [tutorial](tutorial) describes how to build a simple, but full
The basic [tutorial](tutorial.md) describes how to build a simple, but full
fledged example app ([see source code](https://github.com/TurkServer/tutorial).)
Take a look at this app as a working starting point for something you can
deploy online.
Expand All @@ -12,6 +12,6 @@ deploy online.
Although studies of individual behavior are desirable and easy to build,
TurkServer really shines for studying groups, especially for complex
interactions or social behavior among many people over time. For ideas,
inspiration, or just examples, take a look at some [research projects](research)
inspiration, or just examples, take a look at some [research projects](research.md)
that have been built using TurkServer.

8 changes: 8 additions & 0 deletions source/examples/hidden.md
Expand Up @@ -42,3 +42,11 @@ conditions where users can communicate or interact with each other in addition
to choosing how much to contribute.

[pgg-meteor]: https://github.com/cosanlab/PGG_meteor

<!--
Other references, stuff that is not well supported
[SoPHIE][sophie]
[sophie]: http://www.sophie.uni-osnabrueck.de/
http://cogsci.stackexchange.com/questions/109
-->
4 changes: 2 additions & 2 deletions source/examples/research.md
Expand Up @@ -28,7 +28,7 @@ experiment, conducted daily over a *month*, collecting an order of magnitude

## Your paper here!

Want more citations and your work on this page? Add your project here and send us
Show your work on this page. Add your project here and send us
a [pull request][pr]!

[pr]: https://github.com/TurkServer/turkserver.github.io/pulls
[pr]: https://github.com/TurkServer/docs/pulls
13 changes: 6 additions & 7 deletions source/examples/tutorial.md
Expand Up @@ -3,8 +3,7 @@
This tutorial walks you through building a simple, but fully functional app
that you can deploy and that demonstrates many of the features of TurkServer.
If you'd prefer to start hacking from a working example, you can start
poking around [the code][1] directly and follow the [deployment
instructions](quickstart) to get it online.
poking around [the code][1] directly and follow the [deployment instructions](../quick-start.md) to get it online.

[1]: https://github.com/TurkServer/tutorial

Expand All @@ -14,7 +13,7 @@ that you can deploy and that demonstrates many of the features of TurkServer.

For the purposes of this tutorial, we will use the default Meteor "counter" app that comes bundled with every installation. To generate the necessary files, simply type `meteor create app`. To run the app, `cd app` and `meteor`. Then open a browser and navigate to `http://localhost:3000/` to verify that all is working properly. You should see the following:

![screenshot]/img/clickMe.png)
![screenshot](/img/clickMe.png)

Before moving on, don't forget to `meteor remove insecure` and `meteor remove autopublish`.

Expand Down Expand Up @@ -511,9 +510,9 @@ So we've set the bonus value, but we haven't yet approved the worker's assignmen

## Deployment

To continue with the tutorial, follow the [deployment instructions]
(quickstart) so that you can use the actual MTurk API and get the app running
in the MTurk sandbox.
To continue with the tutorial, follow the [deployment instructions](../quick-start.md)
so that you can use the actual MTurk API and get the app running in the MTurk
sandbox.

## HITs and HITTypes

Expand Down Expand Up @@ -557,7 +556,7 @@ At this point we're all set to create our HIT. Click on "HITs" in the navigation
-->

The MTurk Sandbox allows you to test and debug your app without spending real
money. See the [deployment instructions](quickstart) for how to post a HIT
money. See the [deployment instructions](../quick-start.md) for how to post a HIT
and pay your workers.


Expand Down
1 change: 1 addition & 0 deletions source/index.rst
Expand Up @@ -17,6 +17,7 @@ The main documentation for the site is organized into a couple sections:

intro
quick-start
api

.. _example-docs:

Expand Down
10 changes: 6 additions & 4 deletions source/launching/checklist.md
Expand Up @@ -13,30 +13,32 @@ This is a checklist of common things we've done before running experiments.
You should copy it somewhere and augment it with things that are specific to
your experiment.

### General Tasks

- Make sure any [post-experiment cleanup](post-experiment) from your last
session is all done

- Make sure the code on your deployment server is up to date

- Make sure there is enough money in your account

**Email forwarding setup:**
### **Email forwarding setup:**

* Make sure that email forwarding on the account used to make the HIT (more than likely lab dartmouth email) because Turkers *will* contact you during a HIT if something goes wrong
* It's a good idea to have an email signature to appear more professional

**Auto Assignment approval setup:**
### **Auto Assignment approval setup:**

* Make sure that you have some reasonable timer Assignment auto-approval (e.g. 1hr - 24hrs is probably reasonable)
* This is important in case you need to wipe a database or debug an issue and no longer have access to completed assignments in Turkserver

**HIT expiration setup (called Lifetime in TurkServer):**
### **HIT expiration setup (called Lifetime in TurkServer):**

- Make sure that you have a reasonable timer for when a HIT expires (this entirely depends on the nature of your study), i.e. how long it's visible for workers to be able to accept it
- Simple things might be just 24hrs or less (it's probably a good idea to run small batches at different times rather than a large batch all at once for multi-turker matching purposes and to more uniformly sample the Turker pool)
- This is also important in case you run into issues as noted above

**Assignment duration**
### **Assignment duration**

* Make sure to set this to something a little more than the length of your study
* This way if there are problems the a Turker's assignment will automatically end when this time is reached
2 changes: 1 addition & 1 deletion source/launching/launching.md
Expand Up @@ -2,7 +2,7 @@

- When you finally launch your meteor experiment rejoice for you have earned a long-rest and a vacation on the beach
- Actually don't because things aren't as "automatic" as they may seem
- You still need to [manage your active experiment](active-management) to
- You still need to [manage your active experiment](live-management.md) to
deal with
users in real time
- As of this writing (9/24/15) it seems far more advantageous to launch many "small" HITs instead of several large ones
Expand Down
2 changes: 1 addition & 1 deletion source/launching/live-management.md
Expand Up @@ -74,7 +74,7 @@
- Under Manage HITs individually you should see the "Remaining Assignments" value along with other info about your HIT
- This is the "gold standard" regardless of what Turkserver says because it's what Turkers will be able to see as well

### Troubleshooting
# Troubleshooting

- If something is going wrong you should immediately freak out! (ask Eshin he know's all about this)
- Actually don't that. Just figure out what's up by checking the following:
Expand Down
5 changes: 0 additions & 5 deletions source/quick-start.md
Expand Up @@ -380,8 +380,3 @@ Now you can modify the demo, turn off sandbox in settings and launch your own ex

Enjoy your new research experience using TurkServer!

## ***Reference***

Turkserver v0.5.0 API Reference: See [here](https://turkserver.meteorapp.com/)

If you have any question: please contact the developer [here](https://kgao.github.io/#contact)

0 comments on commit 998fae9

Please sign in to comment.