DOC: documentation for counterbalancing (offline and online) and debugging online

docs/source/builder/blocksCounterbalance.rst

@@ -5,22 +5,21 @@ Blocks of trials and counterbalancing
 
 Many people ask how to create blocks of trials, how to randomise them, and how to counterbalance their order. This isn't all that hard, although it does require a bit of thinking!
 
-Blocking
+Blocking similar conditions
 ~~~~~~~~~~~~~
 
 The key thing to understand is that you should not create different Routines for different trials in your blocks (if at all possible). Try to define your trials with a single Routine. For instance, let's imagine you're trying to create an experiment that presents a block of pictures of houses or a block of faces. It would be tempting to create a Routine called `presentFace` and another called `presentHouse` but you actually want just one called `presentStim` (or just `trial`) and then set that to differ as needed across different stimuli.
 
-This example is included in the Builder demos, as of PsychoPy 1.85.
+This example is included in the Builder demos, as of PsychoPy 1.85, as "images_blocks".
 
 You can add a loop around your trials, as normal, to control the trials within a block (e.g. randomly selecting a number of images) but then you will have a second loop around this to define how the blocks change. You can also have additional Routines like something to inform participants that the next block is about to start.
 
 .. image:: blocksImagesFlow.png
 
 So, how do you get the block to change from one set of images to another? To do this create three spreadsheets, one for each block, determining the filenames within that block, and then another to control which block is being used:
 
-  - facesBlock.xlsx
-  - housesBlock.xlsx
-  - chooseBlocks.xlsx
+.. image:: blocksImagesCondFiles.png
+        :scale: 50 %
 
 **Setting up the basic conditions.** The facesBlock, and housesBlock, files look more like your usual conditions files. In this example we can just use a variable `stimFile` with values like `stims/face01.jpg` and `stims/face02.jpg` while the housesBlock file has `stims/house01.jpg` and `stims/house02.jpg`. In a real experiment you'd probably also have response keys andsuchlike as well.
 
@@ -30,39 +29,68 @@ Your `chooseBlocks.xlsx` can contain other values as well, such as useful identi
 
 Variables that are defined in the loops are available anywhere within those. In this case, of course, the values in the outer loop are changing less often than the values in the inner loop.
 
-Counterbalancing
+Counterbalancing similar conditions
 ~~~~~~~~~~~~~~~~~~~~
 
-Counterbalancing is simply an extension of blocking. Usually with a block design you would set the order of blocks to be set randomly. In the example above the blocks are set to occur randomly, but note that they could also be set to occur more than once if you want 2 repeats of the 2 blocks for a total of 4.
+Counterbalancing is simply an extension of blocking. Until now, we have a *randomised block design*, where the order of blocks is set to random. At the moment we also only have one repeat of each block, but we could also present more than one repeat of each block by controlling the number of rows assigned to each block in our 'chooseBlocks' file.
 
 In a counterbalanced design you want to control the order explicitly and you want to provide a different order for different groups of participants. Maybe group A always gets faces first, then houses, and group B always gets houses first, then faces.
 
-Now we need to create further conditions files, to specify the exact orders we want, so we'd have something like `groupA.xlsx`:
+Now we need to create further conditions files, to specify the exact orders we want, so we'd have something like `chooseBlockA.xlsx` and `chooseBlockB.xlsx`:
 
-  +------------------+
-  |  condsFile       |
-  +==================+
-  | housesBlock.xlsx |
-  +------------------+
-  | facesBlock.xlsx  |
-  +------------------+
+.. image:: counterbalanceCondFiles.png
+        :scale: 50 %
 
-and `groupB.xlsx`:
 
-  +------------------+
-  |  condsFile       |
-  +==================+
-  | facesBlock.xlsx  |
-  +------------------+
-  | housesBlock.xlsx |
-  +------------------+
+The last part of the puzzle is how to assign participants to groups. For this you *could* write a Code Component that would generate a variable for you (`if.....: groupFile = "groupB.xlsx"`) but the easiest thing is probably that you, the experimenter, chooses this using the GUI we present at the start of the experiment. So, we add a field to our GUI using experiment settings:
 
-In this case the last part of the puzzle is how to assign participants to groups. For this you *could* write a Code Component that would generate a variable for you (`if.....: groupFile = "groupB.xlsx"`) but the easiest thing is probably that you, the experimenter, chooses this outside of PsychoPy and simply tells PsychoPy which group to assign to each participant.
+.. image:: experimentSettingsGroup.png
+        :scale: 100 %
 
-The easiest way to do that is to add the field `group` to the initial dialog box, maybe with the default value of `A`. If you set the conditions file for the `blocks` loop to be
+Note that entering a *list* as the default input will present us with a dropdown in our GUI.
+
+Finally, we set parameters of our `blocks` loop to use the method 'sequential' (because we are using a predefined order) and we enter the following into the conditions field:
 ```
-$"group"+expInfo['group']+".xlsx"
+$"chooseBlocks"+expInfo['group']+".xlsx"
 ```
-then this variable will be used from the dialog box to create the filename for the blocks file and you.
+This will concatinate the string "chooseBlocks" with our selected group ("A" or "B") and the required file extension (in this case "xlsx") in order to select the correct order.
+
+Even though our outer loop is now sequential, your inner loop still probably wants to be random (to shuffle the image order within a block).
+
+Counterbalancing different subtasks
+~~~~~~~~~~~~~~~~~~~~
+
+The above example is useful when we have multiple blocks where the routines we present would be largely similar (i.e. both blocks present an image component), but what about situations where we have totally different tasks we need to counterbalance (e.g. an auditory stroop and an n-back task). The following method is an extension of the logic used in the 'branchedExp' demo available in PsychoPy builderview.
+A working version of the example we will work through here is available  `here <https://gitlab.pavlovia.org/lpxrh6/counterbalance_multiple_tasks_demo>`_
+
+So, imagine we have 4 very different tasks. Our flow might look something like this:
+
+.. image:: counterbalanceTasksFlow.png
+        :scale: 50 %
+
+Here we have 4 totally different tasks, each with its own loop. Now imagine one participant is presented with these tasks using the order Task1 -> Task2 -> Task3 -> Task 4 (for ease let's call this group, 'ABCD') whilst another is presented with Task2 -> Task3 -> Task4 -> Task 1 (let's call them group 'BCDA').
+
+The loop surrounding each task will look something like this (although here I have stripped the parameters to the bare minimum, you will likely have a conditions file):
+
+.. image:: counterbalanceTasksInnerLoop.png
+        :scale: 50 %
+
+Where the number of times that block is repeated (or occurs at all!) is determined by the outer loop (e.g. Task1 nReps = 'nRepsTask1', Task2 nReps = 'nRepsTask2' and so on).
+
+For our outer loop we will use conditions files that look something like this:
+
+.. image:: counterbalanceTasksCondFile.png
+        :scale: 100 %
+
+Each row corresponds to how many times a subtask routine (or set of routines) will be repeated per itteration of the outer loop. The example conditions file above would be used for a participant in group 'ABCD' (on the first itteration Task 1 will repeat once, on the second itteration Task 2 will repeat once and so on).
+
+Just like before we create a field in our experiment settings called group (but let's say that the group names this time are 'ABCD', 'BCDA' and so on where the content of the conditions file differs).
+Finally, we use the following parameters for our outermost loop to select which, preordered, conditions file we are using.
+
+.. image:: counterbalanceTasksOuterLoop.png
+        :scale: 50 %
+
+Using this method, we could present several subtasks in a counterbalanced order (without having to create new experiment files for each order!).
+
 
-Also, if you're doing this, remember to set the `blocks` loop to use "sequential" rather than "random" sorting. Your inner loop still probably wants to be random (to shuffle the image order within a block) but your outer loop should now be using exactly the order that you specified in the blocks condition file.
+What about going **online** ? Well, things are more difficult there, but not impossible let's talk about :ref:`counterbalancingOnline`

docs/source/online/counterbalancingOnline.rst

@@ -0,0 +1,20 @@
+.. _counterbalancingOnline:
+
+Counterbalancing online
+-----------------------------------
+
+If you are manually recruiting your participants (i.e. sending our your experiment URL to a unique population or group) the methods described in :ref:`blocksCounter` will also work online, and can be used in the same way. However, if you have your experiment URL advertised on a recruitment website, it could be that 10s or hundreds of participants click your link. Manually assigning participants and keeping track of participant groupings in these situations is going to be difficult. So, what do we do?
+
+At the moment PsychoPy and Pavlovia doesn't have an internal method for keeping track of how many participants have already completed your task (and this is needed for counterbalancing). However, some core contributers have developed some excellent tools to help out, in particular this tool developed by `Wakefield Morys Carter <https://moryscarter.com/vespr/pavlovia.php>`_ that generates sequential participant IDs for your task. Although not a counterbalance tool per se, we can use this to assign out participants to specific groups.
+
+Add a code component to the beginning of your task that looks something like this:
+
+.. image:: counterbalanceOnlineCodeComp.png
+        :scale: 100 %
+
+Here we are checking if your participant ID is divisible by 2 (i.e. odd of even) and creating the variable 'group' using that. We would then use the same methods outlined previously in :ref:`blocksCounter` except this time we replace any instance of:
+```expInfo['group']```
+with
+```group```
+
+So the conditions files used is selected based on the participant ID!

docs/source/online/fromBuilder.rst

@@ -7,13 +7,13 @@ PsychoPy can't export all possible experiments to PsychoJS scripts yet. "Standar
 
 These are the steps you need to follow to get up and running online:
 
-  - :ref:`onlineCheckSupported`
-  - :ref:`onlineExpSettings`
-  - :ref:`onlineExportHTML`
-  - :ref:`onlineUploadServer`
-  - :ref:`onlineDebugging`
-  - :ref:`onlineParticipants`
-  - :ref:`fetchingData`
+- :ref:`onlineCheckSupported`
+- :ref:`onlineExpSettings`
+- :ref:`onlineExportHTML`
+- :ref:`onlineUploadServer`
+- :ref:`onlineDebugging`
+- :ref:`onlineParticipants`
+- :ref:`fetchingData`
 
 
 .. _onlineCheckSupported:
@@ -52,28 +52,102 @@ Uploading files to your own server
 
 We really don't recommend this and can only provide limited help if you go this route. If you do want to use your own server:
 
-  - You will need some way to save the data. PsychoJS can output to either:
+You will need some way to save the data. PsychoJS can output to either:
 
-    - `csv` files in `../data` (i.e. a folder called `data` next to the html folder). You'll need this to have permissions so that the web server can write to it
-    - a relational database
-
-  - You should make sure your server is using https to encrypt the data you collect from your participants, in keeping with GDPR legislation
-  - You will need to install the server-side script
-  - You will need to adapt PsychoPy Builder's output scripts (`index.html` and the `<experimentName>.js`) so that the references to `lib/` and `lib/vendors` are pointing to valid library locations (which you will either need to create, or point to original online sources)
+- `csv` files in `../data` (i.e. a folder called `data` next to the html folder). You'll need this to have permissions so that the web server can write to it
+- a relational database
+- You should make sure your server is using https to encrypt the data you collect from your participants, in keeping with GDPR legislation
+- You will need to install the server-side script
+- You will need to adapt PsychoPy Builder's output scripts (`index.html` and the `<experimentName>.js`) so that the references to `lib/` and `lib/vendors` are pointing to valid library locations (which you will either need to create, or point to original online sources)
 
 .. _onlineDebugging:
 
 Debug your online experiments
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-This is going to be trickier for now than the PsychoPy/Python scripts. The starting point is that, as in Python, you need to be able to see the error messages (if there are any) being generated. To do this your browser you can hopefully show you the javascript "console" and you can see various logging messages and error messages there. If it doesn't make any sense to you then you could try sending it to the PsychoPy forum in the `Online` category.
+This is going to be trickier (for now) than the PsychoPy/Python scripts. The main resources we recommend are:
+
+- `The PsychoPy to JS crib sheet <https://docs.google.com/document/d/13jp0QAqQeFlYSjeZS0fDInvgaDzBXjGQNe4VNKbbNHQ/edit>`_ a fantastic resource developed by `Wakefield Morys-Carter <https://twitter.com/Psych_Stats/>`_
+- `The PsychoJS API <https://psychopy.github.io/psychojs/>`_
+- `The forum <https://discourse.psychopy.org/>`_
+
+Common problems
+~~~~~~~~~~~~~~~
+
+The majority of problems and solutions are outlined at `The PsychoPy to JS crib sheet <https://docs.google.com/document/d/13jp0QAqQeFlYSjeZS0fDInvgaDzBXjGQNe4VNKbbNHQ/edit>`_ but some common problems include.
+
+**"My experiment works locally, but doesn't run online"***
+
+This means there is a JS error. Likelt because of one of the following reasons:
+
+- You have a code component where you have used something that can't be translated to JS (e.g. python libraries like 'numpy' and 'pandas').
+    - *Solution* check if the function you are trying to use has been added to the crib sheet. If it has, implement it as below. If it has not, try searching 'JS equivilent of X function' in `stack overflow <https://stackoverflow.com/>`, try to implement it as below, if it works share your solution on discourse so that we can add it to the resources, if it does not, post your issue on discourse and tell us what you tried.
+- You have a non-code component, but one of the parameters has been modified to a value that 'doesn't make sense' in JS.
+    - *Solution* look at the `PsychoJS API <https://psychopy.github.io/psychojs/>`_ to see what types of variable each parameter of each component takes.
+
+**"I have an 'Unknown Resources' Error"***
+
+You can control what resources are loaded using the 'online' tab in your experiment settings.
+
+.. image:: manualRes.png
+        :scale: 50 %
+
+Finding the problem
+~~~~~~~~~~~~~~~
+
+If you encounter an error when getting online and see a red error message box or you are *stuck on initializing* follow these steps:
+
+1. open the developer tools (outlined on crib sheet).
+
+2. look for the red line that corresponds to your task. In the example below, my task was called 'selectShapes' and we can see that there is an error on line 256 of the compiled JS code.
+
+.. image:: devToolsError.png
+        :scale: 50 %
+
+3. Examine what is happening on the line causing the error, either by clicking on the message in developer tools or by opening your .js file (usually automatically created on your local desktop when you synced) in a text editor with line numbering. In this example, I was trying to use a python method  `dir <https://docs.python.org/3/library/functions.html#dir>`_ which cannot be use in JS.
+
+.. image:: errorLine.png
+        :scale: 100 %
+
+Fixing the problem
+~~~~~~~~~~~~~~~
+
+**We do not recommend making edits to your .js script. Make corrections to your builder .psyexp file**
+
+In this example, I would look for the code component where I used the dir() method and remove it I could *a)* keep my code type as Auto->JS, remove it from the left hand (python side)  and it will automatically be removed from the JS side *b* change my code type to 'both' and remove it only from the right hand (JS) side (once this is done, code changes made in python will not be auto translated to JS).
+
+Why do we not recommend making changes to your .js file?
+
+- changes you make in your .js file will not be reflected back in your builder file, it is a one way street.
+- it becomes more difficult to sync your experiment with pavlovia from psychopy (since the creation of experiment pages is mainly done through the builder interface)
+- You might be comfortable using JS, but will future users? If you want future researchers to use your experiment having a visualisation of the experiment in GUI format may be more accessible to researcers less familiar with JS.
+
+**Adding JS functions**
+
+If you have a function you want to use, and you find the equivilent on the crib sheet or stack overflow, add an 'initialization' code component to the start of your experiment. Set code type to be 'JS' and copy and paste the function(s) you want there in the 'Begin experiment' tab. These functions will then be available to be called throughout the rest of the task.
+
+.. image:: initializeJScode.png
 
 .. _activateRecruitment:
 
+Posting the issue on the forum
+~~~~~~~~~~~~~~~
+
+If you are struggling to debug your error you can always ask for help on `the forum <https://discourse.psychopy.org/>`_. So that the team and community can help, it is really helpful if we can see your files either by making your project public or by adding a member of the PsychoPy team as a member to your project.
+
+**Making your task public**
+
+To make your task public navigate to your experiment page then select > View code <> > settings > permissions (set to public)
+
+.. image:: gitlabPublic.png
+        :scale: 100 %
+
+If you cannot make your task public share your problem and what you tried on the discourse and add a member of the team to your project (settings> members>add member - use the user name given by whoever is supporting you)
+
 Activate on Pavlovia
 ~~~~~~~~~~~~~~~~~~~~~~~
 
-This is needed
+Once your experiment is online you will see your experiment in dashboard> experiment, you can read more about the `Experiment page here <https://pavlovia.org/docs/experiments/experiment-page>`_.
 
 .. _onlineParticipants:
 
@@ -82,7 +156,7 @@ Recruiting participants
 
 Once you've uploaded your folder with the correct permissions you can simply provide that as a URL/link to your prospective participants. When they go to this link they'll see the info dialog box (with the same settings as the one you use in your standard PsychoPy study locally, but a little prettier). That dialog box may show a progress bar while the resources (e.g. image files) are downloading to the local computer. When they've finished downloading the 'OK' button will be available and the participant can carry on to your study.
 
-Alternatively you may well want to recruit participants using an online service such as `Prolific Academic`_
+Alternatively you may well want to recruit participants using an online service such as `prolific <https://www.prolific.co/>`_ you can read how to connect your experiment with prolific `here <https://www.psychopy.org/online/prolificIntegration.html>`_.
 
 
 .. _fetchingData:

docs/source/online/index.rst

@@ -22,6 +22,7 @@ Those steps are covered in detail here:
     usingPavlovia
     psychojsCode
     onlineParticipants
+    counterbalancingOnline
 
 but you should also be aware of the following: