FAQ Troubleshooting Participation in an Experiment
This FAQ addresses common issues in running a study with participants.
The participant reports that they cannot see the experiments in My Experiment Invitations (Android) or Find My Experiments (iOS)
-
Make sure that the participant is logged into Paco with the same Google email account that is in the published list of the experiment on the Admin screen. This is the most frequent problem.
-
Make sure that the user has internet access. Have them visit https://www.pacoapp.com/ on their mobile browser.
-
Make sure that the experiment has not ended. If all groups are fixed duration make sure that the end dates have not all passed. Participants will not see experiments that have ended.
-
Make sure that the experiment is marked Published on the Admin tab of the experiment on the server. Only admins can see unpublished experiments.
The participant reports that they cannot see the experiments in the Experiment Hub(Android) or Find Public Experiments (iOS)
-
Make sure that the experiment is published in the Admin tab of the experiment on the server.
-
Make sure that the user has internet access. Have them visit https://www.pacoapp.com/ on their mobile browser.
-
Have they joined the experiment? They should see it on the main screen of Paco. You should see a joined event in the Stats link for the experiment on the server.
-
Is the experiment fixed duration and has the end date for all groups passed? This means the experiment is over and will not schedule anymore.
-
The user may not have heard the signal. Look at the Stats link for the experiment on the server. Is it showing missed reports for the participant? If so, this means they are getting signaled but are not responding. Perhaps their ringer is off. Have them check the ringer.
If they are not online, then the reports will not get uploaded so you will have to get the user to check that they have network access. Paco should start uploading events any time the user has network access. Once the events are uploaded, you can verify that the data contains missed signals.
- If none of the above works, and it is an ESM study, you can check, on Android, what schedule the phone things it is following. In the Paco app (Android-only), tap on the Upper right menu from the main screen. Tap Settings. Tap 'Debug ESM' this will show what schedule the phone has generated.
If there is no schedule, you can try to get the phone to regenerate it by having the user tap the menu next to the experiment and then tap 'Edit Schedule'. Then have the user edit one of the schedules. This will cause Paco to regenerate the ESM schedule.
- How long is your timeout for the notifications? While 15 minutes is standard for ESM, if the participant is not aware of this, they may not have enough time to respond. Let them know up front how long they have to respond. Or, if your research question (what you are trying to study) is amenable, then lengthen the timeout duration on the Action section of the trigger.
There is a known issue with 4g/LTE towers being deployed with incorrect timezone settings. This is at least true in the United States. Paco observes timezone changes in order to make sure that the schedule works correctly as participants travel around. We don't want to signal them in the middle of the night during their travels.
Workaround
We have to wait for the carriers to fix their towers, but, in the meantime, we can prevent this on Android with the following steps:
- Go to the Settings App
- Open Date and Time
- Disable Network Sync of Timezone.
Make sure that the participant's operating system is recent enough. Currently, we support iOS 8+ and Android 3+
Please email the Paco-users mailing list and someone will try to help figure out the problem.