There are a number of ways that you can monitor a live experiment:
dallinger summary --app {#id}
, where {#id}
is the id (w...
) of the application.
This will print a summary showing the number of participants with each status code, as well as the overall yield:
status | count
----------------
1 | 26
101 | 80
103 | 43
104 | 2
Yield: 64.00%
The Dallinger experiment server provides a dashboard view for experiment administrators to monitor running experiments. The dasboard can be found at /dashboard
, and requires login credentials that are provided by the commandline output when launching an experiment using dallinger debug
, dallinger sandbox
, or dallinger deploy
.
When running under dallinger debug
a browser window should open with the dashboard already logged in. The dashboard username and password can also be found in the dashboard_user
and dashboard_password
configuration parameters in the deployed config.txt
configuration file. By default the user is named admin
and the password is generated randomly, but the user name and password can be specified using configuration files.
dallinger.experiment_server.dashboard
You can add custom tabs to the Dallinger Dashboard by adding and registering new Flask routes on the dashboard
Blueprint, and resgistering the view as a dashboard_tab
. For example in your experiment.py
you could add the following code to add a "My Experiment" tab to the dashboard:
from dallinger.experiment_server.dashboard import dashboard, dashboard_tabs
@dashboard.route("my-experiment")
def my_experiment():
return "Hello, World. This is some information about My Experiment"
dashboard_tabs.insert("My Experiment", "my-experiment")
The dashboard also supports nested tab/menus using the ~dallinger.experiment_server.dashboard.DashboardTab
object:
from dallinger.experiment_server.dashboard import dashboard_tabs, DashboardTab
def child_tabs():
return [DashboardTab('Child1', 'child1'), DashboardTab('Child2', 'child2')]
complex_tab = DashboardTab('Title', 'route_name', child_tabs)
dashboard_tabs.insert_tab(complex_tab)
The dashboard_tabs
object supports the following methods for managing the available tabs on your experiment's dashboard:
DashboardTabs
insert
insert_tab
insert_before_route
insert_tab_before_route
insert_after_route
insert_tab_after_route
remove
The ~dallinger.experiment_server.dashboard.DashboardTab
object used by the various insert_tab*
methods provide the following API:
DashboardTab
__init__
The dashboard monitoring view can be extended by adding panes to the sidebar or extending the existing panes. This can be done customizing the ~dallinger.experiment.Experiment.monitoring_panels
and/or ~dallinger.experiment.Experiment.monitoring_statistics
methods of your experiment class. Additionally, you can customize the display of the selected nodes customizing the ~dallinger.experiment.Experiment.node_visualization_html
method, or the ~dallinger.models.SharedMixin.visualization_html
property on your model class.
The dashboard database view can be customized by customizing the ~dallinger.models.SharedMixin.json_data
method on your model classes to add/modify data provided by each model to the dashboard views, or by modifying the DataTables data returned by the ~dallinger.experiment.Experiment.table_data
method in your Experiment
class.
dallinger.experiment
Experiment
monitoring_panels
monitoring_statistics
node_visualization_html
table_data
dashboard_database_actions
You may also add new actions to the dashboard database view by adding additional title
and name
pairs to the ~dallinger.experiment.Experiment.dashboard_database_actions
output along with corresponding methods that process submitted data. The ~dallinger.experiment.Experiment.dashboard_fail
method is an example of such an action.
You can use Papertrail to view and search the live logs of your experiment. You can access the logs either through the Heroku dashboard's Resources panel (https://dashboard.heroku.com/apps/{#id}/resources), where {#id} is the id of your experiment, or directly through Papertrail.com (https://papertrailapp.com/systems/{#id}/events).
You can set up Papertrail to send error notifications to Slack or another communications platform.
- Take a deep breath.
- Open the Papertrail logs.
- Search for the term
error
. - To the right of the search bar, you will see a button titled "+ Save Search". Click it. Name the search "Errors". Then click "Save & Setup an Alert", which is to the right of "Save Search".
- You will be directed to a page with a list of services that you can use to set up an alert.
- Click, e.g., Slack.
- Choose the desired frequency of alert. We recommend the minimum, 1 minute.
- Under the heading "Slack details", open (in a new tab or window) the link new Papertrail integration.
- This will bring you to a Slack page where you will choose a channel to post to. You may need to log in.
- Select the desired channel.
- Click "Add Papertrail Integration".
- You will be brought to a page with more information about the integration.
- Scroll down to Step 3 to get the Webhook URL. It should look something like
https://hooks.slack.com/services/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
. - Copy this link to your clipboard.
- Change anything else you want and then scroll to the bottom and click "Save integration".
- Go back to Papertrail page that you left in Step 7.
- Paste the copied URL into the input text box labeled "Integration's Webhook URL" under the "Slack Details" heading.
- Click "Create Alert" on the same page.
- Victory.