/
A2J Analytics Explanation
90 lines (33 loc) · 5.55 KB
/
A2J Analytics Explanation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
A2J Analytics
This information is provided for organizations and individuals who use A2J Author® to create A2J Guided Interviews® and/or self-host the A2J Viewer® on their own servers to better understand A2J Analytics. It is not intended to be our privacy policy or terms of use. End users should be directed to those policies if they have questions about the A2J Analytics.
Why add analytics to A2J Author at all?
A2J Guided Interviews have been run over 4 million times since 2005. However, only 2 million documents have been generated. This discrepancy between assemblies and runs is not a new phenomenon. In the past, we have assumed that the 2 millions people who started an A2J Guided Interview but did not complete it did so intentionally. We assumed that the users were properly screened out of an A2J Guided Interview that was not right for their situation, or the users were asked a question that required information that they did not have on hand at the time, like a spouse’s Social Security number or the address of their child’s school. However, these were just assumptions. We could not see inside the user’s experience within the A2J Guided Interview. The A2J Viewer, the tool that plays the A2J Guided Interview for the end user, is a single page web application. That means commonly used analytical tools, like Google Analytics, can’t see within the A2J Guided Interview. Common analytical tools could tell the author that a user entered the interview and when they exited it, but not what happened in between those two data points. We needed to add our own internal analytics tool to A2J Author to move out of this data black hole.
Why Matomo?
Matomo (formerly known as Piwik) is an open source analytics platform. We chose to use Matomo, instead of other tools like Google Analytics, because Matomo is free and open sourced. It allows us to customize it and store the data on our own servers, ensuring user privacy protection. The information from user’s interactions with A2J Guided Interviews will reside on the CALI servers. CALI is The Center for Computer-Assisted Legal Instruction, A2J Author’s parent organization and a 501(c)(3) non-profit focused on legal education, technology, and access to justice. Because we can control the analytics engine ourselves, we know what is being tracked, why, and what algorithms are underlying the data. We’ve also used Matomo before. CALI has been running Matomo to track usage of our CALI lessons by law students for several years, so we have extensive experience integrating it into CALI products and interpreting the outcoming data.
What is being tracked?
Our A2J Analytics Tool will track the following user information:
IP address for geolocation purposes and to identify returning users
Time stamp of entering an interview
Time stamp and record of pages clicked
Time stamp and record of Learn Mores and Pop-ups clicked
Time stamp and record of exiting pages
Path through the interview
Device, operating system, and browser used to access the interview
Referral points from external websites (how the user got into the A2J Guided Interview)
Engagement level (how long did they stay on a page, how many times have they visited the interview, how many days since their last visit)
What we won’t be tracking:
User’s personal information (answers to any questions, name, address, etc)
What reports can be pulled?
Matomo allows authors to get a very robust look at how users are interacting with their A2J Guided Interviews. Right now, the A2J Author team is committed to providing quarterly reports to authors about their A2J Guided Interviews. These quarterly reports will start at the beginning of quarter two of 2018 and reports will be sent out at the end of quarter two 2018 (available early July 2018). These reports will include:
Number of users per interview
Devices, operating systems, and browsers used
Visitor logs detailing all the runs of the interview and the paths users moved through them
Referral and exiting points
Flow visualization to see how users generally moved through an interview
The reports will go out to the authors of the A2J Guided Interviews. The AuthorID is generated by the account of the last author to work on the interview before it was uploaded to the A2J Viewer. If organizations have hired consultants to do the development work for them and would like analytics on the interviews that were created for them, they can contact Jessica@cali.org to be added to the reporting group.
In the future, the A2J Author team intends to give access to a custom Matomo dashboard for authors to access and analyze their data directly from their accounts on a2jauthor.org
What do I have to do to use A2J Analytics?
To enable A2J Analytics, all A2J Guided Interviews need to have a custom AuthorID added to them. This allows us to track the interviews and sort the data to specific authors. To enable the AuthorID to be placed on your interviews, all authors must open the interview in their A2JAuthor.org accounts and hit “Save” on the Pages Tab. This places the AuthorID into the xml of the interview. You will then need to re-upload the interview to LawHelp Interactive or download the interview and place it on your server if you are self-hosting the A2J Viewer.
If you are self-hosting the A2J Viewer on your own servers, you’ll need to download the latest version of the A2J Viewer and place that on your server. The A2J Viewer can be found on our Github Repository at https://github.com/CCALI/a2jviewer.
Questions?
Contact A2J Author’s Project Manager, Jessica Frank, at jessica@cali.org