Home
VizScribe is a browser-based framework designed to aid protocol analysis (understanding subjects' cognitive processes by analyzing sequences of observations on their behavior). Typically such observations require audio, video, and transcript data, and more recently, computer usage logs, biometric or sociometric sensor data, etc. VizScribe can help you view time-sequence data from these different sources together to get a coherent view of subjects' behavior.
VizScribe is currently tested to run on Google Chrome, on Windows and OS X systems. This Wiki currently has three sections, with instructions on:
For best results, please use these instructions in sequence.
VizScribe's server uses the Node.js runtime, so it needs to be installed first, in order to install all related libraries.
-
Make sure Node.js is installed on your system
-
Download the source code as a zip file or clone the repository for VizScribe by typing the following command on the terminal:
$ git clone https://github.com/senthilchandrasegaran/vizScribe.git
-
On the Node.js Command Prompt (for Windows) or Terminal (for OS X), navigate to the main folder (the one that contains the file
app.js) and type in:$ npm install
This installs all required modules.
Before running VizScribe, you need to have the following data:
- A video file (mp4 format)
- A corresponding transcript file with timestamps. The format of the transcript should be:
start time,end time,speaker,dialog
00:00:01.5,00:00:03.0,Travis Bickle,You talkin' to me?
Note that the timestamps follow the format hh:mm:ss.s
There are other time-sequenced data that you can upload and link, but the video and transcript are essential.
On the Node.js command prompt or Terminal, navigate to the main folder (the one containing app.js). At the prompt, type in the command:
$ node app.jsLaunch the Chrome browser, type localhost:3000 into the address bar.
The page that loads will look somewhat like this:
Upload the video and transcript as requested, and click on the link below to continue.
The page should direct you to a view of your video and transcript that
looks somewhat like this:
The visualizations that you can see are interactive, and linked to each other. Let's take a closer look at the views, how they link to each other, and the possible interactions with these views.
Apart from the video, you will see 3 main views at this point.
-
The transcript view: An interactive view of the transcript, with color-coded identifiers for each speaker.
-
The interactive speech timeline view: A representation of the transcript as a series of "speech events" on a timeline, color-coded by speaker (see the corresponding colors on the transcript view above).
-
The interactive word cloud view And finally, a word cloud view that is generated from the whole transcript. Individual words are scaled according to the number of times they occur in the transcript. The code used for the word cloud uses Jason Davies' excellent d3-cloud code.
All the above visualizations are interlinked; i.e. you can click on one of the speech event timeline items, and the video will skip to it, while the transcript scrolls to the corresponding utterance.
Alternately, you can also [ctrl-click] a line on the transcript for the video to skip to the corresponding time.
Hovering on, or selecting a word on the word cloud highlights all its occurrences on the transcript and the speech timeline views:
The word cloud's default data is the entirety of the transcript. However, you can [ctrl-click] a speaker's speech timeline, and the word cloud will now only show the words spoken by that person.
You can also select a block of text in the transcript, and the word cloud will update according to the selection.
You can also pull up a "keyword in context" view, or concordance view: [ctrl-click] a word in the word cloud to pull up a view of the words immediately preceding and succeeding the selected word.
This can give some idea of the context in which the word has been used.
Finally, right-click on a word to remove it from the word cloud. At the moment, this action cannot be undone, so try it at your own risk!
As a protocol analyst, you may want to identify certain kinds of behavior or speech and categorize it as such. To create such categorizations or "codes", as they are called, select the "Add/Edit" tab under the "Code" section. Type in your codes, one on each consecutive line, and go back to the "View" tab. Each code will now show a corresponding color next to it.
On the transcript, select a block of text that you need to code,
right-click the mouse to show the list of codes, and assign the
relevant code to the selected text.
A "code timeline" now appears under the speech events timeline, showing the speech events assigned to the code.
To see the text assigned to a particular code, ctrl-click a code timeline item.
This will also filter the word cloud to show only the words assigned to that code.
