Data Analyst Nanodegree Project 6 - Visualize Electric Car Battery Performance in D3.js
Hybrid and fully electric cars are awesome because they will help us save the planet from burning through more dinosaur juice! But temperature of the battery and the way of driving your car is very important in order to have a long-lasting battery with which you can drive over a long range. This Data Analyst Nanodegree Visualization Project shows how electric car batteries State-of-Charge diminishes or increases and how the temperature of such a battery develops during different driving situations. The visualization was created with scrambled R&D data from a german car manufacturer and also contains options to drill down into an explorative mode which will be used by roughly 200 researchers in the field.
For my 6th Udacity Data Analyst Nanodegree Project I wanted to "go big" and create an actual project that can be used at my workplace. I work at a large german car manufacturer and involved a collegue who had some electric car battery data he wanted me to visualize. It is quite common for students in Germany to do final study project or thesis together with a company. The reward is the increased applicability - people may actually use the product that was created. I wanted to take that chance for this Nanodegree as well.
The challenge, of course, is that the scope of the project is much larger and additional stakeholders are involved. But data visualization is a topic dear to my heart (what is the use of all the data in the world if you cannot communicate it?). Also, web application development was my profession 5 years ago - so I was eager to see what had changed in the meantime.
Initial Design Principles
Visual Design Decisions of the Explanatory View
When designing the layout of the explanatory view, the following principles were in my mind:
- Visualize the corellation between Temperature of the Battery and State-of-Charge of the Battery. Correlations are best visualized with a scatter plot.
- Use different colors to visualize different scenarios (winter, fast driving, normal driving). Use a categorical color scale from colorbrewer.org, in particular, associate "red" with "hot" (sportive driving), "blue" with "cold" (winter), "green" with "normal" (range testing).
- Show the values with an alpha value of 0.1 in order to show different data points on top of each other. This gives data series where values change rapidly a "scratchy" trail of scatter points while series, where the values (Temperature and State-of-Charge) remain constant over longer time, have a more "rigid" layout.
- Blur out all values >190 seconds to make the different data sets comparable and focus on the 190 second experiment.
- The additional directionality of the scatter points is visualized with a marker of the fixed checkpoint t = 190 seconds. An arrow was added later to hint at this time-dependent directionality of the data (CHG-2).
- Appealing images were added later to the scatterplot to make the topic understandable to laypersons on first notice (CHG-3).
- x-Axis and y-Axis provide clean, solid outlines and clean, prominently positioned text. Enough whitespace around the axes guarantees that the chart can "breathe" and does not collade with the viewport border. Axes are described clearly and precisely to be understandable even by people who see the chart for the first time and have no preconception about what is visualized (see CHG-3).
- A line is added to signify the freezing point (0°C) (or any other zero-point).
- The chart is accompanied by a "story box" which does not collade with the chart, but modestly draws attention to itself by being slighly raised up with a dropshadow.
- Chart and story box share the space in a 2-to-1 ratio, giving emphasis on the visualization.
- The button we want the user to press after they are done reading the story is highlighted with a prominent color.
- An "Explore" button with a Martini glass invokes the emotion "Hey, click me if you want to be adventurous". :)
- An understated (gray, dashed), nevertheless emphatic (3px bold) box clearly marks an area of the chart that is mentioned by the currently described "story" (see CHG-6).
- The buttons to switch to explorative mode are shown in a larger font to signify that there is "much more" behind this button to explore.
- The user is able to zoom in and out of the chart and explore the exact values of different data points by hovering with the mouse over it. This is signified by using diffierent mouse cursors
Visual Design Decisions of the Explorative View
The explorative view was envisioned before the Explanatory View and where it made sense, decisions made here were carried over to the Explanatory View.
- Instead of a 2-to-1 column grid, this view displays a grid with 2 columns of equal width and two dropdowns on top. This facilitates a more technical, tool-oriented, rational view that gives an impression that "here, you have complete control about two things going on".
- The two columns follow a strict structural hierarchy: the left is about one battery (scenario) and its current state, the right is about one or more metrics that are explored in this battery over time.
- On the right side, multiple metric graphs can be displayed below one other. The graphs take up as much space as they can get but equally share the space among each other. They do have a minimum height below which the left column is scrolled. This makes sure that space is used economically.
- On the left side, a clean and simple display of the battery is shown. Additionally, primitives of the metrics that are currently selected are shown with the value of those metrics displayed alongside. This conveys an idea about how those metrics are measured in the battery and where they occur.
- Some metrics, like temperature and electrical current are color-coded. Temperature uses an intuitive diverging color scale (blue-yellow-orange) and temperature a more technical one (green-white-violette). This is useful especially for those metrics, that have many series attached to them (like temperature: min, mean, max etc.)
- In case of such an additional color-coding, the color scale is shown intuitively on the y-axis of the chart and as a color coding of the marker highlighting the currently selected data point.
- If multiple series are present in one graph, less important series (like limits for voltage) can be colored by a paler color like gray. This color is re-used in the schematics, thereby linking the two visual displays.
- The storybox is now added on the right column. It fits better to the area describing "what is going on in the current status of the battery".
- The visualization also allows to divert from the story alltoghether by starting to move inside of the graph or changing the number of metrics that are displayed. This is especially useful for experienced users. Should a user decide that they want to return to the visualization, they can always click "Return to Guided Tour".
Additional High-Level Requirements
The following high-level requirements were added to the requirements of the Nanodegree rubric in order to create a practical solution for the R&D engineers at my company.
- Data must be stored in JSON file in a pre-defined structure and must be easily replacable or extensible by new data of new battery measurements (scenarios).
- Visualization must contain an explorative component that allows access to any metrics recorded in the data in addition to the explanatory visualization.
- Explanatory visualization for the Data Analyst Nanodegree should be a high-level overview visualization of a conclusion from all battery scenarios. This visualization should be understandable by an unexperienced viewer who is not familiar with the topic of E-Mobility.
- From this explanatory visualization it should be possible to drill-down into the explorative view. Even in the explorative view, there should be some guided storytelling / descriptions to guide unexperienced users.
- Switching between both views should occur seamlessly. Navigation between different views should also be possible at any time using a navigation menu. The navigation menu should be as unobstrusive as possible (maybe just a single button bringing it up / "hamburger menu").
- Each visualization should have a "guided" mode that shows the user certain areas of a graph and guides him through exploring it and a "unguided" mode where the user can explore the graph (or related graphs) on their own. This is to make the visualization appealing both to expert users who may be better suited to draw their own conclusions and to laypersons.
- It should be possible to add additional views later and re-use code and components across views. In fact, as much code as possible should be re-used between explanatory and explorative view. Code should be documented in JSDoc.
- Code artifacts relevant for the Data Analyst Nanodegree, especially components that use D3.js should be encapsulated and understandable on their own when reading the commented source code files (and/or potential technical documentation).
- Whole web application should be realized in responsive design and look good on different screen sizes down to viewport width of 1024px. Charts should re-scale in a way that the x- and y-range stays fixed as well as any font sizes and line strokes unchanged. Spacing and shape size may re-size to fill available space. Scrolling is only permissable if the user explicitly elects to view a large number of charts (>2) below each other in an explorative mode.
- Web application should work well in all modern versions of Internet Explorer, Firefox and Chrome.
Changes after Feedback
The following changes were applied after gathering feedback from a public audience. See Actionable Feedback for their justification and [Feedback Methodology][feedback-methodology] for an explanation on how feedback was gathered:
- CHG-1: Changed the text content of the visualization to state more clearly what we mean if we talk about "battery performance" - what makes a good electric car battery?
- CHG-2: While the scatterplot compares two dimensions, Temperature and State-of-Charge, there is also a third dimension, "time", that doesn't come out easily. "Time" gives those scatter plot points an order to be drawn in and the trail of scatter scatter points a direction. Initially, I tried highlighting this by marking the time point "190 seconds" in the data in order to make the three data sets comparable. Appearantly this was not enough so I added a small arrow next to every data series to hint at the directionality.
- CHG-3: The axes labels were changed from "Temperature" to "Mean Temperature of Electric Car Battery" and from "State-of-Charge" to "State-of-Charge of Electric Car Battery".
- CHG-4: In order to make the image more appealing and provide a first-moment eye catcher about the subject matter, little sticker-images have been added on the scatter plot canvas for every data series.
- CHG-5: The data is obfuscated with a fake-factor and -offset since it was sourced from a corporation and I could not compromise intellectual property. In order not to let people jump to wrong conclusions, an additional disclaimer was added.
- CHG-6: Instead of flashing boxes highlighting certain areas of a chart I am using more modest, dashed rectangles now.
Early, when fleshing out the idea and design, I consulted the Udacity Forum with a YouTube Pitch and a Discussion thread in the Forums. The advice given there guided me to not only focus on the explorative visualization instead of the explorative tooling requested by my company, it also inspired ideas on how to design a comparative overview over all battery scenarios. Unfortunately I ended up with only a subset of the data that I wanted to consider initially, so I could not, for example, compare different battery types. Instead, I focussed on the behaviour of batteries in different driving situations.
Throughout the project, I was in touch with a electric car R&D experts who advised me on certain technical concepts and who gave feedback on the documented texts and visuals. During this development phase the project went through many iterations and changes concerned mainly technical details. You can check out the Work-in-Progress section below to get an idea about this initial back-and-forth.
After the visualization reached the maturity of a release candidate, I requested feedback via a few public channels:
- Interviewing a Udacity Coach (subject A)
- Interviewing uninvolved collegues at work (1 critiques, subject B)
- Emailing friends and family as well as posting the visualization on facebook (2 critiques, subject C & D)
- Creating a video walkthrough on YouTube and posting it on the [Udacity Forums][forums-rc] (no critiques)
- Posting a link on my twitter (no critiques)
The following table summarizes the feedback given by 4 individuals (A-D) during the "review phase" of the release candidate.
Work in Progress
|Screenshot||Date / Link to Snapshot||Description|
|28 Feb 2016||Initial meeting with my friend, the battery expert, to discuss the project and brainstorm ideas. Quite a few sketches came out of that.|
|14 Mar 2016||YouTube pitch for the Udacity Forum: After meeting with my friend and fleshing out ideas, I was asking for feedback from the nanodegree community & coaches.|
|13 Apr 2016||Fail: Using dimple.js as the library for the graph components; realizing that there is no option for me to make these components interactable in any meaningful way, so I will need to design the charts from scratch.|
|18 Apr 2016||Components talking to each other! The battery schematic updates when you move the mouse over the chart; also the layout is more responsive; no thought spent yet on actual real-life data or how to de-clutter the UI.|
|26 Apr 2016||Removed "clutter", made layout more responsive and added a whole deal of interactivity; what did not work in the end: to use the colored gradient as a stroke for the linegraph – too confusing! Instead the color mapping it is now shown as a guide on the y axis & on the data points.|
|29 Apr 2016||Added the storybox, interactivity and cleaned up the site layout, navigation and structure.|
|02 May 2016||Fail: A weird way of setting up an explanatory visualization comparing multiple batteries and displaying every battery in some strange bubbles There was a delay in the final "real-life" battery data being supplied and finally I was reaching a point where I desperately needed it to make design decisions!|
|04 May 2016||Integrated first real-life battery + mockup of battery drawing into explorative view; Result: the graphs behaved really, really slow with the many additional data-points – programming of the event handlers was not performant at all! Also, in order to draw the lines with gradients, the lines had to be split into many segments leading to a very huge DOM.|
|04 May 2016||The graphs are performant now - but: many outliers which make the data look ugly. We do have to do some wrangling :/ Also: the segment-wise drawing of the data in order to make their stroke use gradients makes the lines look "broken".|
|07 May 2016||Added 2 other scenarios and re-styled the gradient functionality. Also added the interactivity in the schematic visualization.|
|13 May 2016||Re-designed the exploratory visualization and requested further feedback from a subject-matter-expert.|
|17 May 2016||Release Candidate: Added markers, annotations etc. etc., completed story - except for minor feedback from other people than the subject-matter-expert, this is the final project. The rest was bug fixing, refactoring for maintainability, commenting, documenting and asking other people for feedback.|
|14 Mar - 17 May 2016||Of course, also during the course of this project a few sketches and notes were taken - mainly for myself.|
|22 May 2016||Request For Feedback in the Udacity Forum: The project was at a high enough maturity that I could come back to the Nanodegree Community and ask for feedback.|
|27 May 2016||First Submission: Incorporated all public feedback (see above), finished technical and project documenation and submitted project to Udacity.|
Here are some challenges and lessons learned I would take into account if I were to re-start the project:
- A problem turned out to be the supply of data: I was starting to design the application long before I even had some real data! Therefore I had to "guess" what chart type would be most applicable in the final explanatory visualization. Luckily a scatter plot worked out just fine and was easy to realize from the line plot code I developed "blindly". The
AppChartclass I designed for the explorative visualization needed just some minor adaptations. If that wasn't possible I would not have been able to complete the project in time and would have had to abandon the idea alltogether in favor of something more simple.
- The data is stored in JSON files in a very "quick & dirty" way. The JSON files store the values alongside with directives for data display, decorative elements like annotations, markers etc. This does not obey a clear separation of concerns.
- The obfuscation of the data values in order to protect intellectual property basically makes quantitative statements on the explanatory visualization useless. The visualization states battery temperatures at about 70°C or higher, which is not realistic - rather confusing. I added a disclaimer for the public version of this visualization, however, one could also envision a visualization that simply hides any quantitative data on its axes and states ideas about patterns and trends in general.
The source code of this project is thouroughly documented but before diving into the comments of the source code, I recommend getting a high level understanding about some concepts that are mentioned throughout the source code. I supply some "tutorial" documents alongside in the JSDoc documentation of this project:
- File & Directory Structure contains a general "what is where" of the repository.
- Terminology describes a number of clearly defined concepts you see on the screen of the application.
- JSON Class Structure of Data Files contains information about the file structure of the visualized data and may be especially useful for readers that want to swap out the data set with other data.
- Components of the Application describes the components you see on the web application. This application uses AngularJS, a framework that lets us define "extensions" to HTML and attach custom behaviours to those new elements. For drawing the charts, dropdowns, battery schematics etc. this is heavily used.
- Interaction of the Components explains how the components described above exchange data so they all update simultaneously and thusly provide a consistent user experience.
- Layers in the AppChart D3.js Graph goes into deeper detail about how our chart implementation's internal SVG layering is designed so that all small things of the chart, like axes, labels, data points etc. all fall into place at the right position.
AppChart which is placed in
/js/AppChart.js. Understanding the source code in this file should not require any knowledge of AngularJS. I would, however, advise to get at least familiar with the articles at point (2.), (3.) and (6.).
Interesting Articles / Tutorials / Showcases
- How to develop D3.js in AngularJS
- D3 Reusable Bar Chart with AngularJS
- D3 Example: Zoom, Pan and Axis Rescale
- D3: Continuous Color Scales with Many Color Values
- Coloring Different Segments of a Line Chart (interesting, but I abandoned the idea)
- SVG Paths With Percentages as Sizes and Fixed Stroke Widths
Frameworks Used & Copyrights
- Bootstrap (MIT License)
- Bootstrap Sass
- jQuery (MIT License)
- AngularJS (MIT License)
- D3.js (BSD License)
- Requirejs (New BSD or MIT License)
- Underscore (License)
- JSDoc (Apache License 2.0)
- JSDoc Bootstrap / Docstrap (License)
- ElementQueries, ResizeSensor
... to my collegue Fabian Friedrichs for supplying me with data and advice. Also thanks to everyone who gave feedback!