D3#3: Data Formats and Files

#Data Formats and Files

data() only accepts arrays. However, D3 has various methods to read other file types and formats and reformat them into arrays.

If you set the range of the scale, the range must be calculated in the callback function, after the data loads:

d3.json("data.json", type, function(error, data) {
  x.domain([0, d3.max(data, function(d) { return d.value; })]);
...

##CSV, TSV, and DSV Files

Here is a sample CSV file, saved as food.csv:

Food,Deliciousness
Apples,9
Green Beans,5
Egg Salad Sandwich,4
Cookies,10
Vegemite,0.2
Burrito,7

The top line consists of the column titles or keys. The following lines are the values.

To load this file:

d3.csv("food.csv", function(data) {
  console.log(data);
});

csv() takes two arguments: the filename (with its path), and a callback function. The callback function exceutes after the data loads. The callback function's parameter is the data that was loaded.

The csv file is converted into an array of objects. Only six objects were created, despite the file having seven lines. The first line becomes the keys of the objects. There are two elements in the first line ("Food," "Deliciousness") so each object has two key-value pairs (properties). The following six lines then provide the values for the properties.

csv() stores all keys and values as strings.

For tab-separated values use tsv().

dsv() allows you to specify the delimiter. For example, you could specify semi-colons delimiting data.

##Error Handling

The callback function can take two arguments: error and data:

d3.csv("food.csv", function(error, data) {
  if (error) {
    console.log(error);
  } else {
    console.log(data);
  }
});

##JSON Files or Objects

To load JSON files or objects:

d3.json("waterfallVelocities.json", function(json) {
  console.log(json);
});

###Exercise: View CSV Data

Load the CSV file food.csv and view it in the browser console:

Food,Deliciousness
Apples,9
Green Beans,5
Egg Salad Sandwich,4
Cookies,10
Vegemite,0.2
Burrito,7

####Solution:

Make a file food.csv:

├── README.md
├── app.js
├── d3.min.js
├── food.csv
├── index.html
├── style.css

d3.csv("food.csv", function(data) {
  console.log(data);
});

###Exercise: Load a CSV file and get an error message:

####Solution:

Rename the file food.csv to notfood.csv.

##Binding JSON Objects, er, Arrays

I thought that JSON objects were enclosed in curly brackets. D3 can only read JSON objects enclosed in square brackets. Here is an example of what D3 thinks is a JSON object:

[
  {
    "Food": "Apples",
    "Deliciousness": 9
  },
  {
    "Food": "Green Beans",
    "Deliciousness": 5
  },
  {
    "Food": "Egg Salad Sandwich",
    "Deliciousness": 4
  },
  {
    "Food": "Cookies",
    "Deliciousness": 10
  },
  {
    "Food": "Vegemite",
    "Deliciousness": 0.2
  },
  {
    "Food": "Burrito",
    "Deliciousness": 7
  }
]

###Exercise: Display one element from a JSON object

####Solution:

Put the JSON object in a file called food.json.

d3.json("food.json", function(error, dataset) {
  if (error) {
    console.log(error);
  } else {
    console.log(dataset);
    d3.select("body")
    .data(dataset)
    .append("p")
    .text(dataset[0].Food);
  }
});

Note that we have two D3 statements. The first statement d3.json() loads the data. When the data is loaded the callback function executes. The second D3 statement is in the callback function. We don't want D3 to make a chart before the data loads.

###Exercise: Display all the foods from the JSON file

####Solution

d3.json("food.json", function(error, dataset) {
  if (error) {
    console.log(error);
  } else {
    console.log(dataset);
    d3.select("body").selectAll("p")
    .data(dataset)
    .enter().append("p")
    .text(function(d) {
      return d.Food;
    });
  }
});

###Exercise: Display all data from a JSON file

####Solution

d3.json("food.json", function(error, dataset) {
  if (error) {
    console.log(error);
  } else {
    console.log(dataset);
    d3.select("body").selectAll("p")
    .data(dataset)
    .enter().append("p")
    .text(function(d) {
      return (d.Food + ", " + d.Deliciousness);
    });
  }
});

#Consuming APIs

Some APIs return JSON objects with curly brackets, some with square brackets. D3 can only read JSON objects with square brackets.

##APIs With Curly Brackets To load an API that returns a JSON object with curly brackets, push your JSON object into an array:

d3.json("http://soda.demo.socrata.com/resource/earthquakes.json?$where=magnitude > 5.5", function(error, dataset) {
  if (error) {
    console.log(error);
  } else {
    console.log(dataset);
    var dataArray = [];
    dataArray.push(dataset);
    console.log(dataArray);
    d3.select("body").selectAll("p")
    .data(dataArray)
    .enter().append("p")
    .text(function(d) {
      return (d.success);
    });
  }
});

###Exercise: Make a bar chart of earthquake magnitudes

Use this API:

http://soda.demo.socrata.com/resource/earthquakes.json?$where=magnitude > 5.5

to make this:

####Solution

Start by loading the API and binding magnitude to the DOM:

d3.json("http://soda.demo.socrata.com/resource/earthquakes.json?$where=magnitude > 5.5", function(error, dataset) {
  if (error) {
    console.log(error);
  } else {
    console.log(dataset);
    d3.select("body").selectAll("p")
    .data(dataset)
    .enter().append("p")
    .text(function(d) {
      return (d.magnitude);
    });
  }
});

Now add in the styling:

d3.json("http://soda.demo.socrata.com/resource/earthquakes.json?$where=magnitude > 5.5", function(error, dataset) {
  if (error) {
    console.log(error);
  } else {
    console.log(dataset);
    d3.select("body").selectAll("p")
    .data(dataset)
    .enter().append("div")
    .attr("class", "bar")
    .style("width", function(d) {
      var barWidth = (d.magnitude -5) * 500;
      return barWidth + "px";
    })
    .text(function(d) {
      return d.region;
    })
  }
});

##Binding Functions as Data

See Data Visualization with D3.js Cookbook, by Nick Qi Zhu, pages 51-54.

###Exercise: Bind data from CSV and JSON files