json_vs_yaml.Rmd

---
title: "JSON versus YAML"
date: "`r Sys.Date()`"
output:
  workflowr::wflow_html:
    toc: false
editor_options:
  chunk_output_type: console
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

JSON and YAML are popular [serialisation](https://en.wikipedia.org/wiki/Serialization) formats.

>In computing, serialization (or serialisation) is the process of translating a data structure or object state into a format that can be stored (e.g. files in secondary storage devices, data buffers in primary storage devices) or transmitted (e.g. data streams over computer networks) and reconstructed later (possibly in a different computer environment).

Install the following packages:

* [jsonlite](https://github.com/jeroen/jsonlite) for parsing JSON
* [yaml](https://github.com/vubiostat/r-yaml) for parsing YAML
* [tidyjson](https://github.com/colearendt/tidyjson) for converting JSON to tidy data frames
* [rjson](https://github.com/alexcb/rjson) for parsing JSON

```{r install}
install.packages(c("jsonlite", "yaml", "tidyjson", "rjson"))
```

Load libraries.

```{r load}
library(jsonlite)
library(yaml)
library(tidyjson)
library(rjson)
```

As a first example, we will convert the `women` data set, which is a small data set with 15 observations for 2 variables.

```{r women}
women
```

Convert `women` to JSON using `jsonlite`.

```{r women_json}
women_json <- jsonlite::toJSON(women, pretty = TRUE)
women_json
```

`read_json` does not parse the output of `toJSON`.

```{r read_json}
jsonlite::write_json(x = women_json, path = "women.json")
tidyjson::read_json(path = "women.json")
```

Converts into list.

```{r rjson_from_json}
str(rjson::fromJSON(women_json))
```

Convert `women` to YAML.

```{r women_yaml}
women_yaml <- as.yaml(women, indent = 3)
writeLines(women_yaml)
```

## To data frame

JSON to data frame.

```{r json_to_df}
jsonlite::fromJSON(women_json)
```

YAML to data frame. This does not work for more complex data structures (see below).

```{r yaml_to_df}
yaml.load(women_yaml, handlers = list(map = function(x) as.data.frame(x) ))
```

## Non-tidy data frame

A data frame containing lists.

```{r my_df}
my_df <- data.frame(
  id = 1:3,
  title = letters[1:3]
)
my_df$keywords = list(
    c('aa', 'aaa', 'aaaa'),
    c('bb', 'bbb'),
    c('cc', 'ccc', 'cccc', 'ccccc')
)

my_df
```

Convert `my_df` to JSON.

```{r my_df_json}
my_df_json <- jsonlite::toJSON(my_df, pretty = TRUE)
my_df_json
```

Convert `my_df` to YAML.

```{r my_df_yaml}
my_df_yaml <- as.yaml(my_df, indent = 3)
writeLines(my_df_yaml)
```

## JSON to YAML and vice versa

Converting from JSON to YAML is easy.

```{r json_to_yaml}
identical(writeLines(as.yaml(jsonlite::fromJSON(my_df_json))), writeLines(my_df_yaml))
```

Converting from YAML to JSON for `my_df` is not as straight-forward because of the different number of keywords.

```{r yaml_load}
my_df_list <- yaml.load(my_df_yaml)
my_df_list
```

This conversion is different from the original data frame to JSON conversion because this creates a single object, where as the original conversion creates an array with three objects.

```{r yaml_to_json_wrong}
jsonlite::toJSON(my_df_list, pretty = TRUE)
my_df_json
```

I could probably write a hacky function to make the conversion but I won't.

## Parsing JSON

The [ffq](https://github.com/pachterlab/ffq) tool generates metadata in JSON:

    ffq SRX079566 > data/SRX079566.json

```{r read_ffq_json}
ffq_json <- jsonlite::read_json(path = "data/SRX079566.json", simplifyVector = TRUE)
str(ffq_json)
```

Use a recursive apply to create a named character vector, which is convenient for plucking values.

```{r rapply_ffq}
test <- rapply(object = ffq_json, f = function(x) x)
class(test)
```

Subset the FTP links.

```{r ftp_url}
test[grepl("ftp.url\\d+$", names(test))]
```