Expressions.Rmd

# Expressions

```{r Expressions-1, include = FALSE}
source("common.R")
```

Attaching the needed libraries:

```{r Expressions-2, warning=FALSE, message=FALSE}
library(rlang, warn.conflicts = FALSE)
library(lobstr, warn.conflicts = FALSE)
```

## Abstract syntax trees (Exercises 18.2.4)

**Q1.** Reconstruct the code represented by the trees below:

```{r Expressions-3, echo = FALSE}
ast(f(g(h())))
ast(1 + 2 + 3)
ast((x + y) * z)
```

**A1.** Below is the reconstructed code.

```{r Expressions-4, eval=FALSE}
f(g(h()))
1 + 2 + 3
(x + y) * z
```

We can confirm it by drawing ASTs for them:

```{r Expressions-5}
ast(f(g(h())))

ast(1 + 2 + 3)

ast((x + y) * z)
```

**Q2.** Draw the following trees by hand and then check your answers with `ast()`.

```{r Expressions-6, eval = FALSE}
f(g(h(i(1, 2, 3))))
f(1, g(2, h(3, i())))
f(g(1, 2), h(3, i(4, 5)))
```

**A2.** Successfully drawn by hand. Checking using `ast()`:

```{r Expressions-7}
ast(f(g(h(i(1, 2, 3)))))

ast(f(1, g(2, h(3, i()))))

ast(f(g(1, 2), h(3, i(4, 5))))
```

**Q3.** What's happening with the ASTs below? (Hint: carefully read `?"^"`.)

```{r Expressions-8}
ast(`x` + `y`)
ast(x**y)
ast(1 -> x)
```

**A3.** The `str2expression()` helps make sense of these ASTs.

The non-syntactic names are parsed to names. Thus, backticks have been removed in the AST.

```{r Expressions-9}
str2expression("`x` + `y`")
```

As mentioned in the docs for `^`:

> \*\* is translated in the parser to \^

```{r Expressions-10}
str2expression("x**y")
```

The rightward assignment is parsed to leftward assignment:

```{r Expressions-11}
str2expression("1 -> x")
```

**Q4.** What is special about the AST below?

```{r Expressions-12}
ast(function(x = 1, y = 2) {})
```

**A4.** As mentioned in [this](https://adv-r.hadley.nz/functions.html#fun-components) section:

> Like all objects in R, functions can also possess any number of additional `attributes()`. One attribute used by base R is `srcref`, short for source reference. It points to the source code used to create the function. The `srcref` is used for printing because, unlike `body()`, it contains code comments and other formatting.

Therefore, the last leaf in this AST, although not specified in the function call, represents source reference attribute.

**Q5.** What does the call tree of an `if` statement with multiple `else if` conditions look like? Why?

**A5.** There is nothing special about this tree. It just shows the nested loop structure inherent to code with `if` and multiple `else if` statements.

```{r Expressions-13}
ast(if (FALSE) 1 else if (FALSE) 2 else if (FALSE) 3 else 4)
```

## Expressions (Exercises 18.3.5)

**Q1.** Which two of the six types of atomic vector can't appear in an expression? Why? Similarly, why can't you create an expression that contains an atomic vector of length greater than one?

**A1.** Out of the six types of atomic vectors, the two that can't appear in an expression are: complex and raw.

Complex numbers are created via a **function call** (using `+`), as can be seen by its AST:

```{r Expressions-14}
x_complex <- expr(1 + 1i)
typeof(x_complex)

ast(1 + 1i)
```

Similarly, for raw vectors (using `raw()`):

```{r Expressions-15}
x_raw <- expr(raw(2))
typeof(x_raw)

ast(raw(2))
```

Contrast this with other atomic vectors:

```{r Expressions-16}
x_int <- expr(2L)
typeof(x_int)

ast(2L)
```

For the same reason, you can't you create an expression that contains an atomic vector of length greater than one since that itself is a function call that uses `c()` function:

```{r Expressions-17}
x_vec <- expr(c(1, 2))
typeof(x_vec)

ast(c(1, 2))
```

**Q2.** What happens when you subset a call object to remove the first element? e.g. `expr(read.csv("foo.csv", header = TRUE))[-1]`. Why?

**A2.** A captured function call like the following creates a call object:

```{r Expressions-18}
expr(read.csv("foo.csv", header = TRUE))

typeof(expr(read.csv("foo.csv", header = TRUE)))
```

As mentioned in the [respective section](https://adv-r.hadley.nz/expressions.html#function-position):

> The first element of the call object is the function position.

Therefore, when the first element in the call object is removed, the next one moves in the function position, and we get the observed output:

```{r Expressions-19}
expr(read.csv("foo.csv", header = TRUE))[-1]
```

**Q3.** Describe the differences between the following call objects.

```{r Expressions-20, results = FALSE}
x <- 1:10
call2(median, x, na.rm = TRUE)
call2(expr(median), x, na.rm = TRUE)
call2(median, expr(x), na.rm = TRUE)
call2(expr(median), expr(x), na.rm = TRUE)
```

**A4.** The differences in the constructed call objects are due to the different *type* of arguments supplied to first two parameters in the `call2()` function.

Types of arguments supplied to `.fn`:

```{r Expressions-21}
typeof(median)
typeof(expr(median))
```

Types of arguments supplied to the dynamic dots:

```{r Expressions-22}
x <- 1:10
typeof(x)
typeof(expr(x))
```

The following outputs can be understood using the following properties:

- when `.fn` argument is a `closure`, that function is inlined in the constructed function call
- when `x` is not a symbol, its value is passed to the function call

```{r Expressions-23}
x <- 1:10

call2(median, x, na.rm = TRUE)

call2(expr(median), x, na.rm = TRUE)

call2(median, expr(x), na.rm = TRUE)

call2(expr(median), expr(x), na.rm = TRUE)
```

Importantly, all of the constructed call objects evaluate to give the same result:

```{r Expressions-24}
x <- 1:10

eval(call2(median, x, na.rm = TRUE))

eval(call2(expr(median), x, na.rm = TRUE))

eval(call2(median, expr(x), na.rm = TRUE))

eval(call2(expr(median), expr(x), na.rm = TRUE))
```

**Q4.** `call_standardise()` doesn't work so well for the following calls. Why? What makes `mean()` special?

```{r Expressions-25}
call_standardise(quote(mean(1:10, na.rm = TRUE)))
call_standardise(quote(mean(n = T, 1:10)))
call_standardise(quote(mean(x = 1:10, , TRUE)))
```

**A4.** This is because of the ellipsis in `mean()` function signature:

```{r Expressions-26}
mean
```

As mentioned in the respective [section](If the function uses ... it’s not possible to standardise all arguments.):

> If the function uses `...` it’s not possible to standardise all arguments.

`mean()` is an S3 generic and the dots are passed to underlying S3 methods.

So, the output can be improved using a specific method. For example:

```{r Expressions-27}
call_standardise(quote(mean.default(n = T, 1:10)))
```

**Q5.** Why does this code not make sense?

```{r Expressions-28, eval = FALSE}
x <- expr(foo(x = 1))
names(x) <- c("x", "y")
```

**A5.** This doesn't make sense because the first position in a call object is reserved for function (function position), and so assigning names to this element will just be ignored by R:

```{r Expressions-29}
x <- expr(foo(x = 1))
x

names(x) <- c("x", "y")
x
```

**Q6.** Construct the expression `if(x > 1) "a" else "b"` using multiple calls to `call2()`. How does the code structure reflect the structure of the AST?

**A6.** Using multiple calls to construct the required expression:

```{r Expressions-30}
x <- 5
call_obj1 <- call2(">", expr(x), 1)
call_obj1

call_obj2 <- call2("if", cond = call_obj1, cons.expr = "a", alt.expr = "b")
call_obj2
```

This construction follows from the prefix form of this expression, revealed by its AST:

```{r Expressions-31}
ast(if (x > 1) "a" else "b")
```

## Parsing and grammar (Exercises 18.4.4)

**Q1.** R uses parentheses in two slightly different ways as illustrated by these two calls:

```{r Expressions-32, eval = FALSE}
f((1))
`(`(1 + 1)
```

Compare and contrast the two uses by referencing the AST.

**A1.** Let's first have a look at the AST:

```{r Expressions-33}
ast(f((1)))
ast(`(`(1 + 1))
```

As, you can see `(` is being used in two separate ways:

- As a function in its own right ``"`(`"``
- As part of the prefix syntax (`f()`)

This is why, in the AST for `f((1))`, we see only one ``"`(`"`` (the first use case), and not for `f()`, which is part of the function syntax (the second use case).

**Q2.** `=` can also be used in two ways. Construct a simple example that shows both uses.

**A2.** Here is a simple example illustrating how `=` can also be used in two ways:

- for assignment
- for named arguments in function calls

```{r Expressions-34}
m <- mean(x = 1)
```

We can also have a look at its AST:

```{r Expressions-35}
ast({
  m <- mean(x = 1)
})
```

**Q3.** Does `-2^2` yield 4 or -4? Why?

**A3.** The expression `-2^2` evaluates to `r -2^2` because the operator `^` has higher precedence than the unary `-` operator:

```{r Expressions-36}
-2^2
```

The same can also be seen by its AST:

```{r Expressions-37}
ast(-2^2)
```

A less confusing way to write this would be:

```{r Expressions-38}
-(2^2)
```

**Q4.** What does `!1 + !1` return? Why?

**A3.** The expression `!1 + !1` evaluates to `r !1 + !1`.

This is because the `!` operator has higher precedence than the unary `+` operator. Thus, `!1` evaluates to `FALSE`, which is added to `1 + FALSE`, which evaluates to `1`, and then logically negated to `!1`, or `FALSE`.

This can be easily seen by its AST:

```{r Expressions-39}
ast(!1 + !1)
```

**Q5.** Why does `x1 <- x2 <- x3 <- 0` work? Describe the two reasons.

**A5.** There are two reasons why the following works as expected:

```{r Expressions-40}
x1 <- x2 <- x3 <- 0
```

- The `<-` operator is right associative.

Therefore, the order of assignment here is:

```r
(x3 <- 0)
(x2 <- x3)
(x1 <- x2)
```

- The `<-` operator invisibly returns the assigned value.

```{r Expressions-41}
(x <- 1)
```

This is easy to surmise from its AST:

```{r Expressions-42}
ast(x1 <- x2 <- x3 <- 0)
```

**Q6.** Compare the ASTs of `x + y %+% z` and `x ^ y %+% z`. What have you learned about the precedence of custom infix functions?

**A6.** Looking at the ASTs for these expressions,

```{r Expressions-43}
ast(x + y %+% z)

ast(x^y %+% z)
```

we can say that the custom infix operator `%+%` has:

- higher precedence than the `+` operator
- lower precedence than the `^` operator

**Q7.** What happens if you call `parse_expr()` with a string that generates multiple expressions? e.g. `parse_expr("x + 1; y + 1")`

**A7.** It produced an error:

```{r Expressions-44, error=TRUE}
parse_expr("x + 1; y + 1")
```

This is expected based on the docs:

> parse_expr() returns one expression. If the text contains more than one expression (separated by semicolons or new lines), an error is issued. 

We instead need to use `parse_exprs()`:

```{r Expressions-45}
parse_exprs("x + 1; y + 1")
```

**Q8.** What happens if you attempt to parse an invalid expression? e.g. `"a +"` or `"f())"`.

**A8.** An invalid expression produces an error:

```{r Expressions-46, error=TRUE}
parse_expr("a +")

parse_expr("f())")
```

Since the underlying `parse()` function produces an error:

```{r Expressions-47, error=TRUE}
parse(text = "a +")

parse(text = "f())")
```

**Q9.** `deparse()` produces vectors when the input is long. For example, the following call produces a vector of length two:

```{r Expressions-48, eval = FALSE}
expr <- expr(g(a + b + c + d + e + f + g + h + i + j + k + l +
  m + n + o + p + q + r + s + t + u + v + w + x + y + z))
deparse(expr)
```

What does `expr_text()` do instead?

**A9.** The only difference between `deparse()` and `expr_text()` is that the latter turns the (possibly multi-line) expression into a single string. 

```{r Expressions-49}
expr <- expr(g(a + b + c + d + e + f + g + h + i + j + k + l +
  m + n + o + p + q + r + s + t + u + v + w + x + y + z))

deparse(expr)

expr_text(expr)
```

**Q10.** `pairwise.t.test()` assumes that `deparse()` always returns a length one character vector. Can you construct an input that violates this expectation? What happens?

**A10** Since R 4.0, it is not possible to violate this expectation since the new implementation produces a single string no matter the input:

> New function `deparse1()` produces one string, wrapping `deparse()`, to be used typically in `deparse1(substitute(*))`

## Walking AST with recursive functions (Exercises 18.5.3)

**Q1.** `logical_abbr()` returns `TRUE` for `T(1, 2, 3)`. How could you modify `logical_abbr_rec()` so that it ignores function calls that use `T` or `F`?

**A1.** To avoid function calls that use `T` or `F`, we just need to ignore the function position in call objects:

```{r Expressions-50, echo=FALSE}
expr_type <- function(x) {
  if (is_syntactic_literal(x)) {
    "constant"
  } else if (is.symbol(x)) {
    "symbol"
  } else if (is.call(x)) {
    "call"
  } else if (is.pairlist(x)) {
    "pairlist"
  } else {
    typeof(x)
  }
}

switch_expr <- function(x, ...) {
  switch(expr_type(x),
    ...,
    stop("Don't know how to handle type ", typeof(x), call. = FALSE)
  )
}

logical_abbr_rec <- function(x) {
  switch_expr(x,
    # Base cases
    constant = FALSE,
    symbol = as_string(x) %in% c("F", "T"),

    # Recursive cases
    pairlist = purrr::some(x, logical_abbr_rec),
    call = {
      if (is_call(x, "T") || is_call(x, "F")) x <- as.list(x)[-1]
      purrr::some(x, logical_abbr_rec)
    }
  )
}
```

Let's try it out:

```{r Expressions-51}
logical_abbr_rec(expr(T(1, 2, 3)))

logical_abbr_rec(expr(F(1, 2, 3)))

logical_abbr_rec(expr(T))

logical_abbr_rec(expr(F))
```

**Q2.** `logical_abbr()` works with expressions. It currently fails when you give it a function. Why? How could you modify `logical_abbr()` to make it work? What components of a function will you need to recurse over?

```{r Expressions-52, eval = FALSE}
logical_abbr(function(x = TRUE) {
  g(x + T)
})
```

**A2.** Surprisingly, `logical_abbr()` currently doesn't fail with closures:

```{r Expressions-53, echo=FALSE}
logical_abbr <- function(x) {
  logical_abbr_rec(enexpr(x))
}
```

To see why, let's see what type of object is produced when we capture user provided closure:

```{r Expressions-54}
print_enexpr <- function(.f) {
  print(typeof(enexpr(.f)))
  print(is.call(enexpr(.f)))
}

print_enexpr(function(x = TRUE) {
  g(x + T)
})
```

Given that closures are converted to `call` objects, it is not a surprise that the function works:

```{r Expressions-55}
logical_abbr(function(x = TRUE) {
  g(x + T)
})
```

The function only fails if it can't find any negative case. For example, instead of returning `FALSE`, this produces an error for reasons that remain (as of yet) elusive to me:

<!-- TODO -->

```{r Expressions-56, error=TRUE}
logical_abbr(function(x = TRUE) {
  g(x + TRUE)
})
```

**Q3.** Modify `find_assign` to also detect assignment using replacement functions, i.e. `names(x) <- y`.

**A3.** Although both simple assignment (`x <- y`) and assignment using replacement functions (`names(x) <- y`) have `<-` operator in their call, in the latter case, `names(x)` will be a call object and not a symbol:

```{r Expressions-57}
expr1 <- expr(names(x) <- y)
as.list(expr1)
typeof(expr1[[2]])

expr2 <- expr(x <- y)
as.list(expr2)
typeof(expr2[[2]])
```

That's how we can detect this kind of assignment by checking if the second element of the expression is a `symbol` or `language` type object. 

```{r Expressions-58}
expr_type <- function(x) {
  if (is_syntactic_literal(x)) {
    "constant"
  } else if (is.symbol(x)) {
    "symbol"
  } else if (is.call(x)) {
    "call"
  } else if (is.pairlist(x)) {
    "pairlist"
  } else {
    typeof(x)
  }
}

switch_expr <- function(x, ...) {
  switch(expr_type(x),
    ...,
    stop("Don't know how to handle type ", typeof(x), call. = FALSE)
  )
}

flat_map_chr <- function(.x, .f, ...) {
  purrr::flatten_chr(purrr::map(.x, .f, ...))
}

extract_symbol <- function(x) {
  if (is_symbol(x[[2]])) {
    as_string(x[[2]])
  } else {
    extract_symbol(as.list(x[[2]]))
  }
}

find_assign_call <- function(x) {
  if (is_call(x, "<-") && is_symbol(x[[2]])) {
    lhs <- as_string(x[[2]])
    children <- as.list(x)[-1]
  } else if (is_call(x, "<-") && is_call(x[[2]])) {
    lhs <- extract_symbol(as.list(x[[2]]))
    children <- as.list(x)[-1]
  } else {
    lhs <- character()
    children <- as.list(x)
  }

  c(lhs, flat_map_chr(children, find_assign_rec))
}

find_assign_rec <- function(x) {
  switch_expr(x,
    # Base cases
    constant = ,
    symbol = character(),

    # Recursive cases
    pairlist = flat_map_chr(x, find_assign_rec),
    call = find_assign_call(x)
  )
}

find_assign <- function(x) find_assign_rec(enexpr(x))
```

Let's try it out:

```{r Expressions-59}
find_assign(names(x))

find_assign(names(x) <- y)

find_assign(names(f(x)) <- y)

find_assign(names(x) <- y <- z <- NULL)

find_assign(a <- b <- c <- 1)

find_assign(system.time(x <- print(y <- 5)))
```

**Q4.** Write a function that extracts all calls to a specified function.

**A4.** Here is a function that extracts all calls to a specified function:

```{r Expressions-60}
find_function_call <- function(x, .f) {
  if (is_call(x)) {
    if (is_call(x, .f)) {
      list(x)
    } else {
      purrr::map(as.list(x), ~ find_function_call(.x, .f)) %>%
        purrr::compact() %>%
        unlist(use.names = FALSE)
    }
  }
}

# example-1: with infix operator `:`
find_function_call(expr(mean(1:2)), ":")

find_function_call(expr(sum(mean(1:2))), ":")

find_function_call(expr(list(1:5, 4:6, 3:9)), ":")

find_function_call(expr(list(1:5, sum(4:6), mean(3:9))), ":")

# example-2: with assignment operator `<-`
find_function_call(expr(names(x)), "<-")

find_function_call(expr(names(x) <- y), "<-")

find_function_call(expr(names(f(x)) <- y), "<-")

find_function_call(expr(names(x) <- y <- z <- NULL), "<-")

find_function_call(expr(a <- b <- c <- 1), "<-")

find_function_call(expr(system.time(x <- print(y <- 5))), "<-")
```

## Session information

```{r Expressions-61}
sessioninfo::session_info(include_base = TRUE)
```