pyramidi.Rmd

---
title: "Midi manipulation with pyramidi"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Midi manipulation with pyramidi}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

After installing pyramidi, you can run the helper function to install its python dependency miditapyr 
in your environment (in this case `"r-reticulate"`):

```{r}
pyramidi::install_miditapyr(envname = "r-reticulate")
```

## Load libraries

When hopefully everything is set up correctly, we'll load some libraries.

```{r setup, message=FALSE}
library(pyramidi)
library(dplyr)
library(purrr)
library(htmltools)
library(tibble)
library(details)
```

## `MidiFramer` class

This class is the main structure of the package allowing to read midi files, 
manipulate the data, and write it back to disk. 
You can also use it to generate the midi data from scratch in R.

### Generate `MidiFramer` object

We'll create an R6 object of class `"MidiFramer"` from a `midi_file_path` by passing it to the  
constructor method `MidiFramer$new()`: 

```{r}
midi_file_path <- system.file("extdata", "test_midi_file.mid", package = "pyramidi")
mfr <- MidiFramer$new(midi_file_path)
```


```{details, details.summary = "Show print output"}
mfr
```


### Fields in `MidiFramer` objects

It is an [R6](https://r6.r-lib.org/) object that contains the following fields:

```{details, details.summary = "Show print output"}
enframe(as.list(mfr))
```

Please refer to `help(MidiFramer)` for more information on this class. The field `mf` is a [`miditapyr.MidiFrames()`](https://miditapyr.readthedocs.io/en/latest/miditapyr.html#miditapyr.midi_frame.MidiFrames) object. After its first element `mfr$mf$midi_file`, a [mido midi file object](https://mido.readthedocs.io/en/latest/midi_files.html) with the mido message data, there are 3 more dataframes:

```{details, details.summary = "Show print output of 30 first rows of these dataframes"}
list(
    mfr$mf$midi_frame_raw,
    mfr$mf$midi_frame_unnested$df,
    mfr$mf$midi_frame_nested$df
  ) %>% 
  map(head, 30) %>% 
  walk(\(x) print(knitr::kable(x)))
```

When `mf` is initialized `midi_frame_raw` and `midi_frame_nested$df` should be the same (except the ordering of the named fields in the `msg` column might differ). 

For a detailed interactive overview with further links showing how the various fields in the `MidiFramer` class are related and calculated see `vignette("package_workflow")`.

### Populating an empty `MidiFramer` object

You could also achieve the same result by first creating an empty `MidiFramer` object like so:

```{r}
mfr <- MidiFramer$new()
```

```{details, details.summary = "Show print output"}
mfr
```

In this case all the dataframe fields are initialized to `NULL` (or `None` in python which reticulate also translates to `NULL` in R).

In order to load a midi file to `mfr$mf` you have to use the miditapyr method [`calc_attributes()`](https://miditapyr.readthedocs.io/en/latest/miditapyr.html#miditapyr.midi_frame.MidiFrames.calc_attributes):

```{r}
mfr$mf$calc_attributes(midi_file_path)
```

Then you can populate the fields of `mfr` with the `MidiFramer` method [`populate_r_fields()`](https://urswilke.github.io/pyramidi/reference/MidiFramer.html#method-populate_r_fields).

```{r}
mfr$populate_r_fields()
```


## Usage

### Modifying midi data

In the `MidiFramer` object, we can modify `mfr$df_notes_wide`, the notes in note-wise wide format
(`note_on` & `note_off` events in the same line). Thus we don't need to worry which 
midi events belong together

Let's look at a small example. We'll define a function to replace every 
note with a random midi note between 60 & 71:

```{r}
mod <- function(dfn, seed) {
  n_notes <- sum(!is.na(dfn$note))
  dfn %>% mutate(note = ifelse(
    !is.na(note),
    sample(60:71, n_notes, TRUE),
    note
  ))
}
```

We could modify the notes in wide format like this:

```{r}
mod(mfr$df_notes_wide)
```

Then we would have to adapt all the following elements of `mfr` that depend on `mfr$df_notes_wide`.

When we call the method `mfr$update_notes_wide()`, all the depending list elements are also automatically updated.

```{r, message=FALSE}
# Apply the modification to mfr$df_notes_wide and all depending dataframes:
mfr$update_notes_wide(mod)
```

The data has also been changed in `mfr$mf` the miditapyr `MidiFrames` object in mfr:

```{r}
mfr$mf$midi_frame_nested$df %>% head()
```


### Writing modified midi files

Thus we can now directly save the modifications to a midi file:

```{r}
midifile <- "mod_test_midi_file.mid"
mfr$mf$write_file(midifile)
```

For a more extensive and less arbitrary demo how to compose your own music with pyramidi, 
see `vignette("compose")`.

### Playing audio

**You need to install [fluidsynth](https://www.fluidsynth.org/) on your computer and the [R package](https://docs.ropensci.org/fluidsynth/index.html) (which should be installed with pyramidi) if you want to do that on your computer.**

If you want to produce audio files from the midi files you can synthesize them
with the convenience function `player()`


```{r}
midifile |> player()
```

As you can hear, the `sample()` function in `mod()` changed all midi notes randomly.

You can also listen to your synthesized files by embedding an audio player for the 
`MidiFramer` object:

```{r}
mfr$play()
```

When using R interactively, this will directly play the generated audio in the console.

### Multiple results

One of the reasons to create this package was that R might help to avoid repetitive work. 
See below how you can use functional programming approaches of [purrr](https://purrr.tidyverse.org/), 
to generate multiple midi files in one call 
(actually two calls for more clarity but you could also put them in one).

You can also generate a list of multiple `MidiFramer` objects (in this case 2) and apply different modifications to each:


```{r}
l_mfr <- 1:2 %>%
  set_names(paste0("test", ., ".mp3")) %>% 
  map(~mfr$clone(deep = TRUE)$update_notes_wide(mod))
```

And if you want to successively add modifications to the same object and store all intermediate results in a list, you could do it like this:

```{r}
l_mfr2 <- 1:2 %>%
  set_names(paste0("test", ., ".mp3")) %>% 
  accumulate(~.x$clone(deep = TRUE)$update_notes_wide(mod), .init = mfr)
```

Please note that we made deep copies (`$clone(deep = TRUE)`) of the `MidiFramer` object in the list `l_mfr`.
Otherwise the field of the python [`miditapyr.MidiFrames`](https://miditapyr.readthedocs.io/en/latest/miditapyr.html#miditapyr.midi_frame.MidiFrames) object `mf` is a shallow copy. 
This means that updates to one of these elements in `l_mfr` (for instance `l_mfr[[1]]$mf`) also change the others, because they all are references to the same object.


This is how you can embed multiple audio files in an rmarkdown document using `purrr::imap()`:

```{r}
tagList(
  imap(
    l_mfr, 
    ~ div(
      h4(paste("audio result", .y)),
      .x$play(audiofile = .y),
      # add 2 line breaks to vertically separate a bit:
      br(), 
      br()
    )
  )
)
```

Compared to the two audio samples in [Playing audio] (which are the same), we 
now have two different results.