Output plots do not support font family + weird default font

[raivivek]

Hi,

It appears that my IRKernel setup is not working as expected. The plot appears to be rough and changing the fonts doesn't work as expected. See below.

However, after using the function from #559, the plot shows up as it should along with font changes as required which tells me that underlying R is working as expected. I am not sure what could be wrong. Shouldn't the first plot look like the second one anyway?

R version 3.5.1 (2018-07-02)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 9 (stretch)

locale:
[1] en_US.UTF-8

attached base packages:
character(0)

other attached packages:
[1] IRdisplay_0.6.2.9000 IRkernel_0.8.14.9000 repr_0.17.9000      

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.19         pillar_1.3.0         compiler_3.5.1      
 [4] plyr_1.8.4           bindr_0.1.1          methods_3.5.1       
 [7] base64enc_0.1-3      utils_3.5.1          tools_3.5.1         
[10] grDevices_3.5.1      digest_0.6.18        uuid_0.1-2          
[13] jsonlite_1.5         evaluate_0.12        tibble_1.4.2        
[16] gtable_0.2.0         pkgconfig_2.0.2      rlang_0.3.0.1       
[19] bindrcpp_0.2.2       withr_2.1.2          dplyr_0.7.8         
[22] RevoUtils_11.0.1     graphics_3.5.1       datasets_3.5.1      
[25] stats_3.5.1          cowplot_0.9.3        grid_3.5.1          
[28] tidyselect_0.2.5     glue_1.3.0           base_3.5.1          
[31] R6_2.3.0             pbdZMQ_0.3-3         ggplot2_3.1.0       
[34] purrr_0.2.5          magrittr_1.5         scales_1.0.0        
[37] htmltools_0.3.6      assertthat_0.2.0     colorspace_1.3-2    
[40] labeling_0.3         RevoUtilsMath_11.0.0 lazyeval_0.2.1      
[43] munsell_0.5.0        crayon_1.3.4         Cairo_1.5-9

[flying-sheep]

it doesn’t look like it should. look at the second pic, the fonts are still pixelated. the only difference is that the second one seems to have higher DPI, and therefore a bigger pixel font was used.

I assume there’s something going on with your fontconfig, and the PNG device seems to select the wrong font to draw. try + theme(family = 'Something you know is installed')

[flying-sheep]

Hmm, does it look correct when using png() from base R? else this isn’t an IRkernel bug.

[raivivek]

If both are using png() in the background, I don't understand why there would be a difference. I also tried with different font-families/sizes and they have no effect in first case. But show_plot function, supposedly doing the same thing, results the correct result.

For instance,

ggplot(iris) + geom_point(aes(Sepal.Width, Petal.Width)) + cowplot::theme_cowplot(font_family='mono')

Doesn't work in the notebook (no font change) but works when I run it inside R.

[flying-sheep]

Things are different because in IRkernel, we need to record and replay the plots. (We create multiple plot formats, and therefore have to replay the plots multiple times). It was a lot of work to find the perfect recording device:

IRkernel/R/environment_shadow.r

Lines 109 to 125 in e38f34a

	init_null_device <- function() {
	# if possible, use a device that
	# 1. prints no warnings for unicode (unlike pdf/postscript)
	# 2. can handle /dev/null (unlike OSX devices)
	# since there is nothing like that on OSX AFAIK, use pdf there (accepting warnings).
	os <- get_os()
	ok_device <- switch(os, win = png, osx = pdf, unix = png)
	null_filename <- switch(os, win = 'NUL', osx = NULL, unix = '/dev/null')
	null_device <- function(filename = null_filename, ...) ok_device(filename, ...)
	if (identical(getOption('device'), pdf)) {
	options(device = null_device)
	}
	}

But yes, this is of course a source of differences.

[raivivek]

So basically if the png device is working, then IRkernel output should work too. Otherwise, it's an issue with the device and/or R installation?

[raivivek]

Okay, so here's what I did. Clearly, there is something off that I am not able to locate. :/

In R terminal:

library(ggplot2)
p <- ggplot(iris) + geom_point(aes(Sepal.Width, Petal.Width)) +
    cowplot::theme_cowplot(font_family='Helvetica-Narrow')
png("test.png", res=320, height=4, width=4, unit='in')
dev.off()

In Jupyter Notebook with Kernel to the same R:

library(ggplot2)
ggplot(iris) + geom_point(aes(Sepal.Width, Petal.Width)) +
    cowplot::theme_cowplot(font_family='Helvetica-Narrow') # no effect whatsoever

[flying-sheep]

So basically if the png device is working, then IRkernel output should work too. Otherwise, it's an issue with the device and/or R installation?

Not exactly, the recording device also makes a difference. The recordedplot object is slightly different depending on it. It’s a big mess, but there is simply no “record everything faithfully” device available.

You could try:

IRkernel:::init_null_device()

# replicate how IRkernel records plots
dev.new()
# plot call goes here
p <- recordPlot()  # I think this goes after the plot call, not sure though
dev.off()

# replicate what repr_*.recordedplot does
png(...)  # or your device of choice
replayPlot(p)
dev.off()

this should reproduce what happens…

[krassowski]

I also experience this issue. Could it be that the option repr.plot.family which is passed to recordPlot overwrites the font family specified in ggplot?.

[flying-sheep]

Well, try setting it! I don’t have any problems, so I need all the help you can give if you want me to fix it.

[greensii]

i was having this same issue. for me, i believe it was related to having r-cairo package, which i had installed via conda. i rolled back my conda environment to before this installation and the plots are all crisp again.

[casparvl]

I have exactly this issue. @raivivek I see that you closed the ticket recently, were you able to come up with a solution?

For completeness, let me just show what I get in various cases:

Plain R

options(bitmapType='cairo')
png('test3.png')
ggplot(iris) + geom_point(aes(Sepal.Width, Petal.Width)) + cowplot::theme_cowplot(font_family='DejaVu Serif')
dev.off()

R, but trying to mimic IRkernel's plotting method, according to @flying-sheep 's instructions

options(bitmapType='cairo')
IRkernel:::init_null_device()
dev.new()
ggplot(iris) + geom_point(aes(Sepal.Width, Petal.Width)) + cowplot::theme_cowplot(font_family='DejaVu Serif')
p <- recordPlot()
dev.off()
png('test3_irkernel.png')
replayPlot(p)
dev.off()

R, but trying to mimic IRkernel's plotting method, according to @flying-sheep 's instructions AND setting repr.font.family

options(bitmapType='cairo')
IRkernel:::init_null_device()
dev.new()
ggplot(iris) + geom_point(aes(Sepal.Width, Petal.Width)) + cowplot::theme_cowplot(font_family='DejaVu Serif')
p <- recordPlot()
dev.off()
options(repr.font.family='DejaVu Serif')
png('test3_irkernel.png')
replayPlot(p)
dev.off()

Jupyter Notebook

options(bitmapType='cairo', repr.plot.width = 10, repr.plot.height = 8, repr.plot.res=96, repr.plot.family='DejaVu Serif')
library('ggplot2')
ggplot(iris) + geom_point(aes(Sepal.Width, Petal.Width)) + cowplot::theme_cowplot(font_family='DejaVu Serif', font_size=25)

What we see is that

• Plain R properly uses the DejaVu Serif font
• Mimicking IRkernel in Plain R, text quality seems good, but it doesn't seem respect the font-family set by the cowplot::theme, it only seems to respect what is passed as repr.font.family option (which I guess makes sense).
• In Jupyter, text quality is really poor and the font used is defintely not DejaVu Serif. I only get similar poor quality in plain R if using the Xlib bitmapType (and thus some X11 font) instead of the cairo bitmap type.

@flying-sheep Do you have any idea what's going on here? Do you see any reason why the IRkernel in the Jupyter environment behaves different from the replication instructions you gave? Is it possible that - somehow - despite my explicit setting of the options(bitmapType='cairo'), the replot is not using the correct bitmaptype? (that is my gut feeling here...)

[casparvl]

Btw, @flying-sheep I'm not sure what rules you have for issues: I replied to a closed issue since it really seems like the same, so you might want to keep everything together. But I'm happy to open a new one (and reference this issue as related) if you prefer that.

[flying-sheep]

I could find this: https://stackoverflow.com/a/17955000/247482

Tbh, the whole situation is pretty messy, and a) many people having wrongly compiled R installations and b) the huge amount of config options influencing it is just too much to handle. If there’s any code I can use to detect and fix this from IRkernel, I’ll add it, but idk how.

[casparvl]

Thanks for having a look! I understand your difficulty in supporting the wild variety of R installations (and optional configurations that R supports). I'm pretty sure though that my R installation is 'sane' enough, since there I can create high-quality plots in every scenario.

Seems that stackoverflow ticket suggests that if cairo is not available for some reason, it might silently fall back to Xlib. However, I'm pretty sure that's not happening: the Jupyter instance is running on a headless node, without X11 support. I've used the workaround suggested by steverweber at #388 to start the kernel with

"argv": ["/opt/r-3.4.1/lib/R/bin/R", "--slave", "-e", "options(bitmapType='cairo') ; IRkernel::main()", "--args", "{connection_file}"],

That works fine, the X11 errors that I get when Xlib is used no longer show up with this.

Anyway, what I'm really looking for is what the difference could be between running

options(bitmapType='cairo')
IRkernel:::init_null_device()
dev.new()
ggplot(iris) + geom_point(aes(Sepal.Width, Petal.Width)) + cowplot::theme_cowplot(font_family='DejaVu Serif')
p <- recordPlot()
dev.off()
options(repr.font.family='DejaVu Serif')
png('test3_irkernel.png')
replayPlot(p)
dev.off()

in plain R, or running

options(bitmapType='cairo', repr.plot.width = 10, repr.plot.height = 8, repr.plot.res=96, repr.plot.family='DejaVu Serif')
library('ggplot2')
ggplot(iris) + geom_point(aes(Sepal.Width, Petal.Width)) + cowplot::theme_cowplot(font_family='DejaVu Serif', font_size=25)

in a Jupyter Notebook, since your earlier comments seemed to suggest these should do the same. I've checked a bit the IRkernel code to see if I could discover more differences, and I see that the initialize class of Executor also calls init_shadowenv() and init_session(). Would I also need to do that if I want to mimic the Jupyter environment more closely?

My rationale is: if I can find the function that causes the text to go bad, I may be able to understand how it can be fixed or circumvented.

[raivivek]

@casparvl I abandoned my efforts to get this to work. If the issue continues to persist for you, maybe I can reopen the issue. Otherwise, you are also welcome to create a new issue and continue the discussion there. Let me know what you prefer.

[flying-sheep]

I think the problem might be that jupyter sets up the capturing device before the options call, so options(bitmapType='cairo', ... somehow results in other stuff being captured.

I assume running the options(bitmapType='cairo') part in a earlier cell would fix this, as it would basically be

# setup code
IRkernel:::init_null_device()

# first cell
dev.new()
options(bitmapType='cairo')
dev.off()

# second cell, now newly created device has that option set
dev.new()
ggplot(iris) + geom_point(aes(Sepal.Width, Petal.Width)) + cowplot::theme_cowplot(font_family='DejaVu Serif')
p <- recordPlot()
dev.off()
# since a plot was created in the cell, it now gets replayed
png('test3_irkernel.png')
replayPlot(p)
dev.off()

---

Regarding bitmapType:

bitmapType:
    (Unix only, incl. macOS) character. The default type for the bitmap devices such as png. Defaults to "cairo" on systems where that is available

I don’t know why on your system it doesn’t default to 'cairo', since your system seems to support cairo? So I don’t feel comfortable setting it manually, because it might cause problems on system where the default actually makes sense.

[casparvl]

I don’t know why on your system it doesn’t default to 'cairo', since your system seems to support cairo? So I don’t feel comfortable setting it manually, because it might cause problems on system where the default actually makes sense.

Well, not setting it is not an option: the default device it picks made me hit this issue, so I have to set it in order to even get a working kernel on my (headless) node.

But: I actually made progress! I managed to get a plot with nice looking axes labels, in my Jupyter IRkernel! What I do:

• Edit the kernel such that options(bitmapType='cairo') is run before starting IRkernel::main() to resolve the initial issue with an unresponsive kernel:

"argv": ["/opt/r-3.4.1/lib/R/bin/R", "--slave", "-e", "options(bitmapType='cairo') ; IRkernel::main()", "--args", "{connection_file}"],

• In my Jupyter notebook, I run:

options(repr.plot.width = 10, repr.plot.height = 8, repr.plot.res=96
        #, repr.plot.family='Times'
       )
library('ggplot2')
library('Cairo')
CairoFonts(
regular="FreeSans:style=Medium",
bold="FreeSans:style=Bold",
italic="FreeSans:style=Oblique",
bolditalic="FreeSans:style=BoldOblique"
)
ggplot(iris) + geom_point(aes(Sepal.Width, Petal.Width)) + cowplot::theme_cowplot(
    # font_family='Times', 
    font_size=25)

And I finally get an image with nice looking axis labels:

I came up with the idea after reading the following in the Cairo docs:

This function sets the fonts for Cairo graphics devices globally; previously opened Cairo graphics devices will also use these fonts

and figured: that should work in this context, even if IRkernel already has a device initialized.

If I change FreeSans into Helvetica I get the same pixelated fonts I had before. So it seems that the root problem is really that - somehow - neither the font_family set with the theme_cowplot nor the repr.plot.family were respected. That resulted in me getting the default font (Helvetica) which is an X11 font and not a truetype font (which is in turn why it showed up pixelated).

Though the above gives me a workaround, I don't (yet) understand is why in my normal R session it did respect repr.plot.family, and in my Jupyter session, it doesn't. I dug into the repr code a bit, and noticed this code section for png: https://github.com/IRkernel/repr/blob/master/R/repr_recordedplot.r#L80 I checked on my machine that is_cairo_installed() evaluates to True, thus, I know the code path it will follow is this https://github.com/IRkernel/repr/blob/master/R/repr_recordedplot.r#L93

Since that is a direct call to Cairo::Cairo, I understand why it would listen to me setting CairoFonts(). What I don't understand is how it is normally supposed to get the font from repr.plot.family, since the repr_png.recordedplot doesn't do anything with repr.plot.family at all.

Anyway, maybe this analysis puts you on track for finding the actual bug in repr (because I think that's where the font should be properly set) that would help resolve this. Until then, I'll use my workaround (and hope that reporting it here means others might also benifit from it).

[flying-sheep]

Yeah, only vector devices use the font family: If you check the docs for Cairo::Cairo regarding family, they refer to the docs of pdf, which in turn refer to postscript.

To fix this we need to understand the differences. I assume that global family setting just sets the default font for a Vector format, basically like the CSS :root { font-family: ... }.

ggplot and base R (text('...', family=...), par(family=...)) seem to be able to draw things with certain fonts for certain elements, so that must be another mechanism. Maybe the way Cairo rasterizes things doesn’t respect that inline font? Maybe the only reason nobody except for you complained yet is because nobody tried?

I initially decided for Cairo because the default rasterizer (at least on linux) looks crappy and even the dots it draws are wonky non-circles. But of course the lack of proper font support makes a reevaluation necessary 🤔

The R Markdown Cookbook gives a list of possible knitr devices, one of them being agg_png, which looks like it might be a great option!

[flying-sheep]

Can you try the ragg branch? IRkernel/repr#138

Looks like mixing and matching fonts is no problem:

Looks a bit more blurry that Cairo, but OK. Might even look better on a high DPI display:

[casparvl]

Hm, silly question maybe: how do I install an R package from a certain git branch? (I normally use install.packages(), but that just pulls from CRAN) I would like to give it a try (though it may take me a few days to find the time).

Actually, instead of changing the backend, I was thinking if this function https://github.com/IRkernel/repr/blob/master/R/repr_recordedplot.r#L80 shouldn't just be altered in order to get the repr.font.family, and call CairoFonts. I.e.:

repr_png.recordedplot <- function(obj,
	width     = getOption('repr.plot.width'),
	height    = getOption('repr.plot.height'),
	bg        = getOption('repr.plot.bg'),
	pointsize = getOption('repr.plot.pointsize'),
	antialias = getOption('repr.plot.antialias'),
	#special
	res       = getOption('repr.plot.res'),
        family = getOption('repr.font.family'),
...) {
	if (!is_cairo_installed() && !check_capability('png')) return(NULL)
	
	dev.cb <- function(tf)
		if (is_cairo_installed())
                        CairoFonts(regular = ..., bold = ..., ...)
			Cairo::Cairo(width, height, tf, 'png', pointsize, bg, 'transparent', 'in', res)
		else
			png(tf, width, height, 'in', pointsize, bg, res, antialias = antialias)
	
	repr_recordedplot_generic(obj, '.png', TRUE, dev.cb)
}

The only challenge here is to translate the repr.font.family (which is a single name) into the different name & style combinations know by fontconfig, i.e. you'd want to translate e.g. FreeSans into the following CairoFonts call:

CairoFonts(
regular="FreeSans:style=Medium",
bold="FreeSans:style=Bold",
italic="FreeSans:style=Oblique",
bolditalic="FreeSans:style=BoldOblique"
)

I did a bit of digging: you could do this with the systemfonts package. The font_info function allows you to request a name, and font face, and it will return a dataframe with information that matches that requested FontConfig font. One of the columns in the dataframe is the FontConfig style. Knowing this style, we can now construct the appropriate CairoFonts call:

CairoFonts(
regular = paste0(repr.font.family, ":style=", font_info(repr.font.family)$style),
bold = paste0(repr.font.family, ":style=", font_info(repr.font.family, bold = TRUE)$style),
italic = paste0(repr.font.family, ":style=", font_info(repr.font.family, italic = TRUE)$style),
bolditalic = paste0(repr.font.family, ":style=", font_info(repr.font.family, bold = TRUE, italic = TRUE)$style),
)

Clearly, I'd grab this in a function set_cairo_fonts(font_family) or something, so that you don't have to repeat these 4 lines for each device that calls Cairo::Cairo.

Maybe I'm missing something here, but wouldn't this be a solution? I don't think setting the CairoFonts should hurt anybody and I do personally think the quality of the Cairo plotting device is one of the best.

[flying-sheep]

how do I install an R package from a certain git branch?

devtools::install_github('https://github.com/IRkernel/repr.git', ref = 'ragg')

Maybe I'm missing something here, but wouldn't this be a solution? I don't think setting the CairoFonts should hurt anybody and I do personally think the quality of the Cairo plotting device is one of the best.

Definitely. I think we should do that!

I also think that going for ragg is a good idea, as it supports per-element font changes.

Maybe we should add a repr.plot.backends = c('ragg', 'Cairo', 'grDevices') option, which defines the order in which backends are tried, with the first one being used that can emit a certain mime type and is installed.

[casparvl]

@flying-sheep Sorry for going dead silent for so long. As these things go, I had my suggested solution for setting the fonts on the Cairo device on my 'TODO' list for a very long time, until I ran into this issue again and finally made a PR for it...

Note that my PR solves a slightly different problem that you PR on using ragg (the one that implements the backend selector): I solve the issue that repr.plot.family previously did not set the Cairo devices' fonts for png type plots - but I'd still need to workaround of setting the options(bitmapType='cairo') in the kernel.json. I'm not sure, but think your runtime selection of the backend would make that workaround obselete, or isn't it?

In any case, I'd say the PRs are orthogonal: we can add mine, and still add support for ragg & backend selection with yours...