Quarto in AWS Lambda?

[mgibbs1259]

Hello! I am currently trying to use quarto (v1.0.37) in AWS Lambda to generate a daily PDF report that is triggered when a file gets uploaded to AWS S3.

In my notebook, I have the following:
execute:
cache: false
echo: false

I set the following environment variables:
DENO_DIR="/tmp"
XDG_CACHE_HOME="/tmp"

The problem I'm running into is that it is trying to make this directory '.cache/quarto' in the current directory, which is not writeable.

[0m�[1m�[31merror�[0m: Uncaught Error: Read-only file system (os error 30), mkdir '.cache/quarto'
--
  | 2022-07-27T13:51:01.616-04:00 | Deno.mkdirSync(dir, {
  | 2022-07-27T13:51:01.616-04:00 | �[0m�[31m ^�[0m
  | 2022-07-27T13:51:01.616-04:00 | at �[0m�[1m�[3mObject.opSync�[0m (�[0m�[36mdeno:core/01_core.js�[0m:�[0m�[33m172�[0m:�[0m�[33m12�[0m)
  | 2022-07-27T13:51:01.616-04:00 | at �[0m�[1m�[3mObject.mkdirSync�[0m (�[0m�[36mdeno:runtime/js/30_fs.js�[0m:�[0m�[33m102�[0m:�[0m�[33m10�[0m)
  | 2022-07-27T13:51:01.616-04:00 | at �[0m�[1m�[3mensureDirSync�[0m (�[0m�[36mfile:///usr/bin/quarto/bin/quarto.js�[0m:�[0m�[33m6493�[0m:�[0m�[33m18�[0m)
  | 2022-07-27T13:51:01.616-04:00 | at �[0m�[1m�[3mquartoDir�[0m (�[0m�[36mfile:///usr/bin/quarto/bin/quarto.js�[0m:�[0m�[33m27661�[0m:�[0m�[33m5�[0m)
  | 2022-07-27T13:51:01.616-04:00 | at �[0m�[1m�[3mquartoCacheDir�[0m (�[0m�[36mfile:///usr/bin/quarto/bin/quarto.js�[0m:�[0m�[33m27653�[0m:�[0m�[33m12�[0m)
  | 2022-07-27T13:51:01.616-04:00 | at �[0m�[36mfile:///usr/bin/quarto/bin/quarto.js�[0m:�[0m�[33m40572�[0m:�[0m�[33m25�[0m

Based on some digging, I believe it is coming from src/command/render/latexmk/pdf.ts lines 38-48 in which it tries to creates a new directory using Deno.mkdirSync(workingDir) based on the current working directory.

Is there a way to change this (maybe via an environment variable?) to make the directory in /tmp?

[mcanouil]

If you copy the qmd file in tmp and render from there, then get the output file back to where you want, I believe it solves your issue.

Also, there is QUARTO_PROJECT_DIR to define the working directory, see https://quarto.org/docs/projects/code-execution.html#working-dir

[micah-svenson]

I ran across the exact same issue when trying to render a quarto report inside of AWS Lambda. Moving the quarto files to /tmp and rendering there did not fully solve the issue though.

I found that quarto uses XDG_* directories to determine where to add internal data such as cache and logs. This doesn't work in the AWS Lambda environment as the HOME environment variable causes XDG directories to resolve to read-only paths.

To solve this issue for the AWS Lambda environment I did the following:

• Set the $HOME environment variable to /tmp/user_home/
• On lambda execution copy the project or .qmd file into /tmp and render it there.

[cscheid]

This doesn't work in the AWS Lambda environment as the HOME environment variable causes XDG directories to resolve to read-only paths.

If that's accurate, then I think it's a bug in AWS Lambda, cf the xdg spec.

[micah-svenson]

I ran a quick test and AWS Lambda doesn't set HOME or any of the other XDG_* environment variables, which is why this failed. This was the only reference I could find documenting the environment variables set by AWS Lambda, which confirms what my test shows.

The good news is it's easy to work around by setting the HOME directory to somewhere in /tmp for Lambda, the bad news is it was very difficult to finally come to that conclusion.