Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use tinytex to compile LaTeX documents #2788

Closed
yihui opened this issue May 15, 2018 · 17 comments · Fixed by #5490 or #5608
Closed

Use tinytex to compile LaTeX documents #2788

yihui opened this issue May 15, 2018 · 17 comments · Fixed by #5490 or #5608
Assignees

Comments

@yihui
Copy link
Member

@yihui yihui commented May 15, 2018

I think the tinytex package is mature enough now (it has been on CRAN for about half a year), so we may consider using it to compile LaTeX documents (including Rnw documents rstudio/tinytex#10). Note that the rmarkdown package has already used it to compile the intermediate .tex files to PDF in rmarkdown::render(). Some tips on how the IDE options should be passed to tinytex:

image

  1. The main function to call is tinytex::latexmk().

  2. The LaTeX engine (pdflatex, xelatex, lualatex) is passed to the engine argument of latexmk().

  3. The Clean auxiliary output option is passed to the clean argument of latexmk().

  4. The Enable shell escape option is passed to latexmk(engine_args = '-shell-escape').

The main advantages of using tinytex is that it will process bibliographies and resolve other cross-references correctly (compile the .tex for a correct number of times). If the user has installed TinyTeX (tinytex::install_tinytex()), missing LaTeX packages can also be installed automatically.

Let me know if you need any other information.

@kevinushey
Copy link
Contributor

@kevinushey kevinushey commented May 15, 2018

@jmcphers this would be nice to get in for RStudio v1.2; do you think we have the time to squeeze it in?

@jmcphers jmcphers added this to the v1.2 milestone May 15, 2018
@jmcphers
Copy link
Member

@jmcphers jmcphers commented May 15, 2018

It'd certainly be nice! I've added it optimistically to the backlog for 1.2 but we should prioritize some of the longer-standing bugs/regressions over this.

@jmcphers jmcphers removed this from the v1.2 milestone Sep 5, 2018
@njtierney
Copy link

@njtierney njtierney commented Jul 24, 2019

Just wanted to chime in here that using tinytex to compile PDF would be really nice!

@kevinushey kevinushey added this to the v1.3-preview milestone Sep 11, 2019
@kevinushey
Copy link
Contributor

@kevinushey kevinushey commented Sep 11, 2019

I believe this has become somewhat higher priority for v1.3. Newer versions of MiKTeX hide the "full" installer somewhat, and only provide the basic installer; at least based on what I see from https://miktex.org/download.

This implies that attempts to compile PDFs will fail by default, even with MiKTeX installed. Normally, MiKTeX will try to prompt you to install missing packages, but we've seen that fail (and of course that will fail if you lack internet connectivity)

2019-09-10 09:28:28,485-0700 INFO  pdflatex - starting with command line: C:\Users\rich\AppData\Local\Programs\MIKTEX~1.9\miktex\bin\x64\pdflatex.exe -halt-on-error -interaction=batchmode cda.tex
2019-09-10 09:28:28,540-0700 INFO  pdflatex - allowing known shell commands
2019-09-10 09:28:28,541-0700 INFO  pdflatex - enabling input (output) from (to) processes
2019-09-10 09:28:28,762-0700 FATAL pdflatex - GUI framework cannot be initialized.
2019-09-10 09:28:28,762-0700 FATAL pdflatex - Info:
2019-09-10 09:28:28,762-0700 FATAL pdflatex - Source: Libraries\MiKTeX\UI\Qt\mikuiqt.cpp
2019-09-10 09:28:28,762-0700 FATAL pdflatex - Line: 77
2019-09-10 09:28:28,762-0700 INFO  pdflatex - finishing with exit code 1

It's possible to request a "full" installation if you use the "net" installer (which is not the most prominently visible installer), and then request a "full" install in that wizard. But that's definitely not the path most users will take.

@yihui
Copy link
Member Author

@yihui yihui commented Oct 11, 2019

#5490 provided a way to install TinyTeX, which is great! What I requested in this issue was actually to use tinytex::latexmk() to compile .tex files.

@retodomax
Copy link

@retodomax retodomax commented Nov 27, 2019

It would be great if the "Compile PDF" button somehow calls tinytex::pdflatex('filename.tex') if tinytex is installed. Otherwise, if just pdflatex is executed, missing LaTeX packages are not installed automatically (in RStudio 1.2.5). Or am I doing something wrong?

Thank you!

RStudio_LaTeX

@kevinushey
Copy link
Contributor

@kevinushey kevinushey commented Nov 27, 2019

This is currently only available in the daily builds of RStudio.

@retodomax
Copy link

@retodomax retodomax commented Nov 27, 2019

I see... Thank you!

@yihui
Copy link
Member Author

@yihui yihui commented Apr 4, 2020

I just realized that #5608 only made it possible to compile .tex files via tinytex::latexmk(), but not .Rnw documents. And I was hoping the intermediate .tex from .Rnw could be compiled through tinytex as well (rstudio/tinytex#10). I'm not familiar with rnw_weave::runWeave below, but it sounds like it does the two steps together ("weave" + "compile to pdf"):

if (isRnw)
{
// remove existing ancillary files + concordance
removeExistingLatexAncillaryFiles(targetFilePath_);
removeExistingAncillary(targetFilePath_, "-concordance.tex");
// attempt to weave the rnw
rnw_weave::runWeave(targetFilePath_,
encoding_,
magicComments_,
enqueOutputEvent,
boost::bind(
&AsyncPdfCompiler::onWeaveCompleted,
AsyncPdfCompiler::shared_from_this(), _1));
}
else if (prefs::userPrefs().useTinytex())
{
runTinytex();
}

So somewhere inside rnw_weave::runWeave, we need to call tinytex::latexmk() if the preference use_tinytex is true.

I was just about to tell the user who asked the question https://stackoverflow.com/q/61029677/559676 to try the daily build, but after I tested it, I realized .Rnw was not compiled through tinytex, so it won't work for this user (references won't be correctly resolved).

@kevinushey kevinushey modified the milestones: v1.3-preview, v1.3-patch Apr 22, 2020
@kevinushey
Copy link
Contributor

@kevinushey kevinushey commented Sep 9, 2020

It looks like we normally run R as a sub-process when weaving .Rnw files, basically invoking the following:

$ ps -p `pgrep -nx R`
  PID TTY           TIME CMD
36278 ttys006    0:00.21 /Library/Frameworks/R.framework/Resources/bin/exec/R --slave --no-save --no-restore -e grDevices::pdf.options(useDingbats~+~=~+~FALSE);~+~utils::Sweave('test.Rnw',~+~encoding='UTF-8')

And this is where that all happens:

void runWeave(const core::FilePath& rnwPath,
const std::string& encoding,
const core::tex::TexMagicComments& magicComments,
const boost::function<void(const std::string&)>& onOutput,
const CompletedFunction& onCompleted)
{
// remove existing concordance file (if any)
rnw_concordance::removePrevious(rnwPath);
// get the R bin dir
FilePath rBin;
Error error = rBinDir(&rBin);
if (error)
{
LOG_ERROR(error);
onCompleted(Result::error(error.getSummary()));
return;
}
// R exe path differs by platform
#ifdef _WIN32
FilePath rBinPath = rBin.completePath("Rterm.exe");
#else
FilePath rBinPath = rBin.completePath("R");
#endif
// determine the active sweave engine
std::string weaveType = weaveTypeForFile(magicComments);
boost::shared_ptr<RnwWeave> pRnwWeave = weaveRegistry()
.findTypeIgnoreCase(weaveType);
// determine the driver (if any)
std::string driver = driverForFile(magicComments);
// run the weave
if (pRnwWeave)
{
std::vector<std::string> args = pRnwWeave->commandArgs(
rnwPath.getFilename(),
encoding,
driver);
// call back-end
Error error = compile_pdf_supervisor::runProgram(
rBinPath,
args,
core::system::Options(),
rnwPath.getParent(),
onOutput,
boost::bind(onWeaveProcessExit,
pRnwWeave, _1, _2, rnwPath, onCompleted));
if (error)
{
LOG_ERROR(error);
onCompleted(Result::error(error.getSummary()));
}
}
else
{
onCompleted(Result::error(
"Unknown Rnw weave method '" + weaveType + "' specified (valid " +
"values are " + weaveRegistry().printableTypeNames() + ")"));
}
}

@yihui, can you share what the right way to invoke tinytex::latexmk() here would be -- would we invoke it instead of utils::Sweave() here? Or in tandem in some way?

@yihui
Copy link
Member Author

@yihui yihui commented Sep 30, 2020

@kevinushey In your case, I think you were using Sweave:

image

If you change Sweave to knitr, you will probably see knitr::knit('test.Rnw') in the process.

Anyway, utils::Sweave('test.Rnw', encoding='UTF-8') should generate a .tex file, instead of generating PDF directly. tinytex::latexmk() is supposed to be called after that step, i.e., after utils::Sweave() or knitr::knit() is called, you get a .tex file, and this file needs to be compiled via tinytex::latexmk() (if users choose to use tinytex). I'm not sure where this step occurs. I feel RStudio might be calling tools::texi2pdf() to compile the .tex somewhere.

Please let me know if you need further explanation. Thanks!

@kevinushey
Copy link
Contributor

@kevinushey kevinushey commented Dec 8, 2020

@yihui sorry for taking so long to respond. You are correct that I was using Sweave; sorry for missing that part. This is where we're building the command to invoke knit():

virtual std::string weaveCommand(const std::string& file,
const std::string& encoding,
const std::string& driver) const
{
std::string format = "require(knitr); ";
if (prefs::userPrefs().alwaysEnableRnwConcordance())
format += "opts_knit$set(concordance = TRUE); ";
format += "knit('%1%'";
std::string cmd = boost::str(boost::format(format) % file);
if (!encoding.empty())
cmd += (", encoding='" + encoding + "'");
cmd += ")";
return cmd;
}

And this is where we run the latex compiler after a successful weave (or knit):

void runLatexCompiler(bool targetWeaved,
const rnw_concordance::Concordances& concordances =
rnw_concordance::Concordances())
{
// configure pdflatex options
pdflatex::PdfLatexOptions options;
options.fileLineError = false;
options.syncTex = !isTargetRnw() || !concordances.empty();
options.shellEscape = prefs::userPrefs().latexShellEscape();
// get back-end version info
core::system::ProcessResult result;
Error error = core::system::runProgram(
string_utils::utf8ToSystem(texProgramPath_.getAbsolutePath()),
core::shell_utils::ShellArgs() << "--version",
"",
core::system::ProcessOptions(),
&result);
if (error)
LOG_ERROR(error);
else if (result.exitStatus != EXIT_SUCCESS)
LOG_ERROR_MESSAGE("Error probing for latex version: "+ result.stdErr);
else
options.versionInfo = result.stdOut;
// compute tex file path
FilePath texFilePath;
if (targetWeaved)
{
texFilePath = targetFilePath_.getParent().completePath(
targetFilePath_.getStem() + ".tex");
}
else
{
texFilePath = targetFilePath_;
}
// remove log files if they exist (avoids confusion created by parsing
// old log files for errors)
removeExistingLatexAncillaryFiles(texFilePath);
// setup cleanup context if clean was specified
if (prefs::userPrefs().cleanTexi2dviOutput())
auxillaryFileCleanupContext_.init(texFilePath);
// run latex compile
// this is our "simulated" texi2dvi -- this was originally
// coded as a sequence of sync calls to pdflatex, bibtex, and
// makeindex. re-coding it as async is going to be a bit
// involved so considering that this is not the default
// codepath we'll leave it sync for now (and then just call
// the (typically) async callback function onLatexCompileCompleted
// directly after the function returns
enqueOutputEvent("Running " + texProgramPath_.getFilename() +
" on " + texFilePath.getFilename() + "...");
error = tex::pdflatex::texToPdf(texProgramPath_,
texFilePath,
options,
&result);
if (error)
{
terminateWithError("Unable to compile pdf: " + error.getSummary());
}
else
{
onLatexCompileCompleted(result.exitStatus,
texFilePath,
concordances);
}
}

So I think we need to use tinytex() when rendering there as well.

@kevinushey kevinushey reopened this Dec 8, 2020
@kevinushey
Copy link
Contributor

@kevinushey kevinushey commented Dec 9, 2020

Fixed via #8570 -- this should ensure that the .tex files weaved via Sweave / knitr are now also rendered by tinytex.

@yihui
Copy link
Member Author

@yihui yihui commented Dec 13, 2020

Awesome! Thanks @kevinushey!

@yihui
Copy link
Member Author

@yihui yihui commented Jan 16, 2021

Here is an .Rnw document that I use to test the "Compile PDF" button in the IDE:

\documentclass{article}

\begin{document}

<<>>=
# tinytex should automatically reinstall this package later
tinytex::tlmgr_remove('framed')

# test bibliography
knitr::write_bib('base', 'test.bib')
@

\cite{R-base}

\bibliography{test.bib}
\bibliographystyle{unsrt}
\end{document}

If this feature is implemented correctly and you have TinyTeX installed (tinytex::install_tinytex()), you should see that 1) the LaTeX package framed should be automatically reinstalled; 2) the bibliography should be generated (\cite{R-base} generates [1] instead of ??).

1) The log

processing file: test.Rnw
  |...............................                                                               |  33%
  ordinary text without R code

  |...............................................................                               |  67%
label: unnamed-chunk-1
[1/0, ??:??/??:??] remove: framed
tlmgr: ultimately removed these packages: framed
running mktexlsr ...
done running mktexlsr.
tlmgr: package log updated: ~/Library/TinyTeX/texmf-var/web2c/tlmgr.log
  |..............................................................................................| 100%
  ordinary text without R code


output file: test.tex

tlmgr: package repository http://ctan.math.utah.edu/ctan/tex-archive/systems/texlive/tlnet (not verified: gpg unavailable)
[1/1, ??:??/??:??] install: framed [8k]
running mktexlsr ...
done running mktexlsr.

2) The PDF output

image

@R-Hannibal R-Hannibal assigned ronblum and unassigned rich-rstudio Jan 20, 2021
@ronblum ronblum added test and removed test labels Jan 21, 2021
@ronblum
Copy link
Contributor

@ronblum ronblum commented Jan 27, 2021

Verified in

  • RStudio Desktop 1.4.1507 on MacOS 11.1
  • RStudio Server 1.4.1512 on Red Hat 8.3 with Chrome on MacOS 11.1

Tested using sample code from http://mally.stanford.edu/~sr/computing/latex-example.html and https://www.overleaf.com/latex/examples/dodecahedron-desk-calendar/mrjbknbbrjhy

@kevinushey
Copy link
Contributor

@kevinushey kevinushey commented Feb 4, 2021

Can also confirm the example in #2788 (comment) works as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
10 participants