New version of Tex Live requires rebuild of images to fix TinyTex

[nuest]

When running Rocker images and using the included TinyTex, I get an error (probably since a few days ago):

$ docker run --rm -it rocker/verse:latest R -q -e 'tinytex::tlmgr_search("amsmath")'
> tinytex::tlmgr_search("amsmath")
tlmgr search --file --global 'amsmath'

tlmgr: Local TeX Live (2020) is older than remote repository (2021).
Cross release updates are only supported with
  update-tlmgr-latest(.sh/.exe) --update
See https://tug.org/texlive/upgrade.html for details.
> 

The same error was the case for rocker/verse:4.0.{0,1,2,3}, but not for rocker/verse:3.6.3 (only a warning there that Tex Live 2018 will not be updated anymore, but search still works).

Some more background is available at rstudio/tinytex#293

A possible solution probably is to wait until all images are rebuild and use a later tinytex installation, though I'm not sure if any pinning/configuration of CTAN mirrors happens in the Rocker image stack. In the meantime, triggering an installation of tinytex seems to work as well:

$ docker run --rm --pull -it rocker/verse:4.0.3 R -q -e 'tinytex::install_tinytex(); tinytex::tlmgr_search("amsmath")'
> tinytex::install_tinytex(); tinytex::tlmgr_search("amsmath")
tlmgr revision 57705 (2021-02-10 23:57:28 +0100)
tlmgr using installation: /usr/local/texlive
TeX Live (https://tug.org/texlive) version 2020
trying URL 'https://yihui.org/tinytex/TinyTeX-1.tar.gz'
Content type 'application/octet-stream' length 67122041 bytes (64.0 MB)
==================================================
downloaded 64.0 MB

tlmgr option sys_bin ~/bin
tlmgr: setting option sys_bin to /root/bin.
tlmgr: updating /root/.TinyTeX/tlpkg/texlive.tlpdb
You may have to restart your system after installing TinyTeX to make sure ~/bin appears in your PATH variable (https://github.com/yihui/tinytex/issues/16).
Warning message:
Detected an existing tlmgr at /usr/local/texlive/bin/x86_64-linux/tlmgr. It seems TeX Live has been installed (check tinytex::tinytex_root()). You are recommended to uninstall it, although TinyTeX should work well alongside another LaTeX distribution if a LaTeX document is compiled through tinytex::latexmk(). 
tlmgr search --file --global 'amsmath'
tlmgr: package repository https://ctan.math.illinois.edu/systems/texlive/tlnet (not verified: gpg unavailable)
00texlive.image:
        tlpkg/tlpsrc/amsmath-it.tlpsrc
        tlpkg/tlpsrc/amsmath.tlpsrc
        tlpkg/tlpsrc/latex-amsmath-dev.tlpsrc
        tlpkg/tlpsrc/onlyamsmath.tlpsrc
amsmath:
        texmf-dist/doc/latex/amsmath/README.md
        texmf-dist/doc/latex/amsmath/ams-external.txt
        texmf-dist/doc/latex/amsmath/ams-internal.txt

[...]

[cboettig]

Yeah, thanks for this. I think all we need to do is to set CTAN_REPO env var, it should have been getting locked in the previous 'frozen' versions, but I just didn't have the chance to figure out the new locked release URLs like we were doing in 3.6.x

Note that our installation is TinyTex compatible, but is not actually TinyTex-sourced: https://github.com/rocker-org/rocker-versioned2/blob/master/scripts/install_texlive.sh . Like TinyTex, it's a custom tlmgr install, but uses system-wide settings and is paired back even more (perhaps too much) in terms of the default packages.

Anyway, I think we just need to dig out the corresponding env var URL and set it in the stacks files. (There's a nice thread on version-locking somewhere in the rocker-versioned issues I think with advice from the maintainer of those).

Meanwhile, note that the update instructions you get from tlmgr when you hit this issue do actually work just fine. (basically says just download and run http://mirror.ctan.org/systems/texlive/tlnet/update-tlmgr-latest.sh).

Pulling in a fresh tinytex is fine, but it needs some extra sharffing to install in a shared dir and not in the home directory of root or a user. (tinytex has directions for this, which worked fine in the debian-based rocker 3.x but I recall having trouble with them when we switched to Ubuntu, so we moved to our own "standard" tlmgr install linked above.)

[cboettig]

Ah found it again. On the frozen images at least, we should be pointing to CTAN_REPO to the corresponding archive from http://www.texlive.info/tlnet-archive/, maintained tirelessly by @norbusan 🙏 .

For example, we should probably just lock the 4.0.4 images to http://www.texlive.info/tlnet-archive/2021/03/30/tlnet . Apologies for letting this regression slip in in the 4.x transition! We'll update the old 4.x images accordingly.

Maybe also relevant to @jeroen ?

[norbusan]

If lots of people pull from my side it will probably crash at some point, not the biggest and strongest and fastest server it is, since it is paid privately by me. If it is just got docket image builds now and then that should be fine.

[cboettig]

Thanks for the heads up @norbusan -- this would only be for our 'frozen' images (i.e. archival images of older versions), which built or used nearly as much.

[cboettig]

(p.s. @norbusan wish we could do something to support your server expenses all the same, but for the moment we're all-volunteer here, though we did receive a short-term supporting grant last year, from CZI who supports maintenance of widely used open source software. Drop me a line if I can at least make a few introductions at least?)

[norbusan]

@cboettig using the server for frozen images etc is absolutely fine, don't worry!!!

[nuest]

Thanks @cboettig for the background. I wasn't aware of the switch to TexLive with Rocker's 4.x images, will keep that in mind.

Happy to test run some things after re-builds, if that helps.

[cboettig]

@nuest just to be clear, it's not a switch to texlive apt packages. We're doing almost exactly the same thing tinytex is doing; we're both following the official https://www.tug.org/texlive/pkginstall.html for installing texlive packages without calling on the distro's package manager. So the 'switch' should (hopefully!) be invisible to end users anyway, it's almost all the same stuff being installed. TinyTex integration, including it's automatic install of dependencies on knitting, should work out of the box on rocker/verse images.

Some testing of that would be awesome! I've just pushed the stack changes, but will take a bit before the stacks rebuild so I can ping you again soon when that's done 🤞 really appreciate it!

[cboettig]

okay, frozen tex versions should be up on R 4.0.0, 4.0.1, 4.0.2, 4.0.3, and 4.0.4. latest/4.0.5 is unchanged, users who had a 4.0.5 container (or any other 4.x container pulled prior to this build) will still see the above message.

[cboettig]

@nuest any testing to confirm these are actually working as expected would be wonderful!