Image doesn't work anymore when using a Texlivefile now that TeXLive 2024 is out

[flyx]

The entrypoint now outputs:

Installing dependencies ...

tlmgr: Local TeX Live (2023) is older than remote repository (2024).
Cross release updates are only supported with
  update-tlmgr-latest(.sh/.exe) --update
See https://tug.org/texlive/upgrade.html for details.

There are two solutions to this:

• do update-tlmgr-latest.sh --update as suggested. However this will move to TeX Live 2024 which is probably not what we want in a build environment that should be somewhat reproducible.

• explicitly give the 2023 repository:

           repo="--repository https://ftp.math.utah.edu/pub/tex/historic/systems/texlive/2023/tlnet-final"
           tlmgr update $repo --self
           xargs tlmgr install $repo < "${SRC_DIR}/${TEXLIVEFILE}"
  

This seems desirable, however I didn't do a PR for this because there is an underlying question on how you want to deal with new TeX Live versions in general. For all I know, the shown repository solution can only be employed once the version has been frozen, so you can't simply release a 2024 image with an explicit repo to avoid this problem in the future.

It might be a good idea to do a check whether tlnet-final is available remotely and if so, fill the repo variable. I don't know TeX Live well enough to be content that this is the good solution.

[reitzig]

Yeah, that is bound to happen whenever TeXlive does this freeze in between releases of the image.

I could just hit the button and release a new version of the image that would then install 2024 to begin with?

[reitzig]

FWIW, I wouldn't want to bake a single mirror into the image. It might go away, or I might accidentally overload it.

[reitzig]

IMHO, getting a ticket once a year and then pressing a button is good enough. For me, anyway.

[reitzig]

✅ reitzig/texlive-docker@release-2024.1

[reitzig]

Or is your intent to be able to keep using 2023 images? 🤔

In that case, I'm open to suggestions that do not involve baking individual mirrors into public images.

[flyx]

Or is your intent to be able to keep using 2023 images?

My primary interest is that I do not want my automation to suddenly break just because a new TeX Live version was released. So my wish is for a 2023 image to continue working as it did before, because even though upgrading the image is a small task, the build randomly breaking when you're in the middle of a release is pretty stressful.

TeX packages generally do a pretty good job with being backwards-compatible, so explicitly using an old version is not that much of an important use-case, I think (though I did encounter breaking changes in packages between major releases).

In that case, I'm open to suggestions that do not involve baking individual mirrors into public images.

I looked around a bit but this question on TeX Stack Exchange says that generally there's no CTAN mirror URL for historic releases.

A possible solution may be to have an action to download the packages in TEXLIVEFILE without installing them. For example:

• add an env variable PKG_DIR, which can point to a directory that will be mounted
• add a command download that fetches each package in TEXLIVEFILE and puts it into PKG_DIR
• in the work command, install each package in PKG_DIR with tlmgr install --file

This would allow the user to collect required packages once and then inject them locally without involvement of the repository. Besides being a solution for TeX Live releasing a new version, this would also make it possible to cache the packages in GitHub actions.

While I did find tlmgr install --file, I didn't find a command to download but not install a package. This should surely be possible somehow. If you find this suggestion viable, I could dig deeper.

[reitzig]

To be honest, I don't follow your proposal. 😅

Let me see if I got this right:

My primary interest is that I do not want my automation to suddenly break just because a new TeX Live version was released. So my wish is for a 2023 image to continue working as it did before

Do I understand correctly that

• you have a fixed (set of) document(s) that
• you(r users) actively work on,
• you want to keep packages mostly the same, most of the time for that specific build, and
• don't mind some work upgrading so long as you get to choose when?

In that case, my clear recommendation would be to build your own image, based on mine (or any other, for that matter). That's only a few lines of Dockerfile and a simple workflow to store the image in GHCR, or whichever image registry is closest to your build.

Happy to provide guidance in the form of a demo in this repo here, if that helps! 🙃

--- (some side notes below)

As far as I understand TeX Live, you're basically forced to work with the packages you have, or upgrade to the latest release/year. That fundamentally clashes with the need to keep a system (here: container image) stable and also add/update package. Luckily, container images enable us to freeze multiple states, one for each of our documents if need be.

❓With tlmgr, can you install a specific version of a package? That is, could we pin package versions in our Dockerfiles, both for documentation and reproducibility?

💡We could add build and ENV vars to choose a fixed mirror/repo at build resp. runtime.
❔Would that help adding/upgrading packages in old images?

❓Would rolling tags for major versions, i.e. :2023 and :2024, help any? Not with the issue above, clearly, but generally?

[flyx]

Do I understand correctly that

• you have a fixed (set of) document(s) that
• you(r users) actively work on,
• you want to keep packages mostly the same, most of the time for that specific build, and
• don't mind some work upgrading so long as you get to choose when?

That sums it up nicely. The number of documents may grow but each document has the same dependencies as they are just translations of each other.

In that case, my clear recommendation would be to build your own image, based on mine

Probably not a bad idea in my case. I do have my own image but it's being built on the fly by a GitHub action as suggested by the docs. I didn't know about GHCR which seems to be the preferable solution, strange that the docs don't mention it there.

Generally, I hoped that there was some obvious solution for the base image to not stop working with a TEXLIVEFILE when a new TeX Live version is released, but that doesn't seem to be the case.

💡We could add build and ENV vars to choose a fixed mirror/repo at build resp. runtime.
❔Would that help adding/upgrading packages in old images?

Having that would indeed be helpful.

Would rolling tags for major versions, i.e. :2023 and :2024, help any? Not with the issue above, clearly, but generally?

I don't see much benefit compared to just directly using :latest, since :2023 would stop working as soon as :2024 becomes available. So there doesn't seem to be a need to differentiate.

---

Thank you very much for your comments, this helps me chart a way forward. I guess I'll go with pushing my own image to GHCR.

I do suggest putting information about this problem in the Readme.

[reitzig]

Thanks for your feedback!

My todos are:

• Add environement variable to the image; proposed name TEXLIVE_REPOSITORY; default unset/empty.
• Update entrypoint.sh so any tlmgr calls use that repository, if set.
• Update README to explain how to keep using "old" images (i.e. by setting the variable).
• Provide an example/demo for the "keep builds stable" scenario, using a custom image persisted to GHCR by a workflow.
  • This may end up being a link to @flyx's setup, if they're inclined to share. 😬

These seem to be non-breaking changes, so there's little risk in just rolling them out.

[flyx]

This may end up being a link to @flyx's setup, if they're inclined to share.

I'm afraid this is in a private company repository which I can't share.

Once I have set it up properly, I can contribute an example based on my setup. This may take some time.

[reitzig]

@flyx, if you find the time to check out #19, what do you think?

[reitzig]

You can backport this change to older images (and set the historic repo) like so:

FROM reitzig/texlive-minimal:2023.3

ENV TEXLIVE_REPOSITORY="https://ftp.math.utah.edu/pub/tex/historic/systems/texlive/2023/tlnet-final"
RUN wget -O "/bin/entrypoint" "https://raw.githubusercontent.com/reitzig/texlive-docker/d16e2669d8dd5dbea4d9875783836a5ebb620723/entrypoint.sh"

[reitzig]

Provide an example/demo for the "keep builds stable" scenario, using a custom image persisted to GHCR by a workflow.

I think I'll pull this out into a separate issue since it's rather unrelated.
➡️ #20

[reitzig]

I decided to merge this in time for the monthly release; still open to feedback, especially for the documentation.

[flyx]

Thanks for improving this!

I think it is worth mentioning in the Readme that building your own image is also a solution to avoid this problem. Currently it does describe the possibility, but not that it solves the problem.

I still want to do a proper example for doing this. Sadly it's always hard to find time for a thing that is not currently broken.

[reitzig]

Is 6ea3353 what you had in mind?

[flyx]

Yes, exactly.