Provide a binary?

[wspr]

Logging this here for further discussion or just so I remember; no urgency.

I think there might be some benefit for us to distribute a script called l3build with chmod+x to more easily document (and add some features) that behaved essentially like texlua build.lua but with some extra possibilities.

Benefits:

• Slightly easier to document how to use l3build, and slightly easier for new users to get started
• Possibility of l3build init which could generate a local build.lua file (a definite plus from my perspective)
• Possibility of l3build newproject which could generate a set of template files in a “standard form” that fits our view of best practice (not sure we could ever agree on this but might be a nice idea...)

None of these are show-stoppers, really, but I do think it would be nice for new users to be able to head to their command line and write l3build and have something sensible happen, even if it's just a sensible error message like what current happens when you write texlua build.lua — the main thing being that you wouldn't already need to have a build.lua file in the current directory!

[josephwright]

For reference, the reason for the current approach was that it is relatively easy to use across platforms and without any particular additions to the TeX system in use. In particular, I'm not sure how one deals with 'local' installs for an executable (can't just go in ~/texmf). On Windows at least there is also a need to have a stub wrapper somewhere to allow the program to run.

[wspr]

Hmm, I see what you mean for local installs. I guess if the wrapper is simple enough then we shouldn't need to change it much; e.g., something like

#!/usr/bin/env texlua

dofile("build.lua")

is ‘enough’ for the dead-simple use case. Anything dealing with new options can still be written as .lua files installed in texmfhome and loaded using the kpse library. I don't know what is needed from the TeX Live / CTAN perspective to indicate a programme to be installed in the bin directory though!

I also hadn't considered the Windows side of things. I suppose that makes the whole effort more difficult...

[davidcarlisle]

If you have #!/usr/bin/env texlua at the top of the build.lua file and make it executable you can use ./build.lua instead of texlua build.lua so you don't really need a separate executable, you don't need the lua extension so you could call it ./foobuild instead (not ./build though unless you change the default build directory:-) the #! probably doesn't do anything in the windows shell, but it doesn't stop its use as texlua foobuild

…

On 23 January 2018 at 22:18, Will Robertson ***@***.***> wrote: Hmm, I see what you mean for local installs. I guess if the wrapper is simple enough then we shouldn't need to change it much; e.g., something like #!/usr/bin/env texlua dofile("build.lua") is ‘enough’ for the dead-simple use case. Anything dealing with new options can still be written as .lua files installed in texmfhome and loaded using the kpse library. I don't know what is needed from the TeX Live / CTAN perspective to indicate a programme to be installed in the bin directory though! I also hadn't considered the Windows side of things. I suppose that makes the whole effort more difficult... — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#50 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABNcAjOLBf0fOzRF5BMWNTZiUdH9t1sxks5tNlqugaJpZM4RpLLj> .

[wspr]

Yes, for that level of convenience I agree having a separate binary offers little benefit — for me, I think an easier approach is to have a bash alias set up as build='texlua build.lua'.

Any advantage of a binary would be that users don't need to write their own build.lua script to get started. (Assuming this then allows you to write something like l3build init, or even if it just gives instructions when l3build is executed without arguments.)

I know right now it's not hard! But every little bit helps I think.

Additionally, I think it's an easier sell to tell people to just run l3build ctan.

[FrankMittelbach]

Am 23.01.2018 um 23:37 schrieb Will Robertson:

Additionally, I think it's an easier sell to tell people to just run |l3build ctan|.

not sure in my opinion. I think it is an easy sell to tell them texlua build.lua ctan and tell them **this works on any platform for you** the only thing that bugs be about the above is that our standard file is called build.lua and we have by default a "build" directory meaning you can't autocomplete on shells I think the platform independence is a goods thing and defining a shell wrapper or alias or anything depending on your system and depending on how often you use this is simple enough (we could provide a short section with examples for this if you really think that is that helpful)

[wspr]

Well, you actually have to tell them: * first create a file called build.lua and populate it with module = ‘foo’ kpse.set_program_name("kpsewhich") dofile(kpse.lookup("l3build.lua”)) * *then* run `texlua build.lua` and this works on any platform But maybe I’m overthinking this. If everyone is happy with the current approach, no need to add additional complexity. Frank’s suggestion to add something to the docs about this is a good one. I’ll leave this open and close when I do so. * * * Should we think seriously about changing the default `build` directory name to help with autocomplete and possible name clashes?

[blefloch]

I'm on Will's side. If providing a cross-platform executable with "init" capability is easy enough, that is.

[josephwright]

On the requirement for a build.lua, we currently would need it even with a binary to deal with module. That can easily be altered.

If we want to go down the binary route, I'd suggest we talk with the TL people and with Christian Schenk, and get l3build added as a 'first-class' tool. We'll likely need a bit of adjustment of how l3build itself gets released, but that is all doable.

[wspr]

Yes, agreed. Maybe I wasn’t quite clear enough about why I thought this would be a good idea. It’s not to simplify or change the way that l3build currently works today. To have it do anything meaningful, you would still need a `build.lua` file in the module folder. (Well, you could have the equivalent of module=os.execute("basename `pwd`”) and then have all other variables as default… but while that might be neat, it's not where I was aiming at.) I see two main advantages to a “binary”: 1. New user comes along, says “I hear l3build is really great” and then types in `l3build` into a command line; something sensible happens like: No build.lua file found. Generate one by typing `l3build init` or type `l3build help` for more information. This is somewhat analogous to typing `make` in a random directory. Providing sensible defaults is one good thing. Providing an easy-to-use template for a user to follow is, I think, an even better feature to add for a general tool. 2. For lazy people like me. I know how to run l3build and want to create a new build.lua script every other week. At the moment, because I’m lazy, I just copy/paste one that’s lying around somewhere. But that’s not very fun. For me, being able to write `l3build init` would save me time and headspace. (Okay okay, after all this I’ll probably write my own little script to do this in the mean time. But I’m sure it’s not just me that would benefit from it!)

[josephwright]

If we want to go for this we should make the decision: it will need some reworking of how existing build.lua files work (both the 'load the core' line and the variable setting will need to change). Should we ask the distro people about how easier it would be for them to add the necessary wrappers?

[davidcarlisle]

On 5 February 2018 at 09:35, Joseph Wright ***@***.***> wrote: If we want to go for this we should make the decision: it will need some reworking of how existing build.lua files work (both the 'load the core' line and the variable setting will need to change). Should we ask the distro people about how easier it would be for them to add the necessary wrappers? —

what does make4ht look like on windows texlive? on cygwin/linux it is just a lua script in the bin directory, with first line #!/usr/bin/env texlua -- Package make4ht. Author Michal Hoftich <michal.h21@gmail.com> -- This package is subject of LPPL license, version 1.3 kpse.set_program_name("luatex") so it executes via texlua. Does it have a binary executable wrapper for windows, or a .bat file or...? David

[josephwright]

@davidcarlisle At least in TeX Live it's a wrapper .exe file: I suspect that is true for MiKTeX too and for all scripts: Windows needs something that is a 'real' executable stub. It's all set up easily enough, just needs some help from the system maintainers.

[josephwright]

The biggest single issue would be that we'd need a new scriptfiles table as we'd need to put the script parts in a different place to the TeX part. All doable (as usual).

Overall, this might well be a good idea now we are getting wider take-up of l3build. I think we probably should raise it with the TL team and Christian Schenk, and agree on when to make the change. There will be a bit of re-working of the way we do over-riding of functions, which will be the most 'interesting' part (basically the entire logic has to get flipped).

[josephwright]

I notice that kpsewhich finds stuff in scripts, so we will be fine moving and having a 'transition', and also allowing local installation.

[josephwright]

Unless I hear otherwise, I'll start on this shortly, probably first by adjusting where the script installs to, then branching for a version that is intended to run directly not by running the build.lua script (so for testing texlua l3build.lua <target> until we have a script runner).

[wspr]

@josephwright — does the logic have to change or do you just think it's a good time to streamline things?

I was thinking more that you might have something like l3build-binary.lua which is the basis for the l3build executable; it might do its own argument parsing for a few small steps but then has something like

build_file = <arg parse> or "build.lua"
dofile(build_file)

and then everything continues as it currently does. Is that too clunky, do you think?

[josephwright]

@wspr The problem there is that current build.lua files end with lines to load l3build: I think we'd end up with issues avoiding double-loading, etc.

We also gain an opportunity to tidy up some of the load order stuff with a binary: it loads first so we get l3build (bulk) then build.lua then (at the end of l3build) fire main(). That means that the build.lua file can rely on functions being defined when it loads, so it would address e.g. the issues you've had with the manifest target. That wouldn't work if you try 'both routes'.

More widely, I guess I'd like a single approach. Looking at the needs for a binary, they are actually pretty small. So we could make everyone's life easier and be able to do

l3build check -epdftex

or similar 'universally'. Would be good learning experience for any future format (where a ConTeXt-like sctipt is I think a must).

[wspr]

@josephwright — not complaints from me on a reordering of the loading order and so on; I just wanted to make sure. I agree it makes life easier for certain parts of the code.

Good point about the ConTeXt-like script! I would always want a default "just run it once", I think, but it make sense to build support in for code to indicate when it needs a re-compile, with a matching tool to make it all happen.

[FrankMittelbach]

Unless I hear otherwise, I'll start on this shortly

I don't have much of an opinion on this topic so please go ahead if you think this is the right move. It would be good though (I think) if there remains a fallback using only core luatex to process in the future.

[josephwright]

@FrankMittelbach To be clear, a binary here is only needed for Windows and is a stub: what we are really talking about is allowing l3build.lua to be a script run directly rather than needing texlua <some name>. The suggested change would still allow

texlua l3build.lua

with the biggest actual alteration being that you then have a fixed name for the config script (more like make).

[FrankMittelbach]

Having just started a few new projects I now see a different "need" for the binary (already mentioned in @wspr original issue): the ability to start a project without the need to write a build.lua first, eg

l3build --name "Frank Mittelbach" 
        --mail frank.mittelbach@latex-project.org
        --license lppl
        --type simple-flat
        --...  (other CL args, eg looks at dtxgen or sty2dtx)
        <module-name>

most of the args could be default, eg mail and name could be $MAIL and $NAME if set, license could default to lppl, etc

This way one could get with a single line a while template structure being build with basic test files, .ins file proper license info etc etc

I think this would be really valuable.

[josephwright]

@FrankMittelbach OK, as we are in the TL freeze period we have a window to do this: I'll action.

[josephwright]

The core actions required here are:

• Move installation target to scripts: probably use a new scriptfiles value for this (will support others)
• Adjust the loading to allow direct calls to the script and to auto-run a build.lua file
• Change --version to --tag (see Rework setversion #40)
• Add some basic form of manpage data (requested by Karl Berry)

[FrankMittelbach]

looks good to me

[josephwright]

I'll probably branch for all of this: I'd rather have somewhere 'safe' to work on the changes.

Two areas to note on build.lua. First, I think we can pick up both current and 'new' usage (so texlua build.lua versus texlua l3build or just l3build. Second, I think it makes sense to load build.lua 'late' such that it can access globals (see @wspr's issues setting up manfest). To do that, I'll arrange to read l3build-variables.lua twice: once 'early' to set up the global names, then again after reading build.lua. Fingers-crossed this will keep the current behaviours but also allow better set ups for other stuff.

[FrankMittelbach]

I'll probably branch for all of this: I'd rather have somewhere 'safe' to work on the changes.

sounds sensible

Two areas to note on |build.lua|. First, I think we can pick up both current and 'new' usage (so |texlua build.lua| /versus/ |texlua l3build| or just |l3build|.

guess so. one needs to clearly define what what incarnation is supposed to do then I guess it falls into place more or less l3build -- display usage l3build --<setup-options> -- setup a new data structure including build.lua, testdir etc l3build <goal> <options> -- like texlua build.lua <goal> <options> if build.lua is in the current dir etc

Fingers-crossed this will keep the current behaviours but also allow better set ups for other stuff.

:-)

[josephwright]

@FrankMittelbach I was thinking you'd have l3build init --<setup-options>, i.e. one always needs a <goal> for something to happen.

[FrankMittelbach]

yes right you are, but that makes it even simpler I guess (except that init should balk if there is already a structure of some sort)

[josephwright]

The first two points of my list are done: they are by far the most important. I think everything still works: could some other people check?

Are we happy sticking with a hard-coded config name of build.lua. I note that things like make use a fixed name: I suspect we are fine but at least want to check. (We have the --config option but that's really meant for something subtly different, though doubtless we use it as an override if required.)

[FrankMittelbach]

Am 06.03.18 um 08:51 schrieb Joseph Wright:

The first two points of my list are done: they are by far the most important. I think everything still works: could some other people check?

I removed the load of l3build in my local build file to see if that works. but how do you set up l3build to actually work (on OSX)? after texlua l3build install it ends up in a local scripts dir but that is not in any path nor would that file be executable if it is had to do texlua `kpsewhich l3build.lua` check which then looked like it worked.

Are we happy sticking with a hard-coded config name of |build.lua|. I note that things like |make| use a fixed name: I suspect we are fine but at least want to check. (We have the |--config| option but that's really meant for something subtly different, though doubtless we use it as an override if required.)

I think a fixed name is fine I'm still a bit unhappy about the fact that "build.lua" is an important part of your project (should not be deleted) but "build" directory is a somewhat tmp object. It is less of a problem as you do no longer need "build-lua" in your command line once this is allready but still ...

[josephwright]

@FrankMittelbach On 'how will this work', we are in a 'flux' state just at the moment. If you run texlua l3build.lua for l3build itself, you will get the new interface. If you install it then do texlua build.lua <target> for say LaTeX2e it will also pick up the new script as that old build.lua construct has the lookup stuff. The only place you have to do a manual lookup is with a new-style build.lua for cases other than l3build itself. Once I upload to CTAN and Karl/Christian install the new code, the script will be directly callable using just l3build <target>, as for other Lua scripts (e.g. lwarpmk). That work (I think) on *nix by having a symlink from <texmf>/bin/<name> to <texmf>/texmf-dist/scripts/<name>. I'm not sure about 'local' installs (one might have to fiddle about a bit with symlinks), but for the bulk of users that will be fine. ('local' installs are really just an issue for the team.)

On the build/ directory, we could just change the standard name to say tmp/.

[FrankMittelbach]

Am 06.03.18 um 10:16 schrieb Joseph Wright:

On the |build/| directory, we could just change the standard name to say |tmp/|.

wouldn't call it tmp as that may be already used (well I do often) perhaps tmpbuild if we want to change at all. as i said, once we really use l3build <target> there isn't much problem left

[josephwright]

I think we are good to go: just need a man page (so we can do a prerelease already). If everything looks OK I'll merge.

[FrankMittelbach]

can't see why not (after understanding why it didn't work here due to version mixup in the local tree)

[josephwright]

Everything should now be in place: I'll get to CTAN to see what issues come up in pre-testing.

Probably we want to set a sunset period for the older approach: various bits of the code can be simplified if we don't need to support loading build.lua 'early'.

[FrankMittelbach]

Everything should now be in place: I'll get to CTAN to see what issues come up in pre-testing.

good work, thanks

Probably we want to set a sunset period for the older approach: various bits of the code can be simplified if we don't need to support loading |build.lua| 'early'.

we probably need to allow for the older approach for a good while

[josephwright]

Sure: was thinking at least a year. I mainly wanted to flag up that there are reasons we might want to consider this longer-term.