"luaotfload-names.luc" remains zero-sized, enforcing the regeneration of the font names database every time

[afborchert]

I've TeXLive 2021 on Solaris 11.4 including luaotfload-tool 3.18:

[dublin] theon$ luaotfload-tool --version

luaotfload-tool:
        Luaotfload font management and diagnostic utility.
        License: GNU GPL v2.0.
        Report problems to <https://github.com/latex3/luaotfload/issues>

luaotfload-tool version: "3.18"
Revision: nil
Lua interpreter: stock; version "Lua 5.3"
Luatex version: 1.13
Platform: type=unix name=sunos
    +  machine: i86pc
    + nodename: theon
    +  release: 5.11
    +  sysname: SunOS
    +  version: 11.4.18.4.0
No database metadata available.
[dublin] theon$ 

Whenever the font names database is regenerated, the file “luaotfload-names.luc” remains zero-sized, causing the regeneration to occur again on the next run. luaotfload-tool itself runs through without any errors (see attached output of “luaotfload-tool -v -vvv -u”). The same problem has been reported here: https://tex.stackexchange.com/questions/567678/luaotfload-always-being-invoke
luaotfload-tool-log.txt

[zauguin]

• What is the path of the zero sized luaotfload-names.luc?
• There should be a luaotfload-names.lua.gz next to it. Does this exists? Is it empty too?
• What happens if you remove both files, then add a file luaotfload.conf next to your TeX file with the context

  [db]
  compress = false

  and try to recompile? Is the newly generated luaotfload-names.lua empty?

[afborchert]

Thanks for helping. The following path is below my home directory:

[dublin] theon$ ls -l .texlive2021/texmf-var/luatex-cache/generic/names
total 522
-rw-rw-r--   1 borchert sai           11 Oct 13 15:57 luaotfload-lookup-cache.lua
-rw-rw-r--   1 borchert sai           88 Oct 13 15:57 luaotfload-lookup-cache.luc
-rw-rw-r--   1 borchert sai       233849 Oct 13 15:56 luaotfload-names.lua.gz
-rw-rw-r--   1 borchert sai            0 Oct 13 15:56 luaotfload-names.luc
[dublin] theon$ 

As you can see luaotfload-names.lua.gz isn't empty.

I've removed both files and added the configuration file. After lualatex I got again a zero-sized luaotfload-names.luc:

[dublin] theon$ ls -l
total 524
-rw-rw-r--   1 borchert sai         4694 Oct 13 17:29 luaotfload-lookup-cache.lua
-rw-rw-r--   1 borchert sai         2954 Oct 13 17:29 luaotfload-lookup-cache.luc
-rw-rw-r--   1 borchert sai       233849 Oct 13 17:29 luaotfload-names.lua.gz
-rw-rw-r--   1 borchert sai            0 Oct 13 17:29 luaotfload-names.luc
[dublin] theon$ 

[zauguin]

Please save the following under the name test.lua and run it with luatex test.lua:

local dumped = string.dump(function()end)
if type(dumped) == 'string' then
  print('Str: ', #dumped)
else
  print('Other: ', type(dumped))
end

What output does this give?

[afborchert]

[dublin] theon$ cat >test.lua
local dumped = string.dump(function()end)
if type(dumped) == 'string' then
  print('Str: ', #dumped)
else
  print('Other: ', type(dumped))
end
[dublin] theon$ luatex test.lua
Str:    91
[dublin] theon$ 

[zauguin]

Thanks. Could you try the same with

local dumped = string.dump(function()end, true)
if type(dumped) == 'string' then
  print('Str: ', #dumped)
else
  print('Other: ', type(dumped))
end

[afborchert]

[dublin] theon$ cat >test2.lua
local dumped = string.dump(function()end, true)
if type(dumped) == 'string' then
  print('Str: ', #dumped)
else
  print('Other: ', type(dumped))
end
[dublin] theon$ luatex test2.lua
Str:    78
[dublin] theon$ 

[zauguin]

Please try using the latest development version and test if it works any better: Clone this repo, checkout the dev branch and run l3build install in the main directory. Afterwards compiling with lualatex-dev should use the development version.

The code writing the .luc file has changed quite a bit, so it might work there. (Or not, I really have no idea what's causing this, but since the new code writes both files in basically the same way it might be different)

[afborchert]

Thanks again for your support but unfortunately the development version fails with following error:

luaotfload | db : Font names database not found, generating new one.
luaotfload | db : This can take several minutes; please be patient....t/t
exmf/tex/latex-dev/luaotfload/luaotfload-database.lua:3518: [string "t"]:16522:
 malformed number near '0x0.0000000000000p+0'.
<to be read again> 
relax 
l.112 \normalsize
               
? 

None of the database files were updated. Apparently, the crash occured within

    if file_luc then
        local compiled = dump(assert(load(serialized, 't')), true)
        file_luc:write(compiled)
        file_luc:close()
    end

within the load.

[zauguin]

Well, at least we get a proper error message now.

Solaris's standard C library has two modes: A C99 conforming mode and a C90 conforming mode. While Lua could work with either one of them, compiling Lua without a C99 conforming standard library requires the LUA_USE_C89 compilation option.

The behavior you see shows that your LuaTeX is built with the C library in C90 mode but LUA_USE_C89 is not set. It should probably be fixed by switching the standard library to C99 mode. Anyway this should be reported to the TeX Live team. (@kberry ?)

[afborchert]

This is indeed the problem. I can confirm this and provide a workaround:

LD_PRELOAD=/usr/lib/32/values-xpg6.o lualatex-dev lecture01.tex

runs through and generates successfully the cache. And the same trick can also be applied for lualatex from TeXLive 2021.

Thank you for your help and your work on luaotfload. Adding error messages is surely an improvement! I hope that this will be fixed for TeXLive 2022 and the next time it would also be great if all this could be compiled with “-m64”. There is no good reason for a restriction to a 32-bit address space.

I will now patch up our TeXLive 2021 installation with wrappers setting LD_PRELOAD.

[kberry]

LuaTeX build setup is determined by Luigi (Scarso). I will write him. -k

[zauguin]

This is indeed the problem. I can confirm this and provide a workaround:

LD_PRELOAD=/usr/lib/32/values-xpg6.o lualatex-dev lecture01.tex

runs through and generates successfully the cache. And the same trick can also be applied for lualatex from TeXLive 2021.

Great, one mystery error off the list.

I will write him. -k

Thanks!

[mojca]

I hope that this will be fixed for TeXLive 2022 and the next time it would also be great if all this could be compiled with “-m64”. There is no good reason for a restriction to a 32-bit address space.

Just to clarify. TL provides two sets of binaries: i386-solaris and x86_64-solaris. Which part exactly seems to be missing the -m64 flag when you use the x86_64 binaries?

[afborchert]

./install-tl installed just i386-solaris, not x86_64-solaris. How to select x86_64-solaris and why isn't this the default? Please note that Solaris 11 supports just 64-bit architectures. There is no reason to have a 32-bit version of an 64-bit-only operating system. 32-bit binaries are still supported but they shouldn't be the first choice.

Right now, config.guess delivers "i386-pc-solaris2.11" as it tries cc first, catching the Oracle Studio compiler cc which, by default, generates 32-bit binaries. Hence, the test at lines 439–441 fails as long as "-m64" isn't added to the flags:

theon$ (echo '#ifdef __amd64'; echo IS_64BIT_ARCH; echo '#endif') | cc -E -
# 1 "<stdin>"
#ident "acomp: Studio 12.5 Sun C 5.14 SunOS_i386 2016/05/31"
theon$ (echo '#ifdef __amd64'; echo IS_64BIT_ARCH; echo '#endif') | cc -E -m64 -# 1 "<stdin>"
# 2
IS_64BIT_ARCH
#ident "acomp: Studio 12.5 Sun C 5.14 SunOS_i386 2016/05/31"
theon$

[mojca]

You have already answered yourself: if config.guess reports i386, then TL will pick the 32-bit binaries.
TL can override the choice, but we probably need a way to check whether 64-bit binaries will work. What I don't know is whether this change may be made in the middle of the year, as that might completely break the existing installations. Most likely this would need to wait for deployment for TL 2022 (but of course it could be written earlier).

@norbusan.

[afborchert]

You can skip this test for Solaris 11 as Solaris 11 does not support any 32-bit architecture. And if you still want to check if 64-bit binaries will work you need to specify "-m64". This is required to generate 64-bit binaries for Oracle Studio compilers, for the gcc provided by Oracle, and for a gcc bootstrapped from its sources using a default configuration.

I do not need a fix for TL 2021 as I have a workaround (see above). But I would be glad if this problem doesn't reappear when I install TL 2022 on Solaris. Thanks for your work!

[norbusan]

Please note that Solaris 11 supports just 64-bit architectures. There is no reason to have a 32-bit version of an 64-bit-only operating system. 32-bit binaries are still supported but they shouldn't be the first choice.

Right now, config.guess delivers "i386-pc-solaris2.11" as it tries cc first, catching the Oracle Studio compiler cc which, by default, generates 32-bit binaries. Hence, the test at lines 439–441 fails as long as "-m64" isn't added to the flags:

So we don't want to patch config.guess since it comes from upstream gnu, that would be futile. THere are a few options:

• send patches to upstream so that config.guess correctly detects the arch
• if solaris2.11 always is guarantueed to mean 64bit is supported, we can adjust the arch detection code on our side (we already do some mangling of the return code of config.guess)

But as @mojca said, doing this during the year sounds a bit dangerous to me ...

[afborchert]

@norbusan I've sent a patch to the folks maintaining https://savannah.gnu.org/projects/config/.

[mojca]

But just to confirm: does the code work correctly with 64-bit binaries, or are there any further changes needed in the TL sources?

You can either force the usage of a different set of binaries, or install additional set(s) of binaries when running the installer, and just adjust the PATH appropriately.

(I tried to request some changes for config.guess a few times, but usually they had a different opinion about what the correct answer was supposed to be. Sometimes you need some luck. If they accept the patch, let us know, so that it can be updated in TL sources as well during the TL 2022 pretest.)

[kberry]

We automatically install all changes to config.* (among other files) when they are released upstream. Hope the current config.* maintainer (Dmitry Levin) likes your patch or fixes it in another way. Sometimes he needs nudging (more than once). Although changing it will cause trouble for users, I am reluctant to turn off config.* updates for several months, or manually merge them. We can figure out some workaround, I suppose. According to https://en.wikipedia.org/wiki/Oracle_Solaris, as well as Andreas's message :), Solaris 11 is indeed only for x86_64 and sparc, but I'm even more reluctant to override results until it's certain we need to do so. --thanks, karl.

[luigiScarso]

@zauguin solaris uses -pedantic -std=c99, but not for Lua5.3, and by default /opt/csw/bin/gcc-5.5 should use c11 .
Could be this the issue ?

[afborchert]

@luigiScarso Please do not depend anywhere on the nowadays obsolete CSW project. This has been a community project to support free software on top of Solaris but not been updated since years. In Solaris 11, gcc comes with the regular packages of the operating system and you have directories for various gcc releases under /usr/gcc and /usr/bin/gcc etc. pointing to the corresponding binaries of the gcc release of your choice.

[mojca]

We did depend on OpenCSW for TL 2021.
Solaris 10 is still kind of supported for two more years. I have no clue how to build any binaries on Solaris 10 without OpenCSW.

We might change the strategy for TL 2022 which would need some investigation first, but that should better be discussed on the TeX Live mailing list.

[afborchert]

Solaris 10 was shipping the SUNWgcc package with gcc 3.4.3 from 2004 which installed in /usr/sfw. Compared with that /opt/csw/bin/gcc-5.5 is indeed newer (from 2017) but I wouldn't expect anything newer coming from OpenCSW. The point is that Solaris 10 is an operating system from 2005 where regular support ended on 31 January 2018. Solaris 10 is mainly of interest for people who want to keep Solaris running but do not have a platform which is supported by Solaris 11 (like earlier instances of the sun4u architecture). However, it is becoming more and more challenging to compile software for Solaris 10. For example, you will not get full C11 support on Solaris 10 even when using the newest gcc as functions of the C11 standard library are missing in libc and in the corresponding header files.

[afborchert]

@norbusan Dmitry V. Levin has kindly applied my patch to config.guess: https://git.savannah.gnu.org/gitweb/?p=config.git;a=commitdiff;h=9e4c79f5fc4ef734b2f3115ab9184b70789e21a3

[plk]

This issue still appears to be around in TL 2022 fully updated on 64-bit Intel Solaris:

$ luaotfload-tool -u
luaotfload | db : Font names database not found, generating new one.
luaotfload | db : This can take several minutes; please be patient.
...texmf-dist/tex/luatex/luaotfload/luaotfload-database.lua:3521: [string "t"]:17380: malformed number near '0x1.4000000000000p+3'

The workaround above seems to work:

LD_PRELOAD=/usr/lib/amd64/values-xpg6.o lualatex ...