Compiling Lua

[ahgamut]

I thought it might be fun to compile a portable version of a language which has C source code.
I picked Lua, and I eventually got it to compile and run on Windows/Linux.

I thought some of the compilation errors might be useful info:

• The following functions/macros in the Lua source clashed names with those in cosmopolitan.h: panic, reverse and isempty.
• strcoll, tanh, sinh, and cosh are unimplemented (leads to linker error)
• UCHAR_MAX in limits.h of the C stdlib is not defined even though SCHAR_MAX is
• Maybe similar to printf doubles with %lf gives nan #58, but floats don't print in the portable Lua console

[jart]

Outstanding! You successfully built Lua as a 336kb Actually Portable Executable using Cosmopolitan! Now we know exactly what features are missing and which things need to change. I intend to get each one of these things fixed. This issue will be updated live as that happens.

[ahgamut]

I forked the Lua Github mirror and created a cosmopolitan branch from Lua 5.4.2.

https://github.com/ahgamut/lua/tree/cosmopolitan

Change COSMO_LIBDIR in the makefile to the appropriate folder containing cosmopolitan.h and friends, and run make.

Just for fun, the git diff due to the above changes (excluding the README and dummy files) is:

23	8	makefile
18	18	ltable.c
6	6	lgc.c
5	5	lapi.c
4	4	lauxlib.c
4	3	lvm.c
3	1	loslib.c
3	0	lmathlib.c # filled in sinh, cosh, tanh
2	2	lvm.h
2	2	ltests.c
2	2	liolib.c
1	1	luaconf.h
1	1	lua.c
1	1	lobject.h
1	1	loadlib.c
1	1	llex.c
1	1	ldo.c
 17 files changed, 78 insertions(+), 57 deletions(-)

[ingenieroariel]

Passing by to ask a somewhat related question, what would need to be changed in order to compile lua as a regular elf with cosmopolitan instead of an ape?

On nixos it gets written to a read only location and the binary does not run, I assume it is related to the "self updating headers":

Failed to execute process '/nix/store/0bxwk7hyscxlc0a6fr3wlrzjba4bcp02-lua-unstable/bin/lua'. Reason:
exec: unknown error (errno was 8)
The file '/nix/store/0bxwk7hyscxlc0a6fr3wlrzjba4bcp02-lua-unstable/bin/lua' is marked as an executable but could not be run by the operating system.

with import <nixpkgs> {};

let
  cosmopolitan = import ./cosmopolitan.nix ;
in stdenv.mkDerivation rec {
  pname = "lua";
  version = "unstable";

  src = fetchFromGitHub {
    owner = "ahgamut";
    repo = "lua";
    rev = "aa20d02dffc4f96d673d30e7378c0c02a6df13f8";
    hash = "sha256-GdXFk4AYE8EP886r7tQk+//rTm4sFMu0x2cVyCyrFgM=";
  };

  nativeBuildInputs = [ makeWrapper cosmopolitan gcc ];


  preBuild = ''
    makeFlagsArray=(SHELL=/bin/sh "MKDIR=mkdir -p" "COSMO_LIBDIR=${cosmopolitan}/lib")
  '';

  installPhase = ''
    runHook preInstall
    mkdir -p $out/{bin,lib/include}
    make
    cp lua.exe $out/bin/lua
    runHook postInstall
  '';
}

with import <nixpkgs> {};


stdenv.mkDerivation rec {
  pname = "cosmopolitan";
  version = "unstable";

  src = fetchFromGitHub {
    owner = "jart";
    repo = "cosmopolitan";
    rev = "cdc54ea1fd86253ced49dd24d45660cadac1d018";
    hash = "sha256-vIDNWaM2+ztwx9hknl5+3ENkx9FxH7/VQ8uUW/sckyg=";
  };

  nativeBuildInputs = [ makeWrapper ];

  preBuild = ''
    makeFlagsArray=(SHELL=/bin/sh "MKDIR=mkdir -p")
  '';

  installPhase = ''
    runHook preInstall
    mkdir -p $out/{bin,lib/include}
    install o/cosmopolitan.h $out/lib/include
    install o/cosmopolitan.h $out/lib
    install o/cosmopolitan.a o/libc/crt/crt.o o/ape/ape.{o,lds} $out/lib
    makeWrapper ${gcc}/bin/gcc $out/bin/cosmoc --add-flags "-O -static -nostdlib -nostdinc -fno-pie -no-pie -mno-red-zone -fuse-ld=bfd -Wl,-T,$out/lib/ape.lds -include $out/lib/include/cosmopolitan.h $out/lib/{crt.o,ape.o,cosmopolitan.a}"
    runHook postInstall
  '';
}

[benwbooth]

Wouldn't you just need to run lua first as part of the nix install? Maybe lua --version or something?

[mnemnion]

Strikes me that low-hanging fruit would be to 'namespace' all the macros and internal functions in cosmopolitan, so that you would have cosmo_panic instead of just panic.

I haven't played around with cosmopolitan/APE (yet) so this could be completely off base.

[jart]

Passing by to ask a somewhat related question, what would need to be changed in order to compile lua as a regular elf with cosmopolitan instead of an ape? @ingenieroariel

Try running make CPPFLAGS=-DSUPPORT_VECTOR=113 which disables Windows + Metal + XNU support. By turning off those three systems, the APE bootloader will choose to produce an ELF binary. See https://justine.lol/cosmopolitan/howfat.html and please note that this functionality got added to Cosmopolitan after the 0.2 release so you'll need to be sure you're using the latest nightly until I can cut a 0.3 release. https://justine.lol/cosmopolitan/download.html

[jart]

@ingenieroariel It's also worth mentioning that the Cosmopolitan Makefile has similarly strict build-time requirements. The way we're currently approaching this problem is by using a wrapper program that creates atomic copies when it's asked to launch a .com binary. Since it uses vfork() and copy_file_range() it's just the fastest thing imaginable. Probably 10x faster than doing the same thing using a shell script.

cosmopolitan/tool/build/compile.c

Lines 523 to 543 in 839e2f4

	/*
	* create temporary copy when launching APE binaries
	* and we help FindDebugBinary to find debug symbols
	*/
	if (!IsWindows() && endswith(cmd, ".com")) {
	comdbg = xasprintf("%s.dbg", cmd);
	cachedcmd = xasprintf("o/%s", cmd);
	if (fileexists(comdbg)) {
	AddEnv(xasprintf("COMDBG=%s", comdbg));
	}
	if (FileExistsAndIsNewerThan(cachedcmd, cmd)) {
	cmd = cachedcmd;
	} else {
	if (startswith(cmd, "o/")) {
	cachedcmd = NULL;
	}
	originalcmd = cmd;
	cmd = xasprintf("%s.tmp.%d", originalcmd, getpid());
	copyfile(originalcmd, cmd, 0);
	}
	}

[jart]

@ahgamut I believe I've addressed everything now in the previous three linked changes. Please take a look and let me know if there's anything else you need. Thank you again for your contribution. The community is very excited about the work we're doing.

[ahgamut]

I have listed out the errors that I have seen/fixed during Lua compilation.
(Unsure which of these are within the scope of Cosmopolitan).

I am using the cosmopolitan branch in my fork of the Lua Github mirror:

• Lua v5.4.2 source
  • added dummy headers
  • modified makefile
  • minimal modifications to source files
• crt.o from the zip file and rest of cosmopolitan compiled from the latest commit.

Errors from earlier that do not appear now:

• UCHAR_MAX is available now
• no issues with setlocale/LC_ALL
• no issues with BUFSIZ/frame size errors
• sinh, cosh, tanh are now available (no linker error)
• no naming clashes for panic, reverse, isempty
• dummy headers for entire C stdlib avoid any #include problems
• created the following dummy headers (I could just comment out the includes in the source)
  • sys/wait.h (lauxlib.c)
  • sys/types.h (liolib.c)
  • unistd.h (loslib.c, lua.c)

Errors that still appear / require adding a fix to the Lua source code for successful compilation:

• strcoll is missing (linker error)
• localeconv()->decimal_point is not available, so just using .
• blocked out dlfcn.h from loadlib.c (it requires dlopen)
• ldo.c expects _setjmp/_longjmp if the LUA_USE_POSIX flag is enabled (I disabled this)
• LUA_USE_POSIX also requires the following functions in liolib.c

#if defined(LUA_USE_POSIX)
#define l_getc(f)		getc_unlocked(f)
#define l_lockfile(f)		flockfile(f) // linker missing the two below functions
#define l_unlockfile(f)		funlockfile(f)
#else
#define l_getc(f)		getc(f)
#define l_lockfile(f)		((void)0)
#define l_unlockfile(f)		((void)0)
#endif

With the above minor changes, lua.exe compiles successfully.
Later I will try some of the tests provided alongside the Lua source.

[jart]

Anything you need can potentially be in scope, so long as it isn't cross-compiling SDL DLLs for ARM with MSVC that run on BSD. Also I totally overlooked strcoll. Thanks for the checkboxes. That makes it much easier to digest. Reopening.

[ahgamut]

I think there might be some weird behavior when calling getc?

I was trying to run a sample file using lua -W sample.lua and got segfaults.
Examining the debug with gdb showed that the first getc call after opening the file was causing the segfault. The weird part is, I replaced getc with fgetc, and no segfault.

static int skipBOM (LoadF *lf) {
  const char *p = "\xEF\xBB\xBF";  /* UTF-8 BOM mark */
  int c;
  lf->n = 0;
  do {
    // this is the first getc call after opening the file successfully
    c = getc(lf->f);
    // segfault
    if (c == EOF || c != *(const unsigned char *)p++) return c;
    lf->buff[lf->n++] = c; 
  } while (*p != '\0');
  lf->n = 0;
  return getc(lf->f); // if above is fgetc, this does not segfault
}

[jart]

getc() is a macro wrapper that inlines some of the functionality of fgetc(). I took a quick glance and the code appears ok. I'm not sure how what you describe would happen unless your build is somehow changing the layout of struct FILE. If you can produce a minimal complete program that lets me reproduce this then I can fix it.

[jart]

On second glance one possible cause is this might need parenthesis:

#define getc(f)    (f->beg < f->end ? f->buf[f->beg++] : fgetc(f))

Try changing that to:

#define getc(f)    ((f)->beg < (f)->end ? (f)->buf[(f)->beg++] : fgetc(f))

[ahgamut]

The macro doesn't seem to be the issue. I tried substituting the macro's generated code.
I also tried using a temporary variable instead of lf->f and that still crashed.
Somehow the value f->buf (as per the above macro) is not an accessible memory location. (Is it optimization-related?)

Using fgetc works, using getc_unlocked also works.

Minimal example (just opening a file and reading) works perfectly with getc and fgetc.
The file containing the above function lauxlib.c has not been changed in any way either.

[ahgamut]

Assuming that's not a major error, here's the current status of testing the compiled lua.exe.
The test scripts are part of the Github mirror, and are also available here.
From their site:

The test suite is not a product; it is just a tool for our internal use. It is available here by popular demand but we cannot give any kind of support for it. You're on your own.

Here's the current performance (partial indicates that Lua's internal APIs have not been tested) :
Passing almost half the tests so far :)

closure.lua          success
vararg.lua           success
locals.lua           success
cstack.lua           success
tpack.lua            success
calls.lua            success
utf8.lua             success
sort.lua             success
goto.lua             success
pm.lua               success
db.lua               success
coroutine.lua        partial
nextvar.lua          partial
errors.lua           partial
gengc.lua            partial
gc.lua               partial
heavy.lua            ignored
constructs.lua       failed
bwcoercion.lua       failed
literals.lua         failed
verybig.lua          failed
strings.lua          failed
bitwise.lua          failed
events.lua           failed
attrib.lua           failed
files.lua            failed
math.lua             failed
main.lua             failed
code.lua             failed
big.lua              failed
api.lua              failed

[ahgamut]

Lua 5.4.2 uses the %g and %Lg format strings for printing double and long double values.

I changed it to %f in luaconf.h and now portable Lua prints float values!

TIL %g is actually a format for printf. %g is defined as the shorter of %e and %f.

[jart]

I've implemented the other posix functions you need. We can't use LUA_USE_DLOPEN because Cosmopolitan statically links everything. We can do %g and %e but I've been holding off temporarily on that since I'd ideally want to do it in a way that keeps Hello World tiny. We export generalized dtoa which can be used for proper float formatting that's more readable than what the POSIX standard for printf() would otherwise allow. See https://justine.lol/cosmopolitan/documentation.html#xdtoa which works great with the gc() garbage collector macro.

[ahgamut]

Using gc works, but requires -std=gnu99 as Lua uses -std=c99 by default.
Can work around it by #define asm __asm__, though.
I'll stick with using %ffor now.

Available on https://github.com/ahgamut/lua/tree/cosmopolitan

Using the latest commit from this repo (3e19b96).
I changed all the #defines related to the header updates.

• Linker error: bad zip cdir that disappeared once I built cosmopolitan from scratch.
• (getc segfault still exists, so I've used fgetc)
• lua.exe compiles without error (1 warning about luaD_throw)
• lua.exe runs without error and prints floats as 2.1000000 (better than g)

testbench results (partial means success, but additional testing requires access to the internal API).
I'll look at individual tests later today to see if the failures are actually relevant (for example a couple of tests need dlopen).

Edit: a few tests (bitwise.lua) were failing because they relied on other files in the same directory.

bwcoercion.lua       success
literals.lua         success (after disabling a locale-changing test)
verybig.lua          success
closure.lua          success
bitwise.lua          success
vararg.lua           success
locals.lua           success
cstack.lua           success
attrib.lua           success
tpack.lua            success
calls.lua            success
utf8.lua             success
sort.lua             success
main.lua             success
goto.lua             success
pm.lua               success
db.lua               success
coroutine.lua        partial
nextvar.lua          partial
events.lua           partial
errors.lua           partial
gengc.lua            partial
gc.lua               partial
heavy.lua            ignored
all.lua              ignored
constructs.lua       failed
strings.lua          failed
files.lua            failed
math.lua             failed
code.lua             failed
big.lua              failed
api.lua              failed
Success %            76.67

[jart]

I just gave it a build. Printf float formatting seems to be integral to the functioning of the language. We're definitely going to need an adequate implementation of %g and %e.

[ahgamut]

The pow function gives nan when the base is negative.
Minimal example:

int main() {
    double x = -3;
    double y = pow(x,3); // -27
    printf("%lf ^ 3 = %lf\n", x, y);
    return 0;
}

Output is

-3.000000 ^ 3 = -nan

[jart]

Note: I've posted an important update on helping to debug language integration issues in #97 (comment) The amalgamation build is now more helpful. You need to start using new compiler flags. You'll benefit with backtraces and function call tracing.

[jart]

I'd recommend reaching out to the Lua developers on their mailing list, and let them know that portable APE binaries for Lua is something you'd like to see happen. With a little bit of luck, the Lua team might like Cosmopolitan enough that they'll choose to adopt it for their releases. That would be the optimal outcome, since Lua binaries should ideally come from the Lua developers.

[niutech]

For now there is portable ULua.

[retsyo]

The problem is that not everybody has Linux, Mac or WSL with GCC toolchain and GNU make, or they don't want to spend time on building it. They may just want to quickly test the portable Lua interpreter, so thank you for allowing it.

you can post the lua.com on your github site, so that windows user, for example I, can download it. Thanks

[niutech]

@retsyo You can find Cosmopolitan binaries here: https://github.com/burggraf/awesome-cosmo

[jart]

That page is over a year old. Please see https://github.com/shmup/awesome-cosmopolitan All the official downloads come from

• https://justine.lol/
• https://redbean.dev/

[niutech]

@jart There should be an APE binary repository (app store), similar to Flathub, AppImagehub or Snapcraft, because not everybody can build its own binaries from source. burggraf/awesome-cosmo was a temporary solution, but it seems no longer updated. Maybe there could be an official app store established under justine.lol?
There are open source projects: snapcraft, kebe, etc.

[jart]

I agree but there's no incentive to creating app store for binaries which would be costly to closely monitor for abuse.

[paulwratt]

I agree but there's no incentive to creating app store for binaries which would be costly to closely monitor for abuse.

@niutech unfortunately the nature of APE binaries means they are easy to abuse. Even with vetting and hashes, to detect something that is intentionally hidden is not a simple task to automate. Just look at how complex JavaScript hacks can get, and those are plain ASCII text files.

From what I have seen in the repo, on the website, and on the Issues, it is not hard (or large) to create your own binaries, outside of those official download locations. To put this in a more familiar context, if we were talking about sandwiches, and I was (your) a parent, you know what my answer would be, right?

It may happen that the possibility of a store type repo with automated protection may become a reality, but that will be far of in the future and requires some foundation work which does not (and will not) exist ATM (I have a TODO that may assist in getting that off the ground, but its not likely to show results in the next 12 months - "my plate has just been filled up" ATM)

[jart]

APE is no different from any binary executable. It's harder to abuse the APE format because it stands out and bad guys want to blend in. Many of the features bad guys want, Cosmopolitan Libc doesn't have.

Right now our App Store is https://github.com/shmup/awesome-cosmopolitan which is a curated list of Cosmo/APE projects we created collaboratively on our Discord. As for burggraf, I don't know who he is.

[niutech]

@jart APE is no different from any binaries, yet app stores for them exist, e.g. AppImageHub, although I agree that it requires additional work in preventing malware. But maybe there is a free C++ compiler in the cloud, where newbies could easily clone an APE repo with Cosmopolitan libc and build it by themselves, ideally by one click, without installing GCC, GNU make and all the toolchain on their PC - something like StackBlitz for Node.js? Or a free CI, which would produce APE binaries from multiple source code repos? That way we would make sure that APE binaries are made from the original open source code.
FYI, @burggraf is Mark Burggraf, Senior Developer at Supabase.

[paulwratt]

where newbies could easily clone an APE repo with Cosmopolitan libc and build it by themselves, ideally by one click, without installing GCC, GNU make and all the toolchain on their PC

You just described a Docker image.

@niutech what you are describing is a custom build. The point @jart was making about her binaries is that they are a know quantity. You need to remember that in all honesty, if the single package bin (executable) is working for you, you dont need to update it. This mean there is practically no need for an update process.

Part of the original idea of a store, was to move the package visibility out of the confines of a repo. Unfortunately, the reality of the situation is that no store has got it right. To the point that the previous version of Ubuntu did not support Snap (out-of-the-box) and they designed that store system themselves. Ftalpak is not installed by default either.

I'll quote Explaining Computer regarding Snap!:

Whilst using Snap! rather than other technologies is potentially beneficial and transparent to many users, other find it controversial.

Take a step back for a minute and ask yourself why and how you ended up at APE & Cosmopolitan.

I understand the energy generated by the realisation of what APE, Cosmopolitan and Redbean can do, and spreading the word or giving it greater exposure is one of the 1st results of that. But I see it in the same light as SerenityOS, its not for mainstream (ie fast food) consumption. People who need to use it will end up finding it. People who contstantly need to update there software because someone else develops it, already have a huge selection of Apps and Locations to get them.

If you keep finding new and amazing APE developments or repo locations, and you want to "spread the word", I'd suggest submitting PR's to the awesome-cosmopolitan repo, not just add suggestions to the Issues there. You dont need top install any tools to do that, you can do it all online. Maybe over time it will expand into something that more resembles a store.

On the other hand, I for one (without an x86_64 machine) would appreciate any effort to provide an (online) on demand APE App compiler, if you would like to give that a crack. Particularly if you are interested in a union between MicroWindows and APE binaries, it might contribute to the vision you have for APE distribution and availability.

[jart]

Whilst using Snap! rather than other technologies is potentially beneficial and transparent to many users, other find it controversial.

Take a step back for a minute and ask yourself why and how you ended up at APE & Cosmopolitan.

@paulwratt I adore you for saying this.

On the other hand, I for one (without an x86_64 machine) would appreciate any effort to provide an (online) on demand APE App compiler

We're considering doing it because it's really hard to get a GCC toolchain that works for us on non-Linux. I've never felt fully comfortable recommending things like Homebrew packages which are maintained by people I don't know. Compiling GCC is frightfully difficult, and when it does compile, there's often minor incompatibilities or new linter warnings introduced between versions that cause folks to have a negative experience. What we ideally want is Cosmopolitan Libc to be good enough that it's ready to build a legacy codebase as complex as GCC and let it work the same on Windows and MacOS. But we're not there yet.

What I'd like to have is a 200kB APE binary that just asks one of the Linux servers I operate to build it for you, using our standard toolchain. That way if you use something like MacOS or Windows you won't need to setup a Linux VM inside WSL/Docker/VirtualBox/etc. before you can hack on Cosmopolitan.

The value prop is (1) is you'd only need to curl cosmocc.com, and (2) we can make builds go faster since we're pooling our resources, can cache things, and can have more cpu cores put to use, since right now our workstation cores are mostly idle while we're coding.

AppImageHub

@niutech That's an anonymous unlisted business with a bunch of white-labeled websites making false claims. I'd be surprised if anyone trusts them as a distribution platform for legitimate open source binaries.

But maybe there is a free C++ compiler in the cloud, where newbies could easily clone an APE repo with Cosmopolitan libc and build it by themselves, ideally by one click, without installing GCC,

That's already the case. If you git clone the cosmopolitan repository, it builds hermetically using a vendored toolchain. The git commits here are signed by my GPG key. It doesn't have dependencies. You don't need to build your artifacts in my cloud to have assurances about provenance. Our project is easier to build than any other project you'll find of similar complexity. While I'd love to offer more online services, we'd need to identify an opportunity that makes economic sense.

[burggraf]

Burggraf here :) I created https://github.com/burggraf/awesome-cosmo about a year ago just as a fan page. If you're maintaining a more up-to-date repo for that now, great. I'm a huge fan of your work, @jart. Thanks for what you're doing.

[jart]

@burggraf Thanks for stopping by and thanks for your work promoting the project last year! Since then we've started a Discord community and we'd be honored to count you as a member. https://discord.gg/Z4PXmtwD

[paulwratt]

( hmm.. my reply go lost somewhere. I can see it in my inbox )

On the other hand, ..
We're considering doing it ..

my comment was more of an "anti-spirit crushing" out ;)

(but is also a side-interest of mine - I'm doing something similar for m68k-atari-mint)

it's really hard to get a GCC toolchain that works for us on non-Linux.

maybe someone could have a look at the rust-musl-cross docker image spinner, the 1st step of which is compiling a gcc-musl
(here) - There might even already be a a Windows binary builder image somewhere, as he is referencing Azure in the final docker update step at the end of the log - also I was just reading another cloned gcc (v9.3.1) repo that had specific MinGW patches for Windows if anyone is interested (btw I am pro- vcc, just so you know)

What I'd like to have is a 200kB APE binary that just asks one of the Linux servers I operate to build it for you

that sounds a lot simpler from a user point of view. BTW I check that Workers server you use, but is seems APE is not possible there (yet), right?

I think over-all (and all round) maybe some more thought about how and what a user that doesn't (or can't) have a toolchain might want out of APE, Cosmopolitan, or Redbean.

But I also think its too soon for something substancial, outside of the awesome-cosmo repo - maybe it can incorporate more specific links (which is what @burggraf had right?), or sub sections - you know, evolve - which is also why I made that suggestion - and while thats happening others can mull over what missing to be able to provide a "give me this app now" APE (and that could become a psuedo- store)

Actually maybe thats the greater part about what was off putting in @niutech language, the context of "what constitutes a store" and "how a store is presented", so maybe the question should be "how to present a store to APE users, that does not look or feel like a store, and that is secure (unlike you average store)" - in which case @niutech might have managed to get some answers

[mingodad]

Out of curiosity what prevent the gcc compiler inside this repository to be built like redbean ?
I mean a standalone executable where all the header, object files needed to compile link are inside a zip file attached to it.

[paulwratt]

yeah, see now thats lateral thinking for you :) if it could at least do something like:

cosmocc.com --tiny myapp.c

and reference that appended zip internally without needing to unpack it (ie a re-binadable/extensible virtual fs - PhysicsFS by Icculus) that could help simplify an initiation into the world of APE's, and provide more advanced users and projects better fs layout opportunities

[mingodad]

I did an experiment trying to compile/buil c2m from https://github.com/vnmakarov/mir and got it to build but when trying to use it then several issues arised see here vnmakarov/mir#297

[paulwratt]

> On the other hand, .. We're considering doing it ..

@jart your busy atm .. let someone else have a play, come back to it when things slow down a bit (unless you can keep an eye on where the "someone else" is going with it) - my comment was more of an "anti-spirit crushing" out ;) (but is also an interest of mine)

it's really hard to get a GCC toolchain that works for us on non-Linux.

maybe someone could have a look at the [rust-musl-cross](https://github.com/messense/rust-musl-cross) docker image spinner, the 1st step of which is compiling a `gcc-musl` ([here](https://github.com/messense/rust-musl-cross/actions/runs/3087155053/jobs/4992234259#step:8:191)) - There might even already be a a Windows binary builder image somewhere, as he is referencing Azure in the final docker update step at the end of the log - also I was just reading another cloned `gcc` (v9.3.1) repo that had specific MinGW patches for Windows if anyone is interested

What I'd like to have is a 200kB APE binary that just asks one of the Linux servers I operate to build it for you

that sounds a _lot_ simpler from a user point of view. BTW I check that Workers server you use, but is seems APE is not possible there (yet), right? I think over-all (and all round) maybe some more thought about _how_ and _what_ a user that doesn't (or can't) have a toolchain might want out of APE, Cosmopolitan, or Redbean. But I also think its too soon for something substancial, outside of the `awesome-cosmo` repo - maybe it can incorporate more specific links (which is what @burggraf had right?), or sub sections - you know, evolve - which is also why I made that suggestion - and while thats happening others can mull over what missing to be able to provide a "give me this app now" APE (and _that_ could become a _psuedo-_store) Actually maybe thats the greater part about what was off putting in @niutech language, the _context_ of "what constitutes a store" and "how a store is presented", so maybe the question should be "how to present a store to APE users, that does not look or feel like a store, _and_ that _is_ secure (unlike you average store)" - in which case @niutech might have managed to get some answers A build on demand would be great, but APE's are static, you dont _need_ to have versions (or maybe you do, based on Cosmopolitan repo updates) - except maybe in certain circumstances, like say RedBean - but even then its not like a _must have_ based on what that is being used for already ( I replied in an email, I'm going to submit and review this a bit more, got longer than I expected )

[jart]

Anyone is free to set up their own platform for distributing APE binaries written by people other than themselves. I have no authority. However it'd be unlikely to earn my support, since if such a thing were to happen, it should be done by the person who created APE. In order for me to be interested in doing it, I'd want to see more people who are using APE to ship their own programs that they've written, on their own. Then once enough people have adopted APE as their preferred format for shipping their releases, then we can build a platform since it'd be solving a logistical problem for our users. Until then I'm happy to focus on creating better build tools, since they'd make APE more attracting to folks adopting it for releases.

[jart]

With that said, app stores aside, here's something that anyone can and should do. You can build your open source distro with APE. Our open source conformance just hit an inflection point yesterday that allows it; see e557058. If you read our toolchain instructions, you'll notice:

cosmopolitan/tool/scripts/cosmocc

Line 17 in 31dab8a

# ./configure --prefix=/opt/cosmos

This /opt/cosmos directory is where software goes after you successfully get it to build and run make install. I think we should turn this into a game where we all try to port as much software to cosmopolitan for ourselves, and then each of us can upload our /opt/cosmos folders to our own web servers with directory listings enabled, or in tarball format, binaries and all, as proof that we did it. Then anyone in the community can download your binaries and run them using the pledge.com sandbox tool.

I think this would be a fun game. If the game is fun enough, then it'd potentially grow up and become its own Linux-distro, except rather than Linux it'd be a meta distro, that each program is its own operating system that runs on all operating systems.

[jart]

that sounds a lot simpler from a user point of view. BTW I check that Workers server you use, but is seems APE is not possible there (yet), right?

It's possible to upload APE binaries to Cloudflare workers. If you're referring to https://worker.jart.workers.dev/ then I that's my CDN for image files and .com.dbg files. I put the .com.dbg files on there because the DWARF data is huge, they're mostly only downloaded by crawlers, and Cloudflare R2 offers free network bandwidth egress. I use GCE for the official domains and Google sends me outrageous bills when stories make the front page of Hacker News, which isn't sustainable for a free project like this. However I keep the .com files on the official domains because I want people to know for certain the software is coming from the source they trust, just in case anyone doesn't know I run the workers domain.

[mingodad]

About gcc with musl I found this https://musl.cc/ that have several read to use distributions of gcc and uses https://github.com/kvinwang/musl-gcc as base to build then.

[stefnotch]

@jart I would have loved to check out the Discord server, but the invite link seems to have expired. Is it possible to get a new one?

(Also, on a somewhat related note: There's this lovely blog post from someone else, which also seems to have an expired Discord link https://ahgamut.github.io/2021/07/13/ape-python/ )

[jart]

Here's the latest Discord invite, which expires in seven days. https://discord.gg/vFdkMdQN