[cmds] Introduce ability to compile and install ELKS applications outside of repo

[ghaerr]

This PR introduces the capability of compiling and installing applications outside of the ELKS source tree and adding them to any of the distribution images currently supported on ELKS.

A new buildext.sh shell script is now run by GitHub CI and can also be run on your host system, which controls building "external" applications. A ExtApplications file operates very similar to the existing Applications file and allows specific binaries, images, man pages etc generated by the buildext.sh script to be copied onto specified disk images during the existing make images process.

The following external applications can now be automatically built, pulled automatically from their respective repositories:
Microwindows (Nano-X)
D-Flat (memopad)
elkirc
When OpenWatcom C is installed (as indicated by WATCOM= set in the environment, the following applications are built:
ELKS OWC Library
C86 / 8086-toolchain (not yet automatically installed, coming)
ELKSDoom
ngircd-elks
(more coming soon, including elksviewer, bobcat, Lua, 6Forth, pc-calc)

Currently, only Nano-X is automatically built by CI, with the basic set of nano-x, nxstart, nxterm, nxcalc, nxclock and nxtetris fitting on 1440k floppies, and the full suite including nxdsktop etc on 2880k and HDD images. See ExtApplications for details, 1232c floppies should also be supported as 1440k but is not yet tested.

To build the entire set of applications above, make sure that WATCOM is installed, and run ./buildext.sh all, which will git clone each repo into the extapps/ directory the first time, then update using git pull subsequently, then build the applications using instructions in buildext.sh. If WATCOM is not installed, then those applications requiring it will be skipped. Likewise, applications specified in ExtApplications that were not built will be skipped for installation, displaying a message instead of an error.

To build a specific external application, use ./buildext.sh microwindows for instance and only it will be built. This is useful for quickly updating work in progress.

Here's a screenshot of the D-Flat memopad and elksdoom applications which are being built and installed onto HDD images.

A subsequent PR will add automatic compilation of all external applications, including WATCOM apps, to GitHub Continuous Integration. This is being skipped for the time being pending further testing.

[toncho11]

Sounds great!!! So, there will be new HDD images such as hdd_minix_32_ext.img ready to download once the github action is added.

[ghaerr]

there will be new HDD images such as hdd_minix_32_ext.img ready to download

Yes - but the image names will remain the same as always, just more applications on them: hd32-minix.img, hd32-fat.img, hd64-minix and MBR versions of the same.

[ghaerr]

We now have the capability of easily producing FD images with Nano-X, and HD images with the C86 devkit and many other external repos :) This can be done locally or automatically on GitHub CI. OpenWatcom must be installed locally to take full advantage. Building Nano-X does not require OpenWatcom.

This PR now builds all the following in GitHub CI automatically on each push, which results in all images being built with applications specified by disk image size in elkscmd/ExtApplications:

#       Name            Compiler        Desc
#       microwindows    ia16-elf-gcc    Nano-X Graphical Windowing Environment
#       dflat           ia16-elf-gcc    D-Flat TUI memopad/library
#       elkirc          ia16-elf-gcc    IRC for ELKS
#       owc_libc        OpenWatcom      ELKS C Library compiled by OWC
#       owc_elkscmd     OpenWatcom      Some elkscmd/ programs compiled by OWC
#       c86_toolchain   OpenWatcom/C86  C86 Toolchain, header files and examples
#       c86_elkscmd     C86             Some elkscmd/ programs compiled by C86
#       doom            OpenWatcom      Doom for ELKS
#       ngircd_elks     OpenWatcom      IRC daemon for ELKS

If OpenWatcom is installed on your local host, all the above can also be compiled, and all images produced, using:

$ ./buildext.sh all
$ cd image; make images

The old buildc86.sh and copyc86.sh scripts are now obsolete. The above will build a full devkit. In order to build only the C86 devkit, the following can be used, for instance:

$ ./buildext.sh owc_libc c86_toolchain
$ cd image; make hd32-minix

To build a single project, just pass the project name to buildext.sh.

Adding more external applications is planned. There are also some problems with doom loading elksdoom.wad, for which I've reached out. Some application changes are needed to put files in the proper places now that we can build distributions. These issues can be discussed in more detail in the future.

[rafael2k]

elks-viewer - for the "full version", lets use the OWC

I just rebased Lua master branch to upstream, so we are in sync with latest lua (5.5 + Roberto fixes) - still need to test it - but this is the full blown version.

lua-5.1i is a special version where I set the default number type to integer - it is very fast and integer only (but be aware with non integer math... as all the math will be integer!)

lua-5.1 is the full Lua 5.1 version

bobcat works, but the handling of the menus in curses is not there yet... but just believe - and it works (network, html, all good), just needs some love in the curses implementation

jove - almost there, but not yet (compiles, but I need to get the tty right)

[ghaerr]

After testing Lua-5.1, we should probably add that to the build. Is there a config file? If so, this should probably be kept in /etc.

Not sure yet about bobcat, as it sounds as it might not be very usable yet?

[toncho11]

After testing Lua-5.1, we should probably add that to the build. Is there a config file? If so, this should probably be kept in /etc.

Not sure yet about bobcat, as it sounds as it might not be very usable yet?

@rafael2k maybe we should select only one version for now and name it 5.X, so that we could add the second one later.
I would love to have my 3D rotation cube available!!!!!!!!!!!!!!!!!!!!!!!!!!!! I think both support it.

[rafael2k]

After testing Lua-5.1, we should probably add that to the build. Is there a config file? If so, this should probably be kept in /etc.

No config file AFAIR.

Not sure yet about bobcat, as it sounds as it might not be very usable yet?

It is usable, but the link navigation is a bit clunky. Lets first add the 100% working software I ported (Lua and zlib first). I'll revisit bobcat and jove soon.

[rafael2k]

After testing Lua-5.1, we should probably add that to the build. Is there a config file? If so, this should probably be kept in /etc.
Not sure yet about bobcat, as it sounds as it might not be very usable yet?

@rafael2k maybe we should select only one version for now and name it 5.X, so that we could add the second one later. I would love to have my 3D rotation cube available!!!!!!!!!!!!!!!!!!!!!!!!!!!! I think both support it.

Lets do like this - 5.1 "stock" will be the default, and 5.5 and 5.1 "integer only" as two variants. Yes, the 3D rotation cube is cool!

[rafael2k]

Also the pcmplayer will be a nice to have app...

[toncho11]

@ghaerr So the 1440 images will only contain nxstart for example and not nxdsktop because there is not enough space? So nxdsktop will be available only on the hdd images for x86 systems?

[ghaerr]

@toncho11 - yes, there isn't the space on 1440k for the whole suite of nxdsktop programs.

BTW, does nxdsktop require nxmsg and nxselect?

[ghaerr]

Lets do like this - 5.1 "stock" will be the default, and 5.5 and 5.1 "integer only" as two variants.

That sounds fine. I would prefer we just have a single lua executable without version number for the distribution to reduce confusion, rather than a suite of lua's, at least until people start using it. Are the integer versions in the same repo and available as compilation options? In that way, users could modify extapps/lua should they really need an integer-only version.

I would love to have my 3D rotation cube available!!!!!!!!!!!!!!!!!!!!!!!!!!!!

The 3D cube demo can be temporarily placed in /root for the time being, while we figure out where demo files/data should be placed more permanently. Keeping them there for the first distribution HDD images might actually prompt more people to use the various sample programs too. (This includes *.jpg, elksdoom.wad, and cube.lua I suppose).

[tyama501]

Hello @ghaerr

Currently the nano-X is only compiled for ibm PC since Makefile.elks in the microwindow repository is fixed config for it, right?

Thank you.

[ghaerr]

nano-X is only compiled for ibm PC

Yes. We need to figure a way to build Nano-X for PC-98, as well as the other ELKS application binaries requiring special compilation, without requiring a complete rebuild each time. I'm thinking something like what's done in buildimages.sh needs to be moved into build.sh. I will work on it.

[tyama501]

Thank you!