Dropping support for GNAT compiler

[Paebbels]

Currently GHDL is compiled for Windows with MinGW and we can proof it works fine with mcode and llvm backends. Compiling GHDL with GNAT (Ada compiler from Ada Core) is hard to maintain. The current scripts are outdated and don't support latest PowerShell versions. See #732. These scripts are also not used in CI runs.

Should we drop PowerShell based compile scripts?

• Main compile script compile.ps1
• GHDL compile script compile-ghdl.ps1
• Library compile script compile-libraries.ps1
• Shared Module shared.psm1

We might want to keep the installer script. This would allow us to ship a *.ps1 file with an internal ZIP container that self-extracts and copies contents to right places.

This functionality exists and is easier to maintain then a MSI file.

Alternatively we could consider shipping GHDL via Chocolaty / NuGet.

---

/cc @umarcor

[umarcor]

For context, we are talking about the scripts currently located in scripts/windows. Until recently, those were located in dist/windows. For some years, those were the only scripts available for building GHDL on Windows (from powershell). Patrick did the initial research for allowing GHDL to be built "easily" on Windows. He also met some of the cliffs in his way (GNAT, GCC backend, Microsoft compiler...).

However, despite being written in powershell, those scripts used MSYS2 under the hood. Some months ago, PKGBUILD recipes were written (now located in scripts/msys2-*). Those recipes are equivalent to powershell scripts, but are the canonical build procedure on MSYS2. Using makepkg-mingw instead of custom scripts does provide several advantages, such as getting dependencies automatically, building for MINGW32 and/or MINGW64 at the same time, generating multiple packages, etc. Those recipes made it easier to upstream the packages to MSYS2's repos. Since a couple of months ago, GHDL is installable through pacman, and it's updated regurlarly. The similarity between the custom recipes here and the upstreamed one makes it easier to handle updates. Those recipes do also make setup-ghdl-ci easy to maintain, because dependencies are metadata of the packages.

At the same time, we have recently been testing how to use MSYS2 packages outside of MSYS2. Patrick has successfully used either mcode or LLVM builds in PyCharm using CPython and pyGHDL. The procedure for getting a zipfile from MSYS2 packages is not automated yet; however, the zipfiles generated through powershell/appveyor scripts do have the same issue.

As a matter of fact, initial versions of #929 first tried to adapt scripts/windows and scripts/windows/appveyor for using them on GitHub Actions. However, doing so while keeping compatibility with both AppVeyor and GHA was considered not to be worth the effort. The next impulse was to remove scripts/windows, but since Patrick was busy at that time, it was temporarily moved to a .deprecated. In the end, the strategy in #929 changed, and migration was done slowly. Most of the jobs were migrated a year ago, but the workflow has been improved/adapted continuously, according to the demands of the project (setup-ghdl-ci, ghdl-yosys-plugin, ghdl-cosim, pyGHDL...). Deprecation of Travis was finally done a few days ago.

AppVeyor is still left as it was 2 years ago, but the deploy stage was removed. This means that zipfiles are no longer pushed to releases by AppVeyor. At the same time, AppVeyor is still running the same testsuites that existed 2 years ago, but not the "new" sanity, vpi, synth and/or pyunit testsuites. That's why AppVeyor is much faster than GHA when running the testsuites. Synth on MINGW64-LLVM is really slow compared to any other backend/platform.

Therefore:

1. Yes, I think we should remove scripts/windows, as it already served its purpose.
2. I think we should consider removing scripts/windows/appveyor too. The same recipes that are currently used on GHA can be used on AppVeyor, because all they need is a working MSYS2 setup. If you analyse jobs Windows-Build and Windows-Test from the current GHA workflow, you'll find that there are four relevant "commands" only:
   2.1 Install/setup MSYS2 and minimal dependencies (base-devel git mingw-w64-${{ matrix.arch }}-toolchain).
   2.2 Build the package with cd scripts/msys2-${{ matrix.pkg }}; makepkg-mingw --noconfirm --noprogressbar -sCLf.
   2.3 Install the package with pacman --noconfirm -U *.zst and Python dependencies with pip3 install -r testsuite/requirements.txt.
   2.4 Run the testsuite with GHDL=ghdl ./testsuite/testsuite.sh.

By using the recipes on AppVeyor, we would be using it for redundancy, which I believe is the only reason to keep it. We might have kept Travis for redundancy too, if their new pricing policy was not problematic.

We might want to keep the installer script. This would allow us to ship a *.ps1 file with an internal ZIP container that self-extracts and copies contents to right places.

This functionality exists and is easier to maintain then a MSI file.

I don't like windows installers, nor self-executing zips. I always download portable versions of the tools, unless the tools do explicitly need to set something on the system. Hence, I do understand why some users might want to have an installer that extracts GHDL to some "expected" location and sets one or a few envvars. However, my approach and recommendation is to use pacman if they want to install and update GHDL with minimum effort. As large as an MSYS2 installation might get, it is negligible compared to vendor tools or the volume of data that is used in testing designs.

GHDL has a Unix look and feel, and it's better used in a Unix ecosystem together with other open source EDA tooling. Being exposed to use bash won't make any harm to windows users. Furthermore, hdl/MINGW-packages is an on-going effort for providing EDA tooling to Windows users through MSYS2. Precisely pacman -S mingw-w64-x86_64-eda allows installing 15 tools, for now (including GHDL, GtkWave, Icarus Verilog, Verilator, KiCad, Yosys with ghdl-yosys-plugin, ngspice, etc.). See https://packages.msys2.org/group/mingw-w64-x86_64-eda and https://hdl.github.io/MINGW-packages/#_packagestools.

Therefore, my idea of an installer would be similar to "Ninite". Let the user select which of those 15 tools wants to have packaged into a self-extracting file. Then, extract a new clean MSYS2, update it, install the desired packages (which will pull dependencies automatically), zip it, add the self-extracting script, and profit. The main advantage of this approach is we don't need to deal with finding the specific DLLs that each tools needs, in order to create standalone zipfiles for each of them. We just use the proven and known MSYS2 features, but we "hide" them so that users don't need to learn bash if they really don't want to. Further thoughts about this:

• Embedding MSYS2 in a self-extracting installer for Windows is a very common approach. Git for Windows, Qt or Ruby, to name a few, do use that approach.
• I think the effort for creating an installer for GHDL only, which includes minimal dependencies only, is actually harder to implement and maintain than offering a solution which would benefit all the community.
• We did aleady test in extended-tests that installing GHDL in a canonical MSYS2 installation and then setting an envvar does allow using it from powershell WITHOUT exposing /msys64*usr/bin in the PATH. That is, Windows users can already use pacman as "a package manager for updating GHDL" only, but keep doing everything in their powershell/command-line workflow.

Alternatively we could consider shipping GHDL via Chocolaty / NuGet.

I believe that distributing GHDL via Chocolatey or NuGet is trivial, once an standalone zipfile is available. It's just a matter of keeping a file with metadata and someone remembering to update/push it. However, we don't have a maintainable procedure for automatically creating standalone zipfiles yet. I honestly did never use Chocolatey / NuGet, as I don't see what value do they provide over Ninite + MSYS2 + Docker (which is what I use). I'd be glad to be illustrated.

[Paebbels]

I think the effort for creating an installer for GHDL only, which includes minimal dependencies only, is actually harder to implement and maintain than offering a solution which would benefit all the community.

Maybe I was unclear. I don't want to write this installer, I did it already some years ago and it was shipped with e.g. GHDL v0.35. See compile.ps1 -CreatePackage -ZIP. In addition, my installer script can already register GHDL in path or unregister it.

I'm asking about keeping this feature only from former PowerShell based compile flow.

---

Many people prefer simple program installations (so programs for a few MB in size) via NuGet (PuTTY, WinSCP, MSYS2, Python3, Notepad++, ...). Compared to MSYS2 / Pacman, these are the advantages:

• No GB in size MSYS2 environment (it grows over time to a monster)
• No (end-)user unfriendly package manager like pacman
  Remembering pacman commands is a nightmare, especially, because it only has characters, is case sensitive and does acount for order and number of characters to distinguish a command !
• As all Windows hosts are nowadays Windows 10, all users have already a NuGet on their machines.
• MSYS2 cannot register GHDL in PATH
• A standalone GHDL is circa 150 MB in size (unzipped).

[umarcor]

I believe that the zipfiles installed by compile.ps1 were created in AppVeyor. Therefore, those were/are equivalent to extracting *.zst packages from MSYS2 builds (or downloading them from either https://packages.msys2.org/package/ or https://github.com/ghdl/ghdl/releases/tag/nightly), and creating a zipfile (for avoiding ZST support on Windows). All of those approaches have the same problem: DLLs are missing. I might be missing something, but I didn't see any function for adding those to the zipfiles that AppVeyor pushed.

Therefore, we can keep the powershell installer (I'm good with that), but it won't be actually usable until we solve how to add DLLs to the zipfiles it consumes. Manually maintaining the list of DLLs that need to be copied is what I want to avoid by using pacman as an installer.

Many people prefer simple program installations (so programs for a few MB in size) via NuGet (PuTTY, WinSCP, MSYS2, Python3, Notepad++, ...). Compared to MSYS2 / Pacman, these are the advantages:

Existing *.zst files can already be installed through simple programs:

curl -fsSL URL_TO_NIGHTLY | tar xf - -C TARGET_LOCATION

We can work around the lack of ZST support on Windows, by providing the content as zipfiles. The problem is unrelated to the package manager of choice, but to the manager not being able to understand the metadata describing the dependencies. pacman is the only one that can do so, because GHDL is only buildable on MSYS2 (for now). Neither of the other procedures/ecosystems provides a workflow which let's us both build GHDL and distribute a binary with dependencies included. Any approach but relying on pacman does involve manual implementation/maintenance.

No GB in size MSYS2 environment (it grows over time to a monster)

MSYS2 can only grow in size, if you let it do so. The installer is 80-90 MB, 320-330 MB after extracting it, and 350-370 MB (unzipped, 165 MB zipped) after running pacman -Syu. That's the overhead. Then, pacman.exe, mingw*.exe, msys2.exe and bash.exe can be removed, just to make sure that no user will be ever able to update it. I don't think that's required, but I hope you get the idea.

The powershell installer can then use that zipfile, set envvars, etc. as if it had nothing to do with MSYS2. Just a bunch of binaries and DLLs in an arbitray location.

If we later find how to further prune that zipfile programmatically, that'd be nice. But I don't think it's a problem given the average size of any engineering tool on Windows.

No (end-)user unfriendly package manager like pacman

Remembering pacman commands is a nightmare, especially, because it only has characters, is case sensitive and does acount for order and number of characters to distinguish a command !

As said, I'm not proposing users to intercat with pacman. I suggest them to do so, but they can avoid/ignore it if they are more familiar with any other CLI for retrieving zipfiles. I'm proposing how we should create those zipfiles, if we really want to do it programatically as of today.

As all Windows hosts are nowadays Windows 10, all users have already a NuGet on their machines.

As correct as that might be, I don't know anyone who uses it and having something installed already doesn't mean users should use it (e.g. Internet Explorer). Hence, I don't have any interest in supporting NuGet. I do have interest in providing a zipfile which can be used without users being aware that MSYS2 exists, neither them having to interact with pacman.

MSYS2 cannot register GHDL in PATH

I do believe it can. MSYS2 tools are native Windows applications. You can do whatever you would do from powershell or the command-line. The difference is syntax only. In extreme cases, you can explicitly call powershell from MSYS2. Therefore post-install steps can be used for this.

A standalone GHDL is circa 150 MB in size (unzipped).

On behalf of honesty, I must say that any zipfile for GHDL with LLVM backend will be larger than those 400 MB I said above. That is because MSYS2 does not provide split packages for runtime and build LLVM libs. Therefore, when the runtime dependencies are installed, many non necessary assets are retrieved. Still, It's 1.8 GB unzipped and 730 MB zipped. It's an order of magnitude larger than the most optimal zipfile we might get; but an order of magnitude below Vivado or Matlab. Moreover, I think it would be sensible to discuss with MSYS2 maintainers whether large packages such as LLVM can be split.

[umarcor]

After talking to @Paebbels, there was a misunderstanding. I thought he was referring to zipfiles in v0.37, v0.36, v0.35, etc. and that all of them were created using the same procedure. However, that was/is not the case. v0.35 was built manually using GNAT and the powershell scripts discussed above. Next releases are the ones generated by AppVeyor which I was referring to as being equivalent to MSYS2's.