Add uninstall target to Makefile

[dominiksalvet]

Is your feature request related to a problem? Please describe.
I am a developer of gim project. It is a simple Git-based installation manager and it can install GHDL using gim install github.com/ghdl/ghdl. Right now it supports only basic ./configure && make && make install/uninstall and make && make install/uninstall approaches.

Gim also can update software. Right now update works as long as gim can install and uninstall a project, which leads me to the problem. I can't uninstall/update GHDL until uninstall target will be introduced. It would be great to have uninstall target in the Makefile if it is not really hard to implement.

Describe the solution you'd like
Make make uninstall command run in the root of the GHDL repository uninstall GDHL from the system.

Additional context
Gim supports installers (dynamically loaded modules used for installation or uninstallation) and approches described above are actually implemented by two installers. Gim doesn't stop here however. I am going to add more installers (such as setup.py, install.sh, cmake, user-delivered installer) and also I am going to transform gim into a simple Git-based package manager.

Right now gim provides (as per its README.md):

• No gim-specific files required
• Easy-to-use for developers and users
• Global and local installations
• Delta updates
• POSIX-friendly environment

If you implement the uninstall target, GDHL will become gim ready project and users can use gim to install it, update it and even uninstall it. Gim updating is based on Git tags, so you don't have to maintain nothing more than now. Also note that gim supports Linux, macOS and Windows (through WSL).

All information (and a little bit more) can be found in the Developer section of gim project.

If I can be of any help with gim integration (except for uninstall implementation for GHDL), do not hesitate to ask me. I have been developing gim for almost one year and I am even prolonging my studies due to it yet it is not perfect. GHDL would be the first official gim ready project not being developed by me (1acheng/INX_Editor has no relation with gim). I would be more than honoured.

[eine]

Regarding grim being able to manage GHDL, bear in mind that GHDL can have a rather complex build procedure. Although, configure && make && make install works, some features are only available with LLVM or GCC backends. See: https://ghdl.readthedocs.io/en/latest/building/Building.html. That's why some GNU/Linux distributions have multiple versions of GHDL available through the package manager. Furthermore, there is a build script that handles the differences in the procedure depending on the backend: https://github.com/ghdl/ghdl/blob/master/dist/travis/build.sh

I think that grim might already do what you want with existing resources:

# Get a specific version
git clone https://github.com/ghdl/ghdl && cd ghdl && git reset --hard <version_tag_or_sha>
# or
git clone -b <version_tag> https://github.com/ghdl/ghdl

# Build
# E.g. mcode
./dist/travis/build.sh -b mcode -p ghdl-mcode.tgz
# E.g. LLVM with default-pic
CONFIG_OPTS="--default-pic" ./dist/travis/build.sh -b llvm-6.0 -p ghdl-llvm-fPIC

# Install
tar -xzf ghdl*.tgz -C /opt/grim/ghdl

# Uninstall
rm -rf /opt/grim/ghdl

Provided that you also echo 'export PATH=$PATH:/opt/grim/ghdl/bin' >> ~/.bashrc.

Using ./dist/travis/build.sh allows you not to handle all the complexity of the build depending on the backend.

Overall, I feel that, in the context of GHDL, make install is not exactly what you think it is. During the build procedure, a lot of intermediate files are generated in the current directory, which are not required by GHDL at runtime. make install gathers runtime dependencies only, and copies them in a separate directory. That's all.

If you use /usr/local as a prefix, make install will, at the same time, install the program on your system. But this is not suggested at all, unless you are ok with having an unmanaged program. Adding/removing content to/from /usr/local should be handled by a package manager, and make is build automation tool.

A package manager is expected to either analyze or have a predefined list of the contents of the tarball. It can also have a list of locations where the program might generate data in the future. So, when you request to uninstall the program, the manager has a list of paths to remove. In this comment you will find references to various threads where we gather information about how is GHDL being packaged for various distributions. The Arch Linux case is very didactic. There are four variants, three of which use the 'latest' from master through git: https://aur.archlinux.org/packages/?O=0&SeB=nd&K=ghdl&outdated=&SB=n&SO=a&PP=50&do_Search=Go. As you see, none of the PKGBUILD files has any specific info about how to uninstall the package: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=ghdl-llvm-git. This is because, the 'uninstall' feature is built into the package manager, pacman: https://wiki.archlinux.org/index.php/pacman#What_happens_during_package_install/upgrade/removal

Summarizing, I think that the request to have a list of paths where GHDL or the companion scripts are expected to generate content is interesting. This can be useful as a makefile target, but also to improve the packaging of GHDL for different distributions (some files might actually be being left behind). However, I would strongly discourage using make install and make uninstall as valid method to handle the installation of programs system-wide.

[dominiksalvet]

Regarding grim being able to manage GHDL, bear in mind that GHDL can have a rather complex build procedure.

You probably meant gim, didn't you? 😃

Although, configure && make && make install works, some features are only available with LLVM or GCC backends.

I know. Gim is simple and gim ready projects should "just work". Therefore, gim relies on default project configuration no matter how many possible options it has. Gim itself can be installed with some additional options as well. Take a look at its help target in Makefile.

Regarding what gim does when installing a project, I can simplify it to the following:

• load dynamic configuration (RC files, root permissions check => global or local installation)
• check given URL and transform it to the canonical form (e.g., https://github.com/dominiksalvet/gim.git -> github.com/dominiksalvet/gim)
• refresh repository cache (pull if exists, otherwise clone new)
• choose preferred commit from master branch to install (latest tag's commit if exists, otherwise latest commit)
• now the installation
  • perform checks (e.g., whether it is possible to uninstall after installation, which is not the case of GHDL => error <NO-TAR-UNINSTALL>, which can be suppressed by -f flag)
  • there are a lot of ways to install projects => gim uses installers, dynamically loadable modules during gim's execution
  • gim asks each installer: "Hey dude, can you install this project?" (i.e., call can_install())
  • the chosen one is then honoured to install the project (in GHDL case, it is cami installer)
• add installation record to status database
• print status

Overall, I feel that, in the context of GHDL, make install is not exactly what you think it is. During the build procedure, a lot of intermediate files are generated in the current directory, which are not required by GHDL at runtime. make install gathers runtime dependencies only, and copies them in a separate directory. That's all.

That's exactly what I want. Note that gim, in its current state, is mostly focused on installations from source code. Later, having more installers, its functionality can be significantly extended, e.g., to installations from precompiled tarballs. The same statement is valid for project-defined installers, which is a planned gim feature (if you really want gim to install GHDL using build.sh, this would allow it).

If you use /usr/local as a prefix, make install will, at the same time, install the program on your system. But this is not suggested at all, unless you are ok with having an unmanaged program. Adding/removing content to/from /usr/local should be handled by a package manager, and make is build automation tool.

As long as I know, /usr/local is not managed by a package manager as opposed to /usr. That's also why I have chosen it as a default prefix provided by gim while installing projects. Gim itself is also installed here. Checkout the following terminal log on my system:

dominik@sleeker:~$ ls -1 /usr/local/bin/
asus-fan-control
ghdl
ghdl1-llvm
gim
idea
ise
ux430ua-jack-volume
dominik@sleeker:~$ ls -1 /usr/bin/ | wc -l
2151

However, I would strongly discourage using make install and make uninstall as valid method to handle the installation of programs system-wide.

Installation is demonstrated in GDHL README.md file this way. Furthermore, I still don't see anything wrong with this installation from source code.

[tgingold]

@1138-4eb I don't understand your point of view. What is wrong with make install ?

I think we should follow the GNU conventions: make install should work, and we should add make uninstall. I agree it wouldn't be possible to install/uninstall multiple backends.

[eine]

[@dominiksalvet]
You probably meant gim, didn't you? 😃

Yes, sorry, I got mixed with Brothers Grimm.

[@dominiksalvet]
* check given URL and transform it to the canonical form (e.g., dominiksalvet/gim.git -> github.com/dominiksalvet/gim)
* choose preferred commit from master branch to install (latest tag's commit if exists, otherwise latest commit)

If you are going to support GitHub only, you might want to consider using https://codeload.github.com/ghdl/ghdl/tar.gz/master and the GitHub API instead.

[@dominiksalvet]
* add installation record to status database

This is where you can save a list of the files that were installed. Instead of making make install copy everything to /usr/local, you can copy it to a temporal dir, get the list of content, and then move it.

[@dominiksalvet]
if you really want gim to install GHDL using build.sh, this would allow it.

I don't want gim to use build.sh per se. I am not the target audience of gim. The step that generates the tarball is almost irrelevant; the utility of build.sh is to complement the makefile by convering the cases that are not as easy as make && make install. However, if you are only going to support mcode backend on Unix systems, it does not matter.

---

[@tgingold]
@1138-4eb I don't understand your point of view. What is wrong with make install ?

[@dominiksalvet]
As long as I know, /usr/local is not managed by a package manager as opposed to /usr. That's also why I have chosen it as a default prefix provided by gim while installing projects. Gim itself is also installed here. Checkout the following terminal log on my system:

The point is not who manages /usr/local, but if you are the only app managing that path. The user might also have other package managers as gim which are also designed considering that no other manager will use /usr/local.

For example, GHDL provides some helper scripts to compile and install vendor libraries: https://github.com/ghdl/ghdl/tree/master/libraries/vendors. This is quite usual, and they can be located anywhere (the location of GHDL, HOME, /opt). The main Makefile in GHDL is not aware of these. So a make uninstall might work or might fail. If it works, it might remove or leave some of the vendor libraries.

Sincerely, at this point, I'd rather make nightly *.deb and *.rpm packages instead of investing the effort on having uninstall targets in the makefiles which can cover corner cases. Then, gim could fall back to the corresponding package manager. However, if either of you want to implement make uninstall, it should be a matter of checking all the install targets and doing the reverse actions.

---

[@dominiksalvet]
Installation is demonstrated in GDHL README.md file this way. Furthermore, I still don't see anything wrong with this installation from source code.

There are two reasons for it (@tgingold, correct me if I'm wrong):

On the one hand, the note below is specific about Windows, where there is no conflict. One reason to show --prefix in the example is to make it easier for Windows users to understand the note. Otherwise, it would probably be ./configure; make; make install. It is seen in the history that the prefix was not added until four years ago.

On the other hand, it is an old convention. The target audience when it was written was users that were used to compiling C programs from sources. In this context, it is well known that /usr/local is an example, and the user is aware that he is responsible for having programs mixed there. Some alternative common location in the workplaces I've been at are /eda, /tools, /opt.

There has been some effort to improve the docs to make it easier for new users, but there are still some inconsistencies in this sense: some very easy to follow parts and other very specific details.

Overall, GHDL has always been and still is, a tool for developers. This means that knowledge and certain proficiency about GNU/Linux, VHDL, C, Ada, etc. are expected/assumed. E.g.:

This manual is the user and reference manual for GHDL. It does not contain an introduction to VHDL. Thus, the reader should have at least a basic knowledge of VHDL. A good knowledge of VHDL language reference manual (usually called LRM) is a plus.

From this point of view, I feel that GHDL might not be the best project to be 'gim compatible'. If a user wants the latest version by executing a single command, I believe she/he expects to get a binary/tarball and install it. So, we'd better go for nightly *.deb or *.rpm. If the user wants to actually build the program locally with a single command, I'm afraid that gim might be missing lots of information that other mainstream package managers handle properly. This is because GHDL is written in Ada, VHDL and C. Hence, each distribution and version might need a different set of packages. We don't have any computer-readable file describing it. We do have https://github.com/ghdl/docker/tree/master/dockerfiles/build, tho.

Nonetheless, if you constrain it to mcode only, Debian-based distributions only (this includes WSL), and uninstall on a 'best static effort' basis, I think it is doable.

---

[@tgingold]
I agree it wouldn't be possible to install/uninstall multiple backends.

This is a different issue. I had not considered this context. I meant that the user might want to install some backend different from mcode, which requires additional arguments to ./configure and more specific package names.

[dominiksalvet]

If you are going to support GitHub only, you might want to consider using https://codeload.github.com/ghdl/ghdl/tar.gz/master and the GitHub API instead.

Nope, I am trying to develop gim so that it depends only on Git from the server's side. I was on GitLab and it worked. Now on GitHub it works as well.

This is where you can save a list of the files that were installed. Instead of making make install copy everything to /usr/local, you can copy it to a temporal dir, get the list of content, and then move it.

Well, you probably don't know what's the content of status file. One line per installed project containing canonical form of URL, installed commit hash and installed prefix. Example on my system from /var/lib/gim/status:

github.com/dominiksalvet/ux430ua-jack-volume 41638de56589aead0c0d75e5b840a9cb633ac49c /usr/local
github.com/dominiksalvet/gim f778f3c009dc2146f871493e8485fb4bac024888 /usr/local
github.com/dominiksalvet/asus-fan-control e09981fd1943961ef8277df963b01f7df2f0e681 /usr/local

However, the idea of temporary directory is great. That would lead to create a map which files to delete during uninstallation even without uninstallation recipe (not mentioning security issues). Nevertheless, some projects don't respect prefix at all (or partially - e.g., bash completion implementation) and I also don't want to maintain rather big database. I stick with relying on projects' uninstallation recipe.

The point is not who manages /usr/local, but if you are the only app managing that path. The user might also have other package managers as gim which are also designed considering that no other manager will use /usr/local.

Right now, gim is just an installation manager, not a package manager. And even after becoming a SIMPLE package manager (it will only install dependencies first if any), it is not going to be the only app managing the path.

Then, gim could fall back to the corresponding package manager.

I hope, gim will never do that. It manages its own database and a package manager manages its own as well.

@1138-4eb I am sorry, but I still don't know why installation from source code is a bad thing.

However, if you are only going to support mcode backend on Unix systems, it does not matter.

Nonetheless, if you constrain it to mcode only, Debian-based distributions only (this includes WSL), and uninstall on a 'best static effort' basis, I think it is doable.

Maybe my message was not clear. Gim is simple. It is simple by design and, frankly, simplicity is the most important characteristics of gim for me. It is also the answer to a lot of your questions (past ones and probably future ones as well).

However, still I don't know why it should be limited to Debian-based distributions only.

[eine]

Nope, I am trying to develop gim so that it depends only on Git from the server's side. I was on GitLab and it worked. Now on GitHub it works as well.

Note the bandwidth, tho.

This is where you can save a list of the files that were installed. Instead of making make install copy everything to /usr/local, you can copy it to a temporal dir, get the list of content, and then move it.

Well, you probably don't know what's the content of status file.

I should have said 'when' instead of 'where. I don't mean to discuss implementation details, but the steps in the process when you can introduce additional features to solve the issue.

Nevertheless, some projects don't respect prefix at all (or partially - e.g., bash completion implementation) and I also don't want to maintain rather big database. I stick with relying on projects' uninstallation recipe.

The size of such a database should be negligible compared to downloading and keeping a cache with the full repo. Anyway, I understand that you don't want to increase the complexity.

Right now, gim is just an installation manager, not a package manager.

An installation manager is a package manager with reduced features; it handles a single package instead of managing dependencies too. Moreover, the functionality of gim (installing a tool from git by providing a configuration file) is already supported by some package managers, such as pacman and a PKGBUILD (ref above), or dpkg-buildpackage and *.dsc (https://packages.debian.org/source/sid/ghdl).

Name it however you want, my suggestion is to look and learn how others have solved the problems you are finding and that you will probably find in the future. Both Arch Linux's and Debian's packaging systems are solid references.

Nonetheless, if you constrain it to mcode only, Debian-based distributions only (this includes WSL), and uninstall on a 'best static effort' basis, I think it is doable.

Maybe my message was not clear. Gim is simple. It is simple by design and, frankly, simplicity is the most important characteristics of gim for me. It is also the answer to a lot of your questions (past ones and probably future ones as well).

However, still I don't know why it should be limited to Debian-based distributions only.

Because you want it to be simple by design. Supporting Debian-based distributions only and mcode only lets you be sure that the dependencies you need are gnat gcc make zlib1g-dev. If a different backend or distribution is used, it is not true anymore. For example, on CentOS/Fedora/RedHat, it's gcc-gnat instead of gcc gnat. For different backends, package names change even between versions.

Hence, to support something different from mcode on Debian, you would need the user to tell which distribution is using or gim should handle it. Both options increase the complexity.

[dominiksalvet]

@1138-4eb You know what? Thanks to your constructive comments, I've been thinking a lot. Gim is not and probably won't be as flexible as I initially intended. I am thinking about a complete redesign (read: create a new repo) with support for interpreted languages-only, which will be essentially different and more simple. Lately, it seems for me that gim is over my head.

Anyway, I would still be at least for some basic and straightforward uninstall target.

[eine]

Lately, it seems for me that gim is over my head.

Well, almost 20 years ago, and more than 20 after writing TeX, Donal Knuth said: "In fact, my main conclusion after spending ten years of my life working on the TEX project is that software is hard. It’s harder than anything else I’ve ever had to do" (ref). Yes, software is hard. But you can turn the computer off and relax whenever you want ;)

@1138-4eb You know what? Thanks to your constructive comments, I've been thinking a lot. Gim is not and probably won't be as flexible as I initially intended.

It is quite common, indeed. When you imagine a project, everything seems simpler. As you start implementing it, complexities arise, and almost always you reach a point when you need to cut features, or just stop.

Nonetheless, if you really like the topic, do not discard it so fast. Think about what you have learned while doing it, and what can you still learn from it. It's always more exciting to apply new concepts/tools to your own project, rather than repeating all the 'Hello World' examples.

I am thinking about a complete redesign (read: create a new repo) with support for interpreted languages-only, which will be essentially different and more simple.

I don't know exactly what you are thinking when you say 'interpreted languages', but I think that this might not be a relevant enough criteria. Let me explain:

Almost all the 'new' languages do already have one or multiple package managers that allow to interact with GitHub repositories easily. Golang has go get github.com/user/repo, and it works not only for Git. Python's pip supports git, mercurial, subversion, etc. The JavaScript ecosystem has e.g. npm, yarn. I don't know much about Rust, but I believe that package managers support version control systems too. Note that Python and JavaScript are interpreted, but Golang and Rust are not. If you focus on 'shell scripts', there are other similar projects too: basherpm/basher, bpkg/bpkg.

My humble suggestion is to focus on an area of knowledge, a niche, instead of trying to draw a line that depends on technical details (the tool to communicate with the server, or the language). For example, focusing back on hardware design and on VHDL (since this is GHDL's repo), and related to package managers/installers, I am not aware of any standardised open source tool to share, test and use free IP cores. These are some of the tools/libraries I know:

• https://github.com/VLSI-EDA/PoC
• https://opencores.org/ and https://github.com/fabriziotappero/ip-cores
• https://www.librecores.org/search?query=&type=projects
• https://github.com/kevinpt/vhdl-extras
• https://github.com/VUnit/vunit
• https://github.com/potentialventures/cocotb
• https://github.com/UVVM/UVVM
• https://github.com/OSVVM/OSVVM
• https://github.com/olofk/fusesoc and https://github.com/olofk/edalize

NOTE: almost all of these are related directly (because they explicitly support it) or indirectly (because any VHDL project can be simulated) with GHDL.

• PoC, OpenCores, LibreCores and VHDL-extras contain all kinds of IP cores for synthesis. From common modules, such as rams, shift registers, UARTs, etc. to complete CPU designs.
• VUnit, cocotb, UVVM and OSVVM all provide resource for (co)simulation and verification. Precisely, all of them provide Verification Components, Bus Functional Models, name_it_whatever.
• PoC, fusesoc and edalize provide integration with the synthesis and P&R tools.

So many resources, and still each of those repos is a kind of isolated island. Some of the tools can coexist, but the integration of most of them depends on the user. As a result, each of the libraries/projects follows a different structure, and it is difficult to just open a new project and try to not drown yourself while diving into the sources. LibreCores and OpenCores are the only ones with a search tool, and still it is hard or impossible to properly filter the results.

What's the solution? Maybe a user-friendly tool to interact with IPXACT, as fusesoc does. Maybe some reduced version, that can be later extended. Maybe some custom checklist, similar to what REUSE does. Or maybe none of them. I don't know, I'm just thinking out loud. It's just an excuse as any other for you to know some other neighbours in the town (in case you didn't knwo them already).

The summary is that you can learn a lot by finding and actively doing small contributions to fix/enhance existing long-running open source projects, instead of doing a titanic effort alone. For example, in this specific issue, should you have tried to implement the make uninstall target yourself, you would probably have reached the same conclusion, without us having this conversation. Many arguments (mcode, Debian, vendor...) are easier to justify and to refute with very specific lines of code to discuss about. Don't take me wrong, I'm good with it. Unfortunately, as we have commented before, not everyone thinks like this, and for many people most of this discussion is just off-topic noise. We need to be respectful to them too. It's ok evey once in a while, but we should focus on what GHDL provides and what is needed to interact with other tools or for other tools to interact with it. Hence, large parts of this thread should belong to grim's repo.

I'm just telling this to you because I also started some apparently simple but no-so-simple software projects in the past which seemed important to me. After learning the basics, and hitting the wall a few times, I discovered how helpful it is to have more experienced users around that tell me when I'm trying something stupid. Tristan's patience and persistence are to be really thankful.

---

Anyway, I would still be at least for some basic and straightforward uninstall target.

Yes, I'm ok with it. As @tgingold commented, GHDL (he) tries to follow GNU conventions, so it should be added. Nevertheless, it is still not suggested to use make install and make uninstall with --prefix=/usr/local without taking care about the comments in this thread. Specially, the limitation to install multiple versions/backends at the same time.

[dominiksalvet]

@1138-4eb Hi! 👋 More than 2 months of a lot of work even during having a full-time summer job and here we are again. I am proud to tell you that GitPack, successor to gim, does very well. As I have indicated, I have completely redesigned gim into GitPack, which is now a really simple package manager. 📦

Simplicity is the key and so far I have spent time equally on thinking about features and actually developing them. And so I got rid of unnecessary features and make GitPack much more simpler compared to gim. It does not support installation and uninstallation multiple ways like gim did, rather it supports its native and simple GitPack installation and uninstallation format inspired by the structure of Debian packages. GitPack is simpler than ever and I am excited about GitPack more than ever. 😃

I know that there might be many already existing solutions like GitPack. But when I needed something really simple just like today's GitPack, i.e., Git-based, language-independent, host-independent, POSIX-friendly, Shell-based and with local and global installation support, I couldn't find anything. So I created my own solution. Gim was the first try. GitPack is the second and I hope it is the last as well.

Recently I have released GitPack 0.3.0 and the roadmap for the first stable release looks promising. I have also transformed former gim projects into GitPack projects. I am really excited about the whole project. 🚀

Once again, a big thank you! ❤️ GitPack as per its current state exists thanks to you. If you have any projects in an interpreted language and you want to provide a way to install, update or uninstall them to end users, give GitPack a shot. Everything you need to know is stated in the GUIDE.md file.

If you have any comments or questions, feel free to ask me here, on my email or create an issue at GitHub. 👍