Relook at /var/qmail for everthing

[mbhangui]

Most linux distribution follow FHS standards. Look at http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html

We should relook at having binaries in /var/qmail/bin, man pages in /var/qmail/man, etc. These directories also cause warnings/errors when you use a packing build service like openSUSE build service. rpmlint also lists such violation of FHS as error instead of warning. One of our goalis should be to make notqmail instlallation trivial for many class of users, something as easy as

yum install dotqmail
or
apt-get install dotqmail

The change to FHS is not easy. Many of the qmail programs have
chdir("/var/qmail");
....
args[0] = "bin/qmail-local";
args[1] = "--";
args[2] = x;

If the goal of dotqmail will never be seeing it installed as a standard (like postfix, sendmail, exim), then this is not required. But if it needs to be made poplular with all class of users, then we should take a relook at /var/qmail for everhing. Even something like /opt/qmail will serve better.

[schmonz]

Having notqmail available in system-native package repositories is an explicit goal, "Being easily packaged by OS integrators". It sounds like you're seeing a conflict with our explicit non-goal, "Breaking compatibility (such as by moving away from /var/qmail)". Thank you for raising this.

I have some experience as a package maintainer in pkgsrc, where packages need to install files in locations consistent with BSD hier(7). I've solved the problem in pkgsrc qmail like so:

$ ls -l /var/qmail
total 0
lrwxr-xr-x  1 root  wheel  20 Jun 30 20:10 alias -> /etc/qmail/alias
lrwxr-xr-x  1 root  wheel  22 Jun 30 20:10 bin -> /opt/pkg/bin
lrwxr-xr-x  1 root  wheel  44 Jun 30 20:10 boot -> /opt/pkg/share/examples/qmail/boot
lrwxr-xr-x  1 root  wheel  22 Jun 30 20:10 control -> /etc/qmail/control
lrwxr-xr-x  1 root  wheel  34 Jun 30 20:10 doc -> /opt/pkg/share/doc/qmail
lrwxr-xr-x  1 root  wheel  22 Jun 30 20:10 man -> /opt/pkg/man
lrwxr-xr-x  1 root  wheel  20 Jun 30 20:10 queue -> /var/spool/qmail
lrwxr-xr-x  1 root  wheel  20 Jun 30 20:10 users -> /etc/qmail/users

qmail's installer follows symlinks, so files get installed to physical paths matching pkgsrc users' expectations, while still being available via logical paths matching qmail users' expectations.

My working hypothesis, then, is that it's not worth paying the considerable cost of breaking /var/qmail compatibility, because preserving compatibility does not present a significant impediment to packagers. Admittedly, my experience is limited to pkgsrc. Can you find out and report back whether rpmlint or openSUSE's package builder still give errors or warnings on a setup like pkgsrc's?

[alanpost]

In my own (unpublished) qmail fork I have a set of patches to do this. I created a separate auto_qmail_bin, auto_qmail_doc, auto_qmail_man which all default to /var/qmail (the same as auto_qmail) and where necessary modified exec calls to use PATH or the install location of an executable.

I'm more than happy to submit a PR for the series of patches supporting FHS, as I demonstrably agree with @mbhangui on this one.

[DerDakon]

One thing we definitely do not need to care about is /var/qmail/man. Seriously, just drop it whereever it belongs. Having to fiddle this out from /var/qmail makes things harder for everyone, if it just moves to the default folders everything should be fine. The only thing we need to look at if there are any name collisions.

[schmonz]

Given our goal of supporting packaging, I agree that it's necessary for us to support FHS-compatible installations, and that manual pages (and commands, for that matter) should not require users to muck with PATH (or MANPATH or man.conf or who knows what) to find them.

I'm not yet convinced that it's necessary for us to change any "qmail core" code to support FHS-compatible installations. My experience suggests otherwise.

[mbhangui]

In my own (unpublished) qmail fork I have a set of patches to do this. I created a separate auto_qmail_bin, auto_qmail_doc, auto_qmail_doc which all default to /var/qmail (the same as auto_qmail) and where necessary modified exec calls to use PATH or the install location of an executable.

Ditto here. But my set of changes have too many controversial changes. We could look at your patches for implementation in notqmail. What @schmonz has done is also a workable solution & requires almost no changes to code. google also puts its packages on /opt/google for linux. I will try his method to build a RPM and see what rpmlint reports. To be more specific this is what I did (full code here)

conf-shared - /usr/share/qmail (boot files, doc)
conf-prefix - /usr (binary)
conf-sysconfdir - /etc/qmail (control files, users, openssl certs, tcp.smtp.cdb, etc)
conf-libexec - /usr/libexec/qmail (z* scripts, x* scripts, config-fast, dnscname, dnsip, dnxmxip, etc)
conf-qmail - /var/qmail (queue, alias, virtual domains directory, etc)

There is no conf file for man pages. They go straight to /usr/share/man

and symbolic links in /var/qmail

bin
control
users

[schmonz]

IRC discussion with @alanpost clarified that "FHS" carries with it two related but distinct goals. Attempting to summarize:

1. "FHS-aware": By default, we want notqmail to install files to FHS-compatible paths (with conf-qmail still defaulting to /var/qmail, and having that be a symlink farm as described above)
2. "FHS-strict": Optionally, off by default, we want notqmail to install files to FHS-compatible paths without any sort of /var/qmail or equivalent (by patching qmail programs to find each other and their config files in new ways)

[jccleaver]

I think 'FHS-strict' might be a bit much... There are (or were) quite a few packages out there that internally still rely on symlinks on some pseudo-HOME directory for dispatch to FHS locations, and the underlying code changes might be more effort than their worth in some spots. (This is especially the case if we still want to support non-FHS installs in this.

I was able to get my /var/qmail/ down to just this in my RPMs, and /var/qmail/queue/ arguably could have gone elsewhere too, but it was common at the time to do weird things with it on a separate partition...

[root@cloud qmail]# ll /var/qmail/
lrwxrwxrwx 1 root root 16 Sep 30 2013 alias -> /etc/qmail/alias
lrwxrwxrwx 1 root root 18 Sep 30 2013 control -> /etc/qmail/control
lrwxrwxrwx 1 root root 14 Sep 30 2013 log -> /var/log/qmail
drwxr-x--- 11 qmailq qmail 4096 Sep 30 2013 queue
lrwxrwxrwx 1 root root 16 Sep 30 2013 users -> /etc/qmail/users

Everything else was FHS-strict.

Also, even with qmail being FHS, many add-on packages (vpopmail, etc) may expect to have a unified QMAILHOME where it might call things out from.

[schmonz]

At least for the near term, and possibly much longer, "FHS-strict" builds won't be the default. This is an excellent reminder that when we introduce the FHS-strict build option in 1.08, we'll need to document its risks and enumerate some known implications. Glad to see you commenting here and giving us some vpopmail context — welcome!

[mbhangui]

I have gone through this FHS difficulty in indimail (the design of which is based on vpopmail). What I had to deal with were the following

1. many programs do chdir("/var/qmail").
2. Many programs are expected in /var/qmail/bin
3. The assign file is in /var/qmail/users
4. To further compound the FHS mess, vpopmail introduces another directory /var/vpopmail
5. The delivery instructions in .qmail-default is /var/vpopmail/bin/vdelivermail "" bounce-no-mailbox
6. .qmail-default in /var/vpopmail/domains/domain_name

I had to do a two stage approach. First was to have the following and appropriate symbolic links

1. Put all user binaries in /usr/bin and system binaries in /usr/sbin
2. Put control files in /etc/indimail/control
3. tcp.smtp.cdb, etc in /etc/indimail

With time, I just removed the symbolic links. Now I have a distribution that builds on opensuse build service without having to supress FHS related errors for the RPM & debian distributions and without having to modify the PATH in the .profile

[josuah]

I am trying to summarize the source code changes required (checked with grep '"<subdir-of-/var/qmail>' *.[ch]):

• alias: never refered directly in the code but rather as the home directory of the alias user -> safe
• bin: only referred as a relative path, so stripping bin/ and using execvp() should suffice -> sed -i 's|"bin/|"|' *.[ch]
• boot, man, doc: only used in hier.c, to install qmail -> safe
• queue: called as "chdir" or directly, relatively to qmail_home -> workaround needed!
• control: only called in hier.c, in end qmail-*.c programs or in rcpthosts_init() (itself only called in qmail-*d.c), and when they are, it is always at the begining of the program, after a chdir(qmail_home), and before any use of the queue -> workaround needed
• users: called by programs that only deal with the users/cdb file, or in qmail-lspawn.c which also chdir to the queue, but calls "users/cdb" first

It should be possible to have FHS behavior with only this:

1. defining qmail_home to /etc/qmail or equivalent, with only control and users in it
2. strip bin/ out of exec calls
3. replace occurences of "queue/[...] to QMAIL_QUEUE"/

Then we can:

• keep the /var/qmail behavior by defining qmail_home to /var/qmail and QMAIL_QUEUE to queue.
• use FHS-style paths by defining qmail_home to /etc/qmail and QMAIL_QUEUE to /var/spool/qmail

This is lightweight on qmail (a few strings replaced only), and exposes few switches that will be easy to integrate. It also does not touch to qmail_home (patch compatibility) or remove/add any line of code.

The rest is a matter of install paths, another topic...

I will check this in practice and report it there.

[DerDakon]

Excellent read. The "bin/" thing actually hardcodes the path so no other binaries of the same name will be called.

For the install woes take a look at #116.

[josuah]

@DerDakon I am coming late in the game, so I indeed have some reading to go. Great work so far!

Reading ongoing work, I prefer #35 to the solution I am proposing, for it is a full solution to the problem and not only a workaround.