Skip to content

Relook at /var/qmail for everthing #2

Open
Open
Relook at /var/qmail for everthing#2
Labels
packaging

Description

@mbhangui
mbhangui
opened on Jul 7, 2019

Most linux distribution follow FHS standards. Look at http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html

We should relook at having binaries in /var/qmail/bin, man pages in /var/qmail/man, etc. These directories also cause warnings/errors when you use a packing build service like openSUSE build service. rpmlint also lists such violation of FHS as error instead of warning. One of our goalis should be to make notqmail instlallation trivial for many class of users, something as easy as

yum install dotqmail
or
apt-get install dotqmail

The change to FHS is not easy. Many of the qmail programs have
chdir("/var/qmail");
....
args[0] = "bin/qmail-local";
args[1] = "--";
args[2] = x;

If the goal of dotqmail will never be seeing it installed as a standard (like postfix, sendmail, exim), then this is not required. But if it needs to be made poplular with all class of users, then we should take a relook at /var/qmail for everhing. Even something like /opt/qmail will serve better.

Activity

  1. schmonz commented on Jul 7, 2019

    @schmonz
    schmonz
    on Jul 7, 2019
    Member

    Having notqmail available in system-native package repositories is an explicit goal, "Being easily packaged by OS integrators". It sounds like you're seeing a conflict with our explicit non-goal, "Breaking compatibility (such as by moving away from /var/qmail)". Thank you for raising this.

    I have some experience as a package maintainer in pkgsrc, where packages need to install files in locations consistent with BSD hier(7). I've solved the problem in pkgsrc qmail like so:

    $ ls -l /var/qmail
total 0
lrwxr-xr-x  1 root  wheel  20 Jun 30 20:10 alias -> /etc/qmail/alias
lrwxr-xr-x  1 root  wheel  22 Jun 30 20:10 bin -> /opt/pkg/bin
lrwxr-xr-x  1 root  wheel  44 Jun 30 20:10 boot -> /opt/pkg/share/examples/qmail/boot
lrwxr-xr-x  1 root  wheel  22 Jun 30 20:10 control -> /etc/qmail/control
lrwxr-xr-x  1 root  wheel  34 Jun 30 20:10 doc -> /opt/pkg/share/doc/qmail
lrwxr-xr-x  1 root  wheel  22 Jun 30 20:10 man -> /opt/pkg/man
lrwxr-xr-x  1 root  wheel  20 Jun 30 20:10 queue -> /var/spool/qmail
lrwxr-xr-x  1 root  wheel  20 Jun 30 20:10 users -> /etc/qmail/users

    qmail's installer follows symlinks, so files get installed to physical paths matching pkgsrc users' expectations, while still being available via logical paths matching qmail users' expectations.

    My working hypothesis, then, is that it's not worth paying the considerable cost of breaking /var/qmail compatibility, because preserving compatibility does not present a significant impediment to packagers. Admittedly, my experience is limited to pkgsrc. Can you find out and report back whether rpmlint or openSUSE's package builder still give errors or warnings on a setup like pkgsrc's?

  2. alanpost commented on Jul 7, 2019

    @alanpost
    alanpost
    on Jul 7, 2019
    Contributor

    In my own (unpublished) qmail fork I have a set of patches to do this. I created a separate auto_qmail_bin, auto_qmail_doc, auto_qmail_man which all default to /var/qmail (the same as auto_qmail) and where necessary modified exec calls to use PATH or the install location of an executable.

    I'm more than happy to submit a PR for the series of patches supporting FHS, as I demonstrably agree with @mbhangui on this one.

  3. DerDakon commented on Jul 7, 2019

    @DerDakon
    DerDakon
    on Jul 7, 2019
    Member

    One thing we definitely do not need to care about is /var/qmail/man. Seriously, just drop it whereever it belongs. Having to fiddle this out from /var/qmail makes things harder for everyone, if it just moves to the default folders everything should be fine. The only thing we need to look at if there are any name collisions.

  4. schmonz commented on Jul 8, 2019

    @schmonz
    schmonz
    on Jul 8, 2019
    Member

    Given our goal of supporting packaging, I agree that it's necessary for us to support FHS-compatible installations, and that manual pages (and commands, for that matter) should not require users to muck with PATH (or MANPATH or man.conf or who knows what) to find them.

    I'm not yet convinced that it's necessary for us to change any "qmail core" code to support FHS-compatible installations. My experience suggests otherwise.

  5. mbhangui commented on Jul 8, 2019

    @mbhangui
    mbhangui
    on Jul 8, 2019
    ContributorAuthor

    In my own (unpublished) qmail fork I have a set of patches to do this. I created a separate auto_qmail_bin, auto_qmail_doc, auto_qmail_doc which all default to /var/qmail (the same as auto_qmail) and where necessary modified exec calls to use PATH or the install location of an executable.

    Ditto here. But my set of changes have too many controversial changes. We could look at your patches for implementation in notqmail. What @schmonz has done is also a workable solution & requires almost no changes to code. google also puts its packages on /opt/google for linux. I will try his method to build a RPM and see what rpmlint reports. To be more specific this is what I did (full code here)

    conf-shared - /usr/share/qmail (boot files, doc)
    conf-prefix - /usr (binary)
    conf-sysconfdir - /etc/qmail (control files, users, openssl certs, tcp.smtp.cdb, etc)
    conf-libexec - /usr/libexec/qmail (z* scripts, x* scripts, config-fast, dnscname, dnsip, dnxmxip, etc)
    conf-qmail - /var/qmail (queue, alias, virtual domains directory, etc)

    There is no conf file for man pages. They go straight to /usr/share/man

    and symbolic links in /var/qmail

    bin
    control
    users

  6. alanpost
    added
    packaging
    on Jul 9, 2019

  7. schmonz commented on Jul 11, 2019

    @schmonz
    schmonz
    on Jul 11, 2019
    Member

    IRC discussion with @alanpost clarified that "FHS" carries with it two related but distinct goals. Attempting to summarize:

    1. "FHS-aware": By default, we want notqmail to install files to FHS-compatible paths (with conf-qmail still defaulting to /var/qmail, and having that be a symlink farm as described above)
    2. "FHS-strict": Optionally, off by default, we want notqmail to install files to FHS-compatible paths without any sort of /var/qmail or equivalent (by patching qmail programs to find each other and their config files in new ways)
  8. alanpost
    added this to the 1.08 milestone on Jul 22, 2019

  9. jccleaver commented on Aug 23, 2019

    @jccleaver
    jccleaver
    on Aug 23, 2019

    I think 'FHS-strict' might be a bit much... There are (or were) quite a few packages out there that internally still rely on symlinks on some pseudo-HOME directory for dispatch to FHS locations, and the underlying code changes might be more effort than their worth in some spots. (This is especially the case if we still want to support non-FHS installs in this.

    I was able to get my /var/qmail/ down to just this in my RPMs, and /var/qmail/queue/ arguably could have gone elsewhere too, but it was common at the time to do weird things with it on a separate partition...

    [root@cloud qmail]# ll /var/qmail/
    lrwxrwxrwx 1 root root 16 Sep 30 2013 alias -> /etc/qmail/alias
    lrwxrwxrwx 1 root root 18 Sep 30 2013 control -> /etc/qmail/control
    lrwxrwxrwx 1 root root 14 Sep 30 2013 log -> /var/log/qmail
    drwxr-x--- 11 qmailq qmail 4096 Sep 30 2013 queue
    lrwxrwxrwx 1 root root 16 Sep 30 2013 users -> /etc/qmail/users

    Everything else was FHS-strict.

    Also, even with qmail being FHS, many add-on packages (vpopmail, etc) may expect to have a unified QMAILHOME where it might call things out from.

  10. schmonz commented on Aug 23, 2019

    @schmonz
    schmonz
    on Aug 23, 2019
    Member

    At least for the near term, and possibly much longer, "FHS-strict" builds won't be the default. This is an excellent reminder that when we introduce the FHS-strict build option in 1.08, we'll need to document its risks and enumerate some known implications. Glad to see you commenting here and giving us some vpopmail context — welcome!

  11. mbhangui commented on Aug 23, 2019

    @mbhangui
    mbhangui
    on Aug 23, 2019
    ContributorAuthor

    I have gone through this FHS difficulty in indimail (the design of which is based on vpopmail). What I had to deal with were the following

    1. many programs do chdir("/var/qmail").
    2. Many programs are expected in /var/qmail/bin
    3. The assign file is in /var/qmail/users
    4. To further compound the FHS mess, vpopmail introduces another directory /var/vpopmail
    5. The delivery instructions in .qmail-default is /var/vpopmail/bin/vdelivermail "" bounce-no-mailbox
    6. .qmail-default in /var/vpopmail/domains/domain_name

    I had to do a two stage approach. First was to have the following and appropriate symbolic links

    1. Put all user binaries in /usr/bin and system binaries in /usr/sbin
    2. Put control files in /etc/indimail/control
    3. tcp.smtp.cdb, etc in /etc/indimail

    With time, I just removed the symbolic links. Now I have a distribution that builds on opensuse build service without having to supress FHS related errors for the RPM & debian distributions and without having to modify the PATH in the .profile

  12. josuah commented on Apr 10, 2020

    @josuah
    josuah
    on Apr 10, 2020

    I am trying to summarize the source code changes required (checked with grep '"<subdir-of-/var/qmail>' *.[ch]):

    • alias: never refered directly in the code but rather as the home directory of the alias user -> safe
    • bin: only referred as a relative path, so stripping bin/ and using execvp() should suffice -> sed -i 's|"bin/|"|' *.[ch]
    • boot, man, doc: only used in hier.c, to install qmail -> safe
    • queue: called as "chdir" or directly, relatively to qmail_home -> workaround needed!
    • control: only called in hier.c, in end qmail-*.c programs or in rcpthosts_init() (itself only called in qmail-*d.c), and when they are, it is always at the begining of the program, after a chdir(qmail_home), and before any use of the queue -> workaround needed
    • users: called by programs that only deal with the users/cdb file, or in qmail-lspawn.c which also chdir to the queue, but calls "users/cdb" first

    It should be possible to have FHS behavior with only this:

    1. defining qmail_home to /etc/qmail or equivalent, with only control and users in it
    2. strip bin/ out of exec calls
    3. replace occurences of "queue/[...] to QMAIL_QUEUE"/

    Then we can:

    • keep the /var/qmail behavior by defining qmail_home to /var/qmail and QMAIL_QUEUE to queue.
    • use FHS-style paths by defining qmail_home to /etc/qmail and QMAIL_QUEUE to /var/spool/qmail

    This is lightweight on qmail (a few strings replaced only), and exposes few switches that will be easy to integrate. It also does not touch to qmail_home (patch compatibility) or remove/add any line of code.

    The rest is a matter of install paths, another topic...

    I will check this in practice and report it there.

  13. DerDakon commented on Apr 11, 2020

    @DerDakon
    DerDakon
    on Apr 11, 2020
    Member

    Excellent read. The "bin/" thing actually hardcodes the path so no other binaries of the same name will be called.

    For the install woes take a look at #116.

  14. josuah commented on Apr 11, 2020

    @josuah
    josuah
    on Apr 11, 2020

    @DerDakon I am coming late in the game, so I indeed have some reading to go. Great work so far!

    Reading ongoing work, I prefer #35 to the solution I am proposing, for it is a full solution to the problem and not only a workaround.

  15. DerDakon
    removed this from the 1.09 milestone on Jan 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    packaging

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions