Test that popular patches still apply cleanly

[schmonz]

The wiki says "Expect your patchset to mostly continue to apply".

To automatically keep paying attention to this, I'm thinking we need an automated regression test:

1. Select a baseline set of popular patches
2. Verify that each patch individually applies and builds
3. Maybe verify certain combinations too

This can be the first of a suite of automated tests that will grow to protect us against more and more mistakes we want to avoid making. (Later, when we have a build farm, we can run them on every commit.)

[alanpost]

I have created the following netqmail branches, corresponding to a variety of published patches:

• netqmail-badmailfrom-wildcard
• netqmail-badmailfrom-x-relayclient
• netqmail-big-concurrency
• netqmail-big-todo
• netqmail-dns-any-to-cname
• netqmail-dns-oversize
• netqmail-ext-todo
• netqmail-smtp-auth
• netqmail-smtp-tls
• netqmail-smtpd-logging
• netqmail-smtpd-spf

Nearly all of these patches required follow-up work. A comprehensive but incomplete checklist of QA due diligence:

• remove trailing whitespace.
• add any new files to FILES.
  ** merge or remove any spurious files.
• if necessary, add any new files to hier.c/install-big.c doc/
• restore Makefile to alphabetical order.
• sync the Makefile shar target with FILES.
• add any new targets to TARGETS.
• remove any ifdef/define lines. The branch contains the feature.
• verify Makefile includes any new header dependencies in .o targets.
• add any missing entries to CHANGES.
• update or add any missing manpages or other documentation.
• stylistic cleanup: make the code match the existing codebase.
• commit any necessary bugfixes.

I have also created a single (sample) notqmail branch:

• notqmail-smtpd-logging

This patch is one of three that appears in Gerrit Pape's Debian package, and in this branch I've made a stylistic change and added a minor feature.

In total this permits us to treat all netqmail patches separately, and particularly lets us amend those patches to meet the quality standards of the codebase. Further, it gives us a means of producing features on top of that work while also letting us rebase from netqmail to notqmail. Finally, it allows us to do integration testing between these branches via merging them together.

[DerDakon]

I would leave some of these points out:

• CHANGES file: this is external work, so no need to document it there
• shar target: we want to kill it, so why bother, it only makes the diff larger
• stylistic/whitespace cleanup: again, this is external work, this would only be necessary if this is going to be merged into notqmail

[josuah]

This page is dedicated to it: https://notqmail.z0.is/patch/

For now, it is possible to send me a list of commits/branch/ref and a list of patches to test. If you prefer it can be integrated somewhere into https://github.com/notqmail/.

I like the idea of applying patches to netqmail versions. Then we can see when a patch break on master.

This would be a decision point:

• merge the patch (by rebasing onto its branch if any or applying one)
• provide an edited patch that can still be applied,
• drop support for this patch (maybe it is not required anymore)

[schmonz]

Wonderful. I’ve linked to this from https://github.com/notqmail/notqmail/wiki/Patches