Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debian's patches #23

Closed
paravoid opened this issue May 5, 2016 · 32 comments
Closed

Debian's patches #23

paravoid opened this issue May 5, 2016 · 32 comments
Assignees
Labels
type:discuss Your views/opinions are requested

Comments

@paravoid
Copy link

paravoid commented May 5, 2016

First of, let me say thank you so much for neomutt. I've been disappointed by the state of mutt for many years now and I find this project to be very refreshing to see and very much appreciated.

As you may probably be aware, Debian, like most downstreams, carries a lot of patches on top of mutt. In Debian's case, a few of them (like the sidebar and a few others) are distributed separately as a "mutt-patched" binary package, generated from the same source. The "stock" mutt package is also heavily patched; e.g. compressed folders is part of it (the line isn't very clear and I think it been based traditionally on stability merits). mutt-kz is also part of Debian, but it's generated from a separate source package and carries its own set of patches; I won't be focusing on that here.

Below you'll find the Debian patches, as can be found in pkg-mutt's git, split by their category, and with their status documented:

features

Patch neomutt equivalent
features/ifdef.patch 11-ifdef
features/trash-folder.patch 14-trash
features/purge-message.patch 14-trash
features/imap_fast_trash.patch 14-trash
features/sensible_browser_position.patch devel/sensible-browser
features/compressed-folders.patch devel/compress
features/compressed-folders.debian.patch (not needed)

misc

Patch neomutt equivalent
misc/am-maintainer-mode.patch (not needed)
misc/define-pgp_getkeys_command.patch (obsolete, not needed)
misc/gpg.rc-paths.patch applies cleanly
misc/smime.rc.patch applies cleanly

upstream

Patch neomutt equivalent Comments
upstream/528233-readonly-open.patch applies cleanly
upstream/228671-pipe-mime.patch applies cleanly
upstream/383769-score-match.patch applies cleanly
upstream/771125-CVE-2014-9116-jessie.patch applies cleanly
upstream/path_max.patch applies cleanly GNU Hurd compatibility
upstream/809802_timeout_hook.patch merge conflict Source: http://wolfermann.org/mutt.html

mutt-patched (only included in the "mutt-patched" binary package)

Patch neomutt equivalent Comments
mutt-patched/sidebar.patch 09-sidebar
mutt-patched/sidebar-delimnullwide.patch (not needed)
mutt-patched/multiple-fcc.patch merge conflict Source: https://www.mandarb.com/mutt/
mutt-patched/sidebar-newonly.patch 09-sidebar
mutt-patched/sidebar-compose.patch (not needed)
mutt-patched/nntp.patch devel/nntp

Debian-specific

Patch Rebase status
debian-specific/Muttrc.patch applies cleanly
debian-specific/Md.etc_mailname_gethostbyname.patch applies cleanly
debian-specific/use_usr_bin_editor.patch applies cleanly
debian-specific/correct_docdir_in_man_page.patch applies cleanly
debian-specific/dont_document_not_present_features.patch applies cleanly
debian-specific/document_debian_defaults.patch applies cleanly
debian-specific/assumed_charset-compat.patch applies cleanly
debian-specific/467432-write_bcc.patch applies cleanly
debian-specific/566076-build_doc_adjustments.patch applies cleanly

Legend:

  • Prefixed numbers are Debian bug reports and may provide more context. Use https://bugs.debian.org/NNNNNN to look those up.
  • "applies cleanly" and "merge conflict" means on top of stock neomutt 20160502.
  • "not needed" means the patch is either obsolete or wrong.
  • Debian-specific are… Debian-specific fixes :); all of these could be argued to all be in the "not needed" category.

I'm not part of the Debian mutt maintainer team (yet :), but I am a Debian Developer, familiar with how Debian works and was briefly in touch with the mutt maintainers earlier this week to discuss neomutt. My recommendation to them was to replace our mutt-patched package (and in the future, mutt-kz as well) straight with neomutt.

For this to happen, we'd need to forward-port all of the above patches that have no neomutt equivalent — and in the case of sensible-browser, compressed folders and NNTP, possibly replace them with your own cleaned-up versions from the devel/ branches. The only ones that don't seem to be applying cleanly so far are the multiple FCC and timeout-hook ones. Hopefully these can found their way to neomutt?

Ideally, now that there is a responsive upstream, we wouldn't have to carry any upstream patches in the Debian package. I hope the above helps you get a few more patches on your radar so that we can avoid dealing with merge-conflicts ourselves and that others can benefit from these :)

@flatcap
Copy link
Member

flatcap commented May 5, 2016

Hi @paravoid

thank you so much for neomutt.

You're welcome. I'm slowly gathering people want Mutt to evolve.

I'm not part of the Debian mutt maintainer team (yet :), but I am a Debian Developer, familiar with how Debian works

Two weeks ago, I wrote to Antonio Radici, Christoph Berg (maintainers of the mutt-patched package). No reply. It's good to hear from a Debian developer.

Debian carries a lot of patches on top of mutt.

That makes me angry. (With Mutt, not Debian).

The "stock" mutt package is also heavily patched
I think it been based traditionally on stability merits

Fabian Groffen (@grobian) has the same situation with Gentoo.

mutt-kz is also part of Debian, but it's generated from a separate source package and carries its own set of patches

mutt-kz branched off from Mutt years ago and was starting to drift away.
After a huge amount of work rebasing, I got mutt-kz up-to-date and split out the sub-features.
Mutt-kz and NeoMutt are now synchronised.

I pull all of Karel Zak's (@karelzak) changes into NeoMutt
I push all dependent-feature commits to mutt-kz.

Debian patches, split by their category, and with their status documented:

Thanks for the breakdown. Very helpful.

I was briefly in touch with the mutt maintainers earlier this week to discuss neomutt.
My recommendation to them was to replace our mutt-patched package straight with neomutt.

Great. Let me know their requirements.

For this to happen, we'd need to forward-port all of the above patches that have no neomutt equivalent

Yes. I'll get to work.

and in the case of sensible-browser, compressed folders and NNTP,
possibly replace them with your own cleaned-up versions from the devel/ branches.

OK, here's the status of [devel/*]

devel/ State of feature
compress Code tidied, but not fully tested
keywords Not tidied, not tested
new-mail WIP Command hook for new mail
nntp Externally maintained. Reliable. Needs docs.
sensible-browser Not tidied. Known bugs. Needs work.
win-sidebar Demo of Sidebar using new Mutt windows. Includes $sidebar_on_right variable!

The only ones that don't seem to be applying cleanly so far are the multiple FCC and timeout-hook ones.
Hopefully these can found their way to neomutt?

I shall have a look at them.

"Bringing together all the Mutt code" -- For now, I need to be a "patch curator".

Ideally, we wouldn't have to carry any upstream patches in the Debian package.

That would be nice.

If it's helpful, I'm happy to add distro-specific dirs to the feature branches. e.g.

  • contrib/debian
  • docs/gentoo

(and give write privs to the devs to keep things up-to-date).
This will also help spread information between the distros.

... so that we can avoid dealing with merge-conflicts ourselves

As I'm finding with Gentoo, there'll always be some conflicts.

and that others can benefit from these :)

Absolutely.

Rich / FlatCap

@flatcap flatcap added the type:discuss Your views/opinions are requested label May 5, 2016
@flatcap flatcap self-assigned this May 5, 2016
@flatcap
Copy link
Member

flatcap commented May 5, 2016

@paravoid

Which versions of Mutt are Debian's patches for?

  • 1.5.24
  • 1.6.1
  • default

@evgeni
Copy link

evgeni commented May 5, 2016

Hey @flatcap, I am one of the current maintainers of mutt in Debian/Ubuntu (@ChristophBerg) did forward me your mail, but I forgot to reply, sorry!

The patches @paravoid linked to are currently against mutt 1.6.0. @mfvescovi wanted to update them to 1.6.1, but I don't think he had the time yet.

I think syncing mutt-patched and neomutt would be a good idea. Or dropping it altogether as mutt itself gains all features... Wishful thinking :)

Anyways, My todo says to try your sidebar instead of ours in the next days, maybe even more. Thanks for all the work you did!

@flatcap
Copy link
Member

flatcap commented May 5, 2016

@evgeni

... forward me your mail, but I forgot to reply, sorry!

No worries :-)

I think syncing mutt-patched and neomutt would be a good idea.

Good. I'll be a happy man when there are .deb's with the new sidebar.

Or dropping it altogether as mutt itself gains all features...

Kevin's new window patches, for Mutt, show he's keen to integrate Sidebar at some point.
However, the changes are in the default branch which could mean that they're not seen for months.

Fabian is keen to cherry-pick/transplant default-branch code into Gentoo, if it gets him cool features.

If there's demand, I might consider maintaining a set of patches against default.
If I don't see any demand, I'll stop supporting 1.5.24.

My todo says to try your sidebar instead of ours in the next days,

Note: Some config has changed since the 2015-11-11 patch.

Any problems, or questions, let me know.

Thanks for all the work you did!

You're welcome.

Lots still to do.

@paravoid
Copy link
Author

paravoid commented May 5, 2016

OK, I had another look. In terms of immediate next steps, to allow us to reuse at least a subset of neomutt's patches for a first stab without any regressions (ifdef, trash, sidebar, nntp):

  • Graduating NNTP into a feature, or at least providing a clean rebase on top of neomutt (or sidebar) — preferrably the latter of course :) I tried to do a merge devel/nntp on top of the neomutt branch and got tons of conflicts.
  • Graduating compressed folders into a feature. Same story, although things are a little better there as the patch that Debian currently carries applies cleanly.

The timeout-hook and multiple-fcc patches would be next in line, followed by the rest of Debian's custom patches — but all these can wait a little bit I think.

@flatcap
Copy link
Member

flatcap commented May 6, 2016

@paravoid
I've just created a new repo integration as a temporary home for some patch sets.
It contains three branches:

  • debian/1.6.0 -- original Debian patches against Mutt 1.6.0
  • debian/1.6.1 -- original Debian patches against Mutt 1.6.1
  • deb-new/1.6.1 -- NeoMutt patches + Debian patches for Mutt 1.6.1

They compile. They appear to work.
I'll keep testing and check that the patches look sane.

@flatcap
Copy link
Member

flatcap commented May 7, 2016

A few fixes here and there and three new patch branches:

  • patch/debian/1.6.0 -- original Debian patches against Mutt 1.6.0
  • patch/debian/1.6.1 -- original Debian patches against Mutt 1.6.1
  • patch/deb-new/1.6.1 -- NeoMutt patches + Debian patches for Mutt 1.6.1

The patches look good and everything compiles.

@RichiH
Copy link

RichiH commented May 10, 2016

I'd just like to say "weeeeeeeeeeeeeeeee" and shall lurk on this issue from now on.

Thanks to all involved :)

@flatcap
Copy link
Member

flatcap commented May 10, 2016

@RichiH

I'd just like to say "weeeeeeeeeeeeeeeee"

Save the celebrations for later, we've got work to do :-)

and shall lurk on this issue from now on.

Any Deb-dev knowledge you can contribute will be welcome.

@RichiH
Copy link

RichiH commented May 10, 2016

I will be more than happy to help (I am a DD as well), but I suspect that @evgeni and @paravoid don't really need much help. If there's anything to do, I will gladly chip in, though.

@paravoid
Copy link
Author

@flatcap, thank you so much for doing this integration work for us. I pushed a wip-neomutt branch into pkg-mutt, partially based on your work. We discussed it again within the Debian maintainers, and we are considering instead to switch to the "pristine" neomutt, with the entirety of the patches and in your order, to replace mutt-patched (and possibly mutt) entirely with it.

What would help us tremendously, I think, would be what I mentioned in my last comment above: get rid of all of our upstream patches in favor of upstream ones, especially compressed folders and NNTP (that are halfway there :)). The rest of our upstream patches can follow after that, assuming you have the time and will to review them and incorporate them :)

By the way, a question that inevitably came up while we were discussing our plans was: where does vanilla mutt stands in all this? We know they have been traditionally slow/reluctant to respond, but some work seems to be happening upstream now (including the windows stuff you already mentioned), so I'm wondering if we can expect some movement there as well, especially considering the curation work that you have been doing.

@flatcap
Copy link
Member

flatcap commented May 14, 2016

@paravoid

I pushed a wip-neomutt branch into pkg-mutt, partially based on your work.

Great stuff.

compressed and NNTP

Message understood :-) I shall make some time this weekend and sort them out.

get rid of all of our upstream patches

I've got a plan for that (open to discussion). Each distro can have their own bug-fix/tweak branch. Anything that's likely to be universally wanted, I'll put into NeoMutt's bug-fix branch and then push upstream to Mutt.

where does vanilla mutt stand in all this?

Good question. Most code pushed to Mutt will go into the default branch. These changes won't be public (merge into the stable branch) until everything in default is stable. That's likely to be many months.

But, that probably doesn't matter too much. Each bug that's accepted will make our jobs easier (just not immediately). Git's perfectly happy to have the same commit in multiple places and it'll do the right thing when everything gets merged.

@evgeni
Copy link

evgeni commented May 16, 2016

just tested the wip branch from @paravoid and loving it so far.

I'd totally make mutt-patched == neomutt in the future and move the non-bugfix patches out of mutt to mutt-patched :)

@RichiH
Copy link

RichiH commented May 17, 2016

@evgeni @paravoid can you toss a signed test package into an appropriate place? I would love to test it (without going through the hassle of a local build, for now).

@evgeni
Copy link

evgeni commented May 17, 2016

@RichiH don't have my debian gear with me, but I think @mfvescovi wanted to do an experimental upload soon.

@mfvescovi
Copy link

On 2016-05-17 at 17:20 (CEST), Evgeni Golov wrote:

@RichiH don't have my debian gear with me, but I think @mfvescovi wanted to do an experimental upload soon.

@RichiH, done. Enjoy!

@RichiH
Copy link

RichiH commented May 23, 2016

I have been using neomutt for a few days now, and there's only one issue I am away of: #29

Note that this might be existent in older versions as well, I never really used sidebar before.

@RichiH
Copy link

RichiH commented May 23, 2016

As per #29 an updated build might make sense, else I will just wait for the next release to test again.

@riesebie
Copy link
Contributor

Wouldn't it make more sense to keep Debian's mutt package as it is and to create a seperate neomutt package? I am afraid, that the pristine mutt will be overpatched and at some point gets non maintainable? I was maintaining mutt-ng in Debian some years ago and the package coexists well to the pristine mutt one. If, well maybe in some decades ;-), pristine mutt will integrate the neomutt patches we can merge the two Debian packages. I can package and maintain neomutt as a seperate Debian package with a replacement to the pristine mutt/mutt-patched package so that the user can decide which one to use. From the mainataining point of view this would make more sense?

@flatcap
Copy link
Member

flatcap commented May 25, 2016

Hi @riesebie, welcome to the discussion.

... keep Debian's mutt package as it is ...

First, it would need to be updated to the latest versions of the patches. They contain bug-fixes and other improvements, like documentation. I don't want to get bug reports for things I can't do anything about.

I am afraid, that the pristine mutt will be overpatched

It's a possibility, but... Most of the patches, out there, are already integrated, so there's not much more to add.

Historically, the Mutt devs have been too conservative to new ideas and this has pushed away lots of developers. I want to encourage people to think of new ideas. I'd be happy to see a NeoMutt-devel branch, full of dangerous ideas!

at some point gets non maintainable

Hopefully that won't happen. We're all volunteers and little time to waste. That's why I've been keen to bring the distros together. It's something the Mutt community should have done years ago.

If you haven't already read it, see my Patch Tree Proposal.

The distros can decide how much they want to include (to a degree).
And, I get to maintain exactly one set of integration patches.

If others want to make their own patch sets, that's fine, but I don't have time to help them.

I can package and maintain neomutt as a separate Debian package

Thanks for the offer, but if @mfvescovi @evgeni are happy with a full NeoMutt, then there's no need.

From the maintaining point of view would this make more sense?

From my point of view, giving everyone the same thing is much easier :-)
Note: several of the larger features can always be turned off at compile-time for a lighter version.

Thanks for your ideas. Keep them coming.

@paravoid
Copy link
Author

@riesebie I've thought of that too (both the danger of having a significant patchset to maintain forward and the possible need for a "vanilla" mutt). The problem is of course that the vanilla mutt package has accumulated quite a few patches already, several of which are invasive but are included in neomutt already (ifdef, trash, compressed folders, etc.) — and that's even without the mutt-patched variant that has a bunch more.

I'm new to this and not very familiar with mutt-ng's history. Could you perhaps elaborate on what happened with it so that we can learn from history and not repeat the same mistakes? @flatcap seems to be doing a great job driving neomutt forward, but also looping in all the downstreams as well, so I'm hopeful this effort will stick even if he disappears for whatever reason. We already have to port forward our large upstream featuresets across mutt's releases, so it seems worthwhile to benefit from each other's work between the distributions and commit into a single repository. (that may be a little naive, I suppose :)

In general, my opinion tends to lean towards dropping the original mutt upstream in favor of neomutt and merge as many debian/patches as we can in this new upstream. This is a LibreOffice/OpenOffice.org or glibc/eglibc deal (or gcc/egcs if you want to go farther out), as far as I can tell.

On that note and to get back to on-topic for neomutt: @flatcap, thanks you so much for working on compressed and NNTP and graduating them to stable, that's awesome to hear :) In terms of user-visiable features, this leaves:

  • sensible-browser (already on your radar as far as I can tell)
  • timeout-hook (seems easy enough)
  • multiple-fcc (which IMHO could be dropped)
    …plus all kinds of other smaller fixes (all the upstream-* ones) that I"m sure you've already seen :)

@flatcap
Copy link
Member

flatcap commented May 26, 2016

even if he disappears for whatever reason

Er... That sounds like you're planning a coup :-)

compressed

There's more testing and tidying I want to do, but it's good enough.

sensible-browser

It sort-of works, but it should take into account the current sort order.
I want it in NeoMutt, but not in its current state.

@karelzak
Copy link
Contributor

karelzak commented Jun 7, 2016

Just note... I prefer to use NeoMutt than directly mutt-kz. I'll keep mutt-kz as git tree, but for end-users and downstream distribution is better to have all integrated to the one package. It would be also nice to have the same [Neo]Mutt in all distributions to minimize difference and distro-specific patches. We need to share things...

@grobian
Copy link
Contributor

grobian commented Jun 7, 2016

/me nods
Perhaps that also gives a stronger incentive to $UPSTREAM to integrate the patches at last.

@flatcap
Copy link
Member

flatcap commented Jun 7, 2016

@karelzak

I prefer to use NeoMutt than directly mutt-kz

Thanks for the vote of confidence :-)

for end-users and downstream distribution is better to have all integrated to the one package

I'm happy with that, but I didn't want to presume. You've worked hard and got a lot of followers. I didn't want to undermine that.

It would be also nice to have the same [Neo]Mutt in all distributions to minimize difference and distro-specific patches. We need to share things...

You may not have read my Patch Tree Proposal and see my Patch Sets. Progress is being made.

@paravoid
Copy link
Author

paravoid commented Jul 9, 2016

With the recent 1.6.1-2 Debian upload, we've come much closer to neomutt (this release is actually neomutt). See https://anonscm.debian.org/cgit/pkg-mutt/mutt.git/log/?h=experimental for the full changelog and https://anonscm.debian.org/cgit/pkg-mutt/mutt.git/tree/debian/patches?h=experimental for the patch list (https://anonscm.debian.org/cgit/pkg-mutt/mutt.git/tree/debian/patches/series?h=experimental is the series file).

So far, in terms of features, we lack:

  • sensible browser (already in progress, AFAIK)
  • timeout hook
  • multiple FCC

Thanks for all of your efforts :)

@flatcap
Copy link
Member

flatcap commented Jul 10, 2016

Great stuff! I'll look through the list of patches soon (it's very late now).

  • sensible browser (already in progress, AFAIK)
  • timeout hook
  • multiple FCC

Got, got, got. I think I got my Debian patches from Sid.
The patches are rebased to apply cleanly over NeoMutt 2016-07-09.
The README lists the order in which they should be applied.

Thanks for all of your efforts :)

And thanks for yours.

@paravoid
Copy link
Author

I've already used these from your branch and they were very helpful :)

At this point, I'm interested in either seeing these get upstreamed to neomutt (and perhaps cleaned up in the process) or to drop them from the package. Debian has been historically accepting upstream patches in the package but nowadays that we have a reasonable upstream (you ;), we should get back to what our job is really about: packaging and integration.

@darnir
Copy link
Contributor

darnir commented Jul 21, 2016

Can we close this now? Or do we still have some pending patches from Debian that can be applied?

@flatcap
Copy link
Member

flatcap commented Jul 21, 2016

Leave it for now, please.
Those three patches need to be cleaned up and migrated to [bugs/common].

@flatcap
Copy link
Member

flatcap commented Aug 7, 2016

I've committed two new branches and merged them into neomutt. They are

I've had a good look at the third patch you wanted, sensible-browser, but it needs more thought.
I plan to close this issue, on the next release, because sensible-browser has its own issue, and the discussion here is over.

@RichiH
Copy link

RichiH commented Aug 7, 2016 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:discuss Your views/opinions are requested
Projects
None yet
Development

No branches or pull requests

9 participants