Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible regression on RPM package from EPEL on EL7 #52

Open
jleafey opened this issue Dec 5, 2018 · 25 comments
Open

Possible regression on RPM package from EPEL on EL7 #52

jleafey opened this issue Dec 5, 2018 · 25 comments

Comments

@jleafey
Copy link

jleafey commented Dec 5, 2018

On several RHEL and CentOS 7 systems we just updated from 0.5.10 to 0.5.12 via the package from the EPEL repository. Since then we have started getting a large number of defunct processes that are children of the incron daemon. Each and every time a particular script runs a "zombie" process is left with the parent PID pointing to the incron daemon.
The incrontab entry in question is as follows:

/oracle/hbanpr/jobs/output IN_CLOSE_WRITE,recursive=false /home/oracle/bin/cp_to_volshare.sh $@/$#

I added "recursive=false" to the line after reading some recent posts on the network. When I look at the zombie processes in the /proc filesystem they have no open files. The script itself is pretty simple, it merely copies the file triggering the event to another location and it works just fine if I invoke it from the command line.
Short of restarting incrond periodically to flush out the defunct processes, is there something else I can do to resolve this?

@jleafey
Copy link
Author

jleafey commented Dec 5, 2018

More testing has shown that the problem only occurs with user incrontab entries, the system incrontab entries in /etc/incron.d/ do not cause processes to be left in the zombie state.

@jleafey jleafey closed this as completed Dec 5, 2018
@jleafey
Copy link
Author

jleafey commented Dec 5, 2018

OK, didn't mean to close this! Still and issue, but at least I can work around it now.

@jleafey jleafey reopened this Dec 5, 2018
@ericjunkermann
Copy link

This was fixed in checkin f45c2f5 on 31 Oct 2017, but it's not in a formal release.

@jleafey
Copy link
Author

jleafey commented Dec 6, 2018

I just downloaded the source RPM from EPEL and compared the sources, it appears that the EPEL package was built from sources that did not include the fix in that checkin. I will take the issue up with EPEL, thanks for the pointer!

@jleafey
Copy link
Author

jleafey commented Dec 6, 2018

This just keeps getting better and better! I rebuilt the RPM using the current sources from Github and reinstalled, but the behaviour did not change, still getting zombies with every user incron event. I then deinstalled the packaged version and built and installed it from sources. I altered the systemd unit file to point to /usr/local/sbin/incron and started it up, no zombies. This seems to point to a difference in how they are being built, so I believe this is a packaging issue, which I will take up with EPEL.

@sharkannon
Copy link

sharkannon commented Dec 10, 2018

I've also had the same issue running the latest from EPEL on Centos 7. I downloaded the previous release and installed it (0.5.10-8.el7) and that seems to have "resolved" the issue.

# cat /etc/redhat-release
CentOS Linux release 7.6.1810 (Core)

Latest EPEL version:
0.5.12-6.el7

Working:
wget https://kojipkgs.fedoraproject.org//packages/incron/0.5.10/8.el7/x86_64/incron-0.5.10-8.el7.x86_64.rpm

yum downgrade /incron-0.5.10-8.el7.x86_64.rpm

#rpm -qa | grep incron
incron-0.5.10-8.el7.x86_64

@sharkannon
Copy link

@jleafey Can you link the ticket you might open with EPEL? I'd like to keep an eye on it.

@jleafey
Copy link
Author

jleafey commented Dec 10, 2018

The ticket number for this issue is 1656939 on Red Hat's Bugzilla system, which covers Fedora and EPEL as well as Red Hat's commercial products. https://bugzilla.redhat.com/show_bug.cgi?id=1656939

@rippiedoos
Copy link

Just for reference, I hit the same bug. I tried moving the user incrontab to /etc/incron.d. That doesn't fix the problem. In my case, the zombie processes where gone but instead the child incron processes where created, but not zombied. So downgrade to 0.5.10 is the only workaround for now. And wait until the EPEL-ticket get's resolved.

@jleafey
Copy link
Author

jleafey commented Dec 19, 2018

As a test I rebuilt incrond and incrontab from source and copied them over the files provided in the EPEL RPM (/usr/sbin/incrond and /usr/bin/incrontab). After restarting the daemon and running my test case again I have seen no defunct processes being generated at all.
I was curious to see if cropped up in the latest Fedora release, which appears to be built from the same source RPM. I built a Fedora 29 VM from scratch, installed incron from the repository, and it does the same as the EPEL RPM, all user-level (and most system-level) incron events leave a defunct process. This looks more and more like a packaging issue.

@robert-scheck
Copy link

From my point of view, this is not a regression in the EPEL package itself, but purely in incron 0.5.12; having an equivalent experience with issues #4 and #32, which just worked perfectly with incron 0.5.10 before.

@jleafey
Copy link
Author

jleafey commented Jan 8, 2019

Fedora has new packages for 0.5.12 in the testing repository for EL7, Fedora 28, and Fedora 29 to include the no-zombie fix from upstream. The new packages are marked 0.5.12-9, the old package was 0.5.12-6. I have only tested it for a few hours but so far I have not seen another defunct process. Once I have tested it more extensively I will note the results here, but it looks good so far.

@slesru
Copy link

slesru commented Jan 16, 2019

I created issue today https://bugzilla.redhat.com/show_bug.cgi?id=1666590
before read this one.
I have +1 incrond process on each run.

ps ax | grep incron
15495 ? Ss 0:00 /usr/sbin/incrond
15506 ? S 0:00 /usr/sbin/incrond
15507 ? S 0:00 /usr/sbin/incrond

0.5.12-9 from testing does not resolve my problem... :-(

@sdwru
Copy link

sdwru commented Jan 18, 2019

Glad I found this discussion. Current EPEL version is definitely broken. Caused some big headaches for us. Downgrading to 0.5.10 fixed it for now. That old version was not that easy to find. Apparently EPEL does not keep old versions around.

incron-0.5.10-8.el7.x86_64.rpm.zip

@philrandal
Copy link

I've experienced the same issue.

0.5.12-9 from testing did not fix it for me, either.

0.5.10 works for me.

@Uatschitchun
Copy link

This was fixed in checkin f45c2f5 on 31 Oct 2017, but it's not in a formal release.

Stumbled upon that one, too!
Running Ubuntu 18.04 which provides 0.5.10 only, no recusive watch is possible. So I downloaded latest release (0.5.12) and built it my own. After running for a while, I notice app. 200 zombies which parent is incron!
As stated above, the corresponding commit is not available in latest release, which is 14 commits behind already and 4 years old! I did build from master and have no zombies anymore! Can't say anything about memory leaks until now.
As even debian has updated their build in sid only lately to 0.5.12, (see: Debian Changelog), it would be nice, to fix up things and do a new release quite soon!

@vlours
Copy link

vlours commented Mar 18, 2019

Tests made as part of the Bugzilla: 1666590 seems to fix the huge number of defuncts (0.5.12-6) or processes (0.5.12-9) in version 0.5.12-10.

Thanks @philfry for the commit.

@sdwru
Copy link

sdwru commented Dec 11, 2019

EPEL stable shows v0.5.12-11 and we definitely have memory leak problems with that version. Downgrading to incron-0.5.10-8.el7.x86_64.rpm fixes it. So it appears there are still memory leak problems.

@sdwru
Copy link

sdwru commented Jan 24, 2020

So is this ever going to get fixed?

@jpfisherabc
Copy link

Finally upgraded to 5.12.. After reviewing /var/log/cron, it appears that our old Comments lines were being interpreted as watch directories. removing the comments solved our problem. Hope this helps.

@Averell7
Copy link

Averell7 commented Apr 3, 2020

This issue is also present in Debian Buster. Using armbian, based on Debian Buster, I have the problem. incron version installed is 5.12-1.

I solved it by downgrading to 5.10.

@Chealer
Copy link

Chealer commented Sep 1, 2021

So is this ever going to get fixed?

This is "fixed" in the VCS, just not in packages. Whether the package will be fixed only depends on whether incron, which looks quite dead, will ever see a new release.

@Chealer
Copy link

Chealer commented Sep 1, 2021

This issue is also present in Debian Buster. Using armbian, based on Debian Buster, I have the problem. incron version installed is 5.12-1.

This bug is tracked in Debian's #930526. It only affects buster (oldstable), where it will likely never be fixed.

@sdwru
Copy link

sdwru commented Sep 1, 2021

Seeing as how this (pretty severe imo) bug will apparently never be fixed is it safe to say this package is abandonware?

@Averell7
Copy link

Averell7 commented Sep 2, 2021

is it safe to say this package is abandonware?

It would be so sad. Incron is a wonderful package, and a key piece of the software we are developing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests