Coverage measurement fails on code containing os.exec* methods

[nedbat]

Originally reported by Anonymous

---

I recently tried to measure coverage of a program that calls os.execvpe, essentially causing the process to be replaced by a different one. This did not record any coverage information at all.

The reason of course is that os.execvpe does not return, so there is no opportunity to call coverage.stop() and coverage.save() as is done if e.g. an exception is thrown. I'd suggest this method could be "monkey-patched" so that such code can be inserted before it.
(and also the other 7 os.exec* methods of course)

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/43
• This issue had attachments: execfunctions.bundle. See the original issue for details.

[nedbat]

Original comment by Geoff Bache (Bitbucket: geoffbache, GitHub: Unknown)

---

Sorry, didn't mean to report this anonymously.

[nedbat]

Original comment by Geoff Bache (Bitbucket: geoffbache, GitHub: Unknown)

---

I have a patch for this one also if you're interested.

[nedbat]

I'd be interested to see a patch. It sounds very involved!

[nedbat]

Original comment by Geoff Bache (Bitbucket: geoffbache, GitHub: Unknown)

---

Patch attached against current trunk. Could be simplified if it assumed that these functions called each other internally, which they do currently.

Tested with a small program as follows

#!python
import os

print "First program..."
os.execvp("./prog2.py", [ "./prog2.py", "-x" ])

which I can now get coverage information out of.

[nedbat]

Original comment by NiklasH (Bitbucket: nh2, GitHub: nh2)

---

What happened to this?

[nedbat]

I haven't applied the patch yet, because I'd only heard one request for it (this ticket), and I am averse to monkeypatching. But I now have a second request, and this is a fairly small patch.

Interestingly, if execvpe would execute the atexit-registered handlers before changing the process over, it would just work. I created http://bugs.python.org/issue16822 to request Python to be fixed.

[nedbat]

Considering this for the next release.

[nedbat]

For anyone looking for Geoff's changes:

diff -r [f7d26908601c (bb)](https://bitbucket.org/ned/coveragepy/commits/f7d26908601c) -r [f3a76cf7aa00 (bb)](https://bitbucket.org/ned/coveragepy/commits/f3a76cf7aa00) coverage/control.py
--- a/coverage/control.py       Sun Nov 07 19:45:54 2010 -0500
+++ b/coverage/control.py       Mon Nov 15 21:36:31 2010 +0100
@@ -360,6 +360,14 @@

         self._harvested = False
         self.collector.start()
+        os.execvpe = self.intercept(os.execvpe)
+
+    def intercept(self, method):
+        def new_method(*args, **kw):
+            self.stop()
+            self.save()
+            method(*args, **kw)
+        return new_method

     def stop(self):
         """Stop measuring code coverage."""

and then:

diff -r [f3a76cf7aa00 (bb)](https://bitbucket.org/ned/coveragepy/commits/f3a76cf7aa00) -r [ba05ad03668e (bb)](https://bitbucket.org/ned/coveragepy/commits/ba05ad03668e) coverage/control.py
--- a/coverage/control.py       Mon Nov 15 21:36:31 2010 +0100
+++ b/coverage/control.py       Mon Nov 15 22:37:22 2010 +0100
@@ -359,15 +359,13 @@
             self.omit_match = FnmatchMatcher(self.omit)

         self._harvested = False
+        for funcName in [ 'execl', 'execle', 'execlp', 'execlpe', 'execv', 'execve', 'execvp', 'execvpe' ]:
+            newFunc = self.intercept(getattr(os, funcName))
+            setattr(os, funcName, newFunc)
         self.collector.start()
-        os.execvpe = self.intercept(os.execvpe)

-    def intercept(self, method):
-        def new_method(*args, **kw):
-            self.stop()
-            self.save()
-            method(*args, **kw)
-        return new_method
+    def intercept(self, method):
+        return StopCoverageDecorator(self, method)

     def stop(self):
         """Stop measuring code coverage."""
@@ -612,6 +610,21 @@
         return info


+class StopCoverageDecorator:
+    inDecorator = False
+    def __init__(self, cov, method):
+        self.cov = cov
+        self.method = method
+
+    def __call__(self, *args, **kw):
+        if not StopCoverageDecorator.inDecorator:
+            StopCoverageDecorator.inDecorator = True
+            self.cov.stop()
+            self.cov.save()
+        self.method(*args, **kw)
+        StopCoverageDecorator.inDecorator = False
+
+
 def process_startup():
     """Call this at Python startup to perhaps measure coverage.

[nedbat]

Original comment by Geoff Bache (Bitbucket: geoffbache, GitHub: Unknown)

---

Looking at this again. There are other circumstances where no coverage information is produced because the atexit handlers are not called:

1. When processes are terminated by a signal
2. When control is not returned to the Python interpreter. I've run into this in several circumstances under Jython, where Java code shuts down the JVM without returning control to Jython or calling any exit handlers.

In the light of this, I'm wondering if a generic, if slightly clumsy, solution would be simply to provide a "process_shutdown" method (in a similar way to process_startup) so that this external handler could be called explicitly in any of these circumstances.

Then it's easy to add code like

#!python

try:
    import coverage
    coverage.process_shutdown()
except:
    pass

and call it in signal handlers, just before os.exec* or in a JVM exit handler to work around all these cases.

Would be better of course if it was detected automatically, but this will avoid the evils of monkey patching and is more generic than the code I posted here. Can provide a patch if you agree to the solution.

[nedbat]

Original comment by Geoff Bache (Bitbucket: geoffbache, GitHub: Unknown)

---

Implementing a solution like this now appears to be difficult, as the "coverage" object is not necessarily stored in a global variable any more.

I resorted to the following in my code, which does not require patching coverage at all, but only works under Python 2 and relies on an undocumented feature of Python's atexit module:

#!python

import atexit
for func, args, kw in atexit._exithandlers:
    if func.__module__.startswith("coverage."):
        func(*args, **kw)

Clearly not ideal but about the only thing I could think of to get my coverage measurement working with the latest version.

[spaceone]

CC.
Still affected in ```
Python 3.7.3 (default, Jul 25 2020, 13:03:44)

import coverage
coverage.version
'4.5.2'

[rgbyrnes]

Here's another hack that seems to work for Python 3, at least with coverage run: it walks the stack to find the Coverage object ...

#!python

if 'COVERAGE_RUN' in os.environ and sys.gettrace() is not None:
    def _find_cov():
        from traceback import walk_stack
        for (f, _) in walk_stack(None):
            for obj in f.f_locals.values():
                if (obj.__class__.__name__ == 'CoverageScript' and
                    obj.__module__ == 'coverage.cmdline'):
                    return obj.coverage
    cov = _find_cov()
    if cov is not None:
        cov.stop()
        cov.save()