Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test suite requires python-2.x #16

Closed
orlitzky opened this issue Mar 28, 2020 · 6 comments
Closed

Test suite requires python-2.x #16

orlitzky opened this issue Mar 28, 2020 · 6 comments

Comments

@orlitzky
Copy link
Contributor

Now that python-2.x is finally being EOLed , for real this time, we swear -- from all Linux distributions -- the rbldnsd test suite needs to be updated to work with python-3.x.

@ammammita
Copy link
Collaborator

Hello,

there is a branch called python3-tests
If you'd be so kind to have a look, it should address this ticket.

@orlitzky
Copy link
Contributor Author

I get a bunch of warnings with python-3.8 on Gentoo, but the test suite does ultimately pass:

Running tests.py
Can not deduce size of pointer. Assuming pointer size of 8.
/usr/lib/python3.8/unittest/case.py:633: ResourceWarning: unclosed file <_io.BufferedRandom name=5>
  method()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
.s.s/usr/lib/python3.8/unittest/case.py:633: ResourceWarning: unclosed file <_io.BufferedRandom name=7>
  method()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
......./usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=3 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=4 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=5 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=6 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=7 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=8 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=9 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=10 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=11 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=12 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=13 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=14 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=15 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=16 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=17 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=18 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/site-packages/DNS/Lib.py:155: ResourceWarning: unclosed file <_io.FileIO name=19 mode='rb+' closefd=True>
  for label in name.split('.'):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/unittest/case.py:633: ResourceWarning: unclosed file <_io.BufferedRandom name=21>
  method()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
............./usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=3 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=4 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=5 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=6 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=7 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=8 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=9 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=10 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=11 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=12 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=13 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=14 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=15 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=16 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=17 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=18 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=19 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=20 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=21 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=22 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=23 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=24 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=25 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.8/tempfile.py:147: ResourceWarning: unclosed file <_io.FileIO name=26 mode='rb+' closefd=True>
  letters = [choose(c) for dummy in range(8)]
ResourceWarning: Enable tracemalloc to get the object allocation traceback
......
----------------------------------------------------------------------
Ran 30 tests in 6.439s

OK (skipped=2)

@orlitzky
Copy link
Contributor Author

Oh, and I noticed you changed a warning from "The py3dns library is not installed..." to "The python3-dns library is not installed..." That misled me for a moment, because AFAIK the name "py3dns" is correct (i.e. that's what I needed to install). This is its project page: https://git.launchpad.net/py3dns

@ammammita
Copy link
Collaborator

i thought the same and i tested the patch with py3dns too.

Then, working on the travis pull request, i noticed it was importing only python-dns (not pydns) and it was working, so i gave a try to python3-dns and that seems a better idea other than py3dns that requires a separate installation.

Regarding the warnings: i noticed them too but i have not enough experience with python to fix them.
A patch would be appreciated.

@orlitzky
Copy link
Contributor Author

I think the warnings are just a matter of calling close() on the temporary files. This makes the errors go away, but is probably not correct since I didn't really pay attention to where/how those files are being used or cleaned up.

diff --git a/rbldnsd.py b/rbldnsd.py
index 79ef6d2..8134047 100644
--- a/rbldnsd.py
+++ b/rbldnsd.py
@@ -27,13 +27,16 @@ $NS 1d ns0.example.org
 
 class ZoneFile(object):
     def __init__(self, lines=None, no_header=False):
-        self._file = NamedTemporaryFile()
+        self._file = NamedTemporaryFile(delete=False)
         if not no_header:
             self._file.write(bytes(DUMMY_ZONE_HEADER, encoding = 'utf-8'))
         if lines is not None:
             self.writelines(lines)
         self._file.flush()
 
+    def __del__(self):
+        self._file.close()
+
     @property
     def name(self):
         return self._file.name
@@ -157,6 +160,8 @@ class Rbldnsd(object):
                     raise DaemonError("can not kill stop rbldnsd")
                 time.sleep(0.1)
 
+        self._stdout.close()
+
         self._daemon = None
         if daemon.returncode != 0:
             raise DaemonError("rbldnsd exited with code %d"
diff --git a/test_acl.py b/test_acl.py
index 84c9f55..f071c08 100644
--- a/test_acl.py
+++ b/test_acl.py
@@ -44,9 +44,10 @@ no_ipv6 = not _have_ipv6()
 def daemon(acl, addr='localhost'):
     """ Create an Rbldnsd instance with given ACL
     """
-    acl_zone = NamedTemporaryFile()
+    acl_zone = NamedTemporaryFile(delete=False)
     acl_zone.writelines(bytes("%s\n" % line, encoding='utf8') for line in acl)
     acl_zone.flush()
+    acl_zone.close()
 
     dnsd = Rbldnsd(daemon_addr=addr)
     dnsd.add_dataset('acl', acl_zone)
diff --git a/test_btrie.py b/test_btrie.py
index 51a61ed..6f6e450 100644
--- a/test_btrie.py
+++ b/test_btrie.py
@@ -32,7 +32,7 @@ def deduce_pointer_size(makefile='./Makefile'):
     cc = make_vars['CC']
     cflags = make_vars['CFLAGS']
 
-    test_c = NamedTemporaryFile(suffix=".c")
+    test_c = NamedTemporaryFile(suffix=".c", delete=False)
     test_c.write(r'''
 #include <stdio.h>
 #ifndef __SIZEOF_POINTER__
@@ -44,6 +44,7 @@ int main () {
 }
 ''')
     test_c.flush()
+    test_c.close()
     src = test_c.name
 
     try:
@@ -123,6 +124,9 @@ class CaptureOutput(object):
     def __init__(self):
         self._file = TemporaryFile()
 
+    def __del__(self):
+        self._file.close()
+
     def fileno(self):
         return self._file.fileno()

@ammammita
Copy link
Collaborator

Thank you Michael.
I applied your patch to the python3-tests branch and now it runs perfectly!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants