Unraiseable exception when running Tests with Python 3.4.1

[ebrown]

Complied with VS2010 on Windows 7 64
.Unraiseable exception <class 'TypeError'>:xUnlock1() missing 3 required positional arguments: 'number', 'of', and 'arguments'
Unraiseable exception <class 'ZeroDivisionError'>:division by zero

Removing C:\Python34\Lib\site-packages\apsw-3.8.5_r1-py3.4.egg-info
Writing C:\Python34\Lib\site-packages\apsw-3.8.5_r1-py3.4.egg-info
running test
Python C:\Python34\python.exe sys.version_info(major=3, minor=4, micro=1, releaselevel='final', serial=0)
Testing with APSW file C:\Python34\lib\site-packages\apsw.pyd
APSW version 3.8.5-r1
SQLite lib version 3.8.5
SQLite headers version 3008005
Using amalgamation True
Not doing LoadExtension test. You need to compile the extension first
gcc -fPIC -shared -o testextension.sqlext -I. -Isqlite3 src/testextension.c
......................................................................Unraiseable exception <class 'TypeError'>:xUnlock1() missing 3 required positional arguments: 'number', 'of', and 'arguments'
File "setup.py", line 863, in
'win64hackvars': win64hackvars}
File "C:\Python34\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\Python34\lib\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
File "C:\Python34\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "setup.py", line 111, in run
result=unittest.TextTestRunner(verbosity=self.show_tests+1).run(suite)
File "C:\Python34\lib\unittest\runner.py", line 168, in run
test(result)
File "C:\Python34\lib\unittest\suite.py", line 87, in call
return self.run(_args, *_kwds)
File "C:\Python34\lib\unittest\suite.py", line 125, in run
test(result)
File "C:\Python34\lib\unittest\suite.py", line 87, in call
return self.run(_args, *_kwds)
File "C:\Python34\lib\unittest\suite.py", line 125, in run
test(result)
File "C:\Python34\lib\unittest\case.py", line 625, in call
return self.run(_args, *_kwds)
File "C:\Python34\lib\unittest\case.py", line 577, in run
testMethod()
File "C:\users\eb15792\Desktop\apsw-3.8.5-r1\tests.py", line 5014, in testVFS
self.assertRaises(apsw.SQLError, self.assertRaisesUnraisable, TypeError, testdb)
File "C:\Python34\lib\unittest\case.py", line 704, in assertRaises
return context.handle('assertRaises', callableObj, args, kwargs)
File "C:\Python34\lib\unittest\case.py", line 162, in handle
callable_obj(_args, *_kwargs)
File "C:\Python34\lib\unittest\case.py", line 183, in exit
traceback.clear_frames(tb)
File "C:\Python34\lib\traceback.py", line 309, in clear_frames
tb.tb_frame.clear()
File "c:\users\eb15792\desktop\apsw-3.8.5-r1\src\vfs.c", line 2174, in apswvfsfile.xUnlock
AddTraceBackHere(FILE, LINE, "apswvfsfile.xUnlock", "{s: i}", "flag", flag);
File "c:\users\eb15792\desktop\apsw-3.8.5-r1\src\pyutil.c", line 158, in Call_PythonMethod
AddTraceBackHere(FILE, LINE, "Call_PythonMethod", "{s: s, s: i, s: O, s: O}",
Unraiseable exception <class 'ZeroDivisionError'>:division by zero
File "setup.py", line 863, in
'win64hackvars': win64hackvars}
File "C:\Python34\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\Python34\lib\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
File "C:\Python34\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "setup.py", line 111, in run
result=unittest.TextTestRunner(verbosity=self.show_tests+1).run(suite)
File "C:\Python34\lib\unittest\runner.py", line 168, in run
test(result)
File "C:\Python34\lib\unittest\suite.py", line 87, in call
return self.run(_args, *_kwds)
File "C:\Python34\lib\unittest\suite.py", line 125, in run
test(result)
File "C:\Python34\lib\unittest\suite.py", line 87, in call
return self.run(_args, *_kwds)
File "C:\Python34\lib\unittest\suite.py", line 125, in run
test(result)
File "C:\Python34\lib\unittest\case.py", line 625, in call
return self.run(_args, *_kwds)
File "C:\Python34\lib\unittest\case.py", line 577, in run
testMethod()
File "C:\users\eb15792\Desktop\apsw-3.8.5-r1\tests.py", line 5016, in testVFS
self.assertRaises(apsw.SQLError, self.assertRaisesUnraisable, ZeroDivisionError, testdb)
File "C:\Python34\lib\unittest\case.py", line 704, in assertRaises
return context.handle('assertRaises', callableObj, args, kwargs)
File "C:\Python34\lib\unittest\case.py", line 162, in handle
callable_obj(_args, *_kwargs)
File "C:\Python34\lib\unittest\case.py", line 183, in exit
traceback.clear_frames(tb)
File "C:\Python34\lib\traceback.py", line 309, in clear_frames
tb.tb_frame.clear()
File "c:\users\eb15792\desktop\apsw-3.8.5-r1\src\vfs.c", line 2174, in apswvfsfile.xUnlock
AddTraceBackHere(FILE, LINE, "apswvfsfile.xUnlock", "{s: i}", "flag", flag);
File "c:\users\eb15792\desktop\apsw-3.8.5-r1\src\pyutil.c", line 158, in Call_PythonMethod
AddTraceBackHere(FILE, LINE, "Call_PythonMethod", "{s: s, s: i, s: O, s: O}",
File "C:\users\eb15792\Desktop\apsw-3.8.5-r1\tests.py", line 4502, in xUnlock2
1/0
.Unraiseable exception <class 'TypeError'>:xUnlock1() missing 3 required positional arguments: 'number', 'of', and 'arguments'
File "setup.py", line 863, in
'win64hackvars': win64hackvars}
File "C:\Python34\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\Python34\lib\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
File "C:\Python34\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "setup.py", line 111, in run
result=unittest.TextTestRunner(verbosity=self.show_tests+1).run(suite)
File "C:\Python34\lib\unittest\runner.py", line 168, in run
test(result)
File "C:\Python34\lib\unittest\suite.py", line 87, in call
return self.run(_args, *_kwds)
File "C:\Python34\lib\unittest\suite.py", line 125, in run
test(result)
File "C:\Python34\lib\unittest\suite.py", line 87, in call
return self.run(_args, *_kwds)
File "C:\Python34\lib\unittest\suite.py", line 125, in run
test(result)
File "C:\Python34\lib\unittest\case.py", line 625, in call
return self.run(_args, *_kwds)
File "C:\Python34\lib\unittest\case.py", line 577, in run
testMethod()
File "C:\users\eb15792\Desktop\apsw-3.8.5-r1\tests.py", line 3960, in testVFSWithWAL
self.testVFS()
File "C:\users\eb15792\Desktop\apsw-3.8.5-r1\tests.py", line 5014, in testVFS
self.assertRaises(apsw.SQLError, self.assertRaisesUnraisable, TypeError, testdb)
File "C:\Python34\lib\unittest\case.py", line 704, in assertRaises
return context.handle('assertRaises', callableObj, args, kwargs)
File "C:\Python34\lib\unittest\case.py", line 162, in handle
callable_obj(_args, *_kwargs)
File "C:\Python34\lib\unittest\case.py", line 183, in exit
traceback.clear_frames(tb)
File "C:\Python34\lib\traceback.py", line 309, in clear_frames
tb.tb_frame.clear()
File "c:\users\eb15792\desktop\apsw-3.8.5-r1\src\vfs.c", line 2174, in apswvfsfile.xUnlock
AddTraceBackHere(FILE, LINE, "apswvfsfile.xUnlock", "{s: i}", "flag", flag);
File "c:\users\eb15792\desktop\apsw-3.8.5-r1\src\pyutil.c", line 158, in Call_PythonMethod
AddTraceBackHere(FILE, LINE, "Call_PythonMethod", "{s: s, s: i, s: O, s: O}",
Unraiseable exception <class 'ZeroDivisionError'>:division by zero
File "setup.py", line 863, in
'win64hackvars': win64hackvars}
File "C:\Python34\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\Python34\lib\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
File "C:\Python34\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "setup.py", line 111, in run
result=unittest.TextTestRunner(verbosity=self.show_tests+1).run(suite)
File "C:\Python34\lib\unittest\runner.py", line 168, in run
test(result)
File "C:\Python34\lib\unittest\suite.py", line 87, in call
return self.run(_args, *_kwds)
File "C:\Python34\lib\unittest\suite.py", line 125, in run
test(result)
File "C:\Python34\lib\unittest\suite.py", line 87, in call
return self.run(_args, *_kwds)
File "C:\Python34\lib\unittest\suite.py", line 125, in run
test(result)
File "C:\Python34\lib\unittest\case.py", line 625, in call
return self.run(_args, *_kwds)
File "C:\Python34\lib\unittest\case.py", line 577, in run
testMethod()
File "C:\users\eb15792\Desktop\apsw-3.8.5-r1\tests.py", line 3960, in testVFSWithWAL
self.testVFS()
File "C:\users\eb15792\Desktop\apsw-3.8.5-r1\tests.py", line 5016, in testVFS
self.assertRaises(apsw.SQLError, self.assertRaisesUnraisable, ZeroDivisionError, testdb)
File "C:\Python34\lib\unittest\case.py", line 704, in assertRaises
return context.handle('assertRaises', callableObj, args, kwargs)
File "C:\Python34\lib\unittest\case.py", line 162, in handle
callable_obj(_args, *_kwargs)
File "C:\Python34\lib\unittest\case.py", line 183, in exit
traceback.clear_frames(tb)
File "C:\Python34\lib\traceback.py", line 309, in clear_frames
tb.tb_frame.clear()
File "c:\users\eb15792\desktop\apsw-3.8.5-r1\src\vfs.c", line 2174, in apswvfsfile.xUnlock
AddTraceBackHere(FILE, LINE, "apswvfsfile.xUnlock", "{s: i}", "flag", flag);
File "c:\users\eb15792\desktop\apsw-3.8.5-r1\src\pyutil.c", line 158, in Call_PythonMethod
AddTraceBackHere(FILE, LINE, "Call_PythonMethod", "{s: s, s: i, s: O, s: O}",
File "C:\users\eb15792\Desktop\apsw-3.8.5-r1\tests.py", line 4502, in xUnlock2
1/0

.........

Ran 80 tests in 85.094s

OK

[rogerbinns]

Thanks for reporting this - I have reproduced it. I was testing on Linux with 3.4.1 where the issue does not occur, and build/test on Windows with 3.4.0 where the issue didn't occur either. It is a Windows specific change to 3.4.1 causing the problem.

The test is verifying correct behaviour when deep within a custom VFS and it has exceptions which can't be bubbled all the way up the call chain (SQLite C code is in the way doing its own error handling). Unless you are writing your own VFS there is nothing to worry about.

[Arfrever]

I have reproduced these exceptions on Gentoo GNU/Linux with Python 3.4 (3.4.2_pre20140608).

[rogerbinns]

The underlying cause is due to garbage collection and the way the test is written. Note that the tests are not actually failing and things are behaving correctly. Python 3.4 added traceback.clear_frames which is called on frames inside unittest.assertRaises.

There is a single function - testdb - that causes every VFS operation to be executed. Some unraisable exceptions intentionally happen during its execution under the hood and are caught by assertRaisesUnraisable. A SQLite level exception is raised based on the SQLite API return code. unittest.assertRaises then runs traceback.clear_frames on that which causes some garbage collection which in turn causes some vfs code to run which then tickles the code we were testing for unraisable in the first place.

[rogerbinns]

Fixed in a6b2192