The shell's .output command gives an error on OS X + Python 2.7

[rogerbinns]

From pasma10@concepts.nl on April 15, 2011 15:29:02

What steps will reproduce the problem? 1.python -c "from apsw import main; main()"
2..output test.out 3. What is the expected output? What do you see instead? Expected is that the output of queries is redirected to the file.
But the command raises IOError, {Errno 9] Bad File Descriptor What version of the product are you using? On what operating system? APSW 3.7.5- r1 Python 2.7
OS/X PPC Please provide any additional information below. The error results from sys.stdin.flush(). I added this in the Python Issue tracker but it is not considered a bug there. (see http://bugs.python.org/issue11851 for more details)

Original issue: http://code.google.com/p/apsw/issues/detail?id=117

[rogerbinns]

From rogerbinns on April 15, 2011 21:38:46

I've been unable to reproduce this on Windows or Linux. My Mac is PPC running 10.3 and I'm trying there but Python 2.7 doesn't compile so I'm having to try older versions. (Incidentally the test suite also tests the shell and requires that .output works correctly in order to do so.)

In order to make progress you'll need to provide more debugging. Before changing output use '.exceptions on' which will show exceptions in their gory detail including variables and should help narrow down why this is happening.

[rogerbinns]

From rogerbinns on April 15, 2011 21:48:46

Ok, I can't test on my Mac because even when I get python to compile SQLite fails to compile as it requires a more recent version of the OS. I could keep trying to work around issues, but a trace from your unspecified environment would be better.

[rogerbinns]

From rogerbinns on April 15, 2011 22:20:41

I forgot to mention you should the shell's source. You can find it in the download as tools/shell.py

When apsw is compiled that file is simplified (whitespace removed) and incorporated into the resulting DLL which means line numbers aren't useful.

[rogerbinns]

From pasma10@concepts.nl on April 16, 2011 03:41:48

Hello Roger,

First here are the versions. And very first please excuse me because the OS version is not 2.3 but 2.4. I hope this has not wasted your time. I'll try to explain how this could happen.

PPC G4
Mac OS X Version 10.4.11
Xcode 2.1
python-2.7.1-macosx10.3.dmg
APSw 3.7.5- r1 What happened is that I was misled by the name of the Python binary, called *2.3.dmg. Moreover this version number also appears during the APSW build process. But is just an indication of the minimum version. The binary applies for the whole range 10.3 through 10.6. So your hint that my environment was still unspecified has been useful.

I produced the trace of the .output command after setting .exceptions ON. To me it is strange that stdin.flush does not appear. Because that is definitly where the error occurs. That can also be reproduced in the Python interpretor (see http://bugs.python.org/issue11851 )

The trace output is at the bottom.

I hope very much that this is reason for a fix. Tnanks anyway,

Edzard Pasma

$ python -c "from shell import Shell; Shell().cmdloop()" test.db
SQLite version 3.7.5 (APSW 3.7.5- r1 )
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .exceptions on
sqlite> .output test.out
[Errno 9] Bad file descriptor

Frame cmdloop in shell.py at line 758
c =
command = u'.output test.out'
intro = u'SQLite version 3.7.5 (APSW 3.7.5- r1 )\nEnter ".help" for instructions\nEnter SQ
old_completer = None
readline = <module 'readline' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/p
self = <shell.Shell object at 0xcf670>
using_readline = True

Frame process_complete_line in shell.py at line 2186
command = u'.output test.out'
self = <shell.Shell object at 0xcf670>

Frame process_command in shell.py at line 908
c = 'test.out'
cmd = [u'output', u'test.out']
fn = <bound method Shell.command_output of <shell.Shell object at 0xcf670>>
self = <shell.Shell object at 0xcf670>

Frame command_output in shell.py at line 1676
cmd = [u'test.out']
self = <shell.Shell object at 0xcf670>

<type 'exceptions.IOError'>: IOError(9, 'Bad file descriptor')
sqlite>

[rogerbinns]

From rogerbinns on April 16, 2011 10:00:44

You keep confusing OS versions and Python versions :-) In any event the only Mac I have runs 10.3 which is why I mentioned that.

Thanks for tracking down the problem. As I mentioned in the Python bug tracker the reason why flush is being called on standard input (nominally a ready only descriptor) is because it is a terminal and readline is being used on it which really does involve writing to your input.

My fix will be adding a try/except.

    if hasattr(self.stdin, "flush"):
        try:
            self.stdin.flush()
        except IOError:
            pass

Can you please verify everything is okay after doing this?

[rogerbinns]

From pasma10@concepts.nl on April 16, 2011 11:27:32

I'm happy this will be solved. The .output command is very useful also if the shell is used embedded in Python. This may produce SQL reports with minimal programming effort.
finally: saturdaynight

[rogerbinns]

From rogerbinns on April 16, 2011 21:11:29

Fixed in rf72362b0f6ca

Status: Fixed