python printing is broken

[linas]

Printing to the python shell is broken. This got broken by pull request opencog/opencog#2311, the refactoring of the network code is not yet complete.

Tor reproduce: telnetto the cogserver, enter the python shell, and execute any python code -- nothing is printed.

[linas]

So, there is now a partial fix in pull req opencog/atomspace#837 -- but more work in the cogserver shell is needed.

[linas]

The original issue is described here: opencog/atomspace#629 and its probably an issue again...

[linas]

I'm pretty sure that issue opencog/opencog#1596 is an issue again. Basically, at this time, all python prints are going to the cogserver shell.

[linas]

This commit: opencog/atomspace@32e8bf0

trashed the previous python stdout-capture code. This was done because python agents or any python inf loop would hang the cogserver, because that code was waiting forever for the output to arrive, slowly growing over time, per comment opencog/opencog#2301 (comment)

The right fix might be to capture stdout in a different python thread, and forward that to the shell prompt. This would require restoring the above commit, and changing it to multi-task and send the print results back in dribbles.

Alternately, PERFORM_STDOUT_DUPLICATION could be re-enabled in the GenericShell.cc

[smigad]

how about redirecting stdout to a pipe and then reading to a variable like _result
I got i working here

/*  REDIRECTING STDOUT TO A PIPE
    because no string representation can be found for the PyObject returned by 
    PyRun_StringFlags after execution of expression, the only option I can see
    is a temporary hack to redirect the stdout to a variable for the brief
    moment PyRun_StringFlags is running and then reconnecting it back 
    when which we can read from that variable to _result so that the execution 
    result is displayed on the users shell and not on the stdout of cogserver
*/
    int _output_pipe[2];
    int _stdout_original = dup(STDOUT_FILENO);

    if(pipe(_output_pipe) == 0)
        #define _PIPE_OPENED_OKAY_
    #ifdef _PIPE_OPENED_OKAY_
        dup2(_output_pipe[1], STDOUT_FILENO); //redirecting stdout to pipe
        close(_output_pipe[1]);
    #endif

    //execute python script
    PyObject* pyResult = PyRun_StringFlags(command,
            Py_file_input, pyRootDictionary, pyRootDictionary,
            nullptr);

    #ifdef _PIPE_OPENED_OKAY_
        fflush(stdout);
        dup2(_stdout_original, STDOUT_FILENO); //return stdout to its rightful place
        char _result_buffer[1];
        int c;
        while(1)
        {
            c = read(_output_pipe[0], _result_buffer, 1);
            if(c == -1 or c== 0 or errno) //if eof or error
                break;
            _result.append<int>(1, _result_buffer[0]);
        }
        //_result.append("\n");

    #endif

    if (pyResult)
        Py_DECREF(pyResult);

[linas]

Hi @Dagiopia the old code did something similar, and it doesn't work, if the print is being issued from an infinite loop. So, for example, some long-running python code is started e.g. a so-called "mind agent" -- which might run for minutes or hours, and it has print statements inside of it. What you propose will either hang or stall, or, if not that, will have the pipe grow to unbounded size.

There is some older code that partly/mostly solves this problem, but it got kind-of trashed recently. It solved teh problem by capturing everything that goes to stdout, and stuffs it into a pipe, and then, in a different thread, it empties that pipe out. This can continue indefinitely.

The code is here:
https://github.com/opencog/opencog/blob/master/opencog/cogserver/shell/GenericShell.cc#L187-L223
https://github.com/opencog/opencog/blob/master/opencog/cogserver/shell/GenericShell.cc#L263-L306

I don't quite recall what it is that makes it buggy or unstable. It used to work -- it almost works now, but there are some kind of issues.

[smigad]

yes @linas I was afraid there may be something it will ruin.
I'll get started on looking into why the first code doesn't work on monday.

[linas]

I've stubbed out line 106 in tests/cython/PyEvalUTest until this issues gets fixed. When this is fixed, that line of code should be placed back.

[inflector]

@linas Are the comments here up to date? I.e. is this still an issue and no one is working on it so therefore an issue that I can grab and handle without stepping on someone else's work?