ValueError: I/O operation on closed file

[den-run-ai]

c:\python\python27\lib\site-packages\ipykernel\__main__.py:5: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
ERROR:tornado.general:Uncaught exception, closing connection.
Traceback (most recent call last):
  File "c:\python\python27\lib\site-packages\zmq\eventloop\zmqstream.py", line 414, in _run_callback
    callback(*args, **kwargs)
  File "c:\python\python27\lib\site-packages\tornado\stack_context.py", line 275, in null_wrapper
    return fn(*args, **kwargs)
  File "c:\python\python27\lib\site-packages\ipykernel\kernelbase.py", line 260, in dispatcher
    return self.dispatch_shell(stream, msg)
  File "c:\python\python27\lib\site-packages\ipykernel\kernelbase.py", line 218, in dispatch_shell
    sys.stdout.flush()
  File "c:\python\python27\lib\site-packages\ipykernel\iostream.py", line 167, in flush
    data = self._flush_buffer()
  File "c:\python\python27\lib\site-packages\ipykernel\iostream.py", line 242, in _flush_buffer
    data = self._buffer.getvalue()
ValueError: I/O operation on closed file
ERROR:tornado.general:Uncaught exception, closing connection.
Traceback (most recent call last):
  File "c:\python\python27\lib\site-packages\zmq\eventloop\zmqstream.py", line 440, in _handle_events
    self._handle_recv()
  File "c:\python\python27\lib\site-packages\zmq\eventloop\zmqstream.py", line 472, in _handle_recv
    self._run_callback(callback, msg)
  File "c:\python\python27\lib\site-packages\zmq\eventloop\zmqstream.py", line 414, in _run_callback
    callback(*args, **kwargs)
  File "c:\python\python27\lib\site-packages\tornado\stack_context.py", line 275, in null_wrapper
    return fn(*args, **kwargs)
  File "c:\python\python27\lib\site-packages\ipykernel\kernelbase.py", line 260, in dispatcher
    return self.dispatch_shell(stream, msg)
  File "c:\python\python27\lib\site-packages\ipykernel\kernelbase.py", line 218, in dispatch_shell
    sys.stdout.flush()
  File "c:\python\python27\lib\site-packages\ipykernel\iostream.py", line 167, in flush
    data = self._flush_buffer()
  File "c:\python\python27\lib\site-packages\ipykernel\iostream.py", line 242, in _flush_buffer
    data = self._buffer.getvalue()
ValueError: I/O operation on closed file
ERROR:tornado.application:Exception in callback None
Traceback (most recent call last):
  File "c:\python\python27\lib\site-packages\tornado\ioloop.py", line 883, in start
    handler_func(fd_obj, events)
  File "c:\python\python27\lib\site-packages\tornado\stack_context.py", line 275, in null_wrapper
    return fn(*args, **kwargs)
  File "c:\python\python27\lib\site-packages\zmq\eventloop\zmqstream.py", line 440, in _handle_events
    self._handle_recv()
  File "c:\python\python27\lib\site-packages\zmq\eventloop\zmqstream.py", line 472, in _handle_recv
    self._run_callback(callback, msg)
  File "c:\python\python27\lib\site-packages\zmq\eventloop\zmqstream.py", line 414, in _run_callback
    callback(*args, **kwargs)
  File "c:\python\python27\lib\site-packages\tornado\stack_context.py", line 275, in null_wrapper
    return fn(*args, **kwargs)
  File "c:\python\python27\lib\site-packages\ipykernel\kernelbase.py", line 260, in dispatcher
    return self.dispatch_shell(stream, msg)
  File "c:\python\python27\lib\site-packages\ipykernel\kernelbase.py", line 218, in dispatch_shell
    sys.stdout.flush()
  File "c:\python\python27\lib\site-packages\ipykernel\iostream.py", line 167, in flush
    data = self._flush_buffer()
  File "c:\python\python27\lib\site-packages\ipykernel\iostream.py", line 242, in _flush_buffer
    data = self._buffer.getvalue()
ValueError: I/O operation on closed file

[takluyver]

What steps caused that?

[den-run-ai]

Long running script with large cell output, cannot share the whole notebook
currently.

On Mon, Jan 25, 2016, 12:42 PM Thomas Kluyver notifications@github.com
wrote:

What steps caused that?

—
Reply to this email directly or view it on GitHub
#9168 (comment).

[takluyver]

So it was something you were running from the notebook? Is it possible that something in it could have called sys.stdout.close()?

[den-run-ai]

Notebook, very unlikely that anything called sys.stdout.close()

BTW, this is not reproducible if I run simulation again.

On Mon, Jan 25, 2016, 2:18 PM Thomas Kluyver notifications@github.com
wrote:

So it was something you were running from the notebook? Is it possible
that something in it could have called sys.stdout.close()?

—
Reply to this email directly or view it on GitHub
#9168 (comment).

[takluyver]

You mean it only comes up once per kernel, or you can't reproduce it at all?

[den-run-ai]

At all

On Tue, Jan 26, 2016, 5:00 AM Thomas Kluyver notifications@github.com
wrote:

You mean it only comes up once per kernel, or you can't reproduce it at
all?

—
Reply to this email directly or view it on GitHub
#9168 (comment).

[takluyver]

OK, closing as we haven't really got much chance of working out what went wrong. Feel free to add info, we can reopen if we identify something.

[stromnov]

I've just encountered similar issue with Jupyter Notebook and massive Keras output (about 700k cell updates in several seconds).

Additional information: remote Jupyter Server (Ubuntu 14.04, Python 2.7.6, latest Jupyter from PyPI [jupyter-client==4.2.2, jupyter-console==4.1.1, jupyter-core==4.1.0, ipython==4.1.2])

Local: OSX, Google Chrome 48.0.

PS. Similar issue: https://groups.google.com/forum/#!msg/keras-users/_2s1PaoSL58/6539n-oOBgAJ

[stromnov]

Another (probably similar) issue and workaround: http://stackoverflow.com/questions/33958905/watch-long-term-process-in-ipython-notebook

[takluyver]

The case in the SO question is simple enough: the user has replaced sys.stdout, the file handle they're using gets closed (by leaving the with block), and they omit to reset sys.stdout to something that works. If you want to redirect or capture stdout, it's simple enough to write a context manager that will switch the object back when it finishes.

We could reset sys.stdout to our own stdout object after each cell is run, but that would break it for anyone who is trying to redirect/capture stdout over more than one cell.

[sakvaua]

Same issue here: Ipython/Jupyter versions exactly the same as stromnov's, but it's Windows 64, Python 2.7.10
ipython 4.1.2
jupyter-console 4.1.1
jupyter-core 4.1.0
jupyter-client 4.2.2

The issue appears only in Ipython/Jupyter notebook. Exporting noteboot to pure python and running it removes the problem.

ValueError                                Traceback (most recent call last)
<ipython-input-9-9b88ef25c7a1> in <module>()
     73         hst=model.fit(X_train, y_train, nb_epoch=1000, batch_size=512,validation_data=(X_test, y_test), 
     74                       show_accuracy=True, verbose=1,shuffle=True,
---> 75                       callbacks=[early_stopping,history]).history['val_acc']
     76 
     77         score=log_loss(y_test, model.predict(X_test))

c:\winpython64\python-2.7.10.amd64\lib\site-packages\keras\models.pyc in fit(self, X, y, batch_size, nb_epoch, verbose, callbacks, validation_split, validation_data, shuffle, show_accuracy, class_weight, sample_weight)
    699                          verbose=verbose, callbacks=callbacks,
    700                          val_f=val_f, val_ins=val_ins,
--> 701                          shuffle=shuffle, metrics=metrics)
    702 
    703     def predict(self, X, batch_size=128, verbose=0):

c:\winpython64\python-2.7.10.amd64\lib\site-packages\keras\models.pyc in _fit(self, f, ins, out_labels, batch_size, nb_epoch, verbose, callbacks, val_f, val_ins, shuffle, metrics)
    321                     batch_logs[l] = o
    322 
--> 323                 callbacks.on_batch_end(batch_index, batch_logs)
    324 
    325                 epoch_logs = {}

c:\winpython64\python-2.7.10.amd64\lib\site-packages\keras\callbacks.pyc in on_batch_end(self, batch, logs)
     58         t_before_callbacks = time.time()
     59         for callback in self.callbacks:
---> 60             callback.on_batch_end(batch, logs)
     61         self._delta_ts_batch_end.append(time.time() - t_before_callbacks)
     62         delta_t_median = np.median(self._delta_ts_batch_end)

c:\winpython64\python-2.7.10.amd64\lib\site-packages\keras\callbacks.pyc in on_batch_end(self, batch, logs)
    187         # will be handled by on_epoch_end
    188         if self.verbose and self.seen < self.params['nb_sample']:
--> 189             self.progbar.update(self.seen, self.log_values)
    190 
    191     def on_epoch_end(self, epoch, logs={}):

c:\winpython64\python-2.7.10.amd64\lib\site-packages\keras\utils\generic_utils.pyc in update(self, current, values)
    102                 info += ((prev_total_width - self.total_width) * " ")
    103 
--> 104             sys.stdout.write(info)
    105             sys.stdout.flush()
    106 

c:\winpython64\python-2.7.10.amd64\lib\site-packages\ipykernel\iostream.pyc in write(self, string)
    315 
    316             is_child = (not self._is_master_process())
--> 317             self._buffer.write(string)
    318             if is_child:
    319                 # newlines imply flush in subprocesses

ValueError: I/O operation on closed file

[esvhd]

Same issue here. RunningAnaconda 4.0 withPython 2.7 on Windows.

Very strange as looking at the trace logs it happened after a print() statement.... Script was running fine on Anaconda 2.5 before. Only encountered problem after upgrading to 4.0.

ADDED: chasing back the stack, at line 40 of ansitowin32.pyc, second block below, the text variable passed to write() appeared to be just a newline character of length 1..

Z:\python_src\matrix\src\gpp.pyc in solveCash(csv_file, cash_limit, src_path, output_dir)
    261                       .format(key, acct_oad[key], final[key]['CASH']))
    262     else:
--> 263         print('Problem happens here')
    264 
    265 #     return neg_cash

C:\Developer\Anaconda2\lib\site-packages\colorama\ansitowin32.pyc in write(self, text)
     38 
     39     def write(self, text):
---> 40         self.__convertor.write(text)
     41 
     42 

C:\Developer\Anaconda2\lib\site-packages\colorama\ansitowin32.pyc in write(self, text)
    139     def write(self, text):
    140         if self.strip or self.convert:
--> 141             self.write_and_convert(text)
    142         else:
    143             self.wrapped.write(text)

C:\Developer\Anaconda2\lib\site-packages\colorama\ansitowin32.pyc in write_and_convert(self, text)
    167             self.convert_ansi(*match.groups())
    168             cursor = end
--> 169         self.write_plain_text(text, cursor, len(text))
    170 
    171 

C:\Developer\Anaconda2\lib\site-packages\colorama\ansitowin32.pyc in write_plain_text(self, text, start, end)
    172     def write_plain_text(self, text, start, end):
    173         if start < end:
--> 174             self.wrapped.write(text[start:end])
    175             self.wrapped.flush()
    176 

C:\Developer\Anaconda2\lib\site-packages\ipykernel\iostream.pyc in write(self, string)
    315 
    316             is_child = (not self._is_master_process())
--> 317             self._buffer.write(string)
    318             if is_child:
    319                 # newlines imply flush in subprocesses

ValueError: I/O operation on closed file

[minrk]

@esvhd can you provide an example script? It's strange that it's getting I/O on closed file on a StringIO object.

[esvhd]

@minrk It appears to be happening at random, but always at some print(...) statement. My function doesn't really do a lot of I/O, mostly reading csv files with pandas.read_csv() and writing some csv files with pandas.DataFrame.to_csv(), no direct file I/O access.

The only obvious and direct I/O in my own code would be a few print() statements, some with str.format(...), some with just a simple string.

Let me see if I can reliability replicate this in a standalone script.

Just to add, this code runs in a single process, no multi-threading or multi-processing in the code.

[minrk]

It's writing to sys.stdout (which print does) that's failing, so that makes sense. I'm just trying to figure out how it's managing to fail. Does it print a whole bunch? Does it start any subprocesses, perhaps with joblib or multiprocessing?

[esvhd]

@minrk So it prints out around 300+ lines of text when it works. It sometimes does work btw.
No subprocs/joblib/ or anything multi-proc/threading. Serial processing of around only 30 csv files.

ADDED: chasing back the stack, at line 40 of ansitowin32.pyc, second block in the trace logs in my first post above, the text variable passed to write() appeared to be just a newline character of length 1..

Also if I run the same script in a console, it works just fine, no errors, obviously.

[thomasj02]

I'm not sure why this issue is marked closed since lots of people are still running into it. I hit it frequently when running Keras, as do other people based on comments above

[goretkin]

In case it helps anyone, I just ran into this issue while using https://github.com/tqdm/tqdm in a version of the ipykernel that doesn't include ipython/ipykernel#123

[lrq3000]

It seems the issue isn't fixed in the latest stable release of IPython notebook for Python 2.7 as reported here: keras-team/keras#2110

Maybe ipython/ipykernel#123 isn't yet available in the stable release? Otherwise, it seems this didn't fix the issue...

[takluyver]

You're right that ipython/ipykernel#123 is not in a release yet: it has milestone 4.4, while the released ipykernel is 4.3.1.

[minrk]

Working on a 4.4 release shortly.

[lrq3000]

Ah great to know, so the issue may be fixed in 4.4, that's good news :)

2016-08-01 17:14 GMT+02:00 Min RK notifications@github.com:

Working on a 4.4 release shortly.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#9168 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABES3iiXFLRut6W4RZBezOcpoAgBCi7kks5qbg1RgaJpZM4HL27p
.

[hobson]

I ran into this when using tqdm in a django manage.py shell_plus console (ipython)