Skip to content
This repository

Show invalid config message on TraitErrors during init #921

Merged
merged 4 commits into from over 2 years ago

2 participants

Min RK Fernando Perez
Min RK
Owner

implemented via @catch_config decorator

Now, the event that was triggered by invalid app config (see --log-level 5) is triggered by bad config at any point during initialization.

This will catch TraitError-raising bugs in IPython itself, but only during initialization.

A less aggressive approach would be to look for all possible TraitError-raising points, and protect only those.

Also includes a tiny commit which unregisters crash_handler in __call__, because it should never be called twice. The fact that it calls exit() demonstrates that if it ever does get called twice, we are in a great big mess, and we should at least avoid printing pages and pages of extra crash messages.

added some commits October 22, 2011
Min RK Show invalid config message on TraitErrors during initialization
implemented via @catch_config decorator

Now, the event that was triggered by invalid app config (see `--log-level 5`) is triggered by bad config at any point during initialization.

This *will* catch TraitError bugs in IPython itself, but only during initialization.

closes gh-908
7ed219c
Min RK unregister crash handler on call
prevents recursive invocation of the crash handler
074918c
Min RK fix base64 code in nbformat.v2
base64 encoding functions were called, but had no effect, because
the notebook already has everything as b64-encoded bytestrings, which
are valid ascii literals on Python 2.

However, the encode/decode logic is actually triggered on Python 3, revealing its errors.

This fixes the base64 functions that had no effect to have their intended effect, but does not use them.  Rather, it is assumed that
bytes objects are already b64-encoded (and thus ascii-safe), which 
assumption was already made in Python 2.
6607706
IPython/config/application.py
@@ -69,6 +71,26 @@ subcommand 'cmd', do: `{app} cmd -h`.
69 71
 # Application class
70 72
 #-----------------------------------------------------------------------------
71 73
 
  74
+@decorator
  75
+def catch_config(method, app, *args, **kwargs):
1
Fernando Perez Owner
fperez added a note October 28, 2011

This looks excellent, my only suggestion would be a name change: catch_config_error instead. It seems to me it will make the use of the decorator throughout the code much more self-documenting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Fernando Perez
Owner

Great work, thanks @minrk. All tests pass here, so once we agree on the final name choice, this one can go in.

Min RK
Owner

That makes sense.

Min RK
Owner

renamed to catch_config_error

Fernando Perez
Owner

Perfect! All looks good, tests pass, thanks for the name change. Merging now.

Fernando Perez fperez merged commit 6fb30ae into from October 28, 2011
Fernando Perez fperez closed this October 28, 2011
Min RK minrk referenced this pull request October 28, 2011
Merged

%config magic #923

Fernando Perez fperez referenced this pull request from a commit January 10, 2012
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 4 unique commits by 1 author.

Oct 22, 2011
Min RK Show invalid config message on TraitErrors during initialization
implemented via @catch_config decorator

Now, the event that was triggered by invalid app config (see `--log-level 5`) is triggered by bad config at any point during initialization.

This *will* catch TraitError bugs in IPython itself, but only during initialization.

closes gh-908
7ed219c
Min RK unregister crash handler on call
prevents recursive invocation of the crash handler
074918c
Min RK fix base64 code in nbformat.v2
base64 encoding functions were called, but had no effect, because
the notebook already has everything as b64-encoded bytestrings, which
are valid ascii literals on Python 2.

However, the encode/decode logic is actually triggered on Python 3, revealing its errors.

This fixes the base64 functions that had no effect to have their intended effect, but does not use them.  Rather, it is assumed that
bytes objects are already b64-encoded (and thus ascii-safe), which 
assumption was already made in Python 2.
6607706
Oct 28, 2011
Min RK catch_config -> catch_config_error d05b990
This page is out of date. Refresh to see the latest.
37  IPython/config/application.py
@@ -26,6 +26,8 @@
26 26
 from copy import deepcopy
27 27
 from collections import defaultdict
28 28
 
  29
+from IPython.external.decorator import decorator
  30
+
29 31
 from IPython.config.configurable import SingletonConfigurable
30 32
 from IPython.config.loader import (
31 33
     KVArgParseConfigLoader, PyFileConfigLoader, Config, ArgumentError, ConfigFileNotFound,
@@ -69,6 +71,26 @@
69 71
 # Application class
70 72
 #-----------------------------------------------------------------------------
71 73
 
  74
+@decorator
  75
+def catch_config_error(method, app, *args, **kwargs):
  76
+    """Method decorator for catching invalid config (Trait/ArgumentErrors) during init.
  77
+
  78
+    On a TraitError (generally caused by bad config), this will print the trait's
  79
+    message, and exit the app.
  80
+    
  81
+    For use on init methods, to prevent invoking excepthook on invalid input.
  82
+    """
  83
+    try:
  84
+        return method(app, *args, **kwargs)
  85
+    except (TraitError, ArgumentError) as e:
  86
+        app.print_description()
  87
+        app.print_help()
  88
+        app.print_examples()
  89
+        app.log.fatal("Bad config encountered during initialization:")
  90
+        app.log.fatal(str(e))
  91
+        app.log.debug("Config at the time: %s", app.config)
  92
+        app.exit(1)
  93
+
72 94
 
73 95
 class ApplicationError(Exception):
74 96
     pass
@@ -173,6 +195,7 @@ def init_logging(self):
173 195
         self._log_handler.setFormatter(self._log_formatter)
174 196
         self.log.addHandler(self._log_handler)
175 197
 
  198
+    @catch_config_error
176 199
     def initialize(self, argv=None):
177 200
         """Do the basic steps to configure me.
178 201
 
@@ -317,6 +340,7 @@ def update_config(self, config):
317 340
         # events.
318 341
         self.config = newconfig
319 342
 
  343
+    @catch_config_error
320 344
     def initialize_subcommand(self, subc, argv=None):
321 345
         """Initialize a subcommand with argv."""
322 346
         subapp,help = self.subcommands.get(subc)
@@ -376,6 +400,7 @@ def flatten_flags(self):
376 400
             flags[key] = (newflag, help)
377 401
         return flags, aliases
378 402
 
  403
+    @catch_config_error
379 404
     def parse_command_line(self, argv=None):
380 405
         """Parse the command line arguments."""
381 406
         argv = sys.argv[1:] if argv is None else argv
@@ -402,18 +427,12 @@ def parse_command_line(self, argv=None):
402 427
 
403 428
         loader = KVArgParseConfigLoader(argv=argv, aliases=aliases,
404 429
                                         flags=flags)
405  
-        try:
406  
-            config = loader.load_config()
407  
-            self.update_config(config)
408  
-        except (TraitError, ArgumentError) as e:
409  
-            self.print_description()
410  
-            self.print_help()
411  
-            self.print_examples()
412  
-            self.log.fatal(str(e))
413  
-            self.exit(1)
  430
+        config = loader.load_config()
  431
+        self.update_config(config)
414 432
         # store unparsed args in extra_args
415 433
         self.extra_args = loader.extra_args
416 434
 
  435
+    @catch_config_error
417 436
     def load_config_file(self, filename, path=None):
418 437
         """Load a .py based config file by filename and path."""
419 438
         loader = PyFileConfigLoader(filename, path=path)
4  IPython/core/application.py
@@ -34,7 +34,7 @@
34 34
 import shutil
35 35
 import sys
36 36
 
37  
-from IPython.config.application import Application
  37
+from IPython.config.application import Application, catch_config_error
38 38
 from IPython.config.configurable import Configurable
39 39
 from IPython.config.loader import Config, ConfigFileNotFound
40 40
 from IPython.core import release, crashhandler
@@ -304,7 +304,7 @@ def stage_default_config_file(self):
304 304
             with open(fname, 'w') as f:
305 305
                 f.write(s)
306 306
 
307  
-
  307
+    @catch_config_error
308 308
     def initialize(self, argv=None):
309 309
         # don't hook up crash handler before parsing command-line
310 310
         self.parse_command_line(argv)
7  IPython/core/crashhandler.py
@@ -110,7 +110,12 @@ def __init__(self, app, contact_name=None, contact_email=None,
110 110
 
111 111
     def __call__(self, etype, evalue, etb):
112 112
         """Handle an exception, call for compatible with sys.excepthook"""
113  
-
  113
+        
  114
+        # do not allow the crash handler to be called twice without reinstalling it
  115
+        # this prevents unlikely errors in the crash handling from entering an
  116
+        # infinite loop.
  117
+        sys.excepthook = sys.__excepthook__
  118
+        
114 119
         # Report tracebacks shouldn't use color in general (safer for users)
115 120
         color_scheme = 'NoColor'
116 121
 
2  IPython/frontend/html/notebook/notebookapp.py
@@ -43,6 +43,7 @@
43 43
 )
44 44
 from .notebookmanager import NotebookManager
45 45
 
  46
+from IPython.config.application import catch_config_error
46 47
 from IPython.core.application import BaseIPythonApplication
47 48
 from IPython.core.profiledir import ProfileDir
48 49
 from IPython.zmq.session import Session, default_secure
@@ -260,6 +261,7 @@ def init_logging(self):
260 261
         # and all of its ancenstors until propagate is set to False.
261 262
         self.log.propagate = False
262 263
 
  264
+    @catch_config_error
263 265
     def initialize(self, argv=None):
264 266
         super(NotebookApp, self).initialize(argv)
265 267
         self.init_configurables()
3  IPython/frontend/qt/console/qtconsoleapp.py
@@ -29,7 +29,7 @@
29 29
 from IPython.external.qt import QtGui
30 30
 
31 31
 # Local imports
32  
-from IPython.config.application import boolean_flag
  32
+from IPython.config.application import boolean_flag, catch_config_error
33 33
 from IPython.core.application import BaseIPythonApplication
34 34
 from IPython.core.profiledir import ProfileDir
35 35
 from IPython.lib.kernel import tunnel_to_kernel, find_connection_file
@@ -516,6 +516,7 @@ def init_colors(self):
516 516
             else:
517 517
                 raise IOError("Stylesheet %r not found."%self.stylesheet)
518 518
 
  519
+    @catch_config_error
519 520
     def initialize(self, argv=None):
520 521
         super(IPythonQtConsoleApp, self).initialize(argv)
521 522
         self.init_connection_file()
5  IPython/frontend/terminal/ipapp.py
@@ -32,7 +32,7 @@
32 32
 from IPython.config.loader import (
33 33
     Config, PyFileConfigLoader, ConfigFileNotFound
34 34
 )
35  
-from IPython.config.application import boolean_flag
  35
+from IPython.config.application import boolean_flag, catch_config_error
36 36
 from IPython.core import release
37 37
 from IPython.core import usage
38 38
 from IPython.core.completer import Completer
@@ -285,7 +285,8 @@ def parse_command_line(self, argv=None):
285 285
             argv[idx] = sub
286 286
 
287 287
         return super(TerminalIPythonApp, self).parse_command_line(argv)
288  
-
  288
+    
  289
+    @catch_config_error
289 290
     def initialize(self, argv=None):
290 291
         """Do actions after construct, but before starting the app."""
291 292
         super(TerminalIPythonApp, self).initialize(argv)
8  IPython/nbformat/v2/nbjson.py
@@ -16,9 +16,8 @@
16 16
 # Imports
17 17
 #-----------------------------------------------------------------------------
18 18
 
19  
-from base64 import encodestring
20 19
 from .nbbase import from_dict
21  
-from .rwbase import NotebookReader, NotebookWriter, base64_decode
  20
+from .rwbase import NotebookReader, NotebookWriter, restore_bytes
22 21
 import json
23 22
 
24 23
 #-----------------------------------------------------------------------------
@@ -26,9 +25,10 @@
26 25
 #-----------------------------------------------------------------------------
27 26
 
28 27
 class BytesEncoder(json.JSONEncoder):
  28
+    """A JSON encoder that accepts b64 (and other *ascii*) bytestrings."""
29 29
     def default(self, obj):
30 30
         if isinstance(obj, bytes):
31  
-            return unicode(encodestring(bytes))
  31
+            return obj.decode('ascii')
32 32
         return json.JSONEncoder.default(self, obj)
33 33
 
34 34
 
@@ -40,7 +40,7 @@ def reads(self, s, **kwargs):
40 40
         return nb
41 41
 
42 42
     def to_notebook(self, d, **kwargs):
43  
-        return base64_decode(from_dict(d))
  43
+        return restore_bytes(from_dict(d))
44 44
 
45 45
 
46 46
 class JSONWriter(NotebookWriter):
56  IPython/nbformat/v2/rwbase.py
@@ -19,31 +19,67 @@
19 19
 from base64 import encodestring, decodestring
20 20
 import pprint
21 21
 
  22
+from IPython.utils.py3compat import str_to_bytes
  23
+
22 24
 #-----------------------------------------------------------------------------
23 25
 # Code
24 26
 #-----------------------------------------------------------------------------
25 27
 
  28
+def restore_bytes(nb):
  29
+    """Restore bytes of image data from unicode-only formats.
  30
+    
  31
+    Base64 encoding is handled elsewhere.  Bytes objects in the notebook are
  32
+    always b64-encoded. We DO NOT encode/decode around file formats.
  33
+    """
  34
+    for ws in nb.worksheets:
  35
+        for cell in ws.cells:
  36
+            if cell.cell_type == 'code':
  37
+                for output in cell.outputs:
  38
+                    if 'png' in output:
  39
+                        output.png = str_to_bytes(output.png, 'ascii')
  40
+                    if 'jpeg' in output:
  41
+                        output.jpeg = str_to_bytes(output.jpeg, 'ascii')
  42
+    return nb
  43
+
  44
+
  45
+# b64 encode/decode are never actually used, because all bytes objects in
  46
+# the notebook are already b64-encoded, and we don't need/want to double-encode
  47
+
26 48
 def base64_decode(nb):
27  
-    """Base64 encode all bytes objects in the notebook."""
  49
+    """Restore all bytes objects in the notebook from base64-encoded strings.
  50
+    
  51
+    Note: This is never used
  52
+    """
28 53
     for ws in nb.worksheets:
29 54
         for cell in ws.cells:
30 55
             if cell.cell_type == 'code':
31  
-                if 'png' in cell:
32  
-                    cell.png = bytes(decodestring(cell.png))
33  
-                if 'jpeg' in cell:
34  
-                    cell.jpeg = bytes(decodestring(cell.jpeg))
  56
+                for output in cell.outputs:
  57
+                    if 'png' in output:
  58
+                        if isinstance(output.png, unicode):
  59
+                            output.png = output.png.encode('ascii')
  60
+                        output.png = decodestring(output.png)
  61
+                    if 'jpeg' in output:
  62
+                        if isinstance(output.jpeg, unicode):
  63
+                            output.jpeg = output.jpeg.encode('ascii')
  64
+                        output.jpeg = decodestring(output.jpeg)
35 65
     return nb
36 66
 
37 67
 
38 68
 def base64_encode(nb):
39  
-    """Base64 decode all binary objects in the notebook."""
  69
+    """Base64 encode all bytes objects in the notebook.
  70
+    
  71
+    These will be b64-encoded unicode strings
  72
+    
  73
+    Note: This is never used
  74
+    """
40 75
     for ws in nb.worksheets:
41 76
         for cell in ws.cells:
42 77
             if cell.cell_type == 'code':
43  
-                if 'png' in cell:
44  
-                    cell.png = unicode(encodestring(cell.png))
45  
-                if 'jpeg' in cell:
46  
-                    cell.jpeg = unicode(encodestring(cell.jpeg))
  78
+                for output in cell.outputs:
  79
+                    if 'png' in output:
  80
+                        output.png = encodestring(output.png).decode('ascii')
  81
+                    if 'jpeg' in output:
  82
+                        output.jpeg = encodestring(output.jpeg).decode('ascii')
47 83
     return nb
48 84
 
49 85
 
15  IPython/nbformat/v2/tests/nbexamples.py
... ...
@@ -1,10 +1,15 @@
  1
+import os
  2
+from base64 import encodestring
  3
+
1 4
 from ..nbbase import (
2 5
     NotebookNode,
3 6
     new_code_cell, new_text_cell, new_worksheet, new_notebook, new_output,
4 7
     new_metadata, new_author
5 8
 )
6 9
 
7  
-
  10
+# some random base64-encoded *bytes*
  11
+png = encodestring(os.urandom(5))
  12
+jpeg = encodestring(os.urandom(6))
8 13
 
9 14
 ws = new_worksheet(name='worksheet1')
10 15
 
@@ -42,8 +47,8 @@
42 47
         output_text=u'<array a>',
43 48
         output_html=u'The HTML rep',
44 49
         output_latex=u'$a$',
45  
-        output_png=b'data',
46  
-        output_jpeg=b'data',
  50
+        output_png=png,
  51
+        output_jpeg=jpeg,
47 52
         output_svg=u'<svg>',
48 53
         output_json=u'json data',
49 54
         output_javascript=u'var i=0;',
@@ -53,8 +58,8 @@
53 58
         output_text=u'<array a>',
54 59
         output_html=u'The HTML rep',
55 60
         output_latex=u'$a$',
56  
-        output_png=b'data',
57  
-        output_jpeg=b'data',
  61
+        output_png=png,
  62
+        output_jpeg=jpeg,
58 63
         output_svg=u'<svg>',
59 64
         output_json=u'json data',
60 65
         output_javascript=u'var i=0;'
2  IPython/parallel/apps/baseapp.py
@@ -29,6 +29,7 @@
29 29
 
30 30
 from subprocess import Popen, PIPE
31 31
 
  32
+from IPython.config.application import catch_config_error
32 33
 from IPython.core import release
33 34
 from IPython.core.crashhandler import CrashHandler
34 35
 from IPython.core.application import (
@@ -144,6 +145,7 @@ def _loop_default(self):
144 145
     aliases = Dict(base_aliases)
145 146
     flags = Dict(base_flags)
146 147
     
  148
+    @catch_config_error
147 149
     def initialize(self, argv=None):
148 150
         """initialize the app"""
149 151
         super(BaseParallelApplication, self).initialize(argv)
3  IPython/parallel/apps/ipclusterapp.py
@@ -31,7 +31,7 @@
31 31
 import zmq
32 32
 from zmq.eventloop import ioloop
33 33
 
34  
-from IPython.config.application import Application, boolean_flag
  34
+from IPython.config.application import Application, boolean_flag, catch_config_error
35 35
 from IPython.config.loader import Config
36 36
 from IPython.core.application import BaseIPythonApplication
37 37
 from IPython.core.profiledir import ProfileDir
@@ -269,6 +269,7 @@ def _daemonize_changed(self, name, old, new):
269 269
     flags = Dict(engine_flags)
270 270
     _stopping = False
271 271
 
  272
+    @catch_config_error
272 273
     def initialize(self, argv=None):
273 274
         super(IPClusterEngines, self).initialize(argv)
274 275
         self.init_signal()
8  IPython/parallel/apps/ipcontrollerapp.py
@@ -41,9 +41,10 @@
41 41
     BaseParallelApplication,
42 42
     base_aliases,
43 43
     base_flags,
  44
+    catch_config_error,
44 45
 )
45 46
 from IPython.utils.importstring import import_item
46  
-from IPython.utils.traitlets import Instance, Unicode, Bool, List, Dict
  47
+from IPython.utils.traitlets import Instance, Unicode, Bool, List, Dict, TraitError
47 48
 
48 49
 from IPython.zmq.session import (
49 50
     Session, session_aliases, session_flags, default_secure
@@ -263,7 +264,9 @@ def init_hub(self):
263 264
             self.factory = HubFactory(config=c, log=self.log)
264 265
             # self.start_logging()
265 266
             self.factory.init_hub()
266  
-        except:
  267
+        except TraitError:
  268
+            raise
  269
+        except Exception:
267 270
             self.log.error("Couldn't construct the Controller", exc_info=True)
268 271
             self.exit(1)
269 272
         
@@ -385,6 +388,7 @@ def forward_logging(self):
385 388
             self.log.addHandler(handler)
386 389
             self._log_handler = handler
387 390
     
  391
+    @catch_config_error
388 392
     def initialize(self, argv=None):
389 393
         super(IPControllerApp, self).initialize(argv)
390 394
         self.forward_logging()
2  IPython/parallel/apps/ipengineapp.py
@@ -34,6 +34,7 @@
34 34
     BaseParallelApplication,
35 35
     base_aliases,
36 36
     base_flags,
  37
+    catch_config_error,
37 38
 )
38 39
 from IPython.zmq.log import EnginePUBHandler
39 40
 from IPython.zmq.session import (
@@ -318,6 +319,7 @@ def init_mpi(self):
318 319
         else:
319 320
             mpi = None
320 321
 
  322
+    @catch_config_error
321 323
     def initialize(self, argv=None):
322 324
         super(IPEngineApp, self).initialize(argv)
323 325
         self.init_mpi()
4  IPython/parallel/apps/iploggerapp.py
@@ -30,7 +30,8 @@
30 30
 
31 31
 from IPython.parallel.apps.baseapp import (
32 32
     BaseParallelApplication,
33  
-    base_aliases
  33
+    base_aliases,
  34
+    catch_config_error,
34 35
 )
35 36
 from IPython.parallel.apps.logwatcher import LogWatcher
36 37
 
@@ -68,6 +69,7 @@ class IPLoggerApp(BaseParallelApplication):
68 69
     classes = [LogWatcher, ProfileDir]
69 70
     aliases = Dict(aliases)
70 71
 
  72
+    @catch_config_error
71 73
     def initialize(self, argv=None):
72 74
         super(IPLoggerApp, self).initialize(argv)
73 75
         self.init_watcher()
4  IPython/zmq/ipkernel.py
@@ -28,7 +28,7 @@
28 28
 
29 29
 # Local imports.
30 30
 from IPython.config.configurable import Configurable
31  
-from IPython.config.application import boolean_flag
  31
+from IPython.config.application import boolean_flag, catch_config_error
32 32
 from IPython.core.application import ProfileDir
33 33
 from IPython.core.error import StdinNotImplementedError
34 34
 from IPython.core.shellapp import (
@@ -730,6 +730,8 @@ class IPKernelApp(KernelApp, InteractiveShellApp):
730 730
         selecting a particular matplotlib backend and loop integration.
731 731
         """
732 732
     )
  733
+    
  734
+    @catch_config_error
733 735
     def initialize(self, argv=None):
734 736
         super(IPKernelApp, self).initialize(argv)
735 737
         self.init_shell()
3  IPython/zmq/kernelapp.py
@@ -26,7 +26,7 @@
26 26
 # IPython imports.
27 27
 from IPython.core.ultratb import FormattedTB
28 28
 from IPython.core.application import (
29  
-    BaseIPythonApplication, base_flags, base_aliases
  29
+    BaseIPythonApplication, base_flags, base_aliases, catch_config_error
30 30
 )
31 31
 from IPython.utils import io
32 32
 from IPython.utils.localinterfaces import LOCALHOST
@@ -275,6 +275,7 @@ def init_kernel(self):
275 275
         )
276 276
         self.kernel.record_ports(self.ports)
277 277
 
  278
+    @catch_config_error
278 279
     def initialize(self, argv=None):
279 280
         super(KernelApp, self).initialize(argv)
280 281
         self.init_blackhole()
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.