Config masks exceptions from ConfigParser

[xiongchiamiov]

I spent a while trying to figure out why boto wasn't authenticating, to find out eventually that it's because it's not supported on Python 3. That was I suppose my fault for not checking, but it ended up being a bit more difficult than it should've because of how boto is swallowing exceptions.

In the Config class, we've got methods like this:

 def get(self, section, name, default=None):         
     try:                                            
         val = ConfigParser.get(self, section, name) 
     except:                                         
         val = default                               
     return val                                      

The ConfigParser line raises an exception in Python 3:

TypeError: get() got an unexpected keyword argument 'raw'

but unfortunately, this is swallowed by the except. As a result, the default value is used, and no indication is ever given to the user that anything is wrong. This is particularly annoying to track down because increasing the debug level in the configuration file is silently ignored (because everything in there is!), so it takes some digging around in the shell to even start on the path to find out what's going on.

I don't know enough about boto to know when an exception is expected, but if someone can provide some historical information on that subject, I'd be glad to prepare a patch to change the excepts throughout Config.

[jamesls]

What version of python 3 are you using? I'm trying to repro the issue. I made a small change to raise exceptions on TypeErrors just to see if I could get the same error:

--- a/boto/pyami/config.py
+++ b/boto/pyami/config.py
@@ -140,7 +140,10 @@ class Config(ConfigParser):
     def get(self, section, name, default=None):
         try:
             val = ConfigParser.get(self, section, name)
-        except:
+        except TypeError as e:
+            print("Got TypeError: %s" % e)
+            raise
+        except Exception as e:
             val = default
         return val

Using this script:

$ cat t.py
import boto
import sys

print("Using python version:", sys.version_info)
s3 = boto.connect_s3()
print("\n")
print(s3.get_bucket('my-bucket'))

I get this output on python 3.4:

$ python t.py
Using python version: 3.4.2 (default, Jan 12 2015, 13:24:20)
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)]


<Bucket: my-bucket>

Boto supports python3, so if there's any issues you're seeing I'd like to get them fixed.

I do agree that we should probably remove the bare except clauses in that module. But first I'd like to track down what's going wrong here.

[xiongchiamiov]

This was on a work computer, and I'm away from it for another week, so I can't verify it immediately. However, it should have been python 3.5.1.

I suspect the issue arose from the change in 3.2:

Changed in version 3.2: Arguments raw, vars and fallback are keyword only to protect users from trying to use the third argument as the fallback fallback (especially when using the mapping protocol).

However, you should have this change in your version. Additionally, looking at the code now, both the excerpt I listed in the issue and the CPython implementation, I'm not sure where that error would come from.

When I get back into work, I'll poke at my machine and see what's happening.

[lrowe]

We've just run into this problem as well. It seems to be related to the configparser changes in Python 3.5.1 as it works on Python 3.5.0.

[kecaps]

I believe the python tracking issue that introduced this bug is http://bugs.python.org/issue21159. There is a comment that it is causing problems in various libraries (including boto): http://bugs.python.org/issue21159#msg257662

It was introduced in python 3.5.1 and python 3.4.4, I think.

[shea-parkes]

Just encountered this too. Very annoying to track down with the blanket exception catching. I suppose we'll try to port to boto3.

[cynipe]

hit the same error.
I'm wondering why #3453 has not been merged, any reason there?