AttributeError related to BeautifulSoup configuration

[aphearin]

I am using astroquery to interface with the CosmoSim database, where I have an active account. When I login using astroquery, I get an AttributeError that appears to be related to bs4, but I'm not sure how to fix it. Traceback below:

In [3]: from astroquery.cosmosim import CosmoSim

In [4]: CS = CosmoSim()

In [5]: CS.login(username="andrewhearin")
WARNING: Using the password found in the keychain for the provided username. [astroquery.cosmosim.core]
/Users/aphearin/anaconda/lib/python2.7/site-packages/astroquery/cosmosim/core.py:106: UserWarning: Authenticating andrewhearin on www.cosmosim.org...
  .format(self.username))
/Users/aphearin/anaconda/lib/python2.7/site-packages/astroquery/cosmosim/core.py:111: UserWarning: Authentication successful!
  warnings.warn("Authentication successful!")
/Users/aphearin/anaconda/lib/python2.7/site-packages/bs4/__init__.py:181: UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("lxml"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.

The code that caused this warning is on line 6 of the file /Users/aphearin/anaconda/bin/ipython. To get rid of this warning, change code that looks like this:

 BeautifulSoup(YOUR_MARKUP})

to this:

 BeautifulSoup(YOUR_MARKUP, "lxml")

  markup_type=markup_type))
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-5-4098c05c150f> in <module>()
----> 1 CS.login(username="andrewhearin")

/Users/aphearin/anaconda/lib/python2.7/site-packages/astroquery/query.pyc in login(self, *args, **kwargs)
    336     def login(self, *args, **kwargs):
    337         with suspend_cache(self):
--> 338             self._authenticated = self._login(*args, **kwargs)
    339         return self._authenticated
    340 

/Users/aphearin/anaconda/lib/python2.7/site-packages/astroquery/cosmosim/core.pyc in _login(self, username, password, store_password, reenter_password)
    127         if authenticated.status_code == 200:
    128             soup = BeautifulSoup(authenticated.content)
--> 129             self.delete_job(jobid=str(soup.find("uws:jobid").string),
    130                             squash=True)
    131 

AttributeError: 'NoneType' object has no attribute 'string'

[duncandc]

FYI, I am running into the same error.

/anaconda2/lib/python2.7/site-packages/astroquery-0.3.7.dev4152-py2.7.egg/astroquery/cosmosim/core.pyc in _login(self, username, password, store_password, reenter_password)
    127         if authenticated.status_code == 200:
    128             soup = BeautifulSoup(authenticated.content)
--> 129             self.delete_job(jobid=str(soup.find("uws:jobid").string),
    130                             squash=True)
    131 

AttributeError: 'NoneType' object has no attribute 'string'

[keflavich]

OK, we should definitely be explicitly specifying a parser, but that's not likely to be the issue here. I think we need an extra validation step because it appears that uws:jobid is not in the returned response for you both.

[duncandc]

I am not sure I understand why this section of the code attempts to delete a job after simply logging in?

[keflavich]

Neither do I. @agroener, can you chime in here?

[agroener]

Back when I first created this package, the model to interface with CosmoSim was called Universal Worker Service (UWS; https://www.cosmosim.org/cms/documentation/data-access/access-via-uws/). I believe a little while ago there were some serious updates to their scripted access to the database. I will take a look to see what the issue may be.

[agroener]

@duncandc In order to log-in to a session with cosmosim credentials, you have to submit an http request. This http request is actually stored in the list of requests for the session (server side). If somebody were to query cosmosim, they would see the request among others. In other words, login attempts should not clutter the http response view for the user - that was the original motivation for immediately deleting the request after a login/attempt.

... Does anyone know if things have things changed for the CosmoSim database? Is there a better way to do all of this? I seem to recall that I was relying heavily on the python package for http requests (called 'requests').

[agroener]

#999 Should address this specific issue. I am working on a few future PR's so fix up the rest of the package.

[bsipocz]

@aphearin - #999 is merged, could you have a look whether you see any more issues?

[aphearin]

This does not seem to have resolved the issue @bsipocz - I still get the same error using the same code. Is astroquery.__version__=0.3.6.post2 the correct version to do the test?

[bsipocz]

@aphearin - Nope, it's still hasn't been released (but will be soon), you need to check it with the dev version.

[aphearin]

Ok, yes, roger that, I just tried out 8196f29 and this resolved the bug for me.

[bsipocz]

Thank you, that case I suggest to close this issue.