Vizier queries seem slow

[keflavich]

Hello,

I use the Vizier module to query SDSS and it seems slow. My general query is,

Vizier.ROW_LIMIT = -1 # Removes row limit on output table
result = Vizier.query_region(<galaxy>, width=1.0*u.deg,
                                 height=1.0*u.deg, catalog='SDSS')

Where can be something like 'NGC3377'. It takes about a minute and a half to run and return results.

I'm not sure if this is something that can be improved but I think it's worth looking into.

Alexa

[keflavich]

Hi @AlexaVillaume - is this the problem you solved & were posting a solution to, or is it still an open problem?

[AlexaVillaume]

Hey Adam,

This is different.

I�ve been meaning to compare how long it takes astroquery to execute the query vs queryvizier, the IDL version. I realize I should have done this before I opened the issue�my sense is queryvizier is faster but I want to double check.

Alexa

On Apr 22, 2015, at 3:57 PM, Adam Ginsburg notifications@github.com wrote:

Hi @AlexaVillaume - is this the problem you solved & were posting a solution to, or is it still an open problem?

�
Reply to this email directly or view it on GitHub.

[keflavich]

OK, cool - can you link to queryvizier so we can inspect the implementation?

[AlexaVillaume]

Here is the source,

http://idlastro.gsfc.nasa.gov/ftp/pro/sockets/queryvizier.pro

On Apr 23, 2015, at 1:49 PM, Adam Ginsburg notifications@github.com wrote:

OK, cool - can you link to queryvizier so we can inspect the implementation?

�
Reply to this email directly or view it on GitHub.

[keflavich]

It seems that queryvizier uses asu-tsv instead of votable, which may account for some of the server-side speed boost.

[keflavich]

cc @sbvickers who has also noted this
https://github.com/sbvickers/vizquery

[AlexaVillaume]

Good to know. Does astroquery use votable for all the different query types? Is there is a reason for that?

On Apr 24, 2015, at 3:03 PM, Adam Ginsburg notifications@github.com wrote:

cc @sbvickers who has also noted this
https://github.com/sbvickers/vizquery

�
Reply to this email directly or view it on GitHub.

[keflavich]

Yes, astroquery has been using VOTABLE for everything. It is easier to read and validate. I've been working to get tsv and fits reading to work, and it looks like there are a whole lot of invalid tables. This is kind of disappointing, but it might be something we can work with the Vizier team to fix.

I'll add a pull request to this issue that will allow different return types. However, it looks like there is no appreciable speedup.

[coveralls]

Coverage decreased (-0.05%) to 66.1% when pulling f0aae4b on keflavich:vizier_efficiency_boost into ebf8de5 on astropy:master.

[keflavich]

I'll merge this now since it provides the possibility of using tsv and FITS returns, though we rely on #525 to help us with benchmarking.