Benchmark astroquery vizier queries with cdsclient command line version

[fred3m]

Check astroquery Vizier speeds vs cdsclient command line interface (what SCAMP uses). I've looked at the cdsclient scripts and plan to run this on similar fields to compare the results.

[keflavich]

#526 is related

http://cdsarc.u-strasbg.fr/doc/cdsclient.html

[keflavich]

I've e-mailed cds-help asking if they can recommend a speedup

[keflavich]

Some very helpful notes:

Hi,
Thank you for your interest to VizieR and your actions on astroquery!

Concerning the efficiency: CSV or TSV are more effiscient.
The cdsclient package contains the vizquery tool and provide result in TSV, VOTable, FITS....
This script is a simple shell script which use wget to query the common vizier URL:
http://vizier.u-strasbg.fr/viz-bin/asu-tsv, http://vizier.u-strasbg.fr/viz-bin/votable....

For large catalogues: SDSS, 2MASS, WISE....

The cdsclient contains some dedicates scripts to query large catalogues.
These scripts query directly the source and are more efficient.

example : findsdss9 : ./findsdss9 -c 0+0 (format aligned TEXT)

For some other large catalog: example ALLWISE
example: ALLWISE : ./find_gen1 ALLWISE -c=0+0 (format TSV)
(or catClient.py)

(see the list of the available catalogues with command : ./find_gen)

I'm available for more details if you neeed it,
Gilles Landais

I just give you some other way to query VizieR catalogues:

• the TAPVizieR capability to query tabless using ADQL (see (http://tapvizier.u-strasbg.fr/adql/, http://tapvizier.u-strasbg.fr/TAPVizieR/tap/) )
  TAPVizieR is efficient for small and midle catalogue size. This capability provides result in CSV,VOTAble and JSON.
• To crossmatch tables:
  • TAPVizieR
  • ( or better) the CDSXmatch service (see http://cdsxmatch.u-strasbg.fr/xmatch/doc/cross-match-API.html, http://cdsxmatch.u-strasbg.fr/xmatch)

[keflavich]

There has been a lot of performance improvement of the various CDMS offerings over the years. I don't think we need benchmarks against other tools any longer.