Timeout when doing many bayestar queries with green18 dustmap

[petigura]

Hey, @danxhuber and @taberger.

Have you guys tried running isoclassify with the CLI in parallel? I've got 12,000 stars I'd like to run through and the following command should ship a whole bunch of commands off to the CLI.

cat isoclassify.tot | grep mkdir | parallel

When I do this I get a whole bunch of 429 errors from running the bayestar web query.

isoclassify/cks1/direct/K00174/output.log:requests.exceptions.HTTPError: 429 Client Error: TOO MANY REQUESTS for url: http://argonaut.skymaps.info/api/v2/bayestar2017/query

Have you seen this too? I could just reduce the parallelism and hope that the bayestar server doesn't return 429. Any other suggestions. It looks you guys have implemented an 'allsky' method which uses mwdust.Combined15() that's preloaded on the Kepler field. Is this an effective workaround for the Kepler field?

[danxhuber]

Yes, @taberger has noticed this issue too. We believe that the bayestar team reduced the number of web queries after implementing the latest version of their dustmap (which now includes Gaia). @taberger is working on a solution, but using "allsky" is a good workaround in the meantime.

[danxhuber]

@taberger I believe this is now fixed?

[taberger]

Yes, this is now fixed. The code now reads in entire dustmaps from mwdust and works with them.