Simbad ConnectionError

[dagnic]

Hello,

I'm trying to query Simbad to get many sources ids. Sometimes, it fails with:

ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

That's the case for example with Simbad.query_objectids('1es0347-121').

It seems that it is time dependant. I waited a moment then retry and the error occur on another source.

Do you have any idea ?

Thanks!

[dagnic]

Maybe that deals with following assertion in the SimbadClass documentation:

Note that SIMBAD suggests submitting no more than 6 queries per second; if you submit more than that, your IP may be temporarily blacklisted.

I will try with time.sleep() between each query.

[dagnic]

Ok, it seems to solve 104 error, but know it fails with

ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

I may use try except to retry on ConnectionError ?

[bsipocz]

There is still work to be done on vectorizing the modules, however e.g. Simbad.query_region already understands vector coordinates while as far as I know query_objectids not yet.

A workaround would be indeed adding some more time between the queries. And a welcomed enhancement for astroquery would be to vectorize query_objectids. I don't think try/except ConnectionError would solve anything as it may be a result of being of your IP being soft-banned on the server side.

[bsipocz]

(Example how it was done for query_region: #833)

[dagnic]

Yes, I thought to work with vector but I need to do source by source since the names have do be tested and adapted (eg: 'scox1' is not recognised by the resolver, so I changed to 'sco x-1' which is understandable by Simbad).

I added 5 seconds between each queries (that solves ConnectionResetError) but not RemoteDisconnected. So I try and retry one time at exception and that works. But retry occurs each ~ 5 reqests.

I will continue to investigate with time.

[aoberto]

Hello The limitation is above 6 queries in one second, so your sleep of 5 second is quite enough. If you give me your IP address I can investigate in Simbad logs to check what happens. ⁣Anaïs​ Le 5 avr. 2019 à 23:33, à 23:33, dagnic <notifications@github.com> a écrit:

…

Yes, I thought to work with vector but I need to do source by source since the names have do be tested and adapted (eg: 'scox1' is not recognised by the resolver, so I changed to 'sco x-1' which is understandable by Simbad). I added 5 seconds between each queries (that solves `ConnectionResetError`) but not `RemoteDisconnected`. So I try and retry one time at exception and that works. But retry occurs each ~ 5 reqests. I will continue to investigate with time. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #1408 (comment)

[dagnic]

Thanks,

I had a many requests within an imported function, in which I forget to add sleep. Now it works perfectly !

[bsipocz]

Thank you @aoberto for the assistance.

As this has been solved/worked around, I'm closing this issue. The TODO item from our side it to vectorize more of the query methods for simbad, but issue #682 already covers that.

[SilvioLeccia]

Hello!
I have the same problem reported by dagnic.
I do not perform more than 1 or 2 queries per second but I have "ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))".
Before Christmas I had not this error and I worked with a list of 1000 sources. Now every 5/10 sources I have to restart the code.
Can someone give me an help on how to solve?
Cheers,
Silvio