Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fetching HMI data fails whenn no attribute vso.Physobs is provided #2284

Closed
mserquet opened this issue Sep 7, 2017 · 9 comments · Fixed by #2621
Closed

Fetching HMI data fails whenn no attribute vso.Physobs is provided #2284

mserquet opened this issue Sep 7, 2017 · 9 comments · Fixed by #2621
Labels
Affects Release An issue/bug that affects a released version (use a version label too) Bug Probably a bug net Affects the net submodule

Comments

@mserquet
Copy link

mserquet commented Sep 7, 2017

When trying to download HMI data without specifying a vso.Physobs attribute, the query fails with the following error:

WebFault: Server raised fault: 'Element '' can't be allowed in valid XML message. Died. at /opt/vso/lib/perl5/SOAP/Lite.pm line 1480.
'

Searching for the data is succesful, and when at least one vso.Physobs attribute is specified, the download is successful.

Example code:

import sunpy
from sunpy.net import Fido, attrs as a

#this queryresult cannot be downloaded
qr = Fido.search(a.Time('2017-09-02 23:52:00', '2017-09-02 23:52:30') 
                 & a.Instrument('HMI'))
print(qr)
Fido.fetch(qr)

#this queryresult can be downloaded
qr = Fido.search(a.Time('2017-09-02 23:52:00', '2017-09-02 23:52:30') 
                 & a.Instrument('HMI')
                 & (a.vso.Physobs('intensity') | a.vso.Physobs('LOS_magnetic_field')))
print(qr)

Traceback:
traceback.txt
@Cadair
Copy link
Member

Cadair commented Sep 7, 2017

Thanks for the bug report @Hackempluf. This looks like a VSO side issue to me, I will try and get some VSO peoples eyes on it.

@Cadair Cadair added Affects Release An issue/bug that affects a released version (use a version label too) Bug Probably a bug net Affects the net submodule labels Sep 7, 2017
@jhourcle
Copy link

jhourcle commented Sep 7, 2017

This is believed to be the message that SunPy was sending to the VSO:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns0="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns1="http://virtualsolar.org/VSO/VSOi" xmlns:ns2="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
   <SOAP-ENV:Header/>
   <ns2:Body>
      <ns1:GetData>
         <body>
            <request>
               <method>
                  <methodtype>URL-FILE_Rice</methodtype>
                  <methodtype>URL-FILE</methodtype>
               </method>
               <datacontainer>
                  <datarequestitem>
                     <provider>JSOC</provider>
                     <fileiditem>
                        <fileid>hmi__ic_45s:17301111:17301111</fileid>
                        <fileid>hmi__m_45s:17301111:17301111</fileid>
                        <fileid>hmi__v_45s:17301111:17301111</fileid>
                     </fileiditem>
                  </datarequestitem>
               </datacontainer>
            </request>
         </body>
      </ns1:GetData>
   </ns2:Body>
</SOAP-ENV:Envelope>

For whatever reason, it's not parsing the name of the element (e.g., ''), as the code that's throwing the error is

die "Element '$tag' can't be allowed in valid XML message. Died."
    if $tag !~ /^$SOAP::Constants::NSMASK$/o;

# note : $NSMASK = '[a-zA-Z_:][\w.\-:]*';

@jhourcle
Copy link

jhourcle commented Sep 7, 2017

I should've mentioned -- $NSMASK likely doesn't matter, as attempting to print $tag gives an empty string. But as $tag is passed into the function, this is going to require a trace to debug it.

And only having 3 HMI records for that time period is correct. (IDL reports 2 series w/out matches for the time period requested):

Fido.search(a.Time('2017-09-02 23:52:00', '2017-09-02 23:52:30') & a.Instrument('HMI'))

@Cadair
Copy link
Member

Cadair commented Sep 11, 2017

@jhourcle Am I to interpret this as a VSO issue or a requirement to change our side?

@getsanjeev
Copy link
Contributor

@Cadair If this is an error related to sunpy (not a VSO fault), I would like to give it a try. Should I start?

@getsanjeev
Copy link
Contributor

@Cadair Is there a way to know how VSO reacts to our queries, I mean how does it parses our query?

@Cadair Cadair added the Upstream Fix Required A fix needs to go upstream to another package label Jan 31, 2018
@Cadair
Copy link
Member

Cadair commented Feb 17, 2018

(We should also add a test for this usecase, i.e. a search involving Physobs.

@getsanjeev
Copy link
Contributor

Okay @Cadair I will do it just when I will be done with my previous issue.

@sudozer sudozer mentioned this issue May 8, 2018
Merged
@Cadair
Copy link
Member

Cadair commented May 24, 2018

🎉 🎆

@Cadair Cadair removed the Upstream Fix Required A fix needs to go upstream to another package label May 24, 2018
This was referenced Apr 16, 2020
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Affects Release An issue/bug that affects a released version (use a version label too) Bug Probably a bug net Affects the net submodule
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Vso client fix sudozer/sunpy
4 participants
@Cadair @jhourcle @mserquet @getsanjeev