usbtmc assertion failure

[deadsy]

Linux, Python3 - talking to a Rigol ds1102e scope.
Most operations complete ok.
Trying to get the channel data from the scope.
ie: rawdata = inst.query_binary_values(":WAV:DATA? CHAN1")[10:]
Which is on the order of 1024 n-bit values. (I think - not really sure)

usbtmc.py croaks.

  File "./test.py", line 42, in <module>
    main()
  File "./test.py", line 29, in main
    rawdata = inst.query_binary_values(":WAV:DATA? CHAN1")[10:]
  File "/usr/local/lib/python3.4/dist-packages/pyvisa/resources/messagebased.py", line 458, in query_binary_values
    block = self.read_raw()
  File "/usr/local/lib/python3.4/dist-packages/pyvisa/resources/messagebased.py", line 283, in read_raw
    chunk, status = self.visalib.read(self.session, size)
  File "/usr/local/lib/python3.4/dist-packages/pyvisa-py/highlevel.py", line 297, in read
    return self.sessions[session].read(count)
  File "/usr/local/lib/python3.4/dist-packages/pyvisa-py/usb.py", line 107, in read
    usb.USBError)
  File "/usr/local/lib/python3.4/dist-packages/pyvisa-py/sessions.py", line 312, in _read
    current = reader()
  File "/usr/local/lib/python3.4/dist-packages/pyvisa-py/usb.py", line 101, in <lambda>
    return self._read(lambda: self.interface.read(1),
  File "/usr/local/lib/python3.4/dist-packages/pyvisa-py/protocols/usbtmc.py", line 327, in read
    response = BulkInMessage.from_bytes(resp)
  File "/usr/local/lib/python3.4/dist-packages/pyvisa-py/protocols/usbtmc.py", line 95, in from_bytes
    assert msgid == MsgID.dev_dep_msg_in
AssertionError

Here are the setup details:

python3 -m visa info
Machine Details:
   Platform ID:    Linux-3.13.0-45-generic-i686-with-Ubuntu-14.04-trusty
   Processor:      i686

Python:
   Implementation: CPython
   Executable:     /usr/bin/python3
   Version:        3.4.0
   Compiler:       GCC 4.8.2
   Bits:           32bit
   Build:          Apr 11 2014 13:05:18 (#default)
   Unicode:        UCS4

PyVISA Version: 1.6.3

Backends:
   ni:
      Version: 1.6.3 (bundled with PyVISA)
      #1: /usr/lib/i386-linux-gnu/libvisa.so.0.0.0:
         found by: auto
         bitness: 32
         Could not get more info:
            VI_ERROR_NSUP_ATTR (-1073807331): The specified attribute is not defined or supported by the referenced object.
   py:
      Version: 0.1
      TCPIP INSTR: Available 
      USB INSTR: Available via PyUSB (1.0.0b2). Backend: libusb1
      ASRL INSTR: Available via PySerial (N/A)

Thanks for PyVisa-py - I've gotten further with it than with librevisa.
Let me know if you need any more details.

Jason H.

[hgrecco]

I am glad that you find it useful. Regarding your error. I guess it might be because the length of the message and the fact that is being partitioned. Can you put a print just before the assert to see the values of msgid and MsgID.dev_dep_msg_in

[deadsy]

As you say - it's in a loop pulling multiple buffers for the query response.
The number of buffers it gets varies.
The unexpected value of msgid varies.

run1:

inst.query() call
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 0 MsgID.dev_dep_msg_in 2

run2:

inst.query() call
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 255 MsgID.dev_dep_msg_in 2

run3:

inst.query() call
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 255 MsgID.dev_dep_msg_in 2

Hope this helps.

Jason H.

[hgrecco]

AFAIK, 0 and 255 are not valid values fro msgid. So it is either not a BulkInMessage or the instrument is sending something wrong. Interestingly, looking around I found the following C code. Notice there is a variable called rigol_quirk and it seems the check is not done if it is rigol. Do you have NI-VISA as well, maybe in another computer? It would be great to compare both. It would be also very useful to capture the USB packages.

[deadsy]

Here's a wireshark packet capture of the transaction.
The request for the waveform data is at packet 125.
https://github.com/deadsy/rigol_usb/blob/master/rigol_usb.pcapng.gz

I'll see if I can get NI-VISA working.

Thanks -jasonh

[deadsy]

Some other rigol/usbtmc references:
http://www.pittnerovi.com/jiri/hobby/electronics/rigol/
https://github.com/ralfstx/rigol-linux

[deadsy]

jasonh@satan:/lib/modules/3.13.0-45-generic/kernel/drivers/usb/class$ strings usbtmc.ko | grep RIGOL
Trying to find if device Vendor 0x%04X Product 0x%04X has the RIGOL quirk
Setting this device as having the RIGOL quirk

So I have a quirk enable usbtmc.ko. The PID/VID matches so we should be running quirk enabled. I guess the next step is to build a custom usbtmc.ko and start playing with bigger buffer sizes.

[hgrecco]

I am closing this as it seems to be outside the code of pyvisa-py. Please reopen if you think we can help from here.

[agricolab]

Linux, Python3 - talking to a Siglent SDG 805 Signal Generator. Identical issue. The SDG runs fine under Win10 with NI-VISA, that's why i am necromancing this issue.

I can write to the device without problems. But reading from it, i run, as reported above, in an issue with the assertion. I can also confirm that this is linked to message partitioning by the device. The first message part is 64 Byte long. The second usually shorter, which appears to cause the assertion error. It seems USBTMC.read() is called twice by messagebased.read_raw() from pyvisa, and this function would indeed concatenate partitioned messages.

As NI-VISA as a backend works, and as pyvisa can concatenate partitioned messages, i therefore decided to tweak pyvisa-py. I solved the issue (for my machine and device) by adding an additional quirky message class constructor for BulkInMessage in usbtmc.py, and calling it whenever the assertion fails. In addition, my device added trailing zeros, which needed to be removed, even if the rest of the message would conform. My working expansion is below:

    @classmethod
    def from_bytes(cls, data):       
        msgid, btag, btaginverse = struct.unpack_from('BBBx', data)
        # throw away trailing zeros
        while data[-1] is 0:
            data =  data[:-1]
        # if message in correct format, proceed as usual
        try:        
            assert msgid == MsgID.dev_dep_msg_in
            transfer_size, transfer_attributes = struct.unpack_from('<LBxxx', data, 4)
            data = data[12:]
            return cls(msgid, btag, btaginverse, transfer_size, transfer_attributes, data)
        # otherwise, construct a correct response from the quirky device        
        except AssertionError as e:
            return BulkInMessage.from_quirky(data)        
    
    @classmethod
    def from_quirky(cls, data):  
        'constructs a correct response for quirky devices'
        msgid, btag, btaginverse = struct.unpack_from('BBBx', data)    
        # check whether it contains a ';' and if throw away the first 12 bytes            
        if ';' in str(data):
            transfer_size, transfer_attributes = struct.unpack_from('<LBxxx', data, 4)            
            data = data[12:]                                                    
        else:                 
            transfer_size = 0
            transfer_attributes = 1                    
        return cls(msgid, btag, btaginverse, transfer_size, transfer_attributes, data)

I am uncertain, how this could conflict with other devices. I will likely be able to test it in February with a RIGOL DG 1062Z and a RIGOL 1052E

[jondoesntgit]

This works on CentOS 7, Pyvisa, on a RIGOL DG812

   @classmethod
    def from_bytes(cls, data):
        msgid, btag, btaginverse = struct.unpack_from('BBBx', data)
        if msgid != MsgID.dev_dep_msg_in:
            return BulkInMessage.from_quirky(data)

        transfer_size, transfer_attributes = struct.unpack_from('<LBxxx', data,
                                                                4)

        data = data[12:]
        return cls(msgid, btag, btaginverse, transfer_size,
                   transfer_attributes, data)

    @classmethod
    def from_quirky(cls, data):
        """Constructs a correct response for quirky devices"""
        msgid, btag, btaginverse = struct.unpack_from('BBBx', data)
        # check whether it contains a ';' and if throw away the first 12 bytes
        if ';' in str(data):
            transfer_size, transfer_attributes = struct.unpack_from('<LBxxx', data, 4)
            data = data[12:]
        else:
            transfer_size = 0
            transfer_attributes = 1
        print(cls)
        return cls(msgid, btag, btaginverse, transfer_size,
                transfer_attributes, data)

[MatthieuDartiailh]

Thanks for testing this on rigol @jondoesntgit.

Based on this thread https://www.eevblog.com/forum/testgear/rigol-usbtmcvisa-interface-is-really-terrible/ it seems that updating the firmware on the instrument may fix the issue.

[jondoesntgit]

I see, thank you!

I also found that there were a bunch of extra b'\x00' bytes in the data strings, that I removed with a data = data.rstrip(b'\x00'), for future reference.