Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

usbtmc assertion failure #20

Closed
deadsy opened this issue Feb 21, 2015 · 11 comments
Closed

usbtmc assertion failure #20

deadsy opened this issue Feb 21, 2015 · 11 comments

Comments

@deadsy
Copy link

@deadsy deadsy commented Feb 21, 2015

Linux, Python3 - talking to a Rigol ds1102e scope.
Most operations complete ok.
Trying to get the channel data from the scope.
ie: rawdata = inst.query_binary_values(":WAV:DATA? CHAN1")[10:]
Which is on the order of 1024 n-bit values. (I think - not really sure)

usbtmc.py croaks.

  File "./test.py", line 42, in <module>
    main()
  File "./test.py", line 29, in main
    rawdata = inst.query_binary_values(":WAV:DATA? CHAN1")[10:]
  File "/usr/local/lib/python3.4/dist-packages/pyvisa/resources/messagebased.py", line 458, in query_binary_values
    block = self.read_raw()
  File "/usr/local/lib/python3.4/dist-packages/pyvisa/resources/messagebased.py", line 283, in read_raw
    chunk, status = self.visalib.read(self.session, size)
  File "/usr/local/lib/python3.4/dist-packages/pyvisa-py/highlevel.py", line 297, in read
    return self.sessions[session].read(count)
  File "/usr/local/lib/python3.4/dist-packages/pyvisa-py/usb.py", line 107, in read
    usb.USBError)
  File "/usr/local/lib/python3.4/dist-packages/pyvisa-py/sessions.py", line 312, in _read
    current = reader()
  File "/usr/local/lib/python3.4/dist-packages/pyvisa-py/usb.py", line 101, in <lambda>
    return self._read(lambda: self.interface.read(1),
  File "/usr/local/lib/python3.4/dist-packages/pyvisa-py/protocols/usbtmc.py", line 327, in read
    response = BulkInMessage.from_bytes(resp)
  File "/usr/local/lib/python3.4/dist-packages/pyvisa-py/protocols/usbtmc.py", line 95, in from_bytes
    assert msgid == MsgID.dev_dep_msg_in
AssertionError

Here are the setup details:

python3 -m visa info
Machine Details:
   Platform ID:    Linux-3.13.0-45-generic-i686-with-Ubuntu-14.04-trusty
   Processor:      i686

Python:
   Implementation: CPython
   Executable:     /usr/bin/python3
   Version:        3.4.0
   Compiler:       GCC 4.8.2
   Bits:           32bit
   Build:          Apr 11 2014 13:05:18 (#default)
   Unicode:        UCS4

PyVISA Version: 1.6.3

Backends:
   ni:
      Version: 1.6.3 (bundled with PyVISA)
      #1: /usr/lib/i386-linux-gnu/libvisa.so.0.0.0:
         found by: auto
         bitness: 32
         Could not get more info:
            VI_ERROR_NSUP_ATTR (-1073807331): The specified attribute is not defined or supported by the referenced object.
   py:
      Version: 0.1
      TCPIP INSTR: Available 
      USB INSTR: Available via PyUSB (1.0.0b2). Backend: libusb1
      ASRL INSTR: Available via PySerial (N/A)

Thanks for PyVisa-py - I've gotten further with it than with librevisa.
Let me know if you need any more details.

Jason H.

@hgrecco

This comment has been minimized.

Copy link
Member

@hgrecco hgrecco commented Feb 22, 2015

I am glad that you find it useful. Regarding your error. I guess it might be because the length of the message and the fact that is being partitioned. Can you put a print just before the assert to see the values of msgid and MsgID.dev_dep_msg_in

@deadsy

This comment has been minimized.

Copy link
Author

@deadsy deadsy commented Feb 22, 2015

As you say - it's in a loop pulling multiple buffers for the query response.
The number of buffers it gets varies.
The unexpected value of msgid varies.

run1:

inst.query() call
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 0 MsgID.dev_dep_msg_in 2

run2:

inst.query() call
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 255 MsgID.dev_dep_msg_in 2

run3:

inst.query() call
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 2 MsgID.dev_dep_msg_in 2
msgid 255 MsgID.dev_dep_msg_in 2

Hope this helps.

Jason H.

@hgrecco

This comment has been minimized.

Copy link
Member

@hgrecco hgrecco commented Feb 22, 2015

AFAIK, 0 and 255 are not valid values fro msgid. So it is either not a BulkInMessage or the instrument is sending something wrong. Interestingly, looking around I found the following C code. Notice there is a variable called rigol_quirk and it seems the check is not done if it is rigol. Do you have NI-VISA as well, maybe in another computer? It would be great to compare both. It would be also very useful to capture the USB packages.

@deadsy

This comment has been minimized.

Copy link
Author

@deadsy deadsy commented Feb 23, 2015

Here's a wireshark packet capture of the transaction.
The request for the waveform data is at packet 125.
https://github.com/deadsy/rigol_usb/blob/master/rigol_usb.pcapng.gz

I'll see if I can get NI-VISA working.

Thanks -jasonh

@deadsy

This comment has been minimized.

Copy link
Author

@deadsy deadsy commented Feb 23, 2015

@deadsy

This comment has been minimized.

Copy link
Author

@deadsy deadsy commented Feb 24, 2015

jasonh@satan:/lib/modules/3.13.0-45-generic/kernel/drivers/usb/class$ strings usbtmc.ko | grep RIGOL
Trying to find if device Vendor 0x%04X Product 0x%04X has the RIGOL quirk
Setting this device as having the RIGOL quirk

So I have a quirk enable usbtmc.ko. The PID/VID matches so we should be running quirk enabled. I guess the next step is to build a custom usbtmc.ko and start playing with bigger buffer sizes.

@hgrecco

This comment has been minimized.

Copy link
Member

@hgrecco hgrecco commented Aug 25, 2015

I am closing this as it seems to be outside the code of pyvisa-py. Please reopen if you think we can help from here.

@hgrecco hgrecco closed this Aug 25, 2015
@agricolab

This comment has been minimized.

Copy link

@agricolab agricolab commented Jan 4, 2018

Linux, Python3 - talking to a Siglent SDG 805 Signal Generator. Identical issue. The SDG runs fine under Win10 with NI-VISA, that's why i am necromancing this issue.

I can write to the device without problems. But reading from it, i run, as reported above, in an issue with the assertion. I can also confirm that this is linked to message partitioning by the device. The first message part is 64 Byte long. The second usually shorter, which appears to cause the assertion error. It seems USBTMC.read() is called twice by messagebased.read_raw() from pyvisa, and this function would indeed concatenate partitioned messages.

As NI-VISA as a backend works, and as pyvisa can concatenate partitioned messages, i therefore decided to tweak pyvisa-py. I solved the issue (for my machine and device) by adding an additional quirky message class constructor for BulkInMessage in usbtmc.py, and calling it whenever the assertion fails. In addition, my device added trailing zeros, which needed to be removed, even if the rest of the message would conform. My working expansion is below:

    @classmethod
    def from_bytes(cls, data):       
        msgid, btag, btaginverse = struct.unpack_from('BBBx', data)
        # throw away trailing zeros
        while data[-1] is 0:
            data =  data[:-1]
        # if message in correct format, proceed as usual
        try:        
            assert msgid == MsgID.dev_dep_msg_in
            transfer_size, transfer_attributes = struct.unpack_from('<LBxxx', data, 4)
            data = data[12:]
            return cls(msgid, btag, btaginverse, transfer_size, transfer_attributes, data)
        # otherwise, construct a correct response from the quirky device        
        except AssertionError as e:
            return BulkInMessage.from_quirky(data)        
    
    @classmethod
    def from_quirky(cls, data):  
        'constructs a correct response for quirky devices'
        msgid, btag, btaginverse = struct.unpack_from('BBBx', data)    
        # check whether it contains a ';' and if throw away the first 12 bytes            
        if ';' in str(data):
            transfer_size, transfer_attributes = struct.unpack_from('<LBxxx', data, 4)            
            data = data[12:]                                                    
        else:                 
            transfer_size = 0
            transfer_attributes = 1                    
        return cls(msgid, btag, btaginverse, transfer_size, transfer_attributes, data)

I am uncertain, how this could conflict with other devices. I will likely be able to test it in February with a RIGOL DG 1062Z and a RIGOL 1052E

@jondoesntgit

This comment has been minimized.

Copy link
Contributor

@jondoesntgit jondoesntgit commented Apr 22, 2019

This works on CentOS 7, Pyvisa, on a RIGOL DG812

   @classmethod
    def from_bytes(cls, data):
        msgid, btag, btaginverse = struct.unpack_from('BBBx', data)
        if msgid != MsgID.dev_dep_msg_in:
            return BulkInMessage.from_quirky(data)

        transfer_size, transfer_attributes = struct.unpack_from('<LBxxx', data,
                                                                4)

        data = data[12:]
        return cls(msgid, btag, btaginverse, transfer_size,
                   transfer_attributes, data)

    @classmethod
    def from_quirky(cls, data):
        """Constructs a correct response for quirky devices"""
        msgid, btag, btaginverse = struct.unpack_from('BBBx', data)
        # check whether it contains a ';' and if throw away the first 12 bytes
        if ';' in str(data):
            transfer_size, transfer_attributes = struct.unpack_from('<LBxxx', data, 4)
            data = data[12:]
        else:
            transfer_size = 0
            transfer_attributes = 1
        print(cls)
        return cls(msgid, btag, btaginverse, transfer_size,
                transfer_attributes, data)
@MatthieuDartiailh

This comment has been minimized.

Copy link
Member

@MatthieuDartiailh MatthieuDartiailh commented Apr 22, 2019

Thanks for testing this on rigol @jondoesntgit.

Based on this thread https://www.eevblog.com/forum/testgear/rigol-usbtmcvisa-interface-is-really-terrible/ it seems that updating the firmware on the instrument may fix the issue.

@jondoesntgit

This comment has been minimized.

Copy link
Contributor

@jondoesntgit jondoesntgit commented Apr 22, 2019

I see, thank you!

I also found that there were a bunch of extra b'\x00' bytes in the data strings, that I removed with a data = data.rstrip(b'\x00'), for future reference.

bors bot added a commit that referenced this issue May 8, 2019
Merge #186
186: Added support for Rigol/"quirky" devices r=MatthieuDartiailh a=jondoesntgit

This pulls in the changes from @agricolab in #20 

Co-authored-by: Jonathan Wheeler <jonathan.m.wheeler@gmail.com>
Co-authored-by: MatthieuDartiailh <marul@laposte.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.