Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[COM] Thorlabs CCS200 spectrometer on Linux? #296

Open
FilipDominec opened this issue Feb 12, 2021 · 16 comments
Open

[COM] Thorlabs CCS200 spectrometer on Linux? #296

FilipDominec opened this issue Feb 12, 2021 · 16 comments

Comments

@FilipDominec
Copy link

FilipDominec commented Feb 12, 2021 •
edited
Loading

Instrument details

  • Model: Thorlabs CCS200 spectrometer
  • Communication: USB [VID: 0x1313 = 4883, PID 0x8088 = 32904], Thorlabs support suggests it communicates over VISA and that it could work on Linux, too
  • Link to the documentation (if available):

Output of pyvisa-info

Machine Details:
   Platform ID:    Linux-4.15.0-124-generic-x86_64-with-Ubuntu-18.04-bionic
   Processor:      x86_64

Python:
   Implementation: CPython
   Executable:     /usr/bin/python3
   Version:        3.6.9
   Compiler:       GCC 8.4.0
   Bits:           64bit
   Build:          Oct  8 2020 12:12:24 (#default)
   Unicode:        UCS4

PyVISA Version: 1.11.3

Backends:
   ivi:
      Version: 1.11.3 (bundled with PyVISA)
      Binary library: Not found
   py:
      Version: 0.5.2
      ASRL INSTR: Available via PySerial (3.5)
      USB INSTR: Available via PyUSB (1.1.1). Backend: libusb1
      USB RAW: Available via PyUSB (1.1.1). Backend: libusb1
      TCPIP INSTR: Available 
      TCPIP SOCKET: Available 
      GPIB INSTR:
         Please install linux-gpib (Linux) or gpib-ctypes (Windows, Linux) to use this resource type. Note that installing gpib-ctypes will give you access to a broader range of funcionality.
         No module named 'gpib'

What I tried first

import pyvisa
pyvisa.util.get_debug_info()
pyvisa.log_to_screen()

rm = pyvisa.ResourceManager('@py')
print(rm)
    # gives: Resource Manager of Visa Library at py

res = rm.list_resources('?*::?*')
print(res)
    # when CCS200 spectrometer is connected, this gives: ('USB0::4883::32904::None::0::RAW',) 

rm.list_resources_info(res[0])
    # gives: {'USB0::4883::32904::None::0::RAW': ResourceInfo(interface_type=<InterfaceType.usb: 7>, interface_board_number=0, resource_class='RAW', resource_name='USB0::4883::32904::None::0::RAW', alias=None)}

ccs200 = rm.open_resource(res[0])

And this fails with the following message, as if the spectrometer returned no information about its serial number, in/out endpoints etc.:

...
2021-02-12 10:16:43,947 - pyvisa - DEBUG - Created library wrapper for py
2021-02-12 10:16:43,948 - pyvisa - DEBUG - Created ResourceManager with session 1285874
Resource Manager of Visa Library at py
('USB0::4883::32904::None::0::RAW',)
2021-02-12 10:16:43,975 - pyvisa - DEBUG - USB0::4883::32904::None::0::RAW - opening ...
Traceback (most recent call last):
  File "./test_visa_raw.py", line 20, in <module>
    ccs200 = rm.open_resource(res[0])
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa/highlevel.py", line 3304, in open_resource
    res.open(access_mode, open_timeout)
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa/resources/resource.py", line 298, in open
    self._resource_name, access_mode, open_timeout
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa/highlevel.py", line 3232, in open_bare_resource
    return self.visalib.open(self.session, resource_name, access_mode, open_timeout)
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa_py/highlevel.py", line 167, in open
    sess = cls(session, resource_name, parsed, open_timeout)
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa_py/sessions.py", line 323, in __init__
    self.after_parsing()
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa_py/usb.py", line 84, in after_parsing
    self.parsed.serial_number,
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa_py/protocols/usbraw.py", line 37, in __init__
    super(USBRawDevice, self).__init__(vendor, product, serial_number, **kwargs)
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa_py/protocols/usbtmc.py", line 195, in __init__
    self.find_devices(vendor, product, serial_number, None, **device_filters)
  File "/home/dominecf/.local/lib/python3.6/site-packages/usb/core.py", line 1285, in device_iter
    if _interop._all(tests) and (custom_match is None or custom_match(d)):
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa_py/protocols/usbutil.py", line 209, in cm
    if not fnmatch(value.lower(), pattern.lower()):
AttributeError: 'NoneType' object has no attribute 'lower'
2021-02-12 10:16:44,043 - pyvisa - DEBUG - Closing ResourceManager (session: 1285874)

What I tried second

I experimentally edited the pyvisa_py/protocols/usbutil.py so that it does not crash trying to lowercase None. Then the initialization continued further, until it failed with "USBRAW device must have both Bulk-In and Bulk-out endpoints". So the error is deeper than just missing serial number.

What I would like to do when we get VISA communication working

A somewhat bulky bundle of different GUIs and interfaces can be downloaded free from Thorlabs (select "Software -> Full Installer")

The ThorlabsOSASW_Full_setup.exe installer can be run under Linux+Wine and before it fails with some error, it unpacks useful files, e.g.:

... Program Files/IVI Foundation/VISA/Win64/include/TLCCS.h
... Program Files/IVI Foundation/VISA/Win64/TLCCS/TLCCS.c

which seem to contain C++implementation of communication.

But without VISA initialization, I cannot start trying to implement any pythonic interface.
@MatthieuDartiailh
Copy link
Member

I gave a quick look at the Thorlabs page and did not see anything related to VISA support. The software tab suggests that there is a Communications Protocol manual that you could use to figure out how to communicate with the instrument over USB without going through Thorlabs software. If you can get this manual, you could try to directly use pyusb to communicate, however I do not think that pyvisa will help you since the instrument does not seem to rely on VISA.

@FilipDominec
Copy link
Author

FilipDominec commented Feb 12, 2021 •
edited
Loading

@MatthieuDartiailh , thank you for your fast response. Focusing on physics, I am rather a newbie to instrumentation.

  1. In the _/Feedback\_ tab, response from December 2017, Thorlabs responds: "The crucial point will be the implementation of the USB connection in Linux. In Windows this is done by using the VISA interface. VISA-like interfaces for Linux (pyvisa-py, LibreVISA) are available."

  2. If I am to implement (a subset of) the TLCCS.c interface, I am probably going to use VISA likewise. It is 670 lines of readable code. But on line 29 it refers to "NI LabWindows/CVI 2012SP1".

Maybe pyvisa is the way to go, maybe libusb is it, maybe both, maybe CCS200 won't work on Linux ever. Searching internet, it seems CCS200+Linux is something new. This is motivating. I would greatly appreciate any hint.

@MatthieuDartiailh
Copy link
Member

Interesting I missed those comments. I would be curious to see the C example Thorlabs is referring at, could you share them with me by email ? The USB interface is not the most stable part of PyVISA-py. So to avoid wasting time I would try to go through a more battle tested VISA implementation first. If you have a Windows machine it should be easy, otherwise you can install for example Keysight VISA on Linux (it is restricted to some kernels though). If you manage to communicate through those but PyVISA-py fails we will be able to try to fix it.

@FilipDominec
Copy link
Author

FilipDominec commented Feb 12, 2021 via email

@FilipDominec
Copy link
Author

FilipDominec commented Feb 12, 2021 via email •
edited
Loading

@FilipDominec
Copy link
Author

And this is how the spectrometer reports in lsusb.

lsusb-v_for_Thorlabs.txt

@MatthieuDartiailh
Copy link
Member

It will take me some time to go through that. I will try to to come back to you before next Wednesday. Going quickly over the TLCCS.c file it looks like communication is going mostly through usb_bulk_in and usb_bulk_out so VISA may help but not much. Also note that RAW USB resources are not part of the VISA standard only of the NI implementation, so using Keysight may not work.

From pyvisa-py side it seems the first things to figure out is why we do not read the serial number.

@FilipDominec
Copy link
Author

I will be here to give you information regarding the hardware.

Meanwhile I am downloading Keysight VISA right now, but installing it might take a lot of prior learning for me, so let's wait until you get some suggestions.

Thank you a lot for your fast reaction!

@MatthieuDartiailh
Copy link
Member

You may want yo have a look at the following resources about configuration of USB RAW resources from NI (https://zone.ni.com/reference/en-XX/help/370131S-01/ni-visa/configuringni-visarecognizerawusbdevice/). Ideally I would like to get the serial number read properly since without that we are unlikely to go anywhere. If you are on an Arch system you could also try https://aur.archlinux.org/packages/ni-visa/.

Note however that this thread suggests that the way NI handles RAW resources is old and may not be well supported on modern distros.

@FilipDominec
Copy link
Author

Matthieu, do you think then it would be cleaner to write a pyUSB driver from scratch?

It could be integrated e.g. into https://github.com/ap--/python-seabreeze, which also contain an example of similar code. But I would have to find or wiretap the binary commands over USB, which is not going to be easy even for such a simple instrument.

@MatthieuDartiailh
Copy link
Member

Honestly I have no idea at this point. I find the idea that the NI protocol may not be properly supported on modern distros a bit worrying. For wiretrapping you may want to have a look at WireShark.

@CompThing
Copy link

Hi Filip, couple of suggestions. I might not be able to help much as it is over a year since I worked with VISA with Linux and Python.
Your output from lsusb with no detail on device classes hints that the device implementation is minimal. Have you tried running command as root, as without setting up appropriate group you can hit permissions issues reading from USB?
When I was working with a USB VISA instrument with Linux on a Raspberry Pi, I had some better results using usbtmc. That might have changed as there have been enhancements to pyVISA-py since then. The Thor Labs instrument you use seems to take material directly from IVI Foundation. You could look here for some ideas: https://www.ivifoundation.org/default.aspx

@FilipDominec
Copy link
Author

@CompThing 1. Yes, to make sure, I ran all tests also as root. And I tested the usual tricks with /etc/udev/rules.d/...

  1. I was somewhat confused by the diversity of all protocols, interfaces and communication layers, so I maybe gave up too early and did not give https://github.com/python-ivi/python-usbtmc a try. Maybe this is the way to go, but I will wait: Meanwhile I asked Thorlabs directly if they could send me some Python+Linux related documentation. They promised to discuss the conditions and get back to me in few days.

@MatthieuDartiailh If all of the above fails, and if I found enought free time, Wireshark+USBPcap will be the last resort. But this can be really tedious.

@FilipDominec
Copy link
Author

Thorlabs support dpt. is friendly, but their answer was discouraging. In short, what I understood so far:

  1. They cannot send me detailed documentation of the protocol, because it would have to be written first (?).
  2. The official driver explicitly depends on NI-VISA implementation of e.g. USB Control Transfer. I don't know anything about its being available among other VISA implementations. For a hypothetical new Linux driver, Thorlabs suggested the messages could be constructed from scratch using libusb and correctly warned me this can be challenging.
  3. The spectrometer loads its firmware upon each connecting to the Thorlabs application. So any rudimentary spectral measurement would require prior binary firmware upload and repeated enumeration after that. Good news is that the firmware seems publicly available in the big package from the official website (namely, it unpacks into C:\\Program Files\Thorlabs\Thorlabs OSA\CCS\inf\). This approach is somewhat similar to cheap flatbed scanners, BTW.

I will have to think about it. If I was to develop a proof-of-concept Linux driver for CCS200, I would have to learn a lot. Currently there are more urgent tasks at my work to be done.

@heath-smith
Copy link

@FilipDominec I realize this topic is over a year old.. did you ever get any more information on this or make anymore attempts at this? I am a Thorlabs employee, currently developing a QC system in which we need the CCS200 on a Raspberry Pi. I think a Linux implementation of the CCS200 would be a great help to many others and I would be happy to work with you towards a solution. I have already reached out to some key folks in tech support to obtain any current info on this topic as well as the Communication Protocol manual.

@FilipDominec
Copy link
Author

FilipDominec commented Nov 23, 2022 •
edited
Loading

@heath-smith Unfortunately, I did not find enough time to proceed with CCS200, which is ocassionally used as handheld device by our PhD student and instead I am currently constructing a new high-performance spectrometer with echelle configuration already tested on my early prototype. Also the rp2daq project will hopefully soon get a TCD1304 CCD interface for simple grating spectrometers. In fact, on my workbench I have several spectrometer designs in different stages of development; still it would be certainly very interesting to access the CCS200 from Python.

If Thorlabs changes their mind and release some docs on communication with CCS200, I will be happy to become a beta-tester!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@FilipDominec @MatthieuDartiailh @CompThing @heath-smith