[discussion] InstrumentKit #407
Replies: 56 comments
-
|
Hey @scasagrande, thanks for mentioning InstrumentKit! We do indeed have a very similar design, which I think is a great sign. I know that there are also a number of other similar projects offering instrument communication. I think these (and PyMeasure) arise from a lack of a central project leading the way, so many scientists wind up reinventing the wheel. I think we could all benefit from a central package that covers instrument control, which would allow PyMeasure to focus on experiment execution and GUI display, instead of the instrument drivers. What do you think about this idea? |
Beta Was this translation helpful? Give feedback.
-
|
I think that is a fantastic idea. Its actually one that I've tried to advocate to people in the past! I've had people ask me if I would consider including GUI display elements into IK, but I just never felt that it was a good mix to have both tied together in the same software package. I'd want to be able to develop, push, and publish them independently of each other. Regarding your comment on "reinventing the wheel" constantly, that's actually why @cgranade and I started IK a few years ago; we got tired of watching our labmates constantly write communication code again and again. Since then I've tried to raise awareness of the project, as I meet relevant parties IRL. Let's keep this conversation going, I like what I've read so far! |
Beta Was this translation helpful? Give feedback.
-
|
Yea, both @grahamrow and I had written essentially the same package for instrument control and that prompted us to combine them into the PyMeasure package. I think a separation of GUI and instrument drivers is a good idea for maintenance -- much more of a Unix style. I will plan to do a deep dive of InstrumentKit to see the comparison. @scasagrande, how flexible is your API at this point? I think the way to make this work is by putting some effort into marketing, since people are reinventing this because they can't find the existing solutions. I think its critical to have some nice landing pages that are not the docs, which show off the features in a modern style. I think a blog that details feature updates is also a great idea, which GitHub can facilitate with Jekyll. Are you open to doing some marketing? There are other nice packages such as Lantz that could be included as well in this discussion. @hgrecco, do you have any thoughts on how to go about putting together a common package for instrument control? |
Beta Was this translation helpful? Give feedback.
-
|
I suppose it depends on what you mean by API, be it the external or internal developer ones. But hey, I'm always open to ideas and suggestions. Re marketing I'm on board. I've been trying to get the word out, but you're right, I haven't made a website for the project (I'd lean more towards Pelican rather than Jekyll in keeping with the Python theme). I applied to speak at SciPy con and Pycon Canada this year, but sadly was rejected for both. I do bring the project up a lot at my local Python group. A few of us have also been considering writing a paper for the project. |
Beta Was this translation helpful? Give feedback.
-
|
Lantz development seems to have largely stopped, sadly. Interestingly, neither pymeasure nor instrumentkit have hit my radar until now! Both look exciting - on the one hand: Procedures! GUI! On the other hand: Units! On both hands: non-dead development! 😀 |
Beta Was this translation helpful? Give feedback.
-
|
Additional thought/suggestion: Maybe it would make sense to combine both your packages into Lantz - it already has units support (pint, created by @hgrecco afaik), but is lacking "Procedures" (which pymeasure offers), and has, in my experience, better marketing/visibility. Also, merging/porting the drivers would also create a bigger driver library. I don't know if the whole thing would work API- or design-wise, though. |
Beta Was this translation helpful? Give feedback.
-
|
Lantz is something that I've looked at in the past. They use pyvisa under the hood for everything, which was the exact thing I was trying to get away from. Their project doesn't seem to be designed to easily accommodate other connection libraries. |
Beta Was this translation helpful? Give feedback.
-
|
By way of agreement with @scasagrande, one of the major goals of InstrumentKit was to be flexible with respect to communication layers. We didn't want to mandate the use of VISA as that's a very large dependency and as that makes a lot of assumptions about instrument connectivity that aren't appropriate for all users. For instance, we've had users successfully deploy IK on Raspberry Pi devices. That same flexibility also makes it easier for users to move instruments from USB+GPIB to Ethernet connectivity as their laboratory needs change. I think these advantages are somewhat unique to our approach, and have been important for our users. Moving forward, I think it's worth thinking about how to realize these usecases in any efforts which build off of IK. |
Beta Was this translation helpful? Give feedback.
-
|
Hi there! The Lantz author here. It is a project very close to my heart that started a long, long time ago. Python 3 was brand new, the ecosystems was still building up and there was not too much out there. It fulfilled its goal and helped me and others control simple and complex experiments using Python. Like many open source projects, it's development was stopped for non-technical reasons (lack of time, other stuff to do, etc). I am really happy that other alternatives have appeared since then. At the end most of us want a vibrant and active scientific python community. Regarding PyVISA, I am moving the development of PyVISA and its related project (pyvisa-py and pyvisa-sim) to the pyvisa organization and giving commit access to other people. |
Beta Was this translation helpful? Give feedback.
-
|
It seems to me that many of these projects are trying to build up collections of instrument specific drivers based on scpi commands. However given the great number of instruments and use cases that might not be a very practical goal.
What might be more useful would be some sort of parser which could extract the scpi commands from the instrument docs and provide a command tree. Perhaps after that the ability to create object models to match instrument capabilities.
…Sent from my iPhone
On May 30, 2017, at 7:41 PM, Chris Granade ***@***.***> wrote:
By way of agreement with @scasagrande, one of the major goals of InstrumentKit was to be flexible with respect to communication layers. We didn't want to mandate the use of VISA as that's a very large dependency and as that makes a lot of assumptions about instrument connectivity that aren't appropriate for all users. For instance, we've had users successfully deploy IK on Raspberry Pi devices. That same flexibility also makes it easier for users to move instruments from USB+GPIB to Ethernet connectivity as their laboratory needs change. I think these advantages are somewhat unique to our approach, and have been important for our users. Moving forward, I think it's worth thinking about how to realize these usecases in any efforts which build off of IK.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
|
Beta Was this translation helpful? Give feedback.
-
|
@mvddf, I agree with your idea for a SCPI parser. I actually started writing a YAML-based parser for this purpose a few years ago, but never pursued it that far. The main goal of PyMeasure is to make it really easy to set up repeatable experiments with live-plotting and nice GUIs. That said, the instrument portion of the package has been largely for convenience. As I discussed with @scasagrande at the beginning of this thread, I think it would be a great idea to join InstrumentKit, Lantz and PyMeasure.instruments into a common package. What do people think about putting in the work to join the packages? Any suggestions for moving forward with a common solution? |
Beta Was this translation helpful? Give feedback.
-
|
I have been programming automation for instruments for about 35 years in many different languages and it seems there have been many ideas that have come and gone over that time. Labview manages to persist mainly because it makes it quick to get going with any instrument. However its not a great language like python.
Most of my time is spent in copy-pasting scpi commands from user guides into code. Then there are always quirks and customizations, but I think they mostlty conform to scpi.
The remainder of time is spent making classes or routines to fit the object model of the device. Then the actual measurement problem can start.
One weakness of python in this space is the time required to build up ui things as compared to labview and thats where it becomes a hard sell in industry.
…Sent from my iPhone
On May 30, 2017, at 9:31 PM, Colin Jermain ***@***.***> wrote:
@mvddf, I agree with your idea for a SCPI parser. I actually started writing a YAML-based parser for this purpose a few years ago, but never pursued it that far.
The main goal of PyMeasure is to make it really easy to set up repeatable experiments with live-plotting and nice GUIs. That said, the instrument portion of the package has been largely for convenience. As I discussed with @scasagrande at the beginning of this thread, I think it would be a great idea to join InstrumentKit, Lantz and PyMeasure.instruments into a common package.
What do people think about putting in the work to join the packages? Any suggestions for moving forward with a common solution?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
Beta Was this translation helpful? Give feedback.
-
|
Hey @hgrecco! I wish I chose Pint way back when I first started added units into IK. Back then Pint was pretty new, so I rolled the dice and went with a different project. Switching over is going to be a pain, but its one that I want to do. @mvddf My work hasn't just been about copy pasting scpi strings. Those are by far the easiest! As I'm sure you know, more modern instruments that actually follow scpi have a lot of overlap between them. This is why I have generic scpi instrument classes. A lot of the work though has gone into all the other instruments: the ones without documentation, older instruments, or ones with complex packet schemes. These are the instruments that our contributors have spent most of their time on. @cjermain What I would imagine is something like this:
|
Beta Was this translation helpful? Give feedback.
-
@cgranade If I am not misreading what you say, I think python-ivi will be an interesting/related project:
@mvddf what do you think, in your experience, about IVI or python-ivi? It seems like a great idea which could probably help with the abstraction you guys want to achieve? I don't have a deep understanding of that space, admittedly. edit: OK, this does not seem to be the case.
This sounds great! The division that @scasagrande suggests along the lines of GUI/presentation layer (maybe add data i/o) vs. Procedures vs. Instruments sounds reasonable. In fact, I think even having a unified/flexible instrument definitions library that absorbs instruments from elsewhere (pymeasure, lantz,...) and that other projects can easily connect to would be of great value -- so far nearly every automation project I stumbled over had its own small set of instruments defined, using their own standards, which is not a great situation and leads to lots of duplication. From my experience in OSS projects and maintaining such a community I think it's important to unify under a common organisation (like @hgrecco tried to do with https://github.com/LabPy/ - maybe that would already be a good banner to unite under). This way it's more probable that a critical mass of maintainers is achieved to keep the ball rolling even if some devs drop out - many of the projects I have encountered seem to have started as someone's thesis, or as the specific solution for a lab/group, but have subsequently withered away when people moved on. Nowadays when I encounter an OSS project, community activity is one of the most important parameters when choosing if I will use the project. |
Beta Was this translation helpful? Give feedback.
-
|
also, pinging @alexforencich (python-ivi dev) who migth be interested in this thread/conversation. |
Beta Was this translation helpful? Give feedback.
-
|
I think you also need a hard core of developers with time enough to discuss in depth the different challenges and solutions. For example one discussion that stopped midway was the standardization of the driver interfaces, because to design such an interface one has to consider multiple instruments from different vendors and come up with a minimal common subset that makes sense. This is quite time consuming but also requires in some cases unintuitive decisions such as imposing to always use channel even if the instrument has a single output and accepting this can take some convincing (I know I was on the wrong team !) |
Beta Was this translation helpful? Give feedback.
-
|
I'm just going to keep plugging away on my chosen path :) however with my recent job switch (I am now a product test engineer at an electronics manufacturing company) I just don't have time to organize or drive any of this. However I am going to be migrating a lot of our stuff to use IK so I'll still be working away at new development. |
Beta Was this translation helpful? Give feedback.
-
|
Also you may want also to contact people from https://github.com/QCoDeS/Qcodes |
Beta Was this translation helpful? Give feedback.
-
|
Wow that looks expansive. It's crazy that there are so many different solutions/tools out there! |
Beta Was this translation helpful? Give feedback.
-
|
So, one thing about the labpy/lantz stuff that I noticed: I wanted to help work on the core code a few times, but the metaclass stuff is a bit obscure and as a result figuring out how objects end up getting put together is not very intuitive, and after poking around for a few days I ended up setting it aside and moving on to other things. So the 'activation barrier' for new developers (and probably folks who would contribute drivers that don't require modifying core code) I think was a bit too high. As a result, I think it is a good idea to stick with a simpler paradigm that's easier to understand, debug, and use. Fancy features such as metaclasses are nice and you can do powerful things with them, but this comes with the downside that they're rather subtle and more difficult to get acquainted with and use effectively. |
Beta Was this translation helpful? Give feedback.
-
|
Hi everyone! Thanks for mentioning us. From what I understood skimming through this long discussion, you/we want to unify all the different python-based measurement and instrument control packages into one common framework. Who could argue against the usefulness and awesomeness of that? Having said that, I have to add that although QCoDeS has a lot of developer power, we are currently also very narrowly focused on the goal of getting tourtopological qubit project on rails. To that end, a large part of QCoDeS is even going to be proprietary (which of course no one outside our organization can see). I would personally like to keep the community part of QCoDeS thriving and useful, and I'd like to be a good citizen and contribute back to e.g. PyVISA that we use heavily. But we are currently not at a stage where we can put effort into uniting QCoDeS with other frameworks. |
Beta Was this translation helpful? Give feedback.
-
|
Just going out on a limb here as someone who doesn't have as much stake at this layer of development. Which i get is horrible compared to some of the solutions here. Which is why I'm saying if I had found a community like LabPy then, which had 4-5 wrappers listed. I could have chosen one an stuck to it and contributed instead of hacking my way through. Once we have the list of repos, it is an easier for outsiders to see the packages available, use them and perhaps contribute. By doing this we would perhaps reduce the additional number of these packages floating around with 1-2 users, and see some actual adoption outside the core contributors. @MatthieuDartiailh. Thanks for your contributions to PyVisa and the community. I have really appreciated PyVisa as without it I would probably be fired from trying to implement it myself and taking way too long. |
Beta Was this translation helpful? Give feedback.
-
|
@pazzarpj yeah, collecting the relevant packages in one place might be a useful first step. Maybe another reduced-scope goal could be to unify the different "instrument-driver" layers floating around into one package, optimally with a featureset that is a union of what is available in the different packages (unit support, validators, testability/unit tests,...). That way at least that part of the measurement&control space is unified (GUI and sequences and automation not considered for now), and the "boring" part of writing a library, i.e. implementing drivers, would hopefully be already done. People could then integrate the package into their own thing, which would bring users and hopefuillyl contributions. |
Beta Was this translation helpful? Give feedback.
-
|
@alexforencich (you're the only visible org member) would labpy be open to hosting such an effort, and accepting more contributors/activity/discussion? |
Beta Was this translation helpful? Give feedback.
-
|
@bilderbuchi, no worries. I'm glad to see the enthusiasm and openness for collaboration. For my part, I am no longer in a research role, so I am not actively working on PyMeasure. I am maintaining the package, and have been excited to see a number of user contributions. I'm happy to help facilitate any consolidation from the PyMeasure side of things. A few years ago we got to the discussion stage of consolidating, with the help of @hinnefe2. We put up a group, https://github.com/pyinstruments/pyinstruments, and listed a bunch of the packages. Unfortunately, that was not enough momentum to get us to work together on the same packages. I think the most important thing is to get as many dedicated developers onto the same repo. With critical mass, I think you can attract a much bigger audience. Best of luck! |
Beta Was this translation helpful? Give feedback.
-
|
Happy to see that people are still working on replacing labview with python :) I am also no longer in a research role, but my labmate @pbnjeff89 has taken over our collection of python instrument drivers (which we called labdrivers). For what it's worth, I agree with @cjermain that marketing is the key to consolidation. Labs that have already heavily invested in a particular library are not going to change, but if you can a) capture all the people like me three years ago who are googling 'python instrument drivers' and b) let people know it's possible to do this with python then my guess is you could eventually get a decent community going. Having good looking documentation (eg readthedocs) and plenty of simple examples is probably a good start. |
Beta Was this translation helpful? Give feedback.
-
|
I didn't see a new discussion in https://github.com/LabPy/labpy-discussion/issues so I thought I'd drop by here. I'm very late to this party, but I'd like to offer whatever help I could give in between writing my thesis/doing lab work. |
Beta Was this translation helpful? Give feedback.
-
|
Hi everybody! I only managed to plant a seed for the "less" ambitious goal of unifying the instrument driver layers (and disregarding sequencing/GUI/... for now) at the issue Unify different instrument driver solutions. I think @hinnefe2's point about marketing being the key point to consolidation is spot on. If we manage to come up with a unified instrument driver library that contains much of the functionality floating around in insular packages, that would be a great step towards some sort of critical mass. |
Beta Was this translation helpful? Give feedback.
-
|
Couple of other projects of interest:
I see us using a combination of pymeasure and home-assistant in our lab |
Beta Was this translation helpful? Give feedback.
-
|
This issue has more of a discussion/usage question character, so I will move it to our new discussion forum, which is a better fit than the issue tracker. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Hello pymeasure devs,
I just wanted to take the time to bring a similar project to your attention, InstrumentKit, which I started several years ago with similar goals. Our code architecture actually looks surprisingly similar, so IK shouldn't be too foreign.
A few things that you will notice that IK has are handling of unitful quantities, extensive test coverage, and additional connection interfaces one can use.
I hope that we can learn from each other so that we can continue to help the instrumentation community.
Beta Was this translation helpful? Give feedback.
All reactions