-
Notifications
You must be signed in to change notification settings - Fork 122
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SANS GUI: new user file format #18826
Comments
Currently it is possible to save the table, but users want to be able to save all of the information from all tabs in the GUI, including the table, into a single file. |
We should look at this together with SANS, who would like something similar - they currently load settings from user files, but can't save state from the GUI. They also want a better format, e.g. json or yaml. There are some notes on this in the Users Workshop minutes (see the the User files and File storage sections) |
Discussion with @Mantid-Matthew has suggested that SANS are keen to use the same file format as Reflectometry. Saving to a file from the SANS GUI is not however a high priority item. So it is likely that Reflectometry will be the first to implement this. The SANS scientists would like to be kept in the loop and involved in discussions over which format is used. Matthew has identified the following additional requirements from the SANS scientists which may or may not be requirements for reflectometry as well:
Bearing these in mind we considered the following formats to start. XML<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<root>
<foo>bar</foo>
<numbers><!-- A comment -->
<number>1</number>
<number>2</number>
<number>3</number>
</numbers>
<subobject some_value="true">
<subsubobject some_other_value="false"></subsubobject>
</subobject>
</root>
JSON{
"foo" : "bar",
"numbers": [ 1, 2, 3 ],
"subobject": {
"some_value": true,
"subsubobject": {
"other_value": false
}
}
}
YAML---
foo: "bar"
numbers: # A comment
- 1
- 2
- 3
subobject:
some_value: true
subsubobject:
other_value: false
...
TOMLfoo = "bar"
numbers = [ 1, 2, 3 ] # A comment
[subobject]
some_value = true
[subobject.subsubobject]
other-value = false
Based on the above, my personal recommendation would be to use TOML, subject to further investigation and discussion with the instrument scientists. As an additional note, it is possible that when implementing this we could consider adding a solution to the feature:
from #21167. |
A little historical perspective from the SANS side... So the way we reduce data in Mantid from a GUI essentially emulates how we used to do things in our previous (VMS based) data reduction programe 'COLETTE' (the name being a corruption of COrrect Laboratoire d'Electronique des Technologies de l'Information, which would have been COLETI; COLETTE just seemed nicer and more memorable. We'd probably get into trouble for it these days!!! LETI made the first LOQ detector). COLETTE actually sat on top of VMS GENIE (the forerunner to OpenGenie used in SECI), which it used for data i/o and plotting. But because GENIE was written by the crystallographers/spectroscopists it knew nothing about 2D detectors, so we (principally Ray Osborn, Richard Heenan & myself) had to create new routines and command sets to augment GENIE to reduce LOQ data. At the same time that gave us the flexibility to employ the VMS screen management library to create what, today, people would recognise as a GUI. But the only human input device back then was a keyboard, so we needed an efficient way of setting up all the reduction parameters. That became a file of COLETTE/GENIE commands we called a 'Mask File'. And it worked very well. So when Mantid came along we had the same concept implemented. The only change has been that for technical reasons we now call the Mask File a 'User File'. But the bottom-line is this is a concept that has been around 30+ years and which our user community understand and are comfortable with. Essentially the User File is a SANS reduction configuration file for Mantid. I'll attach a LOQ example and a SANS2D example. As you'll see, there are many similarities, but also many differences. And now we have LARMOR and ZOOM too. So the time has definitely come to impart some structure on these files! But they absolutely, most definitely, have to be human-readable, and editable in nothing more sophisticated than Notepad (if really needed; I take the point that a language-sensitive editor like, say, Notepad++ might convey many advantages). The User Files are generally created on an experiment-by-experiment basis (usually by copying the last one, renaming it, and making some subtle changes), and any single experiment may have several, depending on the sophistication of the reduction. We give the User Files to the users with their data. Equally, many users will copy/edit their own versions. So LOADING these files into the GUI is critical. Creating a User File by SAVING the state of the GUI is not something which, by its absence, we have ever found to be a problem (after all, how hard is it to copy a file, rename it, and edit it!), but it would be slightly less laborious than the current need to keep reloading an edited User File when you are testing changes. So we would welcome such a feature. It's just not a high priority for SANS. |
In terms of a new format for User Files, I had been wondering about something not unlike a a Windows .INI file. But maybe that is too basic? We use XML for one of our output data formats, and it presents quite nicely, but it is 'wordy'. I have come across JSON and am not a big fan. I don't find the syntax/structure very user-friendly. YAML I've heard of but have no experience. TOML I've never heard of! Does someone want to take one of the User Files I just uploaded and TOML-ise it as a discussion example? |
Re-worded to avoid confusion with project save/load which the linked issue was not about: Re-opened as a SANS issue as this contains relevant information. The SANS requirement is mainly to replace the existing file format which is used for populating the GUI settings from a User File. We should check if creating/updating a User File from the GUI is required (issue #3058 originally requested to do that but I believe it is a low priority compared to loading). |
In the case of ISIS SANS much of the information populating the GUI is, of course, loaded in from the User File, which NR do not (yet) have. So, yes, moving the User File format to TOML is the greater priority for SANS. |
I've reworded my previous comment as it's about save/load of a User File rather than project save/load. Here's an attempt to summarise the differences, and where we're up to: Project save/load/recovery User File save/load Saving is less straightforward due to the requirement to preserve user-friendly formatting and comments. I don't think in the existing SANS workflow we would expect to create a new User File from scratch using the GUI? It sounds like saving might be most useful as a way to update an existing User File to avoid having to re-load it after making changes. If that's the case, maybe a simple Refresh button would be ok, or watching the file for changes so that it automatically updates? This differs from ISIS Reflectometry workflow where the GUI is the starting point. We have implemented functionality to save the entered information to a "batch" file which contains all of the runs and settings for a specific reduction. This a bit like the SANS User File and Batch file combined, but is not intended to be easily user-readable - we will effectively use the GUI like an editor to load and update it. So, we sort of do have a User File, but it's used in a different way. |
The majority of this implementation is done, however mapping the TOML syntax to Sans State objects is revealing some unimplemented features / assumptions I'm having to run past This has lead to concerns from myself and discussed with the instrument scientists about how we might end up releasing something "half-baked". So I'm pushing this off for 5.1 to go into the nightly fairly early on in development |
Closing because the new user file is now implemented. There are a couple of issues left in this epic but they are more follow on tasks so I don't think we need the epic any more. |
This issue is to make it possible to save/load all the information in the GUI to a file.
The text was updated successfully, but these errors were encountered: