we should be able to annotate a state after starting it

[ggggggggg]

Some ideas:

1. Replace experiment_state.txt with a sql_lite database and add some JSON rpc calls to annotate an old state or current state.
2. Extend the experiment_state.txt format to have some comma (or other delimeter) separated fields and add some JSON rpc calls that will edit an older state by writing out a new file, then moving it over the current file.
3. Somehow get dastard to auto populate some spreadsheet that can be edited with a normal spreadsheet editor.

[joefowler]

What kind of "annotations" do you anticipate?

You know my feelings: that a SQLite database is a great tool, when the need is worth the investment. It is super-flexible and capable of later extension into new functions not yet anticipated; readers and writers exist in a hundred different languages; GUI apps to browse such a DB also exist (e.g., SQLiteBrowser); it's possible to browse a DB in a web browser, too, for example via sql.js.

On the other hand, it would come with a non-trivial startup cost. We'd have to learn to use the golang SQLite package (see database/sql) and the Python one (it's in the standard library as sqlite3). In that event, perhaps your hacky but simple solution 2 would be preferred.

Your idea 3 sounds fragile and unpleasant.

[joefowler]

Ping. Is this still a thing? Should we close this? I admit I never quite knew what you meant by "annotate" an experiment state.

[ggggggggg]

This is still a thing. Imagine you're running the EBIT. You change the EBIT settings to 2 kV, 120 mA and injecting Neon. You press new state, and it gets the name "C". Right now we right down "11:22pm C - 2 kV, 120 mA, Neon" in the lab notebook. I'd like to be able to put that info into a gui somewhere and have it stored with the data. But I don't expect to be able to, or think it would be a good idea, to try to write that down before I change the state. I want to add it after I've changed the state, and be able to edit it if I accidentally wrote "11:22pm C - 2 kV, 120 mA, Krypton" to switch Krypton to Neon.

I think it requires Dastard support since DastardCommander could be on a a different machine, so it can't necessarily write to the same disk location.

[joefowler]

By the principle of minimum change, suppose we support a new kind of entry, an alias:

# unix time in nanoseconds, state label
1650479536151600398, START
1650479719837262446, A
1650481560096165344, B
1650481887344987074, C
ALIAS 2 kV, 120 mA, Neon

An alias always refers to the state most recently started, I guess. (What if 2+ aliases were given?) We'd have to teach MASS to use (or at worst, ignore) the aliases. Maybe "description" is a better name than "alias".

Another approach that might be even simpler is to add an optional second file *_experiment_state_alias.txt with lines like C = 2 kV, 120 mA, Neon. Then it is trivial to have MASS ignore a missing alias file.

I guess before we solve the problem, we have to answer the question of how you want to use these after the fact. Do you want them to be mere notes for later manual tracking, or do you want to use the following two lines in MASS as synonyms?

ds.cuts(state="C")
ds.cuts(state="2 kV, 120 mA, Neon")

If they are synonyms, how will we handle having 2+ states with a single alias (supposing you ran the same voltage, current, and element during states C and X)?

[ggggggggg]

I like the separate file approach, I hadn't considered that. You're later questions about how mass should handle this are completely distinct from this change. We can always change how mass interprets this extra info later without changing how dastard write the extra info. The think I like about a separate file is we can just keep a lot of all the aliases, and have mass ignore all but the last of overlapping ones. Very simple to implement, and you don't lose any information... in case the first alias was actually right!

Two files like experiment_state

# unix time in nanoseconds, state label
1650479536151600398, START
1650479719837262446, A
1650481560096165344, B
1650481887344987074, C

experiment_state_annotations

# unix time in nanoseconds, annotation
1650479719837262446, 2kV, 120 mA, Neon
1650481560096165344, 3 kV, 120 mA, Krypton
1650481560096165344, 3 kV, 120 mA, Neon
1650481887344987074, 2 kV, 120 mA, Krypton

Could represent a run with states A,B,C, where state B had wrong info entered intially.

I would really like if we could support a tabular interface for entering and editing this information. Or somehow support opening this file with excel or an excel like program and having new lines show up automatically, but be able to edit the old lines. But that seems like a lot more work than the minimum feature we're talking about here.