artiq_master.py GUI regeneration upon modification of experiment file

[ghost]

Here, testrepo is cloned from a local repository I setup on my machine. I can launch artiq_master and see the experiments in artiq_gui (labeled by git commit ID). OK. Baseline working fine.

rabi2@vboxartiq:~/artiq-dev/artiq/examples/master/testrepo$ ls
arguments_demo.py  flopping_f_simulation.py  pdb.pyon             tdr.py
ddb.pyon           handover.py               photon_histogram.py  transport.py
dds_test.py        mandelbrot.py             speed_benchmark.py

Check if artiq_gui experiment is automatically updated if I edit, commit and push arguments_demo.py

rabi2@vboxartiq:~/artiq-dev/artiq/examples/master/testrepo$ ls
arguments_demo.py  flopping_f_simulation.py  photon_histogram.py  transport.py
artiq_gui.pyon     handover.py               results
ddb.pyon           mandelbrot.py             speed_benchmark.py
dds_test.py        pdb.pyon                  tdr.py
rabi2@vboxartiq:~/artiq-dev/artiq/examples/master/testrepo$ vi arguments_demo.py 
rabi2@vboxartiq:~/artiq-dev/artiq/examples/master/testrepo$ cat arguments_demo.py 
from artiq import *


class ArgumentsDemo(EnvExperiment):
    def build(self):
        self.attr_argument("free_value", FreeValue(None))
        self.attr_argument("boolean", BooleanValue(True))
        self.attr_argument("enum", EnumerationValue(
            ["foo", "bar", "quux"], "foo"))
        self.attr_argument("number", NumberValue(42, unit="s", step=0.1))
        self.attr_argument("string", StringValue("Hello World"))
        self.attr_argument("scan", Scannable(global_max=400, default=NoScan(313313)))

    def run(self):
        print(self.free_value)
        print(self.boolean)
        print(self.enum)
        print(self.number)
        print(self.string)
        for i in self.scan:
            print(i)
rabi2@vboxartiq:~/artiq-dev/artiq/examples/master/testrepo$ git add arguments_demo.py 
rabi2@vboxartiq:~/artiq-dev/artiq/examples/master/testrepo$ git commit -m "new ScanNo is 313313"
[master a5f72d6] new ScanNo is 313313
 1 file changed, 1 insertion(+), 1 deletion(-)
rabi2@vboxartiq:~/artiq-dev/artiq/examples/master/testrepo$ git push
No refs in common and none specified; doing nothing.
Perhaps you should specify a branch such as 'master'.
fatal: The remote end hung up unexpectedly
error: failed to push some refs to '/home/rabi2/testrepo'
rabi2@vboxartiq:~/artiq-dev/artiq/examples/master/testrepo$ git push master
fatal: 'master' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
rabi2@vboxartiq:~/artiq-dev/artiq/examples/master/testrepo$ git log|head -n 15
commit a5f72d6dbff428efdb7cc3802fb179ea691f902d
Author: Joe Britton <joe.britton@gmail.com>
Date:   Wed Aug 12 16:28:03 2015 -0600

    new ScanNo is 313313

commit dfd97a9da84a87471a564acb8d2c922fedbd5126
Author: Joe Britton <joe.britton@gmail.com>
Date:   Wed Aug 12 16:24:04 2015 -0600

    edit

commit 0f99eefd7de4f5febc1400dd8caebe3104c5289f
Author: Joe Britton <joe.britton@gmail.com>
Date:   Wed Aug 12 16:12:32 2015 -0600

Expecting gui to use version a5f72d. OK it is so. However, the experiment Log is still reporting that default for NoScan is 666 instead of 313313. It looks like the GUI didn't get regenerated. I was able to add an additional print() in run() method of arguments_demo.py and see that it gets printed in the Log. So it looks like this is just a GUI issue.

I think this is the desired default behavior. We often tweak experiment recipes and want to re-run them using a particular set of run-time parameters that might be non-default attr_arguments(). However, there needs to be a way of forcing the GUI to reload new defaults or to regenerate itself in case, say, a new attr_argument() gets added. This might be a button in the GUI.

Anyway, let's try adding a new attr_arguments().

rabi2@vboxartiq:~/artiq-dev/artiq/examples/master/testrepo$ cat  arguments_demo.py
from artiq import *


class ArgumentsDemo(EnvExperiment):
    def build(self):
        self.attr_argument("free_value", FreeValue(None))
        self.attr_argument("boolean", BooleanValue(True))
        self.attr_argument("enum", EnumerationValue(
            ["foo", "bar", "quux"], "foo"))
        self.attr_argument("number", NumberValue(42, unit="s", step=0.1))
        self.attr_argument("string", StringValue("Hello World"))
        self.attr_argument("scan", Scannable(global_max=400, default=NoScan(313313)))
    self.attr_argument("joe", StringValue("GI JOE"))

    def run(self):
        print("ARTIQ is cool.")
        print(self.free_value)
        print(self.boolean)
        print(self.enum)
        print(self.number)
        print(self.string)
        for i in self.scan:
            print(i)

Start master

rabi2@vboxartiq:~/artiq-dev/artiq/examples/master/testrepo$ artiq_master -g -r .
WARNING:artiq.master.worker:worker finished with status code 1 (RID None)
WARNING:artiq.master.repository:Skipping file 'arguments_demo.py'
Traceback (most recent call last):
  File "/home/rabi2/anaconda3/lib/python3.4/site-packages/artiq/master/repository.py", line 23, in _scan_experiments
    description = yield from worker.examine(os.path.join(wd, f))
  File "/home/rabi2/anaconda3/lib/python3.4/site-packages/artiq/master/worker.py", line 263, in examine
    "file": file}, timeout)
  File "/home/rabi2/anaconda3/lib/python3.4/site-packages/artiq/master/worker.py", line 203, in _worker_action
    completed = yield from self._handle_worker_requests()
  File "/home/rabi2/anaconda3/lib/python3.4/site-packages/artiq/master/worker.py", line 159, in _handle_worker_requests
    obj = yield from self._recv(self.watchdog_time())
  File "/home/rabi2/anaconda3/lib/python3.4/site-packages/artiq/master/worker.py", line 146, in _recv
    raise WorkerError("Worker ended while attempting to receive data")
artiq.master.worker.WorkerError: Worker ended while attempting to receive data

Looks like artiq_master is now not processing arguments_demo.py. Roll back to an earlier commit prior to when arguments_demo.py (and pdb.pyon) were edited.

rabi2@vboxartiq:~/artiq-dev/artiq/examples/master/testrepo$ git log 0f99eefd7de4f5febc1400dd8caebe3104c5289f
commit 0f99eefd7de4f5febc1400dd8caebe3104c5289f
Author: Joe Britton <joe.britton@gmail.com>
Date:   Wed Aug 12 16:12:32 2015 -0600

    add .pyon

commit 33ca948489522e498db41c47e58d442cfe2e89e4
Author: Joe Britton <joe.britton@gmail.com>
Date:   Wed Aug 12 16:00:05 2015 -0600

    updated a file

commit 61267a76425acd557303ed5a293ebb9e3c0fabc8
Author: Joe Britton <joe.britton@gmail.com>
Date:   Wed Aug 12 15:56:40 2015 -0600

    init repo
rabi2@vboxartiq:~/artiq-dev/artiq/examples/master/testrepo$ git checkout 0f99eefd7de4f5febc1400dd8caebe3104c5289f
HEAD is now at 0f99eef... add .pyon

Restart master.

rabi2@vboxartiq:~/artiq-dev/artiq/examples/master/testrepo$ artiq_master -g -r .

Summary:

• In the case that a program edit modifies a GUI element there's presently a bug.
• It's nice to keep GUI-stored values for attr_argument() when only editing the run() method
• If GUI-dependent element is modified, a GUI regeneration seems mandatory. In this case the regenerated GUI should first use last-known value for attr_argument() or if unavailable default value.
• A button is needed to force use of default values for all attr_argument()s

[sbourdeauducq]

You are using it incorrectly. You need to:

• point -r to the root of a bare Git repository specifically created to hold experiments (not the ARTIQ source repository)
• set up the rescan post-receive hook (see ARTIQ docs)
• push to that new bare repository you created (from a non-bare repository elsewhere)

When doing that, GUI elements are properly added and removed on push.

There is no function to remove the default values stored in the GUI state and use those from the experiment instead, is that really needed?

[ghost]

You are using it incorrectly. You need to:

• point -r to the root of a bare Git repository specifically created to hold experiments (not the ARTIQ source repository)
• set up the rescan post-receive hook (see ARTIQ docs)
• push to that new bare repository you created (from a non-bare repository elsewhere)

OK. This is helpful to know. I was indeed using it wrong. It would be
helpful to flesh out the instructions a bit more to emphasize these
points. I'll repeat my git test in the next couple days.

There is no function to remove the default values stored in the GUI state and use those from the experiment instead, is that really needed?

It's rather helpful to be able to force reloading (or not) of default
values. Here's an example. When conducting an experiment to measure
the center of a resonance the sweep range depends on the pi-time t_pi
and last known f0. The default value for the sweep range is then a
function of Ion Property t_pi and last known Ion Property f0. The
sweep range might be manually set to something other than default as a
diagnostic. The sweep range may also reflect outdated values for f0
and t_pi.

It's helpful if the user can choose whether or not to reload (and
recompute) the default sweep range. A checkbox

    self.attr_argument("reload_default_attr", BooleanValue(True), default=False)

would work. But there needs to be a way of telling the GUI about this choice.

[ghost]

Please link to the following page in the ARTIQ documentation. I didn't
appreciate the difference between git init and git init --bare.

http://www.saintsjd.com/2011/01/what-is-a-bare-git-repository/

On Thu, Aug 13, 2015 at 4:51 PM, Joe Britton joe.britton@gmail.com wrote:

You are using it incorrectly. You need to:

• point -r to the root of a bare Git repository specifically created to hold experiments (not the ARTIQ source repository)
• set up the rescan post-receive hook (see ARTIQ docs)
• push to that new bare repository you created (from a non-bare repository elsewhere)

OK. This is helpful to know. I was indeed using it wrong. It would be
helpful to flesh out the instructions a bit more to emphasize these
points. I'll repeat my git test in the next couple days.

There is no function to remove the default values stored in the GUI state and use those from the experiment instead, is that really needed?

It's rather helpful to be able to force reloading (or not) of default
values. Here's an example. When conducting an experiment to measure
the center of a resonance the sweep range depends on the pi-time t_pi
and last known f0. The default value for the sweep range is then a
function of Ion Property t_pi and last known Ion Property f0. The
sweep range might be manually set to something other than default as a
diagnostic. The sweep range may also reflect outdated values for f0
and t_pi.

It's helpful if the user can choose whether or not to reload (and
recompute) the default sweep range. A checkbox

    self.attr_argument("reload_default_attr", BooleanValue(True), default=False)

would work. But there needs to be a way of telling the GUI about this choice.

[ghost]

"The ARTIQ master fetches the last (atomically) completed commit at the
time of experiment submission and checks it out in a new temporary
folder. This solves the problem of concurrent access." Is also helpful for users to know.

[sbourdeauducq]

1. What you are describing is a lot more complicated than a button that clears the GUI state and needs detailed specification (e.g. where is the code that defines the parameter-dependent values of the widgets? when is it executed? is it executed on the GUI or the master? what data does it have access to?). It also has very little to do with tracking modifications of experiment files. Can you open a new issue or start a mailing list thread so that a specification can be established?
2. The documentation assumes an understanding of Git. I would avoid linking to external websites as much as possible, as those links tend to break within a year.
3. I have added that precision to the documentation.

[ghost]

GUI updates are still broken. For example, if I add two Scannables to flopping_f_simulation.py

        self.attr_argument("frequency_scan", Scannable(
            default=LinearScan(1000, 2000, 100)))
        self.attr_argument("johns_scan", Scannable(
            default=LinearScan(1000, 2000, 100)))

To update the GUI I have to stop artiq_master, delete artiq_gui.pyon, restart artiq_master and restart artiq_gui.

[ghost]

My error. GUI updates are no longer broken.

[SBuercklin]

I'm running into a similar issue now where I can't get the GUI to update despite the experiment updating.

I created new master repository, ~/artiq_master/repository, run git init -bare in the repository folder, and add the artiq_client scan-repository --async hook. Then I create new work repo, ~/artiq_work, then git init in ~/artiq_work and create the appropriate remote into the master repo. I add gui.py to the repo, commit it, push to master.

The contents of gui.py:

import logging
from artiq.experiment import *
class VaryingGui(EnvExperiment):
	def build(self):
		self.setattr_argument("a",BooleanValue(True))
		self.setattr_argument("b",BooleanValue(False))

	def run(self):
		print(self.a)
		if self.b:
			print("b true")
		else:
			print("b false")

This gives me a GUI in the dashboard with a defaulting to True, b defaulting to False, prints a and then whether b is True or False

Now I modify the experiment to include a new variable c under a menu and also print the word "updated":

import logging
from artiq.experiment import *
class VaryingGui(EnvExperiment):
	def build(self):
		self.setattr_argument("a",BooleanValue(True))
		self.setattr_argument("b",BooleanValue(False))
		self.setattr_argument("c",BooleanValue(False),"Fast")
	def run(self):
		print(self.a)
		if self.b:
			print("b true")
		else:
			print("b false")
		print("update")

I commit this and push, the dashboard updates. Reopen the experiment, there is no dropdown labeled Fast and no c variable to set, but running the experiment prints "updated" along with everything else.

I now copy the experiment to new file named gui2.py but otherwise everything is unchanged. Add, commit and push, open the new experiment in Dashboard, c is present and located under the expected dropdown menu, and submitting the experiment prints everything as expected while the gui.py experiment GUI remains unchanged.

I made sure that I'm pointing artiq-master to a bare repository, I'm pushing from a separate non-bare repository, and I verified that the post-receive hook is working.

[sbourdeauducq]

Click 'Recompute all arguments' in the panel that appears after you open the experiment. NB, this is a different issue.