Adding a single-shot cell-mode for use cases with expensive computation?

[oschulz]

Using Pluto for use cases the include long(er) running computations is currently a bit tricky - esp. for demos/tutorials/etc. with less experienced Julia users or Julia newcomers.

(In the following, I use "above" and "below" in the sense in dependency-order, not notebook layout cell-order.)

If some Pluto cells have run times on the order of minutes (e.g. statistical inference, machine learning training, etc.), and those (typicall few) cells depends on several input/configuration/parameter values "above", then the cell will be triggered again and again while users fiddle with those "inputs". This can make their system really slow, drain their battery, or (when run on large systems that use accounting) come with resource costs in terms of compute credits.

We've had the ability to disable cells temporarily for a while now, it's very handy sometimes. But it doesn't fully address the "a few very expensive cells" scenario - one has to manually disable and re-enable the right cells again and again, and while they are disabled everything "below" is grayed out.

Could add "single-shot" cells to Pluto? They would behave similar to cells that immediately disable themselved after they've been run. But in contrast to actually disabled cells, the content "below" would not be grayed out immediately - not until is becomes invalid - i.e. until the "one-shot" cell should be run again because the values "above" have changed. When that happens, the "one-shot" cell could be marked red, get a big "play" button overlay, or something like that, and everything "below" would be grayed-out - clearly showing that what's "below" is not consistent with what's "above" anymore.

I think this would be in keeping with Pluto philosphy - this is not about adding Jupyter-like cells, i's more like the "single-trigger" mode on an oscilloscope (we already have "continuous trigger" and "trigger off", so to speak).

[oschulz]

This would also help a lot with for use cases like #958 (physical control and DAQ systems).

[Pangoraw]

You can wrap the content of an expensive cell in a PlutoHooks.@use_memo() call which depends only on a single variable which can be toggled to rerun the cell.

using PlutoUI, PlutoHooks

@bind toggle CheckBox() # toggle to rerun
@bind z Slider(1:10) # parameter that should not rerun the cell


result = PlutoHooks.@use_memo([toggle]) do
  # do expensive computation here
  do_something_with(z)
end

result

This will not disable the cells depending on result but only recompute the expansive result when the value of toggle changes and not when z changes.

[oschulz]

Thanks - yes, that's a possible workaround, I may end up using this in tutorials.

I still think one-shot cells would still be a valuable feature, though, since they would provide more. With @use_memo, there's no indication that dependent content is outdated, and there's no prompting that that it's time to re-run the expensive computation now. It's also not very intuitive.

[SimonLafran]

I have a macro that puts a warning on a one-shot cell when "above" dependencies are changed, and it does not require checkboxes. It also does not execute the first time it is run, to avoid auto-running when opening a notebook. However, it does not mark below cells as being outdated.

const firstrunregister::Set{UUID} = Set()

"""
Runs `f` only when the cell is explicitly run, but warning it is stale when
`deps` is different from the last run.

Does not run `f` the first explicit run, to avoid being triggered by the
auto-run when opening a notebook.

Useful for expensive computations.
"""
macro use_oneshot(f, deps)
	quote
		deps = $(esc(deps))

		# used to check
		initial_deps = @use_ref(deps)

		# if the dependencies are different from the last explicit run,
		# mark as stale
		if initial_deps[] != nothing && initial_deps[] != deps
			@warn "Stale"
		end
	
		cell_id = @PlutoHooks.give_me_the_pluto_cell_id()

		# this only runs when explicitly run, i.e. on opening the file or
		# clicking the run button, not when a dependency changes.
		@use_memo([]) do 
			# if this cell was already run one time
			if cell_id in firstrunregister
				($(esc(f)))() 
			else
				push!(firstrunregister, cell_id)

				# An unrunned cell has no dependencies
				# Avoids being marked as stale
				initial_deps[] = nothing
				@warn "Not run"
			end
		end
	end
end

If you prefer to have the auto-run when opening the notebook:

"""

Runs `f` only when the cell is explicitly run, but warning it is stale when
`deps` is different from the last run.

Useful for expensive computations.
"""
macro use_oneshot(f, deps)
	quote
		deps = $(esc(deps))

		# used to check
		initial_deps = @use_ref(deps)

		# if the dependencies are different from the last explicit run,
		# mark as stale
		if initial_deps[] != deps
			@warn "Stale"
		end

		# this only runs when explicitly run, i.e. on opening the file or
		# clicking the run button, not when a dependency changes.
		@use_memo($(esc(f)), [])
	end
end