Front-end interface without visibility in core

[nbenn]

Problem

Visibility is a front-end concern, but it sits in core, and demand is split around it:

• The board_server() function owns the required, visible and frozen channels, renders a block only once visible reports it painted, and holds dock's build ledger in the NA/FALSE/TRUE states of visible (Lift block UI build timing into the front-end API blockr.core#349, Let the front-end declare which blocks board_ui paints blockr.core#350).
• Demand is split between the front-end's required channel and the evaluate, sustain and construct payloads (Evaluation demand should be one multi-owner set, not two channels blockr.core#321). Whether a board gates must be known before the first flush, which no payload is.
• Background construction builds dock's view model into core: it holds until every required block is painted (required_fulfilled()), then builds whatever is left.
• Dock builds a block's UI on first visit, after its server has pushed its restored values, so controls off the startup view come up blank (Blocks on a non-startup view lose the restored state of every control fed by update*Input() blockr.core#317).
• Dock ties freezing to hidden inputs and to the lock, so a locked reader's board pins every block (Simplified mode: locked refuses too much to be the reader's board blockr.dock#475), and an open board pins collapsed inputs that the assistant needs to set.

Once solved:

1. Core tracks no visibility. A front-end tells core which blocks may render, which are frozen, and what to build in what order.
2. Per-block state is cheap to add: keyed by block, following the board's blocks as they are added and removed. Core keeps demand in it, dock its visibility.
3. Demand is one owner-keyed set that every owner writes the same way. The front-end's opening demand, and whether the board gates, are known before the first flush.
4. A block's UI exists whenever its server does.
5. Only a locked board freezes, and only the blocks whose inputs the front-end hides. The lock itself refuses board-wide changes.

Proposal

• Evaluation demand is the owner-keyed sustain claim set of Fold front-end demand into the one multi-owner claim set blockr.core#337, with the front-end one owner among several. The front-end declares itself when its callback is registered, together with its opening claim, and core seeds that claim under the callback's owner label before the first flush; a board without such a callback is not gated. The declaration sits with the callback, not the board class, because which callbacks a board was given decides who drives demand. That replaces the synchronous visibility$gate() write and the gate_claimed latch.
• A block renders when it is evaluated and the front-end claims it. The upstream closure and other owners' claims, such as the assistant's, evaluate without rendering. Shiny's own suspension is not enough: it misses outputs hidden with visibility: hidden, which is how dockview hides back tabs.
• Core inserts a block's UI before starting its server, so the server's restore pushes land, and front-ends move that UI rather than re-create it.
• The construct component queues its ids in the order given, and core builds them one per idle tick after what evaluation needs, replacing the background pass and required_fulfilled(). A front-end sends its next request when it is ready: dock asks for the active view, then for the other views once painted.
• Freezing stays a per-block flag core enforces. Dock sets it only on a locked board, for the blocks whose inputs it hides. The lock keeps refusing board updates and option writes at the update gate, and implies no freezing.
• The visibility bundle leaves the callback signature. Demand, construction and freezing go through update, freezing as a component with set/add/rm deltas, and dock keeps paint and its build ledger in its own per-block state.
• Core exports a collection keyed by block id that follows the board's block set, built on the reactives package (Replace trim_rv() and the internal reactives class with the reactives package blockr.core#361), and holds its own per-block state in it.

Work

• Replace trim_rv() and the internal reactives class with the reactives package in blockr.core (Replace trim_rv() and the internal reactives class with the reactives package blockr.core#361)
• Add a per-block state collection that follows the board's blocks to blockr.core (Add a per-block state collection that follows the board's blocks blockr.core#365)
• Unify evaluation demand in the owner-keyed claim set, with the opening claim declared at callback registration, in blockr.core (Evaluation demand should be one multi-owner set, not two channels blockr.core#321)
• Insert a block's UI before its server starts in blockr.core (Blocks on a non-startup view lose the restored state of every control fed by update*Input() blockr.core#317)
• Gate rendering on the front-end's claim instead of visible in blockr.core (Render a block when the front-end claims it, not when visible reports it painted blockr.core#366)
• Queue construct requests in order and pace them, replacing the background pass, in blockr.core (Build construct requests in the order given, paced by core, instead of a background pass blockr.core#367)
• Move freezing into update and drop the visibility bundle from the callback signature in blockr.core (Move freezing into update and drop the visibility bundle from the callback signature blockr.core#368)
• Adopt the claim set, declaring dock's callback as the gating front-end with the active view as its opening claim, in blockr.dock (Adopt core's unified evaluation-demand claim set blockr.dock#417)
• Place the block UI core inserts at construction, instead of building cards on first visit, in blockr.dock (Paint static block UI at block construction, not on first panel visit blockr.dock#438)
• Stop driving core's visible and frozen channels in blockr.dock: request construction in order, freeze through update, and drop the paint report and the build ledger (Stop driving core's visible and frozen channels blockr.dock#486)
• Freeze only on a locked board in blockr.dock (Simplified mode: locked refuses too much to be the reader's board blockr.dock#475)

Notes for implementers

Pointers, blockr.core at 366150a: output_render_observer() and render_gate_observer(), construct_blocks_in_background(), required_fulfilled(), the update gate, insert_block_ui(), is_board_locked(). In blockr.dock at d389603: built_cards(), mark_cards_built() and show_cards(), freeze_hidden_inputs(), defaultRenderer = "always". Core's own front-end, gate_stacks() and the stack accordion, adopts the same interface as dock.

In-flight work: BristolMyersSquibb/blockr.core#337 and BristolMyersSquibb/blockr.dock#420 carry the claim set and move over, with the gate declared at callback registration instead of by writing visibility$gate(). In gate_stacks(), the claim payload's construct of every collapsed block goes, since construct builds all it names in one flush. BristolMyersSquibb/blockr.core#350 and BristolMyersSquibb/blockr.dock#442 are closed as superseded, since the ledger leaves core; whether board_ui() paints the opening screen into the page is left to BristolMyersSquibb/blockr.core#317 and BristolMyersSquibb/blockr.dock#438. BristolMyersSquibb/blockr.dock#438 is the dock half of the UI coming with the server. BristolMyersSquibb/blockr.dock#477 moves dock's view registry onto the same reactives collections. BristolMyersSquibb/blockr.core#341 fixes the same render step: keep a merely hidden block's output, and skip re-assigning an identical value.

Shiny 1.14 counts an output as hidden only if it or an ancestor has display: none (isVisible() in shiny.js), and treats one whose state is not yet reported as hidden (shouldSuspend()). Dockview's always renderer hides a back tab with visibility: hidden on the panel's dv-render-overlay, and a closed Bootstrap .offcanvas is visibility: hidden; display: flex. Measured on blockr.dock d389603 with dockViewR 0.4.0, a back-tab block's output reports .clientdata_output_<id>_hidden as FALSE. The mock session of testServer() renders every output, so render gating stays unit-testable only through core's own gate.

Shiny's client handles server messages one at a time (taskQueue in shiny.js) and awaits an insertUI(), dependencies loaded and inputs bound, before the next. An immediate insert followed by a server's update*Input() pushes therefore arrives in order. The insert has to be immediate: insertUI(immediate = TRUE) writes its message at once, while pushes go out with the flush, and a deferred insert is sent through onFlushed(), after the pushes, which it then misses. Measured with shiny 1.14 in a minimal app, a value pushed in the same flush lands after an immediate insert and is lost after a deferred one. Core's insert_block_ui.board() and dock's card insert already pass immediate = TRUE.

A payload applies at priority = -Inf, after the first flush has decided what to construct (measured in BristolMyersSquibb/blockr.core#337). The update channel is one reactiveVal, so two writes in a flush drop the first, which fold_update() in BristolMyersSquibb/blockr.dock#420 works around. The construct component currently builds every id in the flush that applies it.

Freezing a block before its inputs have reported pins blank values for state held in input$, such as the glue block's text; dock waits for the card's section report for this reason.

Leave room for block code to read demand later (BristolMyersSquibb/blockr.core#240). The exported validate_board_update() generic dispatches on the board, but the update gate calls the internal validators directly, so validate_board_update.dock_board() never runs there; that is BristolMyersSquibb/blockr.core#369.

To check before building: the cost of UI with every server on the largest real board (BristolMyersSquibb/blockr.dock#438 measured 91 blocks at 0.5 s and 274 KB), and that dock moves card UI rather than re-creating it. Land #488 first, since it moves the reason a block cannot run out of output_render_observer().

[nbenn]

One interaction to settle before BristolMyersSquibb/blockr.core#317 and BristolMyersSquibb/blockr.dock#438 meet: dock's insert_block_ui() method does more than place a card. For each block it is handed, it builds the card, opens a new dockview panel with add_block_panel() in a group that determine_panel_pos() picks from the active view's layout, and moves the card in (block-ui.R#L117-L137, utils-dock.R#L71-L80). That fits the one call site core has today, the block-add path (board-server.R#L2194-L2215), where the block is new to the board.

Called for each block before its server starts, the method runs for every block core constructs: at startup, and then through the background pass or, under this proposal, the construct queue. Only a block new to the board wants a new panel. One already in the active view's grid would get a second, and one that belongs to another view would get a panel in the active view. Both an added block and one built for the first time reach the method as a block about to be constructed, with the same arguments, so it cannot tell which of them wants a panel.

Should opening the panel for a new block move out of insert_block_ui(), leaving the method to put the card in the offcanvas?