Async Dashboard Approaches

Comprehensive Guide: Understanding the async implementations in randomwalk

Related Pages:

How Async Dashboard Uses Crew and Targets - Detailed crew patterns
Dynamic Grid Broadcasting Algorithm - Technical deep dive

Overview

The randomwalk package provides multiple execution modes for simulation:

Mode	`workers`	`sync_mode`	Grid Updates	Use Case
Vectorized Batched	0 (inline)	n/a	After each step	WebR dashboard (RECOMMENDED)
Synchronous	0	n/a	After each walker	Native R baseline
Static Async	1+	"static"	None during sim	Production, simplicity
Chunked	1+	"chunked"	Between batches	Native R (RECOMMENDED)
Dynamic	1+	"dynamic"	⚠️ DEPRECATED	N/A

Approach 1: Synchronous (workers=0)

How It Works

All walkers execute sequentially in the main R process. Each walker sees the grid updated by all previous walkers.

result <- randomwalk::run_simulation(
  grid_size = 100,
  n_walkers = 50,
  workers = 0  # Synchronous mode
)

Key Characteristics

WebR/Shinylive compatible: Yes - runs entirely in browser
Grid updates: Real-time (each walker sees previous paths)
Performance: Sequential, but no overhead
Use case: Browser dashboards, baseline comparisons

Approach 2: Crew-Only Async (Static Snapshots)

How It Works

Each worker receives a static copy of the grid at simulation start. Workers execute independently without communication:

Main Process
├─ Create empty grid
├─ Start crew controller
├─ Push walker tasks (each with grid snapshot)
│
│  Worker 1: [Grid copy] → Walker 1 → Walker 3 → Walker 5
│  Worker 2: [Grid copy] → Walker 2 → Walker 4 → Walker 6
│
├─ Wait for all workers
├─ Collect results
├─ Aggregate walker paths onto final grid
└─ Return combined result

Code Example

result <- randomwalk::run_simulation(
  grid_size = 100,
  n_walkers = 50,
  workers = 4,           # Use 4 crew workers
  sync_mode = "static"   # Default: static grid snapshots
)

When Walkers "See" Each Other's Paths

They don't - each walker operates on the initial grid state:

# Walker 1 starts at (50, 50), creates path...
# Walker 2 starts at (30, 30), also uses ORIGINAL grid
# Walker 2 does NOT see Walker 1's path during simulation
# Paths are merged AFTER all walkers complete

Advantages

Simple mental model
No synchronization overhead
Reproducible results
Works with any crew backend

Limitations

Walkers cannot react to each other's paths
Grid state is "stale" during execution
Not WebR compatible (crew requires multiple processes)

Approach 3: Chunked Mode (RECOMMENDED)

How It Works

Processes walkers in batches of 10, with grid synchronization between batches:

Batch 1: Walkers 1-10 execute in parallel
         ↓ Grid updated with all new black pixels
Batch 2: Walkers 11-20 execute (see batch 1 results)
         ↓ Grid updated
Batch 3: ...

Code Example

result <- randomwalk::run_simulation(
  grid_size = 100,
  n_walkers = 50,
  workers = 4,
  sync_mode = "chunked"  # RECOMMENDED
)

Advantages

~3x more black pixels than static mode (walkers see previous batches)
Better collision detection
No socket complexity
Good balance of parallelism and interaction

Why Recommended

The chunked mode provides a good balance:

Walkers in later batches see earlier batches' paths
No complex socket management
Works reliably (no deprecated dependencies)

Approach 4: Nanonext Broadcasting (DEPRECATED)

⚠️ DEPRECATED: Both sync_mode = "dynamic" and sync_mode = "mirai_dynamic" are deprecated because nanonext sockets fail in crew/mirai subprocesses. Use "chunked" instead.

What It Was Designed For

Workers would communicate black pixel updates in near real-time using nanonext publish/subscribe sockets.

Why It's Deprecated

# From R/simulation.R:
if (sync_mode %in% c("dynamic", "mirai_dynamic")) {
  logger::log_warn("sync_mode is DEPRECATED: nanonext sockets fail in crew/mirai subprocesses")
  logger::log_warn("Use sync_mode='chunked' for ~3x more black pixels (RECOMMENDED)")
}

Technical reason: nanonext socket handles cannot be passed to crew/mirai subprocess contexts. The sockets become invalid in child processes.

Message Count (For Reference)

When it worked, the message count was:

O(N) where N = walkers that create black pixels
NOT "10-20 messages" as previously documented
Each walker broadcasts at most ONCE (when it terminates adjacent to a black pixel)

Approach 5: Vectorized Batched (WebR Dashboard)

Added April 2026. This is the current architecture of the browser dashboard at dashboard_comprehensive.qmd.

How It Works

The simulation is fully inlined in the Shinylive app — it does not call run_simulation() from the package. All walkers are stored as matrices and moved simultaneously using vectorized R operations.

R (Web Worker)                    JS (Main Thread)
──────────────                    ────────────────
Process batch (100 steps)
  pos_mat[idx,] += DR[dirs]       (browser idle)
  grid[cbind(r,c)] == 1L?
  padded neighbor check
  ↓
sendCustomMessage("batch_done") → receives message
  (Worker yields)                  setTimeout(50ms)
                                   browser renders, handles clicks
                                   ↓
                                   setInputValue("next_batch") → observeEvent fires
                                                                  Process next batch

Key Design Decisions

Vectorized state: pos_mat (N×2 matrix), steps_vec, active_vec, reason_vec — no per-walker list objects
Padded grid: (gs+2) × (gs+2) matrix with zero border — eliminates all bounds checking in neighbor lookups
JS round-trip batching: invalidateLater() does NOT yield the WebR Web Worker thread. Only sendCustomMessage → JS setTimeout → setInputValue forces a true browser yield.
Adaptive batch size: max(5, 200000 / n_active) — large sims get fewer steps per batch
Inline fallbacks: initialize_grid(), generate_walker_positions(), plot_grid_enhanced() are defined inside the app when the randomwalk package is unavailable

Performance

Grid	Walkers	Per-walker loop	Vectorized	Speedup
100×100	200	~53s	~35s	~1.5x
200×200	5000	~12 hours	~minutes	~10-50x

Limitations

No walker path storage (trade-off for speed)
proc.time() doesn't advance in WASM (can't use time-based batching)
downloadHandler doesn't work in Shinylive (no filesystem)
Walker Paths tab removed (no paths in vectorized mode)

Code Location

All vectorized simulation code is in vignettes/articles/dashboard_comprehensive.qmd inside the {shinylive-r} block — not in the R/ package source.

Comparison Table

Aspect	Vectorized (WebR)	Sync (workers=0)	Static	Chunked	Dynamic
Status	✅ WebR DEFAULT	✅ Active	✅ Active	✅ Native R	⚠️ DEPRECATED
WebR compatible	✅ Yes	✅ Yes	❌ No	❌ No	❌ No
Walker interaction	Full	Full	None	Between batches	N/A
Black pixels	Baseline	Baseline	Few	~3x more	N/A
Speed (WebR)	10-50x faster	Baseline	N/A	N/A	N/A
UI responsive	✅ Yes (JS round-trip)	❌ Blocks	N/A	N/A	N/A
Dependencies	None (inline)	randomwalk pkg	crew	crew	crew + nanonext

Choosing the Right Approach

Decision Flowchart

Running in WebR/Shinylive?
│
├─ Yes → Vectorized batched engine (inline, no package dependency)
│        JS round-trip for non-blocking UI
│
└─ No → Need parallel processing?
         │
         ├─ No → run_simulation(workers = 0)
         │
         └─ Yes → Use sync_mode="chunked" (RECOMMENDED)

Quick Reference

Scenario	Recommended
Browser dashboard (WebR/Shinylive)	Vectorized inline engine (default in dashboard)
Programmatic native R	`run_simulation(workers = 0)`
Production batch processing	`run_simulation(workers = 4, sync_mode = "static")`
Interactive native R exploration	`run_simulation(workers = 4, sync_mode = "chunked")`

Examples

Example 1: WebR/Shinylive Dashboard

# Sync mode - works in browser
result <- randomwalk::run_simulation(
  grid_size = 100,
  n_walkers = 50,
  workers = 0  # Required for WebR
)

Example 2: Production Batch Processing

# Static async - fast, predictable
result <- randomwalk::run_simulation(
  grid_size = 100,
  n_walkers = 100,
  workers = 4,
  sync_mode = "static"
)

Example 3: Interactive Research (RECOMMENDED)

# Chunked mode - best balance
result <- randomwalk::run_simulation(
  grid_size = 200,
  n_walkers = 50,
  workers = 4,
  sync_mode = "chunked"  # Walkers see previous batches
)

References

crew package: https://wlandau.github.io/crew/
targets package: https://docs.ropensci.org/targets/
randomwalk source: R/simulation.R

Async Dashboard Approaches

Async Dashboard Approaches

Table of Contents

Overview

Approach 1: Synchronous (workers=0)

How It Works

Key Characteristics

Approach 2: Crew-Only Async (Static Snapshots)

How It Works

Code Example

When Walkers "See" Each Other's Paths

Advantages

Limitations

Approach 3: Chunked Mode (RECOMMENDED)

How It Works

Code Example

Advantages

Why Recommended

Approach 4: Nanonext Broadcasting (DEPRECATED)

What It Was Designed For

Why It's Deprecated

Message Count (For Reference)

Approach 5: Vectorized Batched (WebR Dashboard)

How It Works

Key Design Decisions

Performance

Limitations

Code Location

Comparison Table

Choosing the Right Approach

Decision Flowchart

Quick Reference

Examples

Example 1: WebR/Shinylive Dashboard

Example 2: Production Batch Processing

Example 3: Interactive Research (RECOMMENDED)

See Also

References

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally