Threading notes

[Mike Stall (MikeStall)]

The threading model for Power Fx hosting is basically: writes are single-threaded, reads can be multi-threaded.

While PowerApps architecture is fundamentally single-process-per-user (and single-threaded), any multi-user host (like PVA, Cards, Dataverse) will want to enable a single engine for multiple users and enable parallelism across users (such as a thread per user).

Practically, this means that:

• creating the config, engine, initial symbol tables, adding variables, etc are done single-threaded.
• using an engine for parsing and getting evaluators can be multi-threaded.
• a single (non-side-effecting) eval can be called many times on multiple threads. Each eval can share an engine and symbols, and gets a per-eval RuntimeConfig.

Regarding running evals in parallel - the Fx infrastructure should be threads-safe, such as the parser, lexer, IR translator, C# interpreter, etc. But the Fx objects themselves are not thread safe. So if 2 evals run, both call patch, and mutate the same object, that is not thread-safe.

For threading issues/PRs, we're filling under the Kind_Threading tag.

The host should have full control over the thread of execution - the engine should never introduce new threads. For example, the engine should never call Task.WhenAll (#954).
This is important because hosts may inject objects (such as a connector http channel or Dataverse IOrganizationService) which have limited lifespans and are only valid on the current thread.

TL;DR

• We have a strong preference for readonly fields and immutable state.
• For new classes, consider adding a [ThreadSafeImmutable] attribute and favoring an immutable pattern.

Design

This principal also extends to class design, such as ReadOnlySymbolTable and ReadOnlySymbolValues. For example, the host can create a SymbolTable (which derives from ReadOnlySymbolTable) and call AddVariable to set it up. Once set up, it can pass the ReadOnlySymbolTable to the binder. Hence, the binder can only read and no longer mutate.

This is a similar pattern in the .Net libraries: Dictionary implements IReadOnlyDictionary. Array implements IReadOnlyList.

Avoid mutable shared state.

If state is shared (such as symbol tables), then make it immutable,

Caches

Caches are conceptually readonly, but are still mutations under the hood, and so they still need protection (such as locks).
This has been the cause for bugs like #1208 and #1511.

Also - for any cache, consider what is the cache lifetime strategy? Does it grow monotonically or does it get invalidated?

Concurrent collections

Dictionary is single-threaded. Using a dictionary across multiple threads requires taking a lock around access. .Net offers concurrent collections, like ConcurrentDictionary, which have the lock built in. However ....

Do not preemptively make something multi-threaded.

Understand if code should be single-threaded or multi-threaded. If single-thread, then use Dictionary.
Also consider, if you need to synchronize across multiple collections, you need to still take a lock.

Enforcement

To help enforce the model above, we have some static analysis.

1. Classes can be marked with attributes to describe their intent. (see https://github.com/microsoft/Power-Fx/tree/main/src/libraries/Microsoft.PowerFx.Core/Annotations). For example ThreadSafeImmutableAttribute marks a class as conceptually immutable (such as ReadOnlySymbolTable). NotThreadSafe marks a class as mutable (such as SymbolTable).
2. We have static analysis tests (https://github.com/microsoft/Power-Fx/blob/main/src/tests/Microsoft.PowerFx.Core.Tests/AnalyzeThreadSafety.cs) that use reflection to enforce these attributes. For example, immutable classes should only have readonly fields of immutable types (or fields that are conceptually readonly and explicitly protected by a lock, such as in caching).

For single-thread region. (like SymbolTable.AddVariable), we have a "Guard" (https://github.com/microsoft/Power-Fx/blob/main/src/libraries/Microsoft.PowerFx.Core/Annotations/GuardSingleThreaded.cs) that will throw when multiple threads try to enter. These are catching hosts bugs. We explicitly do not use a Reader-Writer lock because the goal is not to just make everything multi-threaded. Rather, it is to help the host catch their own bugs and illegal usages.

We also have some stress test that run multiple threaded scenarios to sanity check.