Staging Environments

[shangyian]

It is sometimes necessary to stage a series of changes together and see the effects prior to saving them. This is where staging environments come in and we can allow users to stage a set of changes together, explore the effects, and then save these changes.

In order to support this, we'll want to add environment as an attribute on a NodeRevision, where the value is either primary, referring to the shared, finalized environment that everyone sees after saving a set of changes, or the user's username, referring to their private environment where they can stage changes as they see fit.

Sequence of events:

• Users will enter their staging environment via a toggle in the UI
• They can specify a node or set of nodes that they're modifying
• Everything downstream of those nodes become aliases
• Users can bring in new nodes if they want (and we'll bring in these nodes' downstreams as aliases)
• Only the node's display name, description, query, can be modified (i.e., no tag changes are allowed)
• Every time a change is made, we create a new NodeRevision with environment set to the staging env. The node always has a pointer to the latest staging revision
• No tag changes in staging
• Detect merge conflicts when merging (warning, can overwrite)

Diagrams

Children

We start off with one environment, Primary, the main environment that everyone shares. There are four nodes in this environment, source S, transform T, dimension D, and metric M, with M depending on T and T depending on S and linked to D.

graph TD;
    subgraph Primary
      subgraph S
        ro1(S-v1) -.-> ro(S-v2)
      end
      subgraph T
        ro2(T-v1) -.-> ro3(T-v2)
      end
      ro2 --> S
      ro3 --> S
      subgraph M
        ro4(M-v1) -.-> ro5(M-v2)
      end
      ro4 --> T
      ro5 --> T
      subgraph D
        ro10(D-v1) -.-> ro11(D-v2)
      end
      ro3 -.-> D
    end

Loading

Modifying node with children but no parents

Let's consider what would happen if a user wants to make changes to node S in their own staging environment User1. We would create a new node revision S-v3 for the modified S. We would bring in all of S's children as new node revisions T-v3 and M-v3. These children may have status changes based on modifications to S. The dimension link from T-v3 to D (in the primary env) is also brought in.

graph TD;
    subgraph Primary
      subgraph S
        ro1(S-v1) -.-> ro(S-v2)
      end
      subgraph T
        ro2(T-v1) -.-> ro3(T-v2)
      end
      ro2 --> S
      ro3 --> S
      subgraph M
        ro4(M-v1) -.-> ro5(M-v2)
      end
      ro4 --> T
      ro5 --> T
      subgraph D
        ro10(D-v1) -.-> ro11(D-v2)
      end
      ro3 -.-> D
    end

    subgraph User1
      subgraph S-User1
        ro6(S-v3)
        ro -.-> ro6
      end
      subgraph T-User1
        ro7(T-v3)
        ro3 -.-> ro7
      end
      ro7 --> S-User1
      ro7 -.-> D
      subgraph M-User1
        ro8(M-v3)
        ro9(M-v4)
        ro5 -.-> ro8
        ro8 -.-> ro9
      end
      ro8 --> T-User1
      ro9 --> T-User1
    end

Loading

Modifying node with children and parents

Let's consider what would happen if a user wants to make changes to node T in their own staging environment User1. We would create a new node revision T-v3 for the modified T, which would point to its parent node S outside of the staging env. We would bring in all of T's children into the staging env as new node revisions (M-v3). The dimension link from T-v3 to D (in the primary env) is also brought in.

graph TD;
    subgraph Primary
      subgraph S
        ro1(S-v1) -.-> ro(S-v2)
      end
      subgraph T
        ro2(T-v1) -.-> ro3(T-v2)
      end
      ro2 --> S
      ro3 --> S
      subgraph M
        ro4(M-v1) -.-> ro5(M-v2)
      end
      ro4 --> T
      ro5 --> T
      subgraph D
        ro10(D-v1) -.-> ro11(D-v2)
      end
      ro3 -.-> D
    end

    subgraph User1
      subgraph T-User1
        ro7(T-v3)
        ro3 -.-> ro7
      end
      ro7 --> S
      ro7 -.-> D
      subgraph M-User1
        ro8(M-v3)
        ro9(M-v4)
        ro5 -.-> ro8
        ro8 -.-> ro9
      end
      ro8 --> T-User1
      ro9 --> T-User1
    end

Loading

[agorajek]

@shangyian this is fantastic. My only concern was the potential overlap between this functionality and the node mode (draft, published), but once @samredai proposed to keep it separate and add the additional mode for nodes: draft --> published --> deprecated all of it starts making sense.

Separately, we should also keep in mind that this triple mode would be a good feature to have for columns, even if we don't do any actions based on the column mode.