Redesign of Project Lifecycle

[sammachin]

Background:

The original project lifecycle was created in the early days of FlowForge before we had considerations such as Devices, Snapshots and Billing.
In addition Node-RED 3.0 has introduced a new capability to run with the flows stopped but the editor available, similar to safe mode but a user needs to explicitly exit this mode when they want to run rather than just starting up on the next deploy.
We have had feedback from users that they find the current terminology of Stopped & Suspended to be confusing around what is available and if they would be charged.

I've been looking at the lifecycle of a project and the states it can be in, along with what this should mean for various features.

Proposal

Starting out with a User Centric approach I've identified the following states

My Project is RUNNING ACTIVE, It will process requests, I can make changes to the flows or configuration and I can deploy those changes, I expect to be charged for this time.

My Project is PAUSED, It does not process requests, however I am able to make changes to the flow or configuration and deploy those changes, I can move from this state to RUNNING quickly. I am still charged for this time.
PAUSED has been replaced with a separate Flow State

My Project is STOPED SUSPENDED, It does not process requests, I am unable to make changes to the flows but I can edit the configuration. Moving the project into a ACTIVE state will take a minute or so to spin up.

My Project is ARCHIVED, It does not process requests, I cannot make any changes, I can only view the logs and config. I can create a new project based on a previous snapshot in the archive. The only state this can move to is DELETED
(This is a new concept that could be an additional feature offered in a future release)

My Project is DELETED, There is nothing. I cannot move from this state.

In terms of the features available to a user in each of those states these are described below.

	ACTIVE	SUSPENDED	ARCHIVED	DELETED
Container Running	Y	N	N	N
Editor Running	Y	N	N	N
Flows Running	Y	N	N	N
Devices Running**	Y	N	N	N
Change Stack	Y	Y	N	N
Change Type	Y	Y	N	N
Export Project	Y	Y	Y	N
View logs	Y	Y	Y	N
Edit Settings	Y	Y	N	N
Create Snapshot	Y	N	N	N
Delete Snapshot	Y	Y	N	N
Rollback Snapshot	Y	Y	N	N
Set Target Snapshot	Y	N	N	N
Add Device	Y	N	N	N
Remove Device	Y	Y	N	N
Set Target Snapshot	Y	N	N	N
Delete Project	Y	Y	Y	N
Customer is Billed	Y	N	N***	N

** If a device is unable to communicate to the Forge app it may continue to run until it next connects
*** Archived state may only be available to teams on a specific tier

[knolleary]

The main issue I can see is the mismatch in terminology between FlowForge and Node-RED.

Within Node-RED we now allow the user to start and stop their flows - there is no paused terminology.

With this proposal, we are saying a Paused Project actually means Node-RED running with stopped flows.

Within Node-RED, the user has an option to Stop Flows which actually means Pause Project.... (side note - need to think about how the platform gets notified the user has stopped/started their flows in the editor.... we don't have a hook for that).

We have had feedback from users that they find the current terminology of Stopped & Suspended to be confusing around what is available and if they would be charged.

Given we have never exposed 'Suspended' as an external concept, I'm not sure we have properly validated that feedback. There is certainly cause for confusion in the UI we have today around Start/Stop - but that reflects the limits of what states we could map them to (the state of the NR process).

[sammachin]

After further discussion with Nick here is an amended proposal, it still keeps the same user experience but we present it differently.

Remove the PAUSED Project State (see flows state below)
RUNNING Becomes ACTIVE (terminology change)
STOPPED Becomes SUSPENDED (terminology change)

Introduce a separate concept to the platform of the state of the flows within the projects Node-RED container.
FLOWS STOPPED is the previous PAUSED
FLOWS RUNNING is a new state.

This new flow state would also simplify the question around Node-RED 2.0 stacks as it would only be availible for NR3.0 based projects, a legacy stack would only be able to be RUNNING or STOPPED at the project level.

There is no change to the current safe mode experience where node-red is put into safe mode following repeated restarts.

There is some UX work to be done around how this would be presented to users with the separate controls,

I'll map this over to an updated figma state diagram

The original comment has been updated to reflect these changes in the table and descriptions

[sammachin]

New proposed lifecycle,
https://www.figma.com/file/UG1UurahXUhU7ZgQQgOlUD/Project-Lifecycle---Updated-Aug-2022?node-id=0%3A1

[joepavitt]

For now, I agree with the proposal of Active/Suspended/Archived/Deleted.

However, I do still have a lot of issues around our terminology of state, particularly with regards to devices, deployments and project state.

In an ideal world, I would like to see:

• Project: Node-RED Editor with flows. State of the "Project" is the primary state and overrules below states.
• Deployments: Given a project, I have a single view of my "deployments". I can manage my deployment of the flows to FF Cloud, additionally, I can manage my deployments to Devices. There is then a deployment "state" on a deployment-by-deployment basis.

Complications:

• What does the "Deploy" button in the Node-RED Editor then do?
• Can we customise what that language says? We could have the "Start"/"Stop" state within the editor more explicitly tied to the FF Cloud instance. Then, we need to be able to monitor/control this state from outside of the Editor in FF directly too - in order to create a consistent "Deployments" view.

[Steve-Mcl]

@sammachin how does stopping / starting node-red fit into this?

Example - user updates contrib node & needs to stop/start node-red for the updated module to be reloaded.

[sammachin]

@Steve-Mcl we still have the restart in actions, just the stop.start thats removed.
Restart is just a stop/start under the hood but the user isn't able to leave their node-red in a stopped state now with the container still running.

[joepavitt]

Some illustration for how I could see us using "Deployments" to better communicate/manage devices and cloud/local-based instances of the project running.

We would also need someway to control/set the target snapshot for the devices here too, this is a quick placeholder, but think we could come up with something cleaner:

[hardillb]

This is to replace the project/devices view?

Looks good.

I think we need to explore the concept of snapshots version on the cloud side, but may be out of scope for this.

[joepavitt]

This is to replace the project/devices view?

That's what I'm thinking. We have a lot of confusion around resource consumption (and potentially billing) with the split of deployment in the "Editor" view (for Cloud/local) and then Devices too, and I don't think it's clear at the moment that the "Devices" page for a Project actually controls deployments/pushing of flows to those devices. This is my attempt at making that more explicit.

I think we need to explore the concept of snapshots version on the cloud side, but may be out of scope for this.

Good to know this is on the radar, I've intentionally missed that out for now, but my thinking behind the two table approach was that you could have multiple rows in the top table too, where a snapshot could be a given "deployment"

[sammachin]

I like this, One thing that I've found a bit confusing in the current UI is that the Devices Table is very similar between a Project and Team view. This fixes that.

Couple of minor points:
In the Cloud Deployments we have a Last Deployed I assume this is when deploy was clicked in the editor? do we have that info in the platform? How does this relate to the container being restarted?

Device Deployments says Deployed Snapshot maybe change this to Running Snapshot ? At the moment all devices in a project should be running the same snapshot provided they are communicating to the platform,

In terms of terminology on Cloud/Local I agree its tricky,
I think Local is bad, especially as for the vast majority of users it won't be on the machine that is local to the browser they are accessing it via.
Cloud is good if a little vague,
Other possibles are:
Containers but that might be too technical,
Server - Although in theory a device is a server,
Hosted - too vague?
Platform - As in its on the FlowForge platform??

[hardillb]

In the Cloud Deployments we have a Last Deployed I assume this is when deploy was clicked in the editor? do we have that info in the platform? How does this relate to the container being restarted?

There is a last updated date field on the flows database table, but it's not currently exposed via any API

Device Deployments says Deployed Snapshot maybe change this to Running Snapshot ? At the moment all devices in a project should be running the same snapshot provided they are communicating to the platform,

But it should show the last known deployed snapshot, so if it's offline it should show the old snapshot (and I think we should support having a canary device for testing on...)

[joepavitt]

There is a last updated date field on the flows database table, but it's not currently exposed via any API

This is me "blue sky" thinking to a certain extent, not considered what we do/don't expose via API now, but what I think a user would want/expect to see. But, yes, would be last time the editor was "Deployed"

Hosted - too vague?

I'm not too against this

[sammachin]

@knolleary @joepavitt I think the work around deployments is now separated out into separate stories so this epic has run its course, ok to close it off?

[knolleary]

I had a great discussion with Joe around the deployment concept this morning as I had missed these comments at the time due to holiday. I'm up to speed with the proposal and I can certainly see the value in moving forward with it.

In terms of this specific issue, I agree I think it has run its originally intended course to refine the project lifecycle model and enabled us to introduce the Suspended state properly.

Deployments may lead to further refinements to the lifecycle, but rather than keep this perpetually open, lets close this off. The focus on Deployments will come in #1046 and I'll add some follow on comments there.