Add websockets to provide server notifications. (#668)

Websockets operate on a subscription level. Client sends a subscribe command to the server and then the server will start pushing notifications.
decred · Jan 16, 2019 · 268d6c8 · 268d6c8
1 parent 793ab1b
commit 268d6c8
Show file tree

Hide file tree

Showing 9 changed files with 757 additions and 67 deletions.
diff --git a/go.mod b/go.mod
@@ -25,6 +25,7 @@ require (
 	github.com/gorilla/mux v1.6.2
 	github.com/gorilla/schema v1.0.2
 	github.com/gorilla/sessions v1.1.3
+	github.com/gorilla/websocket v1.2.0
 	github.com/h2non/go-is-svg v0.0.0-20160927212452-35e8c4b0612c
 	github.com/jessevdk/go-flags v1.4.0
 	github.com/jrick/logrotate v1.0.0

diff --git a/politeiawww/api/v1/api.md b/politeiawww/api/v1/api.md
@@ -3,8 +3,9 @@
 # v1
 
 This document describes the REST API provided by a `politeiawww` server.  The
-`politeiawww` server is the web server backend and it interacts with a JSON REST
-API.  It does not render HTML.
+`politeiawww` server is the web server backend and it interacts with a JSON
+REST API.  This document also describes websockets for server side
+notifications.  It does not render HTML.
 
 **Methods**
 
@@ -115,11 +116,22 @@ API.  It does not render HTML.
 - [`PropStatusPublic`](#PropStatusPublic)
 - [`PropStatusAbandoned`](#PropStatusAbandoned)
 
+**Websockets**
+
+See [`Websocket command flow`](#Websocket-command-flow) for a generic
+description of websocket command flow.
+
+- [`WSError`](#WSError)
+- [`WSHeader`](#WSHeader)
+- [`WSPing`](#WSPing)
+- [`WSSubscribe`](#WSSubscribe)
+
 ## HTTP status codes and errors
 
 All methods, unless otherwise specified, shall return `200 OK` when successful,
-`400 Bad Request` when an error has occurred due to user input, or `500 Internal Server Error`
-when an unexpected server error has occurred. The format of errors is as follows:
+`400 Bad Request` when an error has occurred due to user input, or `500
+Internal Server Error` when an unexpected server error has occurred. The format
+of errors is as follows:
 
 **`4xx` errors**
 
@@ -134,6 +146,77 @@ when an unexpected server error has occurred. The format of errors is as follows
 |-|-|-|
 | errorcode | number | An error code that can be used to track down the internal server error that occurred; it should be reported to Politeia administrators. |
 
+## Websocket command flow
+
+There are two distinct websockets routes. There is an unauthenticated route and
+an authenticated route.  The authenticated route provides access to all
+unprivileged websocket commands and therefore a client that authenticates
+itself via the [`Login`](#login) call should close any open unprivileged
+websockets.  Note that sending notifications to unauthenticated users means
+**ALL** unauthenticated users; this may be expensive and should be used
+carefully.
+
+All commands consist of two JSON structures. All commands are prefixed by a
+[`WSHeader`](#WSHeader) structure that identifies the command that follows.
+This is done to prevent decoding JSON multiple times.  That structure also
+contains a convenience field called **ID** which can be set by the client in
+order to identify prior sent commands.
+
+If a client command fails the server shall return a [`WSError`](#WSError)
+structure, prefixed by a [`WSHeader`](#WSHeader) structure that contains the
+client side **ID** followed by the error(s) itself.  If there is no failure the
+server does not reply.  Note that **ID** is unused when server notifications
+flow to the client.
+
+Both routes operate exactly the same way. The only difference is denied access
+to subscriptions of privileged notifications.
+
+**Unauthenticated route**: `/v1/ws`
+**Authenticated route**: `/v1/aws`
+
+For example, a subscribe command consists of a [`WSHeader`](#WSHeader)
+structure followed by a [`WSSubscribe`](#WSSubscribe) structure:
+```
+{
+  "command": "subscribe",
+  "id": "1"
+}
+{
+  "rpcs": [
+    "ping"
+  ]
+}
+```
+
+The same example but with an invalid subscription:
+```
+{
+  "command": "subscribe",
+  "id": "1"
+}
+{
+  "rpcs": [
+    "pingo"
+  ]
+}
+```
+
+Since **pingo** is an invalid subscription the server will reply with the
+following error:
+```
+{
+  "command": "error",
+  "id": "1"
+}
+{
+  "command": "subscribe",
+  "id": "1",
+  "errors": [
+    "invalid subscription pingo"
+  ]
+}
+```
+
 ## Methods
 
 ### `Version`
@@ -2547,3 +2630,81 @@ A proposal credit allows the user to submit a new proposal.  Proposal credits ar
 | price | uint64 | The price that the credit was purchased at in atoms. |
 | datepurchased | int64 | A Unix timestamp of the purchase data. |
 | txid | string | The txID of the Decred transaction that paid for this credit. |
+
+## Websocket methods
+
+### `WSHeader`
+| Parameter | Type | Description | Required |
+|-|-|-|-|
+|Command|string|Type of JSON structure that follows the header|yes|
+|ID|string|Client settable ID|no|
+
+**WSHeader** is required as a prefix to every other command on both the client
+and server side.
+
+### `WSError`
+| Parameter | Type | Description | Required |
+|-|-|-|-|
+|Command|string|Type of JSON structure that follows the header|no|
+|ID|string|Client settable ID|no|
+|Errors|array of string|All errors that occurred during execution of the failed command|yes|
+
+**WSError** always flows from server to client.
+
+**example**
+```
+{
+  "command": "error",
+  "id": "1"
+}
+{
+  "command": "subscribe",
+  "id": "1",
+  "errors": [
+    "invalid subscription pingo"
+  ]
+}
+```
+
+### `WSSubscribe`
+| Parameter | Type | Description | Required |
+|-|-|-|-|
+|RPCS|array of string|Subscriptions|yes|
+
+Current valid subscriptions are `ping`.
+
+Sending additional `subscribe` commands will result in the old subscription
+list being overwritten and thus an empty `rpcs` cancels all subscriptions.
+
+**WSSubscribe** always flows from client to server.
+
+**Example**
+```
+{
+  "command": "subscribe",
+  "id": "1"
+}
+{
+  "rpcs": [
+    "ping"
+  ]
+}
+```
+
+
+### `WSPing`
+| Parameter | Type | Description | Required |
+|-|-|-|-|
+|Timestamp|int64|Server timestamp|yes|
+
+**WSPing** always flows from server to client.
+
+**example**
+```
+{
+  "command": "ping"
+}
+{
+  "timestamp": 1547653596
+}
+```
diff --git a/politeiawww/api/v1/v1.go b/politeiawww/api/v1/v1.go
@@ -17,49 +17,51 @@ const (
 	CsrfToken = "X-CSRF-Token"    // CSRF token for replies
 	Forward   = "X-Forwarded-For" // Proxy header
 
-	RouteUserMe                 = "/user/me"
-	RouteNewUser                = "/user/new"
-	RouteVerifyNewUser          = "/user/verify"
-	RouteResendVerification     = "/user/new/resend"
-	RouteUpdateUserKey          = "/user/key"
-	RouteVerifyUpdateUserKey    = "/user/key/verify"
-	RouteChangeUsername         = "/user/username/change"
-	RouteChangePassword         = "/user/password/change"
-	RouteResetPassword          = "/user/password/reset"
-	RouteUserProposals          = "/user/proposals"
-	RouteUserProposalCredits    = "/user/proposals/credits"
-	RouteVerifyUserPayment      = "/user/verifypayment"
-	RouteUserPaymentsRescan     = "/user/payments/rescan"
-	RouteUserDetails            = "/user/{userid:[0-9a-zA-Z-]{36}}"
-	RouteManageUser             = "/user/manage"
-	RouteEditUser               = "/user/edit"
-	RouteUsers                  = "/users"
-	RouteLogin                  = "/login"
-	RouteLogout                 = "/logout"
-	RouteSecret                 = "/secret"
-	RouteProposalPaywallDetails = "/proposals/paywall"
-	RouteProposalPaywallPayment = "/proposals/paywallpayment"
-	RouteAllVetted              = "/proposals/vetted"
-	RouteAllUnvetted            = "/proposals/unvetted"
-	RouteNewProposal            = "/proposals/new"
-	RouteEditProposal           = "/proposals/edit"
-	RouteProposalDetails        = "/proposals/{token:[A-z0-9]{64}}"
-	RouteSetProposalStatus      = "/proposals/{token:[A-z0-9]{64}}/status"
-	RoutePolicy                 = "/policy"
-	RouteVersion                = "/version"
-	RouteNewComment             = "/comments/new"
-	RouteLikeComment            = "/comments/like"
-	RouteCensorComment          = "/comments/censor"
-	RouteCommentsGet            = "/proposals/{token:[A-z0-9]{64}}/comments"
-	RouteAuthorizeVote          = "/proposals/authorizevote"
-	RouteStartVote              = "/proposals/startvote"
-	RouteActiveVote             = "/proposals/activevote" // XXX rename to ActiveVotes
-	RouteCastVotes              = "/proposals/castvotes"
-	RouteUserCommentsLikes      = "/user/proposals/{token:[A-z0-9]{64}}/commentslikes"
-	RouteVoteResults            = "/proposals/{token:[A-z0-9]{64}}/votes"
-	RouteAllVoteStatus          = "/proposals/votestatus"
-	RouteVoteStatus             = "/proposals/{token:[A-z0-9]{64}}/votestatus"
-	RoutePropsStats             = "/proposals/stats"
+	RouteUserMe                   = "/user/me"
+	RouteNewUser                  = "/user/new"
+	RouteVerifyNewUser            = "/user/verify"
+	RouteResendVerification       = "/user/new/resend"
+	RouteUpdateUserKey            = "/user/key"
+	RouteVerifyUpdateUserKey      = "/user/key/verify"
+	RouteChangeUsername           = "/user/username/change"
+	RouteChangePassword           = "/user/password/change"
+	RouteResetPassword            = "/user/password/reset"
+	RouteUserProposals            = "/user/proposals"
+	RouteUserProposalCredits      = "/user/proposals/credits"
+	RouteVerifyUserPayment        = "/user/verifypayment"
+	RouteUserPaymentsRescan       = "/user/payments/rescan"
+	RouteUserDetails              = "/user/{userid:[0-9a-zA-Z-]{36}}"
+	RouteManageUser               = "/user/manage"
+	RouteEditUser                 = "/user/edit"
+	RouteUsers                    = "/users"
+	RouteLogin                    = "/login"
+	RouteLogout                   = "/logout"
+	RouteSecret                   = "/secret"
+	RouteProposalPaywallDetails   = "/proposals/paywall"
+	RouteProposalPaywallPayment   = "/proposals/paywallpayment"
+	RouteAllVetted                = "/proposals/vetted"
+	RouteAllUnvetted              = "/proposals/unvetted"
+	RouteNewProposal              = "/proposals/new"
+	RouteEditProposal             = "/proposals/edit"
+	RouteProposalDetails          = "/proposals/{token:[A-z0-9]{64}}"
+	RouteSetProposalStatus        = "/proposals/{token:[A-z0-9]{64}}/status"
+	RoutePolicy                   = "/policy"
+	RouteVersion                  = "/version"
+	RouteNewComment               = "/comments/new"
+	RouteLikeComment              = "/comments/like"
+	RouteCensorComment            = "/comments/censor"
+	RouteCommentsGet              = "/proposals/{token:[A-z0-9]{64}}/comments"
+	RouteAuthorizeVote            = "/proposals/authorizevote"
+	RouteStartVote                = "/proposals/startvote"
+	RouteActiveVote               = "/proposals/activevote" // XXX rename to ActiveVotes
+	RouteCastVotes                = "/proposals/castvotes"
+	RouteUserCommentsLikes        = "/user/proposals/{token:[A-z0-9]{64}}/commentslikes"
+	RouteVoteResults              = "/proposals/{token:[A-z0-9]{64}}/votes"
+	RouteAllVoteStatus            = "/proposals/votestatus"
+	RouteVoteStatus               = "/proposals/{token:[A-z0-9]{64}}/votestatus"
+	RoutePropsStats               = "/proposals/stats"
+	RouteUnauthenticatedWebSocket = "/ws"
+	RouteAuthenticatedWebSocket   = "/aws"
 
 	// VerificationTokenSize is the size of verification token in bytes
 	VerificationTokenSize = 32
@@ -1106,3 +1108,37 @@ type ProposalsStatsReply struct {
 	NumOfPublic          int `json:"numofpublic"`          // Counting number of public proposals
 	NumOfAbandoned       int `json:"numofabandoned"`       // Counting number of abandoned proposals
 }
+
+// Websocket commands
+const (
+	WSCError     = "error"
+	WSCPing      = "ping"
+	WSCSubscribe = "subscribe"
+)
+
+// WSHeader is required to be sent before any other command. The point is to
+// make decoding easier without too much magic. E.g. a ping command
+// WSHeader<ping>WSPing<timestamp>
+type WSHeader struct {
+	Command string `json:"command"`      // Following command
+	ID      string `json:"id,omitempty"` // Client setable client id
+}
+
+// WSError is a generic websocket error. It returns in ID the client side id
+// and all errors it encountered in Errors.
+type WSError struct {
+	Command string   `json:"command,omitempty"` // Command from client
+	ID      string   `json:"id,omitempty"`      // Client set client id
+	Errors  []string `json:"errors"`            // Errors returned by server
+}
+
+// WSSubscribe is a client side push to tell the server what RPCs it wishes to
+// subscribe to.
+type WSSubscribe struct {
+	RPCS []string `json:"rpcs"` // Commands that the client wants to subscribe to
+}
+
+// WSPing is a server side push to the client to see if it is still alive.
+type WSPing struct {
+	Timestamp int64 `json:"timestamp"` // Server side timestamp
+}
diff --git a/politeiawww/cmd/politeiawwwcli/commands/commands.go b/politeiawww/cmd/politeiawwwcli/commands/commands.go
@@ -53,6 +53,7 @@ type Cmds struct {
 	Secret             SecretCmd             `command:"secret" description:"ping politeiawww"`
 	SetProposalStatus  SetProposalStatusCmd  `command:"setproposalstatus" description:"(admin) set the status of a proposal"`
 	StartVote          StartVoteCmd          `command:"startvote" description:"(admin) start the voting period on a proposal"`
+	Subscribe          Subscribe             `command:"subscribe" description:"subscribe to all websocket commands and do not exit tool."`
 	Tally              TallyCmd              `command:"tally" description:"fetch the vote tally for a proposal"`
 	UpdateUserKey      UpdateUserKeyCmd      `command:"updateuserkey" description:"generate a new identity for the user"`
 	UserDetails        UserDetailsCmd        `command:"userdetails" description:"fetch a user's details by his user id"`

diff --git a/politeiawww/cmd/politeiawwwcli/commands/help.go b/politeiawww/cmd/politeiawwwcli/commands/help.go
@@ -59,6 +59,8 @@ func (cmd *HelpCmd) Execute(args []string) error {
 		fmt.Printf("%s\n", VersionCmdHelpMsg)
 	case "edituser":
 		fmt.Printf("%s\n", EditUserCmdHelpMsg)
+	case "subscribe":
+		fmt.Printf("%s\n", SubscribeCmdHelpMsg)
 	default:
 		fmt.Printf("invalid command\n")
 	}