Home Page Plugin API

[robertlong]

This PR marks the start of the Hubs Client SDK. Building out a SDK for Hubs is a huge task, so we're starting small and focusing on the ability to customize the home page.

The primary goal of the Hubs Client SDK is to enable customizing the client while still being able to receive automatic updates to your Hubs Cloud server. In order to accomplish this, we are looking into a plugin API. Plugins can be loaded at runtime and do not require rebuilding the client. A webpack compile can take a while and can be extremely resource intensive so we don't want to build on upgrade because that could result in unnecessary downtime. We could add a CI service for rebuilding the client, but we felt that that would be overly complex for what we are trying to accomplish. So instead of rebuilding the client, plugins are built at deploy time and are written against a versioned API.

When experimenting with this idea we quickly came across Project Gutenberg, the new Wordpress editor written in React. Gutenberg has a plugin API with very similar requirements and an elegant solution. Gutenberg uses a series of npm packages that plugins can require and these dependencies are dynamically loaded at runtime. This means that plugins can pull in just the code they need and these dependencies can be shared across plugins. The core of this solution is the @wordpress/dependency-extraction-webpack-plugin package. This package collects all of the dependencies of a specific package or plugin that start with @wordpress/ or are common libraries (react, react-dom, etc.). At package/plugin build time it marks these packages as externals before bundling. Externals are exposed and imported as global variables. So import React from "react"; actually ends up importing from window.React.

For the Hubs Client SDK this solution would work quite well. We have dependencies that should only be loaded when they are needed. If you want to override the home page completely, why should you require all the original home page code? And most importantly, these plugins can be installed with zero downtime and can be upgraded automatically.

Here's what has been done so far:

There are three packages that have been created @hubs/core, @hubs/react, and @hubs/home-page. Right now these are not actually separate packages, they are just aliases in the webpack config. However, you can start to see how they could be broken up into separate packages.

@hubs/core

The core package contains the Hubs singleton. This is an object that contains the store (our interface into localStorage where we store the session and user configuration), config (what features are enabled and other configuration options for your Hubs Cloud server), and the Reticulum API.

There are methods for authentication:

• Hubs.signin(email: string): Promise<void>
• Hubs.verify(params: { topic: string, token: string, payload: string }): Promise<void>
• Hubs.signOut(): Promise<void>
• Hubs.store.isAuthenticated(): boolean
• Hubs.store.isAdmin(): boolean
• Hubs.store.getEmail(): string
• Hubs.store.getAccountId(): string
• Hubs.store.getAuthToken(): string

And for fetching/creating rooms:

• Hubs.createRoom(params?: {}): Promise<RoomResponse>
• Hubs.getPublicRooms(cursor: number): Promise<RoomsResponse>
• Hubs.getFavoriteRooms(cursor: number): Promise<RoomsResponse>

For fetching configuration variables:

• Hubs.config.feature(key: string): any
• Hubs.config.image(key: string, cssUrl?: boolean): string
• Hubs.config.link(key: string, defaultValue?: string): string

And finally for registering plugins themselves:

• Hubs.registerPlugin(hook: string, exports: {})

@hubs/react

The React package is mostly a React utility wrapper around the @hubs/core API. There are hooks and higher order components for working with these APIs in a react context and a few core components for inheriting the base styling for the navigation and page.

• useStore(): { store: Store } lets your UI react to changes in the Hubs store. For example when logging in you may want to rerender the email. Note that store itself doesn't change, but the wrapping object will when the store updates so you can use it as a dependency to another hook.
• usePublicRooms(): { isLoading: boolean, hasMore: boolean, results: Room[], error?: Error, cursor: number, nextCursor?: number, loadMore: () => void } Loads all of the public rooms.
• useFavoriteRooms(): { isLoading: boolean, hasMore: boolean, results: Room[], error?: Error, cursor: number, nextCursor?: number, loadMore: () => void } Loads all of the user's favorite rooms if they are signed in.
• useInstallPWA(): { supported: boolean, installed: boolean, installPWA: () => void } Install the Hubs PWA
• useCreateAndRedirectToRoom(): { creatingRoom: boolean, error?: Error, createRoom: (roomParams: {}) => void

@hubs/home-page

The home page package holds all of the custom react components needed to create the default home page. If you want to completely restyle the homepage you probably would avoid importing this to reduce the page size. However, if you want to use some of the component's we've already created for the default Hubs homepage, you can import them from this package.

• <HomePage /> The default HomePage component. If you wanted to wrap the HomePage component in a higher order component you could do that. For now you can't pass any additional props to HomePage.
• <CreateRoomButton /> A styled button that creates a new room using the createAndRedirectToRoom hook from the React package.
• <PWAButton /> A styled button for installing the Hubs Progressive Web App (PWA). Doesn't render if it's already installed or if the browser doesn't support PWAs.
• <RoomTile room={room} /> Renders a tile for the provided room with the room name, thumbnail, last visited, favorite status, and current occupancy. Clicking on the tile navigates to that room.
• <MediaGrid>{children}</MediaGrid> A responsive CSS grid wrapper component for RoomTiles.
• styles A CSS Module for the home page. Includes styles specific to just the home page.
• discordLogoUrl The url for the small Discord logo icon that we use on the homepage.

Next Steps

This small API should suffice for a wide variety of home page customizations, but there's still likely more that developers want to do on their home page. We want to ship an early version of this API to developers and get feedback from all of you, but first there is some remaining work to be done.

• We need to discuss this API and ensure that it is at least on the right path to what we see for the larger Hubs plugin story.
• This PR needs to be reviewed, there's a lot of refactored code here, we don't want to break our site or anyone else's.
• We need to be able to develop, package, and deploy plugins. I've started work on this in the hubs-plugin-api repository. We need to make sure that it's easy to get setup with a development environment for working on plugins, we need to make sure that the bundler is simple enough for our target audience, and we need to create a way to deploy plugins to your Hubs Cloud instance.
• Dynamic dependency loading for plugins isn't finished yet. We need to look into how the @hubs/core, @hubs/react and @hubs/home-page packages should be loaded at runtime to minimize code when these dependencies aren't needed. We also need to make sure the latency introduced by this system is acceptable.

[robertlong]

My thoughts on plugin deployment. Copied from Discord:

I'm gonna try to come up with a game plan for deploying these plugins.
My initial thoughts have been to create an endpoint in Reticulum that allows you to upload a zipped plugin package or point it at a plugin manifest via url.
All of the plugin's files would be put in a directory in the server's file system or in the s3 bucket depending on whether you are running on AWS or DO.
Then the question is how does the client load those files. Right now the plugin manifest looks like this:

{
  "name": "my-plugin",
  "version": "0.0.1",
  "hooks": {
    "home-page": [
      {
        "type": "js",
        "url": "index.plugin.js",
        "dependencies": [
          "react",
          "@hubs/home-page",
          "react-intl",
          "classnames",
          "@hubs/media-browser",
          "@hubs/react",
          "@hubs/core"
        ]
      }
    ]
  }
}

So plugins can define scripts and stylesheets to load on a per hook basis.
Some hooks will be 1-to-1 with pages like home-page or avatar-page.
Some wont. I think if you wanted to expose plugins to ECSY for example there'd be a components hook.
And then we need to think about how multiple plugins that define the same hooks are resolved.
I think you could have hooks that use the first plugin it finds like home-page and other hooks like components that load scripts from all the plugins.
I don't think we need to make it more complicated than that yet.
So, with that all laid out, you'd have the following structure:

• Plugins are files that are loaded at runtime based on the hooks defined in their plugin-manifest.json file.
• You install plugins through a Reticulum API that can point to a plugin-manifest.json url or you can upload a .zip file with a plugin-manifest.json file in it.
• The contents of the .zip file or all the referenced files from the plugin-manifest.json are uploaded to the Hubs Cloud server's storage, (S3, or local file system)
• Reticulum also registers the plugins in the database. The database stores what plugins are installed/enabled.
• A naive implementation of this with fairly good performance could regenerate a single combined plugin manifest json file every time a plugin was installed, uninstalled, enabled, or disabled. Reticulum would parse each plugin's manifest, and combine them into one file, then upload it to storage.
• In the APP_CONFIG we would store the url to this complete plugin manifest. If it exists, the client will fetch this file at runtime and then use the resulting file to load additional resources for each plugin hook.
• A slightly better implementation would just embed the resulting json in the APP_CONFIG to reduce the number of round trips needed to fetch client dependencies.
  And that's pretty much it. I'd add a set of REST APIs for managing plugins at /api/v1/plugins. You'd need to be an admin to access it. It'd have all the CRUD actions.

The plugins table would have the following fields:

id
name
version
enabled

It'll get the manifest url using the name of the plugin and the commonly known plugin-manifest.json file.
When you are developing a plugin, I think you would fetch the plugin manifest from the Hubs Cloud server you are working against and merge in your local plugin manifest.

[joshmarinacci]

next step is to figure out the deployment process. @robertlong please link the relevant issue.

[robertlong]

We'll revisit this in the future with a new effort.