[BD-03] Frontend Plugins ADR

[xitij2000]

The relevant commits are: https://github.com/edx/frontend-app-discussions/pull/5/files/84d60d7a712de737bb341b9b25a12cb9b76857fa..4d28d47a865d605780ad592556280bcc7a892c73

[openedx-webhooks]

Thanks for the pull request, @xitij2000! I've created BLENDED-578 to keep track of it in Jira.

When this pull request is ready, tag your edX technical lead.

[xitij2000]

@stvstnfrd This is the document about frontend app plugins, along with a some code to show it in action.

CC: @davidjoy

[bradenmacdonald]

@xitij2000 Thanks for writing up an ADR. Does this allow for multiple plugins to put content into the same "slot"? e.g. multiple tabs on the instructor dashboard, multiple widgets on the Learner dashboard, etc. I think that's important.

I would also like to surface this more widely - can we mention in here in extension_points.rst#Theming Microfrontends (where it mentions "Frontend Plugins"? i.e. say that we're experimenting with it here, and that it may become an OEP for wider adoption if it works well.

[xitij2000]

@bradenmacdonald Yes, I think the usecase you mentioned can be supported.

I've whipped up a quick example in my own repo: xitij2000/frontend-plugins-test#1

This will automatically discover and load tabs installed to the plugins directory. I've added a lot of comments there.

[davidjoy]

Hi @xitij2000 and @bradenmacdonald,

Regarding frontend pluggability, there's a lot to unpack here.

I want to first draw a distinction between component overrides and plugins.

Component overrides, I think, should be a build-time mechanism built into frontend-build that allows you to specify an overrides directory in your app (or as a peer to it, perhaps) which will use webpack resolve aliases to override specific component/code paths with custom versions. This PR is pretty close to that, but I think the mechanism can be even more seamless.

It shouldn't be necessary to use dynamic imports to do this. It works with normal import statements, as webpack is switching out the file at the last moment and statically linking it in at build time.

As it so happens, we recently added a mechanism to frontend-build that does exactly this with dependent packages instead of individual components, but in principle it should work for files too:

https://github.com/edx/frontend-build#local-module-configuration-for-webpack

This allows you to provide a package name, e.g., "@edx/paragon" and a local place to resolve that dependency. It allows local development of frontend-platform, paragon, headers, footers, etc., in the context of an MFE where you want to use them. It uses a module.config.js file where you define the overrides you want to use. This file is .gitignore'd.

So I'd totally support extending this mechanism to allow individual file overrides - I think that'd be a boon to the Open edX community. Component overrides are a developer-centric thing, to be clear, to be used as a last resort to modify behavior that can't be modified in any other way (plugins, branding or configuration). It's going to be brittle because the override file needs to fulfill the same imports/exports/interface as the one it's replacing. But it's a perfectly valid power-operator feature for us to add, with the understanding that it should be used sparingly and that edX can't support broken custom components as the MFE code evolves.

Okay, so that's component overrides - a low-level way of freely modifying behavior without forking.

As for frontend plugins (adding new behavior), this is a related, but different, thing, which can be done by the above mechanism, but which we can make a much more configurable, approachable, usable extension point.

I think the mechanism described here is very similar to a class of "privileged" plugins that the Teaching and Learning squad here at edX are planning in our epic of work related to frontend pluggability. Details of that epic are here: https://openedx.atlassian.net/browse/TNL-7294

Plugins are a supported, documented extension mechanism, not a last resort. They allow operators and even course teams to extend MFE behavior without hacking the code or being aware of its internal APIs.

Privileged plugins are plugins allowed to exist in the page context without a sandbox because they're trusted. These are going to be edX-supported features modeled as plugins, or custom-made features that are trusted by an operator because they made them. This mechanism is NOT recommended for any third-party code and will never be used on edx.org in that capacity, as it represents a huge security and personally identifiable information security risk when misused.

So that said, there are a few differences in how we envision privileged plugins vs. what's here in this PR's ADR:

1. Runtime application plugin configuration - we expect that we'll have other classes of non-privileged plugins that course teams will want to configure into their apps at runtime (such as different discussions integrations!) We expect to build out this runtime configuration mechanism for those plugins, but that also means that 'privileged' plugins that are allowed to exist in the page context un-sandboxed will also be able to take advantage of it.

2. Independent deployments of plugin code - This prevents us from needing to re-deploy the application when it's plugin configuration changes. It also helps protect the application build process from issues in its plugins.

These plugins ARE imported via dynamic import() - like the example in your ADR - because they're hosted outside the MFE and are not bundled in its static assets. The MFE loads a configuration document describing the plugins it should load in each slot, and then uses import() to go fetch the code for privileged plugins, or an iframe for non-privileged plugins. (The majority of the tasks in our epic are around these un-privileged plugins). You can think of a plugin as its own little MFE, in a way, that is independently deployed and served, which can then be pulled into MFEs that depend on it at runtime.

For completeness sake, here's some early exploration that informed those tasks: https://openedx.atlassian.net/wiki/spaces/AC/pages/1162904064/Frontend+Pluggability

That document is a completionist/pedantic description of all the various aspects of plugins, what information they share, how they could be loaded, etc. After writing that, we've settled on a sub-set of all the options in there, as laid out in the epic of tasks.

So, all that said. I'd love to iterate on this PR as an answer to component overrides. In the short term, it gives us a viable, low-level way of doing "plugins" until a more complete, friendly plugin mechanism is underway.

What do you think?

[xitij2000]

@davidjoy

Component overrides, I think, should be a build-time mechanism built into frontend-build that allows you to specify an overrides directory in your app (or as a peer to it, perhaps) which will use webpack resolve aliases to override specific component/code paths with custom versions. This PR is pretty close to that, but I think the mechanism can be even more seamless.

I think the mechanism in this PR can definitely be used for that purpose. WebPack can, at build time, look at the overrides directory and add any folders there as aliases, so that imports from there will override imports from the same named package.

It shouldn't be necessary to use dynamic imports to do this. It works with normal import statements, as webpack is switching out the file at the last moment and statically linking it in at build time.

Yup, for the above purpose, dynamic imports are not necessary, since we know exactly what to import when coding, and webpack is the one switching around components.

However, if we add a privileged extension point where a plugin adds to the list of available option, we need a way to enumerate them dynamically. So, it's dynamic in the sense that we don't know the list of option at coding time. We do know them at build time and run time.

So I'd totally support extending this mechanism to allow individual file overrides - I think that'd be a boon to the Open edX community. Component overrides are a developer-centric thing, to be clear, to be used as a last resort to modify behavior that can't be modified in any other way (plugins, branding or configuration). It's going to be brittle because the override file needs to fulfill the same imports/exports/interface as the one it's replacing. But it's a perfectly valid power-operator feature for us to add, with the understanding that it should be used sparingly and that edX can't support broken custom components as the MFE code evolves.

That wasn't what I was going for here, but this does allow for it. Although in that case as you mentioned we can use static imports since the structure is known at coding time, it's just that we don't know what it will resolve to.

However, allowing component overrides like this will require that we use a consistent import style in all places and don't do relative imports. Otherwise, an external alias cannot override an internal one AFAIK.

I think the mechanism described here is very similar to a class of "privileged" plugins that the Teaching and Learning squad here at edX are planning in our epic of work related to frontend pluggability. Details of that epic are here: https://openedx.atlassian.net/browse/TNL-7294

Yup, definitely, that is what I was going for here, though I was not aware of the existing terminology.

The idea is, if a developer creates a django app plugin that adds a new API, or extends an existing API, and we need to add related functionality to the frontend, we can do that through this mechanism. It would also be nice (as you describe) for the LSM to perhaps expose that plugin as a bundled file that can be imported from the LMS at runtime instead of build time, however in that case we should also create a mechanism to build individual components of importable apps that don't run independently but can be loaded into an MFE.

Part of the approach here could be used. For instance, we could modify the webpack config a bit to bundle each plugin into a separate .js file that can be dynamically imported. These could then be exposed by the LMS via a config file. They would still be somewhat privileged, though.

Completely non-privileged plugins could come from any domain, but privileged ones would need to be loaded from a trusted domain.

I'm still processing some of the links you've shared so will share more thoughts as I explore that.

[xitij2000]

Closing this in favour of the module federation approach.