Create small MFE base image using runtime MFE config system

[kdmccormick]

Context

The MFE runtime config API has been added to frontend-platform and edx-platform! Issue: openedx/wg-frontend#103

We would like to make use of that API in the tutor-mfe plugin so that we can statically build a minimal tutor-mfe image, which can be used by anyone along with runtime configuration.

Acceptance

• Ensure all built-in MFEs have upgraded frontend-platform and specify an APP_ID
• ...
• Update mfe plugin to use dynamic config API instead of static config
• Build & publish an upstream mfe image.
• TODO: add remaining steps
• ...
• Remove --build flag from tutor dev start, as it's no longer needed.

[kdmccormick]

@ghassanmas , @arbrandes and I recently talked about this. We needed to figure out a way to allow the MFEs to communicate with the dynamic configuration API without hard-coding the MFE_CONFIG_API_URL (which varies based on the LMS_HOST Tutor config setting) into the image. What we came up with was:

• Set MFE_CONFIG_API_URL to a particular relative URL, for example /config-api. The relative URL would cause the MFE Config API request to go to the mfe nginx container, not the lms app container.
• Add an nginx routing rule that would redirect /config-api requests to the proper lms API. This rule would be templated based on LMS_HOST.

[kdmccormick]

This is blocked by completion of: openedx/wg-frontend#103

regardless, I'm starting work on a prototype over here: kdmccormick/tutor-mfe#1

[ghassanmas]

@kdmccormick thats great to hear, let me know if I can help with anything.

I was doing some discovery lately regarding module federation which would be also added on top of this. Simply put the vision is just as we have config variables for settings, we can have config variables of location of bundle. e.g. if I don't want to use the default (footer, header, paragon). I could add a config to overide it. That config is a url which points to a sepearte bundle of a which can be hosted on mfe-host or any CDNs... I don't know the possbilty of this yet just though its a path worth discovering.

[kdmccormick]

@ghassanmas Interesting, would that be an alternative to the current system for overriding components, where you must npm-install a package (like https://github.com/edx/frontend-component-footer-edx) so that it replaces the exports of an existing package (like https://github.com/openedx/frontend-component-footer)? Let me know when you have a prototype, I am interested to see.

[kdmccormick]

Something I noticed while working on this:

The Dockerfile uses staged builds to progressively build a base image and and intermediate prod image for every single installed MFE. I think the intention is that rebuilding, say, gradebook-dev, would only require the gradebook stages to be rebuilt. It seems though that Docker does not do this: it rebuilds every stage including and above the gradebook-dev stage, including learning, learning-dev, learning-src, account-dev, account-src, and so on. I believe that this significantly slows down the startup time for the MFEs in dev mode.

[kdmccormick]

I created overhangio/tutor-mfe#58 to address ^

[ghassanmas]

@ghassanmas Interesting, would that be an alternative to the current system for overriding components, where you must npm-install a package (like https://github.com/edx/frontend-component-footer-edx) so that it replaces the exports of an existing package (like https://github.com/openedx/frontend-component-footer)? Let me know when you have a prototype, I am interested to see.

Yes your are exactly right. I haven't created a prototype that exactly uses those pacakges per say, but I tried to do a POOF of concpent with basic webpack/react and it worked, what I did exactly is that the app at runtime was able to require a module (a module can be a library, compnenet, pacakge...etc) from a somewhere else. What I used is WebpackModuleFederatoinPlugin, where I had to configure both the module (that was imported at run time) and the app that used it.

[ghassanmas]

Something I noticed while working on this:

The Dockerfile uses staged builds to progressively build a base image and and intermediate prod image for every single installed MFE. I think the intention is that rebuilding, say, gradebook-dev, would only require the gradebook stages to be rebuilt. It seems though that Docker does not do this: it rebuilds every stage including and above the gradebook-dev stage, including learning, learning-dev, learning-src, account-dev, account-src, and so on. I believe that this significantly slows down the startup time for the MFEs in dev mode.

I haven't used tutor-mfe in development mode, or I didn't rely on it. I think there is area of improvment in the Dockerfile, Now I am not sure about what exactly you did, I know docker stage build is meant to optimize the cahching or buildtime...etc for example I always enable docker buildkit so that those stages would built in parrales (but it's gonna consume network/RAM at a high scale) On the other hand there is somehting that can enhance buildtime (consecutive built) which I though about before I will link what I mean in the PR you opend.

[kdmccormick]

Update: This is in review on overhangio/tutor-mfe#69

[kdmccormick]

Moving this to the build-test-release-wg repo, since this is a big concern for the Olive release.

[ghassanmas]

[Update]: From my end one of the current blocker is this openedx/frontend-app-gradebook/pull/290

[arbrandes]

Gradebook is no longer a blocker, and it looks like images are going to start being built by CI! I think we can close this.