More info on this plugin and the MFEs

[hedgerh]

Hey Zack, I've been really interested in some of the things you've working on! Was hoping to find out more about this plugin.

Say my architecture consists of a single SPA that pulls in 15 individual React components that are each published as their own npm packages. (Each component is a whole section of the site.) This results in a bottleneck, where the whole SPA needs to be deployed in order for updated components to be deployed.

I believe this plugin would instead allow me to deploy components individually, and have them pulled in from external URLs at runtime, correct?

I'm curious how we'd manage common dependencies that are used across the SPA/individual components. Currently, the SPA declares them as actual dependencies, while the components declare them as peerDependencies, so the SPA is the source of truth for a common dep and its version. Interested to hear how we'd approach this when the components aren't being bundled in the context of the entire SPA.

Finally, I haven't dug deep in to MFEs. Are there solutions that even come close to doing what this plugin does?

Thanks for humoring my questions. Let me know if you need any docs written!

[ScriptedAlchemy]

Hey! thanks for sending a message!

So right now (this very minute) I'm working on getting this thing in shape. It's very much in development.

HOWEVER! after the set of code I'm about to push to that open PR, will provide a working demo with the MFEs

This copy, I'm planning to test out on some projects. Right now, I got noooo docs and to be honest I write piss poor documentation.

Getting to your actual message...

Okay, so when I started architecting MFEs (it's kinda my sell to companies. Seems its a rare skill) The main thing all the companies have raised is exactly this. Having to re-deploy everything! If you make an API change, you got to make sure it's not going to break anyone else, or if you are updating a component. Big changes are hard when everyone needs to move as one, and risk is high when one little mistake can bring an entire system to its knees. Monolithic anything isn't good, and a frontend can be a monolith.

This plugin allows for a little more than just pulling external URLs in. It allows you to inject and/or merge multiple webpack manifests together, allowing the frontend to once again act as a SPA.

From what you need to do, Id suggests a slightly different approach. Don't make a page a component and still load them in via external resources. Still to the one SPA. Rather have that page serve its own application. Important for SSR!

Heres how I build these. Let's say I have an application, the app can be broken up into Orders, Transfers, Invoicing, History. All these need APIs some may share APIs. Each section mentioned above runs completely standalone and usually has a few pages (split by user flow verticle), it self-hosts, builds, and serves its own APIs to its frontend. Iv one gots down, the others say online --- they are totally independent systems. This tool is actually more systems like that. True MFEs.

However, in your case. It should work as well, or be very easy to modify. My main design has been for pulling other webpack resources from other builds. Go in your app and console.log(__webpack_modules__) then check the output. My system allows for aliasing something in there and getting it easily on the other end.

Heres some code.
there's 2 MFE's, right. Both run as COMPLETELY separate apps.

WEBSITE 2: http://localhost:3002

NOTE: I put a magic comment in this file - externalize:TitleComponent
This tells website2's build must alias this file as a module with module.id = externalize:TitleComponent
This IS NOT externalizing one function, its externalizing the whole module

// Title.js
import React from 'react';

export const Title = ({title}) => {
  return (<h1>{title}</h1>)
}


/*externalize:TitleComponent*/

There are a few things that go on...

1. files are hashed so how can I get them in a predictable way?
   Each site has my plugin, the plugin writes a small manifest which can be embedded on another site.
2. the importManifest.js (which is cache-busted by a timestamp in the client)

if(!window.entryManifest) {window.entryManifest = {}}; window.entryManifest["website-two"] = {
  "Title.js": "Title.0kj43ib2 luh43.js",
  "hello-world.js": "hello-world.0j47g283v34gh.js",
  "main.js": "main.o4i5nb6v4n3k.js",
  "index.html": "index.html"
}

I then embed this easily, or use my plugin to inject it dynamically.
NOW website 1 wants to render Title.js, or get some function and execute it
WEBSITE 1: http://localhost:3001

// i load the manifest somewhere( this is website 2's manifest)
import('http://localhost:3002/importManifest.js').then(() => {
      this.setState({manifestLoaded: true})
})

// once it's loaded, I want to actually get Title.js from website2
import(/* importUrl */'http://localhost:3002/' + window.entryManifest['website-two']['Title.js']).then(({TitleComponent}) => {
//TitleComponent is the MODULE/FILE... its not the export const Title (refer to title.js in website 2)
        console.log('got module, will render it in 2 seconds')
        this.Component = TitleComponent.Title
        setTimeout(() => {
          this.setState({loaded: true})
        }, 2000)
});

// Obviously this sucks for react. So i just made a React component which is the consumer. Its the same as above by JSX

```render() {
    const {manifestLoaded} = this.state
    const helloWorldUrl = manifestLoaded && 'http://localhost:3002/' + window.entryManifest['website-two']['Title.js']

    return (
      <div>
        <HelloWorld/>
        { manifestLoaded && <ExternalComponent src={import(/* importUrl */ helloWorldUrl)} module="TitleComponent" export='Title' title={'Some Heading'}/>}
        {this.renderDynamic()}
      </div>
    )```

// Lets say i want to just execute from function, not a react component
import(/* importUrl */'http://localhost:3002/' + window.entryManifest['website-two']['hello-world.js']).then(({someFunction}) => {
//someFunction is not
        console.log('got module, will render it in 2 seconds')
        someFunction.externalFunction()
        setTimeout(() => {
          this.setState({loaded: true})
        }, 2000)
});

Peer dependencies, nah that's easy, but little more manual.

You can split chunks how you want. So you can split out the common things into their own files, even some node modules..

Check this manifest, where everything is split out. because im merging manifests together on the client, when other apps start up and need, say react. It'll just check if it got ./node_modules/src/react/dist/index.js, and if another build is on the page, it will see that that one has the exact same file in its manifest... becaaaausee I sync the manifests, that builds dependencies are its dependencies too. It just uses someone else's. Forget manually managing that, work just like as if it was all built on one webpack build.

Heres me being extreme and splitting all npm modules, just because.

 splitChunks: {
      chunks: 'all',
      maxInitialRequests: Infinity,
      minSize: 0,
      cacheGroups: {
        vendor: {
          test: /[\\/]node_modules[\\/]/,
          name(module) {
            // get the name. E.g. node_modules/packageName/not/this/part.js
            // or node_modules/packageName
            const packageName = module.context.match(/[\\/]node_modules[\\/](.*?)([\\/]|$)/)[1];
            // npm package names are URL-safe, but some servers don't like @ symbols
            return `npm.${packageName.replace('@', '')}`;
          },
        },
      },
    },

Which results in a manifest like this, hashes can be added. Which will allow for multiple versions of react, or whatever to not conflict.

if(!window.entryManifest) {window.entryManifest = {}}; window.entryManifest["website-two"] = {
  "Title.js": "Title.js",
  "hello-world.js": "hello-world.js",
  "main.js": "main.js",
  "manifest.js": "manifest.js",
  "npm.ansi-html.js": "npm.ansi-html.js",
  "npm.ansi-regex.js": "npm.ansi-regex.js",
  "npm.events.js": "npm.events.js",
  "npm.fast-levenshtein.js": "npm.fast-levenshtein.js",
  "npm.hoist-non-react-statics.js": "npm.hoist-non-react-statics.js",
  "npm.html-entities.js": "npm.html-entities.js",
  "npm.lodash.js": "npm.lodash.js",
  "npm.loglevel.js": "npm.loglevel.js",
  "npm.node-libs-browser.js": "npm.node-libs-browser.js",
  "npm.object-assign.js": "npm.object-assign.js",
  "npm.prop-types.js": "npm.prop-types.js",
  "npm.querystring-es3.js": "npm.querystring-es3.js",
  "npm.react.js": "npm.react.js",
  "npm.react-dom.js": "npm.react-dom.js",
  "npm.react-hot-loader.js": "npm.react-hot-loader.js",
  "npm.react-is.js": "npm.react-is.js",
  "npm.react-lifecycles-compat.js": "npm.react-lifecycles-compat.js",
  "npm.scheduler.js": "npm.scheduler.js",
  "npm.scriptjs.js": "npm.scriptjs.js",
  "npm.shallowequal.js": "npm.shallowequal.js",
  "npm.sockjs-client.js": "npm.sockjs-client.js",
  "npm.strip-ansi.js": "npm.strip-ansi.js",
  "npm.url.js": "npm.url.js",
  "npm.webpack.js": "npm.webpack.js",
  "npm.webpack-dev-server.js": "npm.webpack-dev-server.js",
  "index.html": "index.html"
}

Finally, your last question. Are there solutions that even come close to doing what this plugin does?

I like to pride myself on building things that have never been done publically. Working in the field and doing this for about 10 years means I've developed many proprietary systems. These are trade secret coupled systems. I'm sure other companies have custom built ones. Amazon has their own one as well

As far as open-source, I don't build things others have made - I wrote this because nothing like it exists. Microfrontends are cumbersome, but they are so new that there's still a lot of tech needed.
As far as I know, this is the only tool of its kind - I've never found anything like this, have searched for 3 years before finally just building it.

This is very hard to explain if you want to do a call or skype. I'll gladly explain how it works and how these problems are solved/answer any other questions. I get a lot of calls like this, so its no bother.

Hope this helps. This is one of the pieces to a much larger project being worked on.

As I said, there's a lot of problems we need to overcome in MFEs and React Scalability. Things we shouldn't have to deal with. https://github.com/faceyspacey/respond-framework/blob/master/README.md

[ScriptedAlchemy]

Run this. Go to localhost:3001. #11

from root directory. npm run manual
npm install, and npm install inside the manual folder

[hedgerh]

finishing up your post now and about to spin it up

[ScriptedAlchemy]

A warning. The code is a little crappy, it works, but dozens of attempts were made.

The MFE is bare bones mostly to make sure I didn’t break stuff. But the concept should get through. And while the code is a little crap under the hood right now. It actually works 😂

[hedgerh]

thank you for the detailed writeup and the offer to do a call! you're awesome haha. would love to chat sometime. i think im getting the gist. im gonna play around with it a bit.

im not currently on the horizontal architecture team at work, but its interesting to learn new strategies for architecting frontends that are either very large or split up across many teams. may need to chat with them to see where MFEs are on their radar.

speaking of Respond, i think that project is also super interesting, but i havent quite gotten a complete picture of what makes it so great, yet. i created an issue on that repo earlier offering to help with docs, btw.

there any better mediums to message you, by any chance?

[ScriptedAlchemy]

Respond would require a call.

Okay so to your situation, this tool will still work for your need. Assuming you don’t server side render. If you do then there last a little more complexity.

As you see in the demo I can load jquery or something. So it works for getting scrips on the page. It’ll depend how you access them once they are on the page, you’ll need to actually reference the function you need to envoke if it’s commonjs bundles, then _ _ webpack_require _ should grab the library name. Or you can always make it self envoking which exposes itself as a window global.

However the best way would still be to build it with webpack on one side and pull the module out like I’m doing.

What you need, with current architecture, this thing can do

[ScriptedAlchemy]

Here’s a better communication medium. I got a slack channel

https://join.slack.com/t/scriptedalchemy/shared_invite/enQtNjI2NjMwMzgzMjAwLTE5YTVjNjE4M2IzMzI3MzI4OTNhYmUxMjlhMGYxNDQyOTBmZjRhNGQ0ZmUyMzhmN2Y1NmI3NDJhYzZmZmM0NWI

[ScriptedAlchemy]

I can also show stuff that’s experimental. Stuff that’s not ready for public posting

[ScriptedAlchemy]

This has progressed significantly #38

[ScriptedAlchemy]

Bumping again. Some major updates to it and now at a stage where I’m using it on actually projects, as are others 😊

[ScriptedAlchemy]

https://link.medium.com/y94FdvpL41

[88kami88]

webpack/webpack#10352

This proposal to merge into webpack is the most comprehensive and cohesive description of the problem and solution. I believe this issue can be closed.