Implementation of new potential skinning mechanism #3723

turt2live · 2019-12-13T02:43:39Z

Reviewer: Please look at this for sanity. If it looks sane, it'll get merged and replicated to the rest of the project. It's an open question if we should apply this to all components or just the ones riot-web overrides.

Based upon #3722 - true diff: pull/3722/head...pull/3723/head
Requires element-hq/element-web#11660

Note: This is set to merge against travis/sourcemaps because it breaks all the downstream builds.

Also sorry for the long description, this is basically a science experiment.

With a switch to Only One Webpack™ we need a way to help developers generate the component index without a concurrent watch task. The best way to do this is to have developers import their components, but how do they do that when we support skins? The answer in this commit is to change skinning.

Skinning now expects to receive your list of overrides instead of the react-sdk+branded components. For Riot this means we send over only the Vector components and not Vector+react-sdk.

Components can then be annotated with the replaceComponent decorator to have them be skinnable. The decorator must take a string with the dot path of the component because we can't reliably calculate it ourselves, sadly.

The decorator does a call to getComponent which is where the important part of the branded components not including the react-sdk is important: if the branded app includes the react-sdk then the decorator gets executed before the skin has finished loading, leading to all kinds of fun errors. This is also why the skinner lazily loads the react-sdk components to avoid importing them too early, breaking the app.

The decorator will end up receiving null for a component because of the getComponent loop mentioned: the require() call is still in progress when the decorator is called, therefore we can't error out. All usages of getComponent() within the app are safe to not need such an error (the return won't be null, and developers shouldn't use getComponent() after this commit anyways).

The AuthPage, being a prominent component, has been converted to demonstrate this working. Changes to riot-web are required to have this work.

The reskindex script has also been altered to reflect these skinning changes - it no longer should set the react-sdk as a parent. The eventual end goal is to get rid of getComponent() entirely as it'll be easily replaced by imports.

This breaks the tests

We're switching to Jest anyways, and getting these things to run is basically impossible at the moment.

They rely on a working riot-web build, which this is not.

MatrixClientPeg in particular doesn't work very well with this.

For matrix-org/matrix-react-sdk#3723

With a switch to Only One Webpack™ we need a way to help developers generate the component index without a concurrent watch task. The best way to do this is to have developers import their components, but how do they do that when we support skins? The answer in this commit is to change skinning. Skinning now expects to receive your list of overrides instead of the react-sdk+branded components. For Riot this means we send over *only* the Vector components and not Vector+react-sdk. Components can then be annotated with the `replaceComponent` decorator to have them be skinnable. The decorator must take a string with the dot path of the component because we can't reliably calculate it ourselves, sadly. The decorator does a call to `getComponent` which is where the important part of the branded components not including the react-sdk is important: if the branded app includes the react-sdk then the decorator gets executed before the skin has finished loading, leading to all kinds of fun errors. This is also why the skinner lazily loads the react-sdk components to avoid importing them too early, breaking the app. The decorator will end up receiving null for a component because of the getComponent loop mentioned: the require() call is still in progress when the decorator is called, therefore we can't error out. All usages of getComponent() within the app are safe to not need such an error (the return won't be null, and developers shouldn't use getComponent() after this commit anyways). The AuthPage, being a prominent component, has been converted to demonstrate this working. Changes to riot-web are required to have this work. The reskindex script has also been altered to reflect these skinning changes - it no longer should set the react-sdk as a parent. The eventual end goal is to get rid of `getComponent()` entirely as it'll be easily replaced by imports.

t3chguy · 2019-12-13T13:49:38Z

The eventual end goal is to get rid of getComponent() entirely as it'll be easily replaced by imports.

+1000

dbkr

OK - I realised why I was completely failing to understand this which was because I was expecting the replaceComponent decorator to be on the replacing component, but in fact the decorator presumably goes on both the replaced and the replacing component. Any reason for not having the riot-web side of the PR? I appreciate it's probably trivial but think it might still have helped me understand.

The PR description is also really useful, but if it could find a home in the codebase somewhere it would help people reading the code in future to understand it without having to dig out the original PR.

In general this seems fine. Biggest problem is probably the one you point out with everything exploding if things don't get imported in the right order. It's a shame we can't detect the failure mode better. I think it would be entirely fine, if not desirable even, for the app to explode on load if the imports are in the wrong order.

Having to have the decorators on the components in order for them to be replaceable might be another thing to go wrong and get forgotten but probably isn't the end of the world.

I should say btw that the reskindex running in the build step is a recent addition: previously we just re-ran reskindex whenever a component was added and committed it to git. We could go back to this as a simple way of getting around having nowhere to run the reskindex watcher.

dbkr · 2019-12-16T12:57:42Z

.buildkite/pipeline.yaml

+#    plugins:
+#      - docker#v3.0.1:
+#          image: "node:10"
+#          propagate-environment: true


when can they end to end tests come back?

They are removed by a different PR. Plan is shortly.

dbkr · 2019-12-16T13:06:50Z

.babelrc

-        "transform-runtime",
-        "add-module-exports",
-        "syntax-dynamic-import"
+        ["@babel/plugin-proposal-decorators", { "legacy": true }],


I'm missing why we need legacy decorators, especially given we're only just writing the code now. Shouldn't we be using the stage 2 version?

We use legacy decorators elsewhere - it turns out we need this regardless. The modern decorators also aren't compatible with this sort of thing just yet (we need more ES6 code).

dbkr · 2019-12-16T13:07:39Z

.eslintrc.js

@@ -25,6 +25,7 @@ module.exports = {
    parserOptions: {
        ecmaFeatures: {
            jsx: true,
+            legacyDecorators: true,


Just to note it looks like this has been removed in babel-eslint v11 so adding it now seems nonideal.

we have to add it now otherwise eslint complaints aggressively :(

dbkr · 2019-12-16T13:10:52Z

src/Skinner.js

        }

+        // Just return nothing instead of erroring - the consumer should be smart enough to
+        // handle this at this point.


Because the only consumer now is going to be a wrapper or something?

It could still be a component, but null is just as clear as a component missing in terms of blowing up.

turt2live · 2019-12-16T15:17:02Z

This was pointed at the wrong branch ._.

turt2live · 2019-12-16T16:00:58Z

@dbkr This is only needed on the component to be replaced. The decorator is executed at runtime, and only decorates classes (which may be a problem for some of our codebase, but that bridge hasn't been crossed yet). The lifecycle is roughly:

App loads matrix-react-sdk's index.js
App calls loadSkin() with their component-index of undecorated components.
The react-sdk caches all of their components and records what they override.
The app continues to load, referencing a component from the react-sdk.
The react-sdk now finally gets to execute its decorator, which checks against the app's pre-loaded skin and if it finds something it replaces the component to be loaded. If it doesn't find something it just returns the component unaltered.
The app finally gets rendered to the user.

The decorator here is supposed to be used for adding properties/functions to things. One example is adding a decorator for putting an EventEmitter on something (if you had a limitation that prevented you from just extending it normally, like a third party library). Here we replace the entire class instead, which is technically not what's going on: we're actually overriding the component's members with the skinned component's members. This works similar to a replacement because we ultimately override the constructor, lifecycle handlers, and render method.

The end goal is to remove the react-sdk's component index because it doesn't really serve a purpose once we decorate the entire project. The current issue with it is that it needs generating, so though we could remove the watcher and run it manually it's almost certainly going to be forgotten about by developers. If we use imports instead it just magically works and we don't need to worry about "registering" the component in some index somewhere.

dbkr · 2019-12-16T16:36:28Z

Ah right - understood (presumably if you then wanted to override the overriding classes from a third project though, the ones in riot-web would need to be decorated). I would maybe rename @replaceComponent as the current name implies it goes on the replacing component. @replaceableComponent maybe? Or maybe even just say put it on all components for simplicity, then it can just be @component?

The current issue with it is that it needs generating, so though we could remove the watcher and run it manually it's almost certainly going to be forgotten about by developers

Yep, I get it - just saying that an alternative would be to go back to running reskindex only when we add components and versioning it, then we only have to run it when adding a component. Yeah, people will still forget at that point but would allow us to fix source maps without having to rewrite the skinning stuff, although it looks like this is mostly done now.

Main thing though - could all the excellent explanatory notes and doc in this PR go in tree somewhere, so we don't have to dig this PR out to understand how it works? :)

turt2live · 2019-12-16T16:40:35Z

Renaming it to something helpful can easily be done. @replaceableComponent sounds like the least likely to conflict with something weird later on.

Docs are being written. I have a fear of forgetting how this works myself ._.

Sourcemaps aren't necessarily blocked on this, though it is a massive quality of life improvement (in that we don't have to remember to do things).

We're making an assumption here that the decorator is actually all over the app when it's not.

dbkr

lgtm then I guess, although still a bit worried about the failure mode on old getComponent calls when the component doesn't exist (ie. react will just start whining about nulls deep within its guts) but I guess this is an incentive to remove all the getComponent()s.

turt2live · 2019-12-17T15:09:07Z

Yup, per the description the plan is to open a subsequent PR that does the conversion for everything else

#3723 removed the prepare script which was how the SDK got built before being published. Add it back as a more modern prepublish script.

turt2live added 8 commits December 12, 2019 14:35

Remove unused jenkins.sh

25f5cca

Copy typescript configs from js-sdk

c02beb9

Upgrade to babel@7 and support typescript

97af040

This breaks the tests

Convert SdkConfig to TypeScript as a proof of concept

a5dadda

yarn.lock changes for upgrade

ff584d1

Disable tests for new TypeScript build

59ddefd

We're switching to Jest anyways, and getting these things to run is basically impossible at the moment.

Disable end-to-end tests too

f5264db

They rely on a working riot-web build, which this is not.

Fix exports for rate limited functions and MatrixClientPeg

225695a

MatrixClientPeg in particular doesn't work very well with this.

turt2live force-pushed the travis/babel7-reskindex branch from 22ccf2d to f6791fa Compare December 13, 2019 02:46

turt2live added a commit to element-hq/element-web that referenced this pull request Dec 13, 2019

Remove now-retired package.json property

c143b9b

For matrix-org/matrix-react-sdk#3723

turt2live mentioned this pull request Dec 13, 2019

Remove now-retired package.json property element-hq/element-web#11660

Merged

turt2live force-pushed the travis/babel7-reskindex branch from f6791fa to 20a6153 Compare December 13, 2019 02:48

turt2live requested a review from a team December 13, 2019 02:53

dbkr reviewed Dec 16, 2019

View reviewed changes

turt2live changed the base branch from develop to travis/sourcemaps December 16, 2019 15:16

turt2live added 2 commits December 16, 2019 16:10

Rename decorator for clarity

0a9985f

Add a bunch of docs

9865ce8

We're making an assumption here that the decorator is actually all over the app when it's not.

turt2live requested a review from dbkr December 16, 2019 23:34

dbkr approved these changes Dec 17, 2019

View reviewed changes

turt2live merged commit d06f476 into travis/sourcemaps Dec 17, 2019

turt2live deleted the travis/babel7-reskindex branch December 17, 2019 15:10

dbkr added a commit that referenced this pull request Jan 20, 2020

Add prepublish script

34c69a5

#3723 removed the prepare script which was how the SDK got built before being published. Add it back as a more modern prepublish script.

dbkr mentioned this pull request Jan 20, 2020

Add prepublish script #3876

Merged

dbkr added a commit that referenced this pull request Jan 20, 2020

Add prepublish script

d3ae656

#3723 removed the prepare script which was how the SDK got built before being published. Add it back as a more modern prepublish script.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implementation of new potential skinning mechanism #3723

Implementation of new potential skinning mechanism #3723

turt2live commented Dec 13, 2019 •

edited

Loading

t3chguy commented Dec 13, 2019

dbkr left a comment

dbkr Dec 16, 2019

turt2live Dec 16, 2019

dbkr Dec 16, 2019

turt2live Dec 16, 2019

dbkr Dec 16, 2019

turt2live Dec 16, 2019

dbkr Dec 16, 2019

turt2live Dec 16, 2019

turt2live commented Dec 16, 2019

turt2live commented Dec 16, 2019

dbkr commented Dec 16, 2019

turt2live commented Dec 16, 2019

dbkr left a comment

turt2live commented Dec 17, 2019

Implementation of new potential skinning mechanism #3723

Implementation of new potential skinning mechanism #3723

Conversation

turt2live commented Dec 13, 2019 • edited Loading

t3chguy commented Dec 13, 2019

dbkr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

turt2live commented Dec 16, 2019

turt2live commented Dec 16, 2019

dbkr commented Dec 16, 2019

turt2live commented Dec 16, 2019

dbkr left a comment

Choose a reason for hiding this comment

turt2live commented Dec 17, 2019

turt2live commented Dec 13, 2019 •

edited

Loading