Support for WebComponents API

[jimfb]

We are trying to use jsdom for our unit tests for React core (http://facebook.github.io/react/).

Unfortunately, the web components spec is not natively supported by jsdom, and the webcomponents.js polyfill does not run on jsdom. This issue requests the addition of WebComponent support (custom elements, shadow dom, html imports, etc).

[domenic]

It'd be interesting to look into which APIs webcomponents.js uses that jsdom doesn't support. If I had to guess, that will be much easier to implement than the full web components spec.

That said, it would be pretty cool to implement web components. Probably not as hard as one might think---the specs are relatively small.

[Sebmaster]

Just had time to dig into this a bit:

First off, we don't have Window defined in the window scope. I just patched this with this.Window = this.prototype in the Window constructor.
Second, webcomponentsjs expects Window to have another prototype, i.e. the EventTarget prototype, which we don't implement as a seperate entity.

Just a bit of info, because I had a bit of time.

[domenic]

Nice. Should be able to expose Window pretty easily. EventTarget prototype is a bit trickier but seems doable given how we currently implement that stuff; it's been a TODO of mine.

[Sebmaster]

Okay, patches so far are rather easy:

• this.Window = Window; in the Window constructor
• inherits(dom.EventTarget, Window, dom.EventTarget.prototype); after the definition of Window

The next crash of webcomponents.js happens due to us not implementing HTMLUnknownElement (#1068), after shiming that we need to implement the SVGUseElement. That's what I'm currently blocked on, because webcomponents.js apparently doesn't like the SVGUseElement shimmed by a HTMLDivElement and throws in an assert.

[Sebmaster]

Okay I checked into the Polyfill some more, we need to implement/you need to shim the following:

• HTMLUnknownElement Add HTMLUnknownElement #1068
• SVGUseElement
• window.CanvasRenderingContext2D
• Range APIs (including: document.getRange(), window.getSelection(), window.Range, window.Selection; Text Selection API for Input and TextArea #804 might be a start)
• npm i canvas

(non-exhaustive list for now)

A start is something like the following:

jsdom.env({
  file: __dirname + '/index.htm', // refers to webcomponent.js
  created: function (err, window) {
    jsdom.getVirtualConsole(window).sendTo(console)

    window.document.createRange = function () { }
    window.getSelection = function () { }
    window.Range = function () { }
    window.Selection = function () { }
    window.CanvasRenderingContext2D = function () { } // Object.getPrototypeOf(require("canvas")(0,0).getContext("2d")) might be better
    window.SVGUseElement = window.HTMLUnknownElement
  },
  done: function (err, window) {
    console.log(err[0].data.error);
    console.log(window.CustomElements)
  },
  features: {
    ProcessExternalResources: ['script']
  }
});

That done, there's some bug in our HTMLDocument constructor, which leads to a maximum call stack error. The constructor is at the moment only for internal use, however it's valid that some script on the site makes calls to it so we need to make that constructor available for public consumption.

[bedeoverend]

+1 Would love to see WebComponents on jsdom, particularly as Polymer gains in popularity, would be great to be able to test custom elements on a headless system.

[domenic]

Right now there is no cross-browser definition of web components, so it'd be premature to implement. (We're not just going to copy Chrome.) In the meantime, you can try using Polymer with jsdom.

[bedeoverend]

@domenic fair enough. Well it's more the support for the WebComponents.js polyfill that I'm after, as that's what Polymer depends on - or webcomponents-lite (polyfills all of them barring Shadow DOM) at the moment. Made a few attempts to get Polymer working on jsdom, but no luck so far - I'm assuming @Sebmaster's tasks in the comment above will at least need to be patched first.

[domenic]

My understanding is that there are three separate polyfills in question. The one in the OP is separate from Polymer. Then there's the webcomponents.org polyfills, which used to be used in old-Polymer. Then in Polymer 1.0, they have their own polyfills, I think, which aren't really polyfills, but instead alternate libraries that do things kinda web-component-ish. Maybe that is webcomponents-lite though.

[bedeoverend]

On the WebComponentsJS repo, it says that the webcomponentsjs-lite is a variant, providing polyfills for all but Shadow DOM, which Polymer then independently attempts to shim using their Shady DOM system. So from that I'm pretty sure Polymer relies on WebComponents as much as it can, with the WebComponentsJS polyfill doing the grunt work. The lite version is supposed to be significantly less weight (funnily enough..) so I'll see if I can pinpoint what it is that jsdom needs for the lite version. What do you think the chances are of getting the polyfill (lite or full) working in jsdom is?

[domenic]

It's really hard to say without some investigation... looking forward to what you find out.

[Sebmaster]

Yeah, I think my list of todo tasks is still applicable and required to use the shims. Getting #1227 merged in might make us a lot quicker with implementing standards-compliant interfaces so we can fix the missing ones more quickly.

[matthewp]

I've (probably naively) started working on adding CustomElementsRegistry as a way to understand how jsdom is structured. I added "custom-elements/custom-elements-registry/define.html" to the web platform tests list and it passes when it shouldn't (i haven't implemented nearly enough yet). I'm pretty sure the test isn't really running as even adding a throw at the top of the test won't prevent it from passing. So I've obviously missed something; aside from adding the test in test/web-platform-tests/index.js is there anything else I need to do?

[Sebmaster]

Seems like that's caused because we fail in the initial const testWindow = iframe.contentDocument.defaultView; line because contentDocument is undefined for some reason. Might be an issue with our loading order vs. script execution, but haven't dug into that. Hope that helps you work around that. We might have to simplify the test for our purposes (for now).

[matthewp]

That helps very much, thanks! I'll see if I can figure out what is going on there, and if not I'll create a simplified test as you recommended.

[matthewp]

@Sebmaster Just in case your interested, I did a bit of research into what is going on with that test and the results are surprising to me.

The test is using the named access feature of html. This means you can do stuff like:

<div id="foo"></div>
<script>
  console.log(window.foo === document.getElementById('foo'));
</script>

However, if the element has a nested browsing context, the global should point to that instead (see the linked spec). For iframe's that's the contentWindow. jsdom gets this right, there's even a test. Safari gets it right too.

What's crazy is that Chrome and Firefox get this wrong; the global points to the iframe, not it's contentWindow. Seeing this, I assumed it was a jsdom bug and did some hunting, eventually finding that test, which led me to the spec.

tldr; working on jsdom is very educational and you guys do an amazing job.

Going to file bugs in the respective browsers. Also will send a PR to web-platform-tests, I found some other mistakes in the test as well.

[domenic]

This is even more motivation to upstream tests like https://github.com/tmpvar/jsdom/blob/master/test/living-html/named-properties-window.js to WPT. Thank you for posting! It makes me feel really great about jsdom ^_^

[solkimicreb]

Hi!

I managed to make Custom Elements polyfill work with jsdom by combining

• this repo: https://github.com/RisingStack/nx-seo
• with this custom element polyfill: https://github.com/WebReflection/document-register-element

Note: the repo uses jsdom 8.5.0. The reason is that I only had success with a MutationObserver polyfill, that uses Mutation Events internally. Mutation Events were removed after 8.5.0 due to bad performance. If native Mutation Observer comes out I will remove the polyfill and update to the latest jsdom.

[lastmjs]

I've got the latest jsdom, and https://github.com/WebReflection/document-register-element is working for me! I've been experimenting with the more official polyfills, and I'm having trouble for some reason. My goal is to get at least custom elements and html imports to work...it would be awesome if we could get Polymer to work as well.

[lastmjs]

I can get the Polymer scripts to run without error. I can even create a component and pass it to the Polymer constructor. After that it fails silently. I think shadow DOM is the issue.

I've been trying to get the webcomponentsjs HTML imports polyfill to work. I can get the script to run, and I believe my HTML imports execute an xmlhttprequest, but it doesn't seem like the scripts in my imports get run.

[snuggs]

Care to share an example @lastmjs? I'm currently knee deep in web components myself. If I can be of help i'd gladly contribute with you.

[lastmjs]

@snuggs Thanks! Give me a day or two, I'm in the middle of some pressing things at the moment.

[FaBeyyy]

@FaBeyyy
So I found the setup which works for me. I had to add polyfill for MutationObserver. I'm using JSDOM for testing porpoise, with Jest, and the working setup is:

// package.json
{  ...
  "jest": {
    "transform": {
      "^.+\\.(mjs|jsx|js)$": "babel-jest"
    },
    "setupFiles": [
      "<rootDir>/node_modules/babel-polyfill/dist/polyfill.js",
      "<rootDir>/node_modules/mutationobserver-shim/dist/mutationobserver.min.js",
      "<rootDir>/node_modules/document-register-element/build/document-register-element.node.js"
    ]
  }
... 
}

//.bablerc
{
    "presets": [
        ["@babel/preset-env", { "modules": "commonjs"}]
    ]
}

Nice, Thanks!

[mgibas]

@majo44 this is not required with latest jsdom. When working with Jest (which is using jsdom v11) you can just use updated environment: https://www.npmjs.com/package/jest-environment-jsdom-fourteen

[majo44]

@mgibas thanks, with jest-environment-jsdom-fourteen it is also works fine and mutation observer polyfill is not required (but version is 0.1.0, single commit package :) )

[calebdwilliams]

Is there a breakdown of which of the web components APIs are currently supported by JSDOM? Seems like shadow DOM is supported, but not custom elements (at least in the release branch/repo)?

[NMinhNguyen]

npm install @tbranyen/jsdom@13.0.0 --save-dev

@tbranyen do you have the source code for your fork available somewhere? Would be curious to look at the diff 🙂

[nweldev]

I'm using jest-environment-jsdom-fifteen like @majo44 suggested, and the babel-polyfill and document-register-element (see @mgibas answers). But I still get an error when I try to retrieve my web component shadow dom for tests.

el.shadowRoot is null with:

const el;
beforeEach(async() => {
  const tag= 'my-component'
  const myEl = document.createElement(tag);
  document.body.appendChild(myEl);
  await customElements.whenDefined(tag);
  await new Promise(resolve => requestAnimationFrame(() => resolve()));
  el = document.querySelector(tag);
}

it(() => {
  const fooContent = el.shadowRoot.querySelectorAll('slot[name=foo] > *');
})

Any idea of a workaround? FYI, it was already tested with Karma, Mocha, Chai & Jasmine.

Nothing special in my component:

customElements.define(
  'my-component',
  class extends HTMLElement {
    constructor() {
      super();

      const shadowRoot = this.attachShadow({ mode: 'open' });
      ...
  }
})

Edit

I did some debugging with jsdom 15.1.1 in order to better understand my issue.
Still, I don't understand why it's null here...

So, Element.shadowRoot is implemented since 88e72ef

jsdom/lib/jsdom/living/nodes/Element-impl.js

Lines 388 to 415 in 1951a19

	if (this._shadowRoot !== null) {
	throw new DOMException(
	"Shadow root cannot be created on a host which already hosts a shadow tree.",
	"InvalidStateError"
	);
	}
	const shadow = ShadowRoot.createImpl([], {
	ownerDocument: this.ownerDocument,
	mode: init.mode,
	host: this
	});
	this._shadowRoot = shadow;
	return shadow;
	}
	// https://dom.spec.whatwg.org/#dom-element-shadowroot
	get shadowRoot() {
	const shadow = this._shadowRoot;
	if (shadow === null || shadow.mode === "closed") {
	return null;
	}
	return shadow;
	}

After document.createElement, this._shadowDom is ok at https://github.com/jsdom/jsdom/blob/15.1.1/lib/jsdom/living/nodes/Element-impl.js#L403. And every element in the shadow dom is created (Element constructor called with the right values).

But when I call el.shadowDom immediately after document.body.appendChild(el) (https://github.com/jsdom/jsdom/blob/15.1.1/lib/jsdom/living/nodes/Element-impl.js#L408), this. _shadowRoot is null!

Same thing after

 await customElements.whenDefined(tag);
 await new Promise(resolve => requestAnimationFrame(() => resolve()));

Or even if I use the following instead of document.

document.body.innerHTML = `
  <my-component id="fixture"></my-component>
`:

For reproduction, see:
https://github.com/noelmace/devcards/tree/jest

[nweldev]

@NMinhNguyen I guess you can find the source code of the fork made by @tbranyan here https://github.com/tbranyen/jsdom/tree/initial-custom-elements-impl

[fernandopasik]

I'm trying to test web components made with lit-html and lit-element and I noticed this difference when creating the elements.

const myElem = new MyElem();

document.body.appendChild(myElem);
await myElem.updateComplete;

console.log(myElem.shadowRoot) // exists and has the rendered markup

and when I use the document.createElement

const myElem = document.createElement('my-elem');

document.body.appendChild(myElem);
await myElem.updateComplete;

console.log(myElem.shadowRoot) // null

For configuring jest I only use one polyfill that is : setupFiles: ['document-register-element']

Seems that the render method in myElem never gets called. Debugging a bit further I've discovered that the method initialize that is in lit-element never gets called.
So the 2nd example would work if I do

const myElem = document.createElement('my-elem');
myElem.initialize();

document.body.appendChild(myElem);
await myElem.updateComplete;

console.log(myElem.shadowRoot) //  exists and has the rendered markup

[capricorn86]

I have created an alternative DOM that supports web components. I first tried to make a PR, but the way JSDOM works made it hard for me to solve my needs there. You are free to use it or look at the code.

DOM:
https://www.npmjs.com/package/happy-dom

Jest environment:
https://www.npmjs.com/package/jest-environment-happy-dom

[motss]

Looks awesome. Thank you.

…

On Mon, Oct 7, 2019, 1:08 AM capricorn86 ***@***.***> wrote: I have created an alternative DOM that supports web components. I first tried to make a PR, but the way JSDOM works made it hard for me to solve my needs there. You are free to use it and/or look at the code. DOM: https://www.npmjs.com/package/happy-dom Jest environment: https://www.npmjs.com/package/jest-environment-happy-dom — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1030?email_source=notifications&email_token=ACQ5ZD5QUEITPND4SXWOHW3QNILSRA5CNFSM4A4G5SF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAOO5ZA#issuecomment-538767076>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACQ5ZDYU465DXI4KHBQH4KTQNILSRANCNFSM4A4G5SFQ> .

[TechQuery]

@capricorn86
Your work makes my Test environment simple, thanks!
https://github.com/EasyWebApp/WebCell/tree/v2/MobX

[capricorn86]

@capricorn86
Your work makes my Test environment simple, thanks!
https://github.com/EasyWebApp/WebCell/tree/v2/MobX

Thank you @TechQuery!

[capricorn86]

Looks awesome. Thank you.
…
On Mon, Oct 7, 2019, 1:08 AM capricorn86 @.***> wrote: I have created an alternative DOM that supports web components. I first tried to make a PR, but the way JSDOM works made it hard for me to solve my needs there. You are free to use it and/or look at the code. DOM: https://www.npmjs.com/package/happy-dom Jest environment: https://www.npmjs.com/package/jest-environment-happy-dom — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1030?email_source=notifications&email_token=ACQ5ZD5QUEITPND4SXWOHW3QNILSRA5CNFSM4A4G5SF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAOO5ZA#issuecomment-538767076>, or mute the thread https://github.com/notifications/unsubscribe-auth/ACQ5ZDYU465DXI4KHBQH4KTQNILSRANCNFSM4A4G5SFQ .

Thank you @motss!

[alfechner]

Is there a breakdown of which of the web components APIs are currently supported by JSDOM? Seems like shadow DOM is supported, but not custom elements (at least in the release branch/repo)?

I'd be interested in this as well :)

[Hlulani]

@capricorn86 I was trying to follow your suggestion. and still having a persistent issue with "Custom element not defined"

I get a customElements is not defined with the error below:
Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure
your tests are not affected.
ReferenceError: customElements is not defined
› node_modules/
@lit/reactive-element
/src/decorators/custom-element.ts:60:7
› __decorateClass src/main/components/form/error-message.ts:11:57
12| .hidden {
13| display: none;
14| }
| ^
15|
16| div {
› src/main/components/form/error-message.ts:7:8
› VitestExecutor.runModule node_modules/vite-node/dist/client.mjs:362:5
› VitestExecutor.directRequest node_modules/vite-node/dist/client.mjs:346:5
› VitestExecutor.cachedRequest node_modules/vite-node/dist/client.mjs:189:14
› VitestExecutor.dependencyRequest node_modules/vite-node/dist/
client.mjs:233:12
› src/main/pages/login/login-page.ts:18:32
› VitestExecutor.runModule node_modules/vite-node/dist/client.mjs:362:5
› VitestExecutor.directRequest node_modules/vite-node/dist/client.mjs:346:5
This error originated in "test/unit/main/pages/page.test.ts" test file. It doesn't
mean the error was thrown inside the file itself, but while it was running.
This error was caught after test environment was torn down. Make sure to
cancel any running tasks before test finishes:
cancel timeouts using clearTimeout and clearInterval
wait for promises to resolve using the await keyword
I added this mock on my setup.ts file:
class MockCustomElementRegistry {
define() {}
get() {}
upgrade() {}
whenDefined() {
return Promise.resolve();
}
}
Object.defineProperty(global, 'customElements', {
value: new MockCustomElementRegistry(),
writable: true,
configurable: true,
});
What could I be missing?

[capricorn86]

@capricorn86 I was trying to follow your suggestion. and still having a persistent issue with "Custom element not defined"

I get a customElements is not defined with the error below: Vitest caught 1 unhandled error during the test run. This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected. ReferenceError: customElements is not defined › node_modules/ @lit/reactive-element /src/decorators/custom-element.ts:60:7 › __decorateClass src/main/components/form/error-message.ts:11:57 12| .hidden { 13| display: none; 14| } | ^ 15| 16| div { › src/main/components/form/error-message.ts:7:8 › VitestExecutor.runModule node_modules/vite-node/dist/client.mjs:362:5 › VitestExecutor.directRequest node_modules/vite-node/dist/client.mjs:346:5 › VitestExecutor.cachedRequest node_modules/vite-node/dist/client.mjs:189:14 › VitestExecutor.dependencyRequest node_modules/vite-node/dist/ client.mjs:233:12 › src/main/pages/login/login-page.ts:18:32 › VitestExecutor.runModule node_modules/vite-node/dist/client.mjs:362:5 › VitestExecutor.directRequest node_modules/vite-node/dist/client.mjs:346:5 This error originated in "test/unit/main/pages/page.test.ts" test file. It doesn't mean the error was thrown inside the file itself, but while it was running. This error was caught after test environment was torn down. Make sure to cancel any running tasks before test finishes: cancel timeouts using clearTimeout and clearInterval wait for promises to resolve using the await keyword I added this mock on my setup.ts file: class MockCustomElementRegistry { define() {} get() {} upgrade() {} whenDefined() { return Promise.resolve(); } } Object.defineProperty(global, 'customElements', { value: new MockCustomElementRegistry(), writable: true, configurable: true, }); What could I be missing?

Hi @Hlulani!

I believe that the correct procedure would be to add a ticket at Happy DOM (if you are using Happy DOM) or Vitest.

To answer your question:
You should not have to create your own CustomElementRegistry (unless you want to for mocking). The reason for why customElements isn't defined is probably because the environment has been torn down (closed) and no global properties are available anymore. In other words, you have code running after all tests has completed that tries to access customElements.

You need to look into your tests and try to find what could cause code to continue running after the tests has completed.

[Hlulani]

Hi @capricorn86 Thanks a lot for your response and I definitely will raise issues in the right places next time,

regarding the issue, I was able to solve this by adding adding testTimeout: 70000, on the config..tx

[capricorn86]

Hi @capricorn86 Thanks a lot for your response and I definitely will raise issues in the right places next time,

regarding the issue, I was able to solve this by adding adding testTimeout: 70000, on the config..tx

70000 is a long time. I would recommend looking into the tests that are taking such a long time and try to reduce it (perhaps by mocking if they are unit tests).