Issues with render function returning a resolved promise

[kalinchernev]

Hi, I'm trying to make a major version upgrade in this pull request ec-europa/ecl-twig#279

The issue I'm having is that the twig templates transformed by the loader is now a resolved promise which does not have the .render() function.

The behavior is as if rendering happens on compile although it is definitely not the case.

I tried to check for existing issues in the repo, but there are none, as well as no notes for breaking changes in the releases.

Here's a sample usage: https://github.com/ec-europa/ecl-twig/blob/5cc7ca6d41f52343210021762b6a7b26f07f26cf/src/ec/packages/ec-component-accordion/accordion.story.js#L31

This is the error message you'll be able to see checking out the branch:

And here's a debug statement which proofs that an environment is loaded and the context is correctly passed as desired:

What could be the reason for this error?

Thanks!

[ericmorand]

Are you relying on the Webpack loader to import the template?

[kalinchernev]

Yes, it's an import syntax https://github.com/ec-europa/europa-component-library/blob/v2-dev/src/systems/ec/implementations/react/storybook/.storybook/manager-head.html using twing-loader

[kalinchernev]

In the same pull request, I made a few changes to transpile the code coming from twing and twing-loader because IE11 doesn't like some of the syntax published by the library such as string literals and arrow functions.

I'm not an expert in babel, but adding the plugin for the arrow functions and updating the webpack config I reached a place where no vendor code contains ES6 features such as the arrow function.

However, now the same template code is problematic:

const env = require('/home/kalata/Projects/ec/@ecl/ecl-twig/src/ec/.storybook/environment.js');
let templatesModule = (() => {
let module = {
    exports: undefined
};

module.exports = (TwingTemplate) => {
    return new Map([
        [0, class extends TwingTemplate {
            constructor(env) {
                super(env);

                this.sourceContext = new this.Source(``, `/home/kalata/Projects/ec/@ecl/ecl-twig/src/ec/packages/ec-component-accordion/ecl-accordion.html.twig`);

                let aliases = new this.Context();
                
                this.blockHandlers = new Map([
                    ['content', async (context, blocks = new Map()) => {
                        let aliases = this.aliases.clone();
                        this.echo((context.has(`_content`) ? context.get(`_content`) : null));
                    }]
                ]);
            }

            async doDisplay(context, blocks = new Map()) {
                let aliases = this.aliases.clone();

                this.startOutputBuffer();
                this.echo(`
`);
                this.echo(`
`);
                this.echo(`
`);
                context.proxy[`_css_class`] = `ecl-accordion`;
                context.proxy[`_extra_attributes`] = `data-ecl-auto-init="Accordion"`;
                context.proxy[`_items`] = (((context.has(`items`))) ? (await this.env.getFilter('default').traceableCallable(30, this.getSourceContext())(...[(context.has(`items`) ? context.get(`items`) : null), new Map([])])) : (new Map([])));
                this.echo(`
`);
                this.echo(`
`);
                if (!!((context.has(`extra_classes`)) && !(await this.env.getTest('empty').traceableCallable(34, this.getSourceContext())(...[(context.has(`extra_classes`) ? context.get(`extra_classes`) : null)])))) {
                    this.echo(`  `);
                    context.proxy[`_css_class`] = (this.concatenate((this.concatenate((context.has(`_css_class`) ? context.get(`_css_class`) : null), ` `)), (context.has(`extra_classes`) ? context.get(`extra_classes`) : null)));
                }
                this.echo(`
`);
                if (!!(!!((context.has(`extra_attributes`)) && !(await this.env.getTest('empty').traceableCallable(38, this.getSourceContext())(...[(context.has(`extra_attributes`) ? context.get(`extra_attributes`) : null)]))) && await this.env.getTest('iterable').traceableCallable(38, this.getSourceContext())(...[(context.has(`extra_attributes`) ? context.get(`extra_attributes`) : null)]))) {
                    this.echo(`  `);
                    context.set('_parent', context.clone());

                    await (async () => {
                        let c = this.ensureTraversable((context.has(`extra_attributes`) ? context.get(`extra_attributes`) : null));

                        if (c === context) {
                            context.set('_seq', context.clone());
                        }
                        else {
                            context.set('_seq', c);
                        }
                    })();

                    await this.iterate(context.get('_seq'), async (__key__, __value__) => {
                        context.proxy[`_key`] = __key__;
                        context.proxy[`attr`] = __value__;
                        this.echo(`    `);
                        if (await this.traceableMethod(this.getAttribute, 40, this.getSourceContext())(this.env, context.get(`attr`), `value`, new Map([]), `any`, true, true, false)) {
                            this.echo(`      `);
                            context.proxy[`_extra_attributes`] = (this.concatenate((this.concatenate((this.concatenate((this.concatenate((this.concatenate((context.has(`_extra_attributes`) ? context.get(`_extra_attributes`) : null), ` `)), await this.traceableMethod(this.getAttribute, 41, this.getSourceContext())(this.env, context.get(`attr`), `name`, new Map([]), `any`, false, false, false))), `="`)), await this.env.getFilter('e').traceableCallable(41, this.getSourceContext())(...[this.env, await this.traceableMethod(this.getAttribute, 41, this.getSourceContext())(this.env, context.get(`attr`), `value`, new Map([]), `any`, false, false, false)]))), `"`));
                            this.echo(`    `);
                        }
                        else {
                            this.echo(`      `);
                            context.proxy[`_extra_attributes`] = (this.concatenate((this.concatenate((context.has(`_extra_attributes`) ? context.get(`_extra_attributes`) : null), ` `)), await this.traceableMethod(this.getAttribute, 43, this.getSourceContext())(this.env, context.get(`attr`), `name`, new Map([]), `any`, false, false, false)));
                            this.echo(`    `);
                        }
                        this.echo(`  `);
                    });
                    (() => {
                        let parent = context.get('_parent');
                        context.delete('_seq');
                        context.delete('_iterated');
                        context.delete('_key');
                        context.delete('attr');
                        context.delete('_parent');
                        context.delete('loop');
                        for (let [k, v] of parent) {
                            if (!context.has(k)) {
                                context.set(k, v);
                            }
                        }
                    })();
                }
                this.echo(`
`);
                this.echo(`
<div
  class="`);
                this.echo(await this.env.getFilter('escape').traceableCallable(51, this.getSourceContext())(...[this.env, (context.has(`_css_class`) ? context.get(`_css_class`) : null), `html`, null, true]));
                this.echo(`"`);
                this.echo((context.has(`_extra_attributes`) ? context.get(`_extra_attributes`) : null));
                this.echo(`
  data-ecl-accordion="true"
>
  `);
                if (!(await this.env.getTest('empty').traceableCallable(54, this.getSourceContext())(...[(context.has(`_items`) ? context.get(`_items`) : null)]))) {
                    this.echo(`    `);
                    context.set('_parent', context.clone());

                    await (async () => {
                        let c = this.ensureTraversable((context.has(`_items`) ? context.get(`_items`) : null));

                        if (c === context) {
                            context.set('_seq', context.clone());
                        }
                        else {
                            context.set('_seq', c);
                        }
                    })();

                    context.set('loop', new Map([
                      ['parent', context.get('_parent')],
                      ['index0', 0],
                      ['index', 1],
                      ['first', true]
                    ]));
                    if ((typeof context.get('_seq') === 'object') && this.isCountable(context.get('_seq'))) {
                        let length = this.count(context.get('_seq'));
                        let loop = context.get('loop');
                        loop.set('revindex0', length - 1);
                        loop.set('revindex', length);
                        loop.set('length', length);
                        loop.set('last', (length === 1));
                    }
                    await this.iterate(context.get('_seq'), async (__key__, __value__) => {
                        context.proxy[`_key`] = __key__;
                        context.proxy[`_item`] = __value__;
                        this.echo(`      <h`);
                        this.echo(await this.env.getFilter('escape').traceableCallable(56, this.getSourceContext())(...[this.env, ((await this.traceableMethod(this.getAttribute, 56, this.getSourceContext())(this.env, context.get(`_item`), `level`, new Map([]), `any`, true, true, false)) ? (await this.env.getFilter('default').traceableCallable(56, this.getSourceContext())(...[await this.traceableMethod(this.getAttribute, 56, this.getSourceContext())(this.env, context.get(`_item`), `level`, new Map([]), `any`, false, false, false), 3])) : (3)), `html`, null, true]));
                        this.echo(` class="ecl-accordion__title">
        `);
                        this.echo(await this.include(context, this.getSourceContext(), {}, new Map([[`label`, await this.traceableMethod(this.getAttribute, 58, this.getSourceContext())(this.env, await this.traceableMethod(this.getAttribute, 58, this.getSourceContext())(this.env, context.get(`_item`), `toggle`, new Map([]), `any`, false, false, false), `label`, new Map([]), `any`, false, false, false)], [`variant`, `ghost`], [`type`, `button`], [`icon`, await this.traceableMethod(this.getAttribute, 61, this.getSourceContext())(this.env, await this.traceableMethod(this.getAttribute, 61, this.getSourceContext())(this.env, context.get(`_item`), `toggle`, new Map([]), `any`, false, false, false), `icon`, new Map([]), `any`, false, false, false)], [`icon_position`, `before`], [`extra_classes`, `ecl-accordion__toggle`], [`extra_attributes`, new Map([[0, new Map([[`name`, `aria-controls`], [`value`, await this.traceableMethod(this.getAttribute, 65, this.getSourceContext())(this.env, context.get(`_item`), `id`, new Map([]), `any`, false, false, false)]])], [1, new Map([[`name`, `data-ecl-accordion-toggle`]])]])]]), false, false, 57));
                        this.echo(`      </h`);
                        this.echo(await this.env.getFilter('escape').traceableCallable(69, this.getSourceContext())(...[this.env, ((await this.traceableMethod(this.getAttribute, 69, this.getSourceContext())(this.env, context.get(`_item`), `level`, new Map([]), `any`, true, true, false)) ? (await this.env.getFilter('default').traceableCallable(69, this.getSourceContext())(...[await this.traceableMethod(this.getAttribute, 69, this.getSourceContext())(this.env, context.get(`_item`), `level`, new Map([]), `any`, false, false, false), 3])) : (3)), `html`, null, true]));
                        this.echo(`>
      <div
        class="ecl-accordion__content"
        hidden
        id="`);
                        this.echo(await this.env.getFilter('escape').traceableCallable(73, this.getSourceContext())(...[this.env, await this.traceableMethod(this.getAttribute, 73, this.getSourceContext())(this.env, context.get(`_item`), `id`, new Map([]), `any`, false, false, false), `html`, null, true]));
                        this.echo(`"
        role="region"
      >`);
                        context.proxy[`_content`] = ((await this.traceableMethod(this.getAttribute, 76, this.getSourceContext())(this.env, context.get(`_item`), `content`, new Map([]), `any`, true, true, false)) ? (await this.env.getFilter('default').traceableCallable(76, this.getSourceContext())(...[await this.traceableMethod(this.getAttribute, 76, this.getSourceContext())(this.env, context.get(`_item`), `content`, new Map([]), `any`, false, false, false), ``])) : (``));
                        await this.traceableDisplayBlock(77, this.getSourceContext())('content', context.clone(), blocks);
                        this.echo(`</div>
    `);
                        (() => {
                            let loop = context.get('loop');
                            loop.set('index0', loop.get('index0') + 1);
                            loop.set('index', loop.get('index') + 1);
                            loop.set('first', false);
                            if (loop.has('length')) {
                                loop.set('revindex0', loop.get('revindex0') - 1);
                                loop.set('revindex', loop.get('revindex') - 1);
                                loop.set('last', loop.get('revindex0') === 0);
                            }
                        })();
                    });
                    (() => {
                        let parent = context.get('_parent');
                        context.delete('_seq');
                        context.delete('_iterated');
                        context.delete('_key');
                        context.delete('_item');
                        context.delete('_parent');
                        context.delete('loop');
                        for (let [k, v] of parent) {
                            if (!context.has(k)) {
                                context.set(k, v);
                            }
                        }
                    })();
                    this.echo(`  `);
                }
                this.echo(`</div>

`);
                this.echo(this.getAndCleanOutputBuffer().replace(/>\s+</g, '><').trim());
            }

            isTraitable() {
                return false;
            }

        }],
    ]);
};

    return module.exports;
})();

env.registerTemplatesModule(templatesModule, '/home/kalata/Projects/ec/@ecl/ecl-twig/src/ec/packages/ec-component-accordion/ecl-accordion.html.twig');

let template = env.loadTemplate('/home/kalata/Projects/ec/@ecl/ecl-twig/src/ec/packages/ec-component-accordion/ecl-accordion.html.twig');

module.exports = (context = {}) => {
    return template.render(context);
};

I haven't dig deep into twing, but it feels like this part of the code is added dynamically on some event and is not an actual code coming from what we take from node_modules, otherwise it should have been transpiled.

If helpful, I'm pasting here what the webpack config looks like after storybook and all, debugger after all the loaders setup

{
  "mode": "development",
  "bail": false,
  "devtool": "#cheap-module-source-map",
  "entry": [
    "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/@storybook/core/dist/server/common/polyfills.js",
    "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/@storybook/core/dist/server/preview/globals.js",
    "/home/kalata/Projects/ec/@ecl/ecl-twig/src/ec/.storybook/config.js",
    "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/webpack-hot-middleware/client.js?reload=true&quiet=true"
  ],
  "output": {
    "path": "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/@storybook/core/dist/public",
    "filename": "[name].[hash].bundle.js",
    "publicPath": ""
  },
  "plugins": [
    {
      "options": {
        "template": "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/@storybook/core/dist/server/templates/index.ejs",
        "templateContent": false,
        "filename": "iframe.html",
        "hash": false,
        "inject": false,
        "compile": true,
        "favicon": false,
        "minify": "auto",
        "cache": true,
        "showErrors": true,
        "chunks": "all",
        "excludeChunks": [],
        "chunksSortMode": "none",
        "meta": {},
        "base": false,
        "title": "Webpack App",
        "xhtml": false,
        "alwaysWriteToDisk": true
      },
      "version": 4
    },
    {
      "definitions": {
        "process.env": {
          "NODE_ENV": "\"development\"",
          "NODE_PATH": "\":/home/kalata/.nvm/versions/node/v10.17.0/lib/node_modules/ndb/lib/preload\"",
          "PUBLIC_URL": "\".\""
        },
        "NODE_ENV": "\"development\""
      }
    },
    {
      "nodeModulesPath": "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules"
    },
    {
      "options": {},
      "fullBuildTimeout": 200,
      "requestTimeout": 10000
    },
    {
      "options": {},
      "pathCache": {},
      "fsOperations": 0,
      "primed": false
    },
    {
      "profile": false,
      "modulesCount": 500,
      "showEntries": false,
      "showModules": true,
      "showActiveModules": true
    },
    {
      "definitions": {}
    },
    {
      "resourceRegExp": {}
    }
  ],
  "module": {
    "rules": [
      {
        "test": {},
        "use": [
          {
            "loader": "babel-loader",
            "options": {
              "cacheDirectory": "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/.cache/storybook",
              "presets": [
                [
                  "@babel/preset-env",
                  {
                    "shippedProposals": true,
                    "useBuiltIns": "usage",
                    "corejs": 3
                  }
                ]
              ],
              "plugins": [
                "@babel/plugin-transform-arrow-functions"
              ],
              "babelrc": false
            }
          }
        ],
        "include": [
          "/home/kalata/Projects/ec/@ecl/ecl-twig",
          "/home/kalata/Projects/ec/@ecl/ecl-twig"
        ],
        "exclude": {}
      },
      {
        "test": {},
        "use": [
          {
            "loader": "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/raw-loader/dist/cjs.js"
          }
        ]
      },
      {
        "test": {},
        "use": [
          {
            "loader": "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/html-loader/index.js"
          }
        ]
      },
      {
        "test": {},
        "sideEffects": true,
        "use": [
          "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/style-loader/index.js",
          {
            "loader": "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/css-loader/dist/cjs.js",
            "options": {
              "importLoaders": 1
            }
          },
          {
            "loader": "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/postcss-loader/src/index.js",
            "options": {
              "ident": "postcss",
              "postcss": {}
            }
          }
        ]
      },
      {
        "test": {},
        "loader": "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/file-loader/dist/cjs.js",
        "query": {
          "name": "static/media/[name].[hash:8].[ext]"
        }
      },
      {
        "test": {},
        "loader": "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/url-loader/dist/cjs.js",
        "query": {
          "limit": 10000,
          "name": "static/media/[name].[hash:8].[ext]"
        }
      },
      {
        "test": {},
        "loader": "twing-loader",
        "options": {
          "environmentModulePath": "/home/kalata/Projects/ec/@ecl/ecl-twig/src/ec/.storybook/environment.js"
        }
      }
    ]
  },
  "resolve": {
    "extensions": [
      ".mjs",
      ".js",
      ".jsx",
      ".json"
    ],
    "modules": [
      "node_modules",
      "/home/kalata/.nvm/versions/node/v10.17.0/lib/node_modules/ndb/lib/preload"
    ],
    "alias": {
      "babel-runtime/core-js/object/assign": "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/core-js/es/object/assign.js",
      "react": "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/react",
      "react-dom": "/home/kalata/Projects/ec/@ecl/ecl-twig/node_modules/react-dom"
    }
  },
  "optimization": {
    "splitChunks": {
      "chunks": "all"
    },
    "runtimeChunk": true,
    "minimizer": [
      {
        "options": {
          "test": {},
          "extractComments": false,
          "sourceMap": true,
          "cache": true,
          "parallel": true,
          "terserOptions": {
            "output": {
              "comments": {}
            },
            "mangle": false,
            "keep_fnames": true
          }
        }
      }
    ]
  },
  "performance": {
    "hints": false
  }
}

[kalinchernev]

Just to let you know, I found out that some parts of the template are not transformed to code for IE11 because are in strings, for example in this patch.

I've reached a point to understand that there is quite a lot of code which is ES6+ which is not going to be run in IE11. (classes, promises, etc. which are not transpiled)

Do you think it'd be feasible to have these changes in the loader? Would it be realistic to aim to run twing in IE11?

[ericmorand]

I'm way more worried about the async / await support that Twing 4 templates use. I don't see how a transpiler could do anything about that.

Twing 3 templates should be transpilable for IE11 though. Not sure this is worth it though.

The issue, fundamentaly, comes from Storybook itself. It let the responsibility of rendering to the browser. We use a custom-written tool in my company that we wrote explicitly to be able to use Twig as our templating language but don't rely on the browser to compile/render them - in other words, compilation and rendering is done locally by nodejs and the browser just have to render the resulting HTML. It allows us to test our designs on any browser, even IE11.

[kalinchernev]

Thanks @ericmorand for the feedback!

I'm still learning for the toolchain and i'm not sure I understand you when you mention async/await support in twig templates. Is it a feature in twig itself or the compiled template function? If it's a twig feature which is to be supported because of the spec, then it's obviously not realistic to expect support for IE11.

I'll try to downgrade to v3 (on Monday) and see if it changes something. If it does then I'll lock it to this version and communicate that this is the highest which is to be supported in IE11.

In our case the support for IE11 is based on the fact that the usage is in a corporate infrastructure and many of the computers which people use are having IE11...

I'm new to Storybook as well and will need some time to figure for the latter rendering part if we can have a SSR for the production build, thanks for the idea!

[JohnAlbin]

@kalinchernev Did you ever get around the fact that the webpack loader is returning a Promise instead of a JS function (compiled from the Twig source)?

I think the twing-loader should be modified to resolve the Promise before giving it to webpack.

[kalinchernev]

No, never got it working, unfortunately.

[ericmorand]

Working on it right now. Should have something by the end of the day.

[ericmorand]

Tracked by NightlyCommit/twing-loader#21

[MichaelAllenWarner]

@kalinchernev @JohnAlbin

I don't know if this will be helpful to you, but I believe I've found a way to get Storybook working with twing-loader's Promises:

NightlyCommit/twing-loader#33 (comment)