HTML broken since 2.10.0

[gartmeier]

Parsing a HTML file throws the following error

$ npx gettext-extract po.html
[easygettext] extracting: 'po.html
[easygettext] could not read: 'po.html
Trace: SyntaxError: Unexpected token (1:0)
    at Object.pp$4.raise (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:2825:15)
    at Object.pp.unexpected (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:689:10)
    at Object.pp$3.parseExprAtom (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:2270:12)
    at Object.parseExprAtom (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn-dynamic-import/lib/index.js:75:117)
    at Object.parseExprAtom (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn-import-meta/index.js:17:75)
    at Object.pp$3.parseExprSubscripts (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:2089:21)
    at Object.pp$3.parseMaybeUnary (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:2066:19)
    at Object.parseMaybeUnary (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn-private-class-elements/index.js:141:29)
    at Object.pp$3.parseExprOps (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:2010:21)
    at Object.pp$3.parseMaybeConditional (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:1993:21)
    at Object.pp$3.parseMaybeAssign (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:1968:21)
    at Object.pp$3.parseExpression (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:1933:21)
    at Object.pp$1.parseStatement (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:877:47)
    at Object.parseStatement (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn-dynamic-import/lib/index.js:63:118)
    at Object.parseStatement (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn-import-meta/index.js:39:22)
    at Object.pp$1.parseTopLevel (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/acorn/dist/acorn.js:746:23) {
  pos: 0,
  loc: Position { line: 1, column: 0 },
  raisedAt: 1
}
    at /Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/easygettext/src/extract-cli.js:74:13
    at Array.forEach (<anonymous>)
    at Object.<anonymous> (/Users/joshua/Documents/Workspace/pi_js_sdk/node_modules/easygettext/src/extract-cli.js:60:7)
    at Module._compile (internal/modules/cjs/loader.js:1147:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:10)
    at Module.load (internal/modules/cjs/loader.js:996:32)
    at Function.Module._load (internal/modules/cjs/loader.js:896:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    at /usr/local/lib/node_modules/npm/node_modules/libnpx/index.js:268:14

If I install version 2.10.0 everything works as expected:

$ npx gettext-extract po.html                        
[easygettext] extracting: 'po.html
msgid ""
msgstr ""
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: easygettext\n"
"Project-Id-Version: \n"

#: po.html:31
msgid "Broken strings are joined"
msgstr ""

#: po.html:14
msgctxt "According to..."
msgid "Hello World"
msgstr ""

#. My comment...
#: po.html:12
#: po.html:16
#: po.html:18
#: po.html:20
#: po.html:24
#: po.html:26
#: po.html:28
msgid "Hello World"
msgid_plural "Hello worlds"
msgstr[0] ""
msgstr[1] ""

#: po.html:42
msgid "Something …"
msgstr ""

I believe it is caused by the typescript support added in 2.11.0. It always tries to parse JavaScript, if the file is not TypeScript:

po.html

<div translate>Hello World</div>

<div translate translate-context="According to...">Hello World</div>

<div translate translate-comment="My comment...">Hello World</div>

<div translate translate-plural="Hello worlds">Hello World</div>

<div placeholder="{{ 'Hello World' | translate }}"></div>

<div placeholder="{{ scopeVariable || ('Hello World' | translate) }}"></div>

<get-text>Hello World</get-text>

<i18n>Hello World</i18n>

<translate>Hello World</translate>

<!--  The following becomes 'Broken strings are joined'  -->
<span ng-bind="{{ 'Broken '
 + 'strings ' +
 'are ' +
 'joined' |translate}}"></span>

<span ng-bind="'Bed n\'' + ' breakfast' |translate"></span>

<!-- JavaScript expressions are parsed and compiled -->
<span ng-bind="true ? 'Always' : 'Never' |i18n "></span>

<!--  By supplying the  --filterPrefix '::' parameter  -->
<span>{{:: 'Something …' |translate}}</span>

<!--  The default delimiters '{{' and '}}' must be changed to empty strings to handle these examples  -->
<span ng-bind=":: 'Something …' |translate"></span>

<div placeholder="'Hello World' | translate"></div>

[Shin--]

@torfeld6 Sorry, likely my fault. I tried to find the cause of the bug and it wasn't so much that parseJavascript was the fallback (it was the same before), but that preprocessScript always returns content and language now. Before we passed an empty string to preprocessScript.

Since you have already digged into the code, can you check if changing line 260 in extract.js to

      ({content, lang}) => {
        if (lang) {
          (lang === 'ts' || (!lang  && ext === 'ts'))
            ?  this.parseTypeScript(filename, content)
            :  this.parseJavascript(filename, content)
        }
      }

solves your problem? I would do a PR, but I only added some html templates to my project to reproduce your problem (it solves it and adds the strings to the .po files) and am not sure if there aren't some other cases I might have missed.

[gartmeier]

Hi @Shin-- thank you for the quick reply. I tried to address the issue in the PR #86

[janlazo]

@torfeld6 Is this resolved in the latest release?

[vperron]

@torfeld6 do you think you resolved the issue ? Or do you think you can fix this one as well as #93 in your incoming fix ? 🙏

[gartmeier]

@torfeld6 do you think you resolved the issue ? Or do you think you can fix this one as well as #93 in your incoming fix ? 🙏

The HTML issue has been resolved with the release of 2.14. And here's the pull request #94 for #93