Sourcemap from bundle is not utf-8 encoded

[SimenB]

This can be seen in sourcemap

This is the actual bundle generated

The letter å is wrong. Not sure if it's Webpack's fault or Sourcemap's?

Using Linux if that matters. Using devtool: '#source-map'

This is a letter only used in comments, not actual source, so shouldn't be a real problem.

[sokra]

https://github.com/webpack/webpack/blob/master/lib/SourceMapDevToolPlugin.js#L155

"utf-8" missing here. Would you like the write a PR?

[SimenB]

@sokra Sure, but I'm unsure where you mean I should put utf-8? Default encoding is utf-8 (https://nodejs.org/api/buffer.html#buffer_new_buffer_str_encoding & https://nodejs.org/api/buffer.html#buffer_buf_tostring_encoding_start_end)

[disjukr]

i think same issue here: #859

[rjruizes]

I came across this issue because I had a Unicode code point in my javascript that was being scrambled as â–¾. My main.js and the outputted bundle.js were like so:
main.js: console.log("\u25BE").
bundle.js (UTF-8): console.log("▾");

My workaround: console.log(String.fromCodePoint(0x25BE))
The workaround prevents webpack from unescaping the code point, and makes it to display correctly.

[sokra]

Did you send the correct UTF-8 HTTP Header from your server? Or did you add the utf-8 meta tag to the HTML page?

[rickihastings]

I'm still getting this issue. Specifically some files being produced are non-utf8 so they cannot be understood by Sentry when I'm uploading them as release artifacts.

That aside, from what I can gather the PR #1797 just seems to fix them when they're data-uri sourcemaps? How can I ensure that all files webpack outputs are utf-8 (using Windows)?

[bebraw]

Is this valid still? I can see the related PR was merged.

[SimenB]

Seeing as @rickihastings stil had the issue a couple of months later, probably not.

I don't have a code base available to reproduce on

[jordanmkoncz]

I'm also having this issue, and like @rickihastings, I came upon this issue due to my source map not being readable by Sentry because the file is not utf8. I'm currently using Webpack 1.13.0 on macOS Sierra. I'm going to try updating to Webpack 1.14.0 and seeing if that resolves this issue - if it does then I'll report back here.

[yuri1992]

@jordanmkoncz did it worked for you ?

[katranci]

@bebraw I believe this issue is still valid. I'm using webpack@2.5.1 and I've got a multi-entry configuration. When I run file -bi *.map I can see that not all sourcemap files are in utf-8.

text/plain; charset=us-ascii
text/plain; charset=us-ascii
text/plain; charset=utf-8
text/plain; charset=us-ascii
text/plain; charset=utf-8
text/plain; charset=us-ascii
text/plain; charset=utf-8

What could cause a sourcemap file to be encoded in non-utf8 character set?

[mikeaustin]

Not sure if this is related, but I just fond I need to add charset="utf-8" to the bundle.js script tag:

<script src="bundle.js" type="text/javascript" charset="utf-8"></script>

Without the charset attribute, I get weird characters even with just &#9632;

[paul-sachs]

Also running into this. Doesn't seem to be any pattern, i have files either encoded as utf-8 or us-ascii.

0.e6063d8a791459c43093.chunk.js.map:                 text/plain; charset=us-ascii
1.e6063d8a791459c43093.chunk.js.map:                 text/plain; charset=utf-8
10.e6063d8a791459c43093.chunk.js.map:                text/plain; charset=utf-8
11.e6063d8a791459c43093.chunk.js.map:                text/plain; charset=utf-8
12.e6063d8a791459c43093.chunk.js.map:                text/plain; charset=utf-8
13.e6063d8a791459c43093.chunk.js.map:                text/plain; charset=utf-8
14.e6063d8a791459c43093.chunk.js.map:                text/plain; charset=utf-8
15.e6063d8a791459c43093.chunk.js.map:                text/plain; charset=utf-8
16.e6063d8a791459c43093.chunk.js.map:                text/plain; charset=us-ascii

In fact, this is not limited to source maps. My bundles are also scattered, some utf-8, some us-ascii. I'd like them to all be utf-8 encoded. Can this be forced? I tried webpack-encoding-plugin but it didn't work.

Using webpack 3.2

[sokra]

They are always utf-8 encoded. If you only use characters from the ascii range, us-ascii and utf-8 encoding look equal.

> Buffer.from("abc", "utf-8")
<Buffer 61 62 63>
> Buffer.from("abc", "ascii")
<Buffer 61 62 63>

> Buffer.from("abcä", "ascii")
<Buffer 61 62 63 e4>
> Buffer.from("abcä", "utf-8")
<Buffer 61 62 63 c3 a4>

[paul-sachs]

hmm, interesting. I am trying to use a tool (sentry-cli) that uploads sourcemaps and it explodes saying "stream did not contain valid UTF-8". Might be something specific on their end in that case.

[newtonsheikh]

Thank you. Was facing the same problem text/plain; charset=utf-8 solved similar problem in our code. Cheers!

[webpack-bot]

This issue had no activity for at least half a year.

It's subject to automatic issue closing if there is no activity in the next 15 days.

[webpack-bot]

Issue was closed because of inactivity.

If you think this is still a valid issue, please file a new issue with additional information.