Issues with UTF-8 characters

[tctruckscience]

I've come across an issue when trying to create a PDF that is written in Vietnamese. If you try this object in the playground you will see that the text on page 4 does not look right. If you copy the text from the generated PDF and paste it into Notepad then you will see that the text will be correct in Notepad.

var dd = {
"content":[
    {"style":"tableExample","table":{"widths":["*",200],"body":[["",""],["",""],["",""],["",""],["",{"text":"","margin":[0,0,0,20]}],["","05/01/2015"]]},"layout":"noBorders","margin":[0,20]},
    {"style":"tableExample","table":{"widths":[100,"*"],"body":[[{"text":"Chào giá sơ bộ","bold":true},{"text":"","bold":true}],["",{"text":"","bold":true}]]},"margin":[0,20],"layout":"noBorders"},
    {"text":"Vui lòng tìm bản tóm tắt của để tham khảo. Phiên bản sử dụng cho 30 ngày. Nếu bạn có thắc mắc nào, vui lòng liên lạc với chúng tôi.","margin":[0,20]},
    {"text":"Chào thân ái","margin":[0,20,0,5]},
    {"text":"","margin":[0,5]},{"text":"Job Title","margin":[0,20,0,5]},
    {"text":"Công ty: ","margin":[0,5]},
    {"text":"Di động: ","margin":[0,5]},
    {"style":"tableExample","table":{"widths":["*"],"body":[[{"text":"Xe của bạn","bold":true,"fontSize":16,"margin":[0,5]}]]},"margin":[0,0,0,10],"layout":{},"pageBreak":"before"},
    {"text":"Sơ lược xe ","style":"page2Header"},
    {"style":"tableExample","table":{"widths":[250,"*"],"body":[["",{"style":"tableExample","table":{"widths":[300],"body":[[{"text":"","bold":true}],[{"text":""}],[{"text":"","margin":[0,0,0,20]}],[{"style":"tableExample","table":{"widths":["*","*"],"body":[[{"text":"Tổng trọng tải:","bold":true},{"text":""}],[{"text":"Tải trọng:","bold":true},{"text":""}],[{"text":"Công suất:","bold":true},{"text":""}],[{"text":"Cơ cấu trục:","bold":true},{"text":""}]]},"layout":"noBorders"}]]},"layout":"noBorders"}]]},"margin":[0,10],"layout":"noBorders"},
    {"text":"Cấu hình xe","style":"page2Header"},
    {"style":"tableExample","table":{"widths":["*",500,"*"],"body":[[[""],{"style":"tableExample","table":{"widths":["*","*","*","*"],"headerRows":1,"body":[["",{"text":"Trước","bold":true},{"text":"Sau","bold":true},{"text":"Tổng cộng","bold":true}],["Tự trọng bản thân","","",""],["Tải trọng","","",""],["Tổng tải trọng","","",""],["Tổng tải trọng cho phép","","",""]]},"layout":{}},[""]]]},"margin":[0,10],"layout":"noBorders"},
    {"style":"tableExample","table":{"widths":["*"],"body":[[{"text":"Chi phí của bạn","bold":true,"fontSize":16,"margin":[0,5]}]]},"margin":[0,0,0,10],"layout":{},"pageBreak":"before"},
    {"style":"tableExample","table":{"widths":["*",130,130],"body":[[[{"text":"Chi phí xe (gồm cabin & khung gầm)","style":"page3Bold","alignment":"left"}],[""],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[[{"text":"","style":"page3Bold"},{"text":"","style":"page3Bold"}]]},"layout":"noBorders"}]],[[{"text":"Chi phí thùng và phụ kiện","style":"page3Bold","alignment":"left"}],[""],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[[{"text":"","style":"page3Bold"},{"text":"","style":"page3Bold"}]]},"layout":"noBorders"}]],[[""],[""],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[["",""]]},"layout":{}}]],[[{"text":"Tổng chi phí vốn","style":"page3Bold","alignment":"left"}],[""],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[[{"text":"","style":"page3Bold"},{"text":"","style":"page3Bold"}]]},"layout":"noBorders","margin":[0,0,0,20]}]],[[{"text":"Chi phí đi đường ","style":"page3Bold","alignment":"left"}],[""],[""]],[["Phí trước bạ"],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[[{"text":"","alignment":"right"},{"text":"","alignment":"right"}]]},"layout":"noBorders"}],[""]],[[""],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[["",""]]},"layout":{}}],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[[{"text":"","alignment":"right"},{"text":"","alignment":"right"}]]},"layout":"noBorders"}]],[[""],[""],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[["",""]]},"layout":{}}]],[[""],[""],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[[{"text":"","style":"page3Bold"},{"text":"","style":"page3Bold"}]]},"layout":"noBorders","margin":[0,0,0,20]}]],[[{"text":"Chi phí tài chính ước tính","style":"page3Bold","alignment":"left"}],[""],[""]],[["Tổng chi phí vốn"],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[[{"text":"","alignment":"right"},{"text":"","alignment":"right"}]]},"layout":"noBorders"}],[""]],[["Thời hạn hoàn trả"],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[["",{"text":"","alignment":"right"}]]},"layout":"noBorders"}],[""]],[["Lãi suất"],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[["",{"text":"","alignment":"right"}]]},"layout":"noBorders"}],[""]],[["Đặt cọc"],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[["",{"text":"","alignment":"right"}]]},"layout":"noBorders"}],[""]],[["Phần còn lại (dư ra) Phần thanh toán thêm"],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[["",{"text":"","alignment":"right"}]]},"layout":"noBorders"}],[""]],[["Tổng số tiền"],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[[{"text":"","alignment":"right"},{"text":"","alignment":"right"}]]},"layout":"noBorders","margin":[0,0,0,20]}],[""]],[[{"text":"Số tiền thanh toán hàng tháng","style":"page3Bold","alignment":"left"}],[""],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[[{"text":"","style":"page3Bold"},{"text":"","style":"page3Bold"}]]},"layout":"noBorders"}]],[[{"text":"Tổng chi phí tài chính ","style":"page3Bold","alignment":"left"}],[""],[{"style":"tableExample","table":{"widths":[30,"*"],"body":[[{"text":"","style":"page3Bold"},{"text":"","style":"page3Bold"}]]},"layout":"noBorders"}]]]},"margin":[0,20],"layout":"noBorders"},
    {"style":"tableExample","table":{"widths":["*"],"body":[[{"text":"Thông số kĩ thuật cụ thể","bold":true,"fontSize":16,"margin":[0,5]}]]},"margin":[0,0,0,10],"layout":{},"pageBreak":"before"},
    {"style":"tableExample","table":{"widths":["*","210"],"body":[[[{"style":"tableExample","table":{"widths":["*"],"body":[[{"text":"TRỌNG LƯỢNG","style":"page4Header"}],[[{"style":"tableExample","table":{"widths":[180,40,"*"],"body":[["Tổng trọng tải","",""],["Tự trọng bản thân","",""],["Trọng lượng thùng và tải trọng ","",""]]},"margin":[10,0,0,0],"layout":"noBorders"}]],[{"text":"KÍCH THƯỚC","style":"page4Header"}],[[{"style":"tableExample","table":{"widths":[180,40,"*"],"body":[["Chiều dài cơ sở","mm","3.350"],["Chiều dài tổng thể","mm","5.885"],["Chiều rộng tổng thể","mm","1.870"],["Chiều cao tổng thể","mm","2.055"],["Chiều dài sắt-xi lắp thùng","mm","4.303"],["Bán kính quay vòng nhỏ nhất","mm","6.600"]]},"margin":[10,0,0,0],"layout":"noBorders"}]]]},"layout":"noBorders"}],[{"style":"tableExample","table":{"widths":["*"],"body":[[""],[""]]},"layout":"noBorders","margin":[0,30,0,0]}]]]},"layout":"noBorders"},
    {"text":"ĐỘNG CƠ","style":"page4Header"},
    {"style":"tableExample","table":{"widths":[180,40,"*"],"body":[["Loại động cơ ","","Động cơ diesel 4 xylanh, 4 kì, bơm nhiên liệu trực tiếp, làm mát bằng nước với turbo tăng áp"],["Công suất cực đại (PS @ rpm)","","110 @ 2.900 - 2.900"],["Moment xoắn cực đại (Nm @ rpm)","","275 @ 1.600 - 1.600"],["Đường kính & hành trình pit-tô ","mm","104 x 115"],["Dung tích xi lạnh","cc","3.908"],["Tiêu chuẩn khí thải ","","Euro 2  "]]},"margin":[10,0,0,0],"layout":"noBorders"},
    {"text":"Hộp số ","style":"page4Header"},
    {"style":"tableExample","table":{"widths":[180,40,"*"],"body":[["Loại hộp số","","5 số tiến và 1 số lùi, bộ đồng tốc từ số 1 đến số 5"]]},"margin":[10,0,0,0],"layout":"noBorders"},
    {"text":"HỆ THỐNG PHANH","style":"page4Header"},
    {"style":"tableExample","table":{"widths":[180,40,"*"],"body":[["Hệ thốnh phanh chính","","Hệ thống phanh thủy lực mạch kép, trợ lực phanh bằng chân không, "]]},"margin":[10,0,0,0],"layout":"noBorders"},
    {"text":"LỐP XE","style":"page4Header"},
    {"style":"tableExample","table":{"widths":[180,40,"*"],"body":[["Thông số lốp (trước/sau)","","7.00-16-12PR"]]},"margin":[10,0,0,0],"layout":"noBorders"},
    {"text":"Hiệu suất ","style":"page4Header"},
    {"style":"tableExample","table":{"widths":[180,40,"*"],"body":[["Tốc độ tối đa","km/h","101"],["Khả năng leo dốc","%","47,0"]]},"margin":[10,0,0,0],"layout":"noBorders"}
],
"defaultStyle":{"color":"black","fontSize":10},
"tableExample":{"margin":[0,5,0,15]},
"styles":{"page2Header":{"bold":true,"fontSize":12,"margin":[0,10]},"page3Bold":{"bold":true,"alignment":"right"},"page4Header":{"bold":true,"margin":[0,10,0,5],"fontSize":12}},
"pageMargins":40
};

If you compare page 4 of the above object with the one below which has removed the other 3 pages you will see that most of the text will display fine but there is still some problems near the bottom.

var dd = {
"content":[
    {"style":"tableExample","table":{"widths":["*"],"body":[[{"text":"Thông số kĩ thuật cụ thể","bold":true,"fontSize":16,"margin":[0,5]}]]},"margin":[0,0,0,10],"layout":{}},
    {"style":"tableExample","table":{"widths":["*","210"],"body":[[[{"style":"tableExample","table":{"widths":["*"],"body":[[{"text":"TRỌNG LƯỢNG","style":"page4Header"}],[[{"style":"tableExample","table":{"widths":[180,40,"*"],"body":[["Tổng trọng tải","",""],["Tự trọng bản thân","",""],["Trọng lượng thùng và tải trọng ","",""]]},"margin":[10,0,0,0],"layout":"noBorders"}]],[{"text":"KÍCH THƯỚC","style":"page4Header"}],[[{"style":"tableExample","table":{"widths":[180,40,"*"],"body":[["Chiều dài cơ sở","mm","3.350"],["Chiều dài tổng thể","mm","5.885"],["Chiều rộng tổng thể","mm","1.870"],["Chiều cao tổng thể","mm","2.055"],["Chiều dài sắt-xi lắp thùng","mm","4.303"],["Bán kính quay vòng nhỏ nhất","mm","6.600"]]},"margin":[10,0,0,0],"layout":"noBorders"}]]]},"layout":"noBorders"}],[{"style":"tableExample","table":{"widths":["*"],"body":[[""],[""]]},"layout":"noBorders","margin":[0,30,0,0]}]]]},"layout":"noBorders"},
    {"text":"ĐỘNG CƠ","style":"page4Header"},
    {"style":"tableExample","table":{"widths":[180,40,"*"],"body":[["Loại động cơ ","","Động cơ diesel 4 xylanh, 4 kì, bơm nhiên liệu trực tiếp, làm mát bằng nước với turbo tăng áp"],["Công suất cực đại (PS @ rpm)","","110 @ 2.900 - 2.900"],["Moment xoắn cực đại (Nm @ rpm)","","275 @ 1.600 - 1.600"],["Đường kính & hành trình pit-tô ","mm","104 x 115"],["Dung tích xi lạnh","cc","3.908"],["Tiêu chuẩn khí thải ","","Euro 2  "]]},"margin":[10,0,0,0],"layout":"noBorders"},
    {"text":"Hộp số ","style":"page4Header"},
    {"style":"tableExample","table":{"widths":[180,40,"*"],"body":[["Loại hộp số","","5 số tiến và 1 số lùi, bộ đồng tốc từ số 1 đến số 5"]]},"margin":[10,0,0,0],"layout":"noBorders"},
    {"text":"HỆ THỐNG PHANH","style":"page4Header"},
    {"style":"tableExample","table":{"widths":[180,40,"*"],"body":[["Hệ thốnh phanh chính","","Hệ thống phanh thủy lực mạch kép, trợ lực phanh bằng chân không, "]]},"margin":[10,0,0,0],"layout":"noBorders"},
    {"text":"LỐP XE","style":"page4Header"},
    {"style":"tableExample","table":{"widths":[180,40,"*"],"body":[["Thông số lốp (trước/sau)","","7.00-16-12PR"]]},"margin":[10,0,0,0],"layout":"noBorders"},
    {"text":"Hiệu suất ","style":"page4Header"},
    {"style":"tableExample","table":{"widths":[180,40,"*"],"body":[["Tốc độ tối đa","km/h","101"],["Khả năng leo dốc","%","47,0"]]},"margin":[10,0,0,0],"layout":"noBorders"}
],
"defaultStyle":{"color":"black","fontSize":10},
"tableExample":{"margin":[0,5,0,15]},
"styles":{"page2Header":{"bold":true,"fontSize":12,"margin":[0,10]},"page3Bold":{"bold":true,"alignment":"right"},"page4Header":{"bold":true,"margin":[0,10,0,5],"fontSize":12}},
"pageMargins":40
};

Is this a problem with how the object is structured?

[minhtuanna]

i has same error, please support!

[tomconneely]

I've come across the same issue. Will UTF-8 be supported in the future?

[jthoenes]

The problem is known and needs to be fixed in PDFkit, the library pdfmake is using for actual pdf generation. Have a look at these issues:

• Embedded font subset needs to be sharded to support pdf file with more than 127 glyphs foliojs/pdfkit#114
• Font Subset Failure with Multiple Languages foliojs/pdfkit#244 (comment)

[wwwmir3cq]

on 30 Jul 2014, pdfkit had been released the version v0.6.3 (https://github.com/devongovett/pdfkit/releases)
in the version,Support for using ArrayBuffers and Uint8Arrays for loading fonts in the browser,it had resolved the problem,your can demo it on http://pdfkit.org/demo/browser.html
like this coe:
var zzzTxt = "";
for(var zzz = 126; zzz < 160; zzz++){
zzzTxt = zzzTxt + String.fromCharCode(zzz);
}
doc
.fontSize(14)
.text(zzzTxt)
.text('1234567890!@#$%^&*()abcdefghijklmnopqrstuvwxyz')
.text('ABCDEFGHIJKLMNOPQRSTUVWXYZ');
you will be found it work fine!

but! in your pdfmake,your still use the new old pdfkit version(old than v0.6.3 )!
can you change it?but not just say like 'The problem is known and needs to be fixed in PDFkit, the library pdfmake is using for actual pdf generation. Have a look at these issues' ?

[jthoenes]

Hi @wwwmir3cq,

Thanks for the suggestion. We can upgrade the pdfkit library but the issue will not go away.

Why? There are two ways to use fonts in pdfs.

• Using a standard font - which is supported directly by your PDF viewer.
• Using a custom font embedded into the PDF.

The fix you present uses a standard font. But pdfmake only uses embedded fonts.

Using embedded fonts is easier for a library like pdfmake, because we can use it to calculate the width of a text.

However, I'm happy to accept a pull request when someone wants to offer using a standard font.

[minhtuanna]

how to fix this bug????

[wwwmir3cq]

Hi jthoenes
The first,Thanks for your great work!
I am not familiar with PDF data characteristics, and therefore does not know the standard font refers to what you say
(I found a STANDARDFONTS, in the pdfkit do not know what you said Is it right? Here)
Through your document, I know how to pass the nodejs TTF font files into the bytes stream data into vfonts.js
But I don't know how to put TTF into the standard font you mentioned.

I came directly realized by pdfkt lib, because pdfmake made packages on pdfkit, modify the pdfmake lib more complex
I try to study pdfkit fonts algorithm, but found it too complicated, needs to spend big energy to study, I give up,
I don't know why pdfkit author of the improved algorithm bug is so difficult to handle, remove the cache function OK? Or optimization, such as more than 127 different characters, we clear the cache!
Look for a simple but can solve my problem solution (this is a temporary solution)
1、modify the code
PDFFont.font = function(){
if (font = this._fontFamilies[this._font.name]) {
//annotation the following two lines code
//this._font = font;
// return this;
delete this._fontFamilies[this._font.name];
}
}
2、in my logic codes like this:
for(var i = 0; i < docDefine.content.length; i++){
_ref = docDefine.content[i];

if(_ref.text){
    _text = _ref.text;
    for(var j = 0; j < _text.length; j++){
        _refChrCode = _text.charCodeAt(j);
        if(!tmpMap[_refChrCode]){
            tmpMap[_refChrCode] = true;
            tmpCnt++

            if(tmpCnt % 80 == 0){
                tmpMap = {};
                tmpCnt = 0;
                var newFontName = 'msyh' + fontCnt;
                doc.registerFont(newFontName, arrayBuffer);
                doc.font(newFontName);
                fontCnt++;
            }
        }
    }
           doc.text(_text,x,y,{width,height,.......} )

But the way, this on the word of the relatively small number of cases can solve the problem, if the number of words is large, there is a serious performance issues ,because, arrayBuffer is very big,when we do doc.registerFont(newFontName, arrayBuffer) A lot of times，it will Occupy many memory.
(my scene is relatively simple, is to generate orders print templates)

[jthoenes]

There is a pull request #211 which potentially fixes this issue. Please review the code!

[JohnnieFucker]

i checkout this branch ,but the issue is still not resolved -_-!

[jthoenes]

@JohnnieFucker Have you made sure to run grunt build before checking it?

[gnibeda]

Seems that issue not resolved. Pdfmake builded from latest sources:

var dd = {
    content: [
        'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ',
        'абвгдежзиклмнопрстуфхйчшщьъэюяАБВГДЕЖЗИКЛМНОПРСТУФХЙЧШЩЬЪЭЮЯ',
        '1234567890'
    ]
}

[gnibeda]

any news about this issue?

[devongovett]

Hey guys, I maintain PDFKit, the library that pdfmake uses to actually write PDFs. I've been working on a new font engine called fontkit which will power PDFKit's font support going forward. It supports tons of advanced typographical features, including ligatures, kerning, contextual substitutions, TrueType and PostScript outlines, etc. in both OpenType and AAT fonts. There's a branch of PDFKit using fontkit currently in progress. It embeds fonts as PDF CID fonts, which allows for much better support for large character sets.

I hope to release it very soon (working on a few final issues), but if you're feeling adventurous, please try out the fontkit branch of PDFKit and see if it works with pdfmake. It is likely that some modifications will be needed here if pdfmake accesses font properties directly, so I thought I'd let you all know so you know where we're heading. Let me know if you have any issues!

[gnibeda]

I have an idea of partial workaround in current version of pdfmake.
I know that embedded fonts used to perform calculations, but can we make pdf without embedding font? Just use vs_fonts for internal calculations but does not embed font in final pdf? As I know, problem with unicode for non-embedded fonts was solved in pdfkit now.
This temporary workaround can help to work with unicode right now.

[bpampuch]

@devongovett sounds great

[jthoenes]

@gnibeda Can you check with the 0.1.18.

[gnibeda]

In 0.1.18. issue was resolved. Tested on example above and in real application. Thank you!

Can you take a look at:
#247
Or tell me where to look in code, part that responsible for cell drawing in repeatable header?

[tctruckscience]

@jthoenes I tested out my example and it looks to be working correctly. That's great news. Thanks.

[lvarayut]

@jthoenes , thanks so much. I confirmed that the 0.1.18 version worked like a charm!

[Ranth67]

I am using pdfmake to write down documents in danish, and I have problems using the danish letters æøå. I don't really see how you get it to work here in this post, can someone explain it?

Btw, really nice library!

[loverdrive]

I'm using 0.1.18, but still have problems with japanese.
The character also change depending on which viewer i use to open generated pdf. In Chrome internal viewer, '8' and '9' are changed with a japanese character, but if i copy that character and paste in another place (text editor), i see the correct '8' character. If i open the same pdf with Adobe Reader, i view a black small square insted of '8' and '9'.
Also, in Adobe Reader i see the same black small square instead of 'x', but in chrome internal viewer i see x correctly.

I'm using Japanese fonts, but if i generate a pdf only with latin character, i see all ok. If i generate a pdf with also a japanese character, i see that problem i described above.

[javier-dominguez]

I have problems with fonts on v0.2.0-alpha.0 and v0.1.71 versions in Chrome

I'm using default font (ROBOTO) and custom fonts and in both case I've the same result.

The font is renders correctly when the PDF is generated in Firefox, but not in Chrome.

avión año (Firefox)
avión año (Chrome)

It's not working when I create a pdf in local using (Chrome) but if I run the code in Firefox or in http://pdfmake.org/playground.html it's working perfectly.

I'm working on MacOS.
what Am I supposed to do?
I suspect it's a pdfmake issue.