RTL and Script Unicode Font (ttf), and ligatures not supported

[sh0umik]

Describe the bug
Bangla Unicode Fonts gets Broken after generation of the PDF. The Unicode font works well in web. But its got broken when

I made a repository to demonstrate the problem. ( font included ). I have tried with 12 different Bangla Unicode font. Same result for all.
Just run the program and you will see that the String in Text widget didnt get rendered properly.

https://github.com/sh0umik/flutter_pdf_test.git

Expected behaviour
Expected behaviour is exactly like the test in Text widget

Screenshots
Got this (broken)

Expected this

Flutter Doctor
Paste the output of running flutter doctor -v here.

[✓] Flutter (Channel beta, v1.12.13+hotfix.6, on Mac OS X 10.14.6 18G103, locale en-BD)
    • Flutter version 1.12.13+hotfix.6 at /Users/diablo/flutter
    • Framework revision 18cd7a3601 (3 weeks ago), 2019-12-11 06:35:39 -0800
    • Engine revision 2994f7e1e6
    • Dart version 2.7.0

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
    • Android SDK at /Users/diablo/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-29, build-tools 29.0.2
    • Java binary at: /Users/diablo/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/191.6010548/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.1, Build version 11A1027
    • CocoaPods version 1.8.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 3.5)
    • Android Studio at /Users/diablo/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/191.6010548/Android Studio.app/Contents
    • Flutter plugin version 42.1.1
    • Dart plugin version 191.8593
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)

[!] IntelliJ IDEA Ultimate Edition (version 2019.3)
    • IntelliJ at /Users/diablo/Applications/JetBrains Toolbox/IntelliJ IDEA Ultimate.app
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • For information about installing plugins, see
      https://flutter.dev/intellij-setup/#installing-the-plugins

[✓] VS Code (version 1.41.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.7.1

[✓] Connected device (3 available)
    • ONEPLUS A6010 • 192.168.0.100:5555 • android-arm64  • Android 10 (API 29)
    • Chrome        • chrome             • web-javascript • Google Chrome 79.0.3945.88
    • Web Server    • web-server         • web-javascript • Flutter Tools

Desktop (please complete the following information):

• [*] iOS
• [*] Android
• [*] Browser

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. all browser, sa]
• Version [e.g. 22]

Additional context
Many thing on my flutter web app depends on this plugin. Need to find a solution ASAP.
Also i am converting the pdf.save() into a Uint8List so that it could be sent through firebase.
Could this be a problem ? List to Uint8List conversation ?

[DavBfr]

Sorry, I don't really see what's wrong in the rendering. I don't read Bengali and can't really spot the missing features.
or is it just the bold face?

[sh0umik]

There is ! Look at the picture. The broken text are joined text its like two latter joined and formed a single one. The formed/joined text is broken.

Flutter widget Text renders it correctly in web, andorid, ios .. but yours pdf lib widget Text does not render it correctly.

[sh0umik]

Similar Problem is mentioned here .. But its for PHP pdf generation lib.

https://stackoverflow.com/questions/32421564/bangla-unicode-font-not-rendering-correctly-in-tcpdf

[DavBfr]

Ok, I guess it's about GSUB support for OTF fonts:
https://docs.microsoft.com/en-us/typography/opentype/spec/gsub

[sh0umik]

I guess so.

https://docs.microsoft.com/en-us/typography/script-development/bengali

This is the problem I am getting whats left side fo -> but i need the output which is at the right side of the ->

[sh0umik]

@DavBfr i tried the arabic-fonts branch. It didnt work for me :(

[DavBfr]

No, the arabic-fonts branch uses some specific character replacement for Arabic only. GSUB is not yet implemented.

If you're willing to implement it in ttf_parser.dart I'll be happy to help.

Try something like this:

diff --git a/pdf/lib/src/ttf_parser.dart b/pdf/lib/src/ttf_parser.dart
index 3108a48..fc7cce0 100644
--- a/pdf/lib/src/ttf_parser.dart
+++ b/pdf/lib/src/ttf_parser.dart
@@ -54,6 +54,9 @@ class TtfParser {
     _parseCMap();
     _parseIndexes();
     _parseGlyphs();
+    if (tableOffsets.containsKey(gsub_table)) {
+      _parseGsub();
+    }
   }

   static const String head_table = 'head';
@@ -64,6 +67,7 @@ class TtfParser {
   static const String maxp_table = 'maxp';
   static const String loca_table = 'loca';
   static const String glyf_table = 'glyf';
+  static const String gsub_table = 'GSUB';

   final UnmodifiableByteDataView bytes;
   final Map<String, int> tableOffsets = <String, int>{};
@@ -368,4 +372,20 @@ class TtfParser {
       components,
     );
   }
+
+  void _parseGsub() {
+    print(fontName);
+    print(tableOffsets);
+
+    final int basePosition = tableOffsets[gsub_table];
+    print('GSUB Version: ${bytes.getUint32(basePosition).toRadixString(16)}');
+    final int scriptListOffset =
+        bytes.getUint16(basePosition + 4) + basePosition;
+    final int featureListOffset =
+        bytes.getUint16(basePosition + 6) + basePosition;
+    final int lookupListOffset =
+        bytes.getUint16(basePosition + 8) + basePosition;
+    print(
+        'GSUB Offsets: $scriptListOffset $featureListOffset $lookupListOffset');
+  }
 }

And see if your font contains any useful information.

[sh0umik]

I am ready to implement it. Just tell me what to write ? It would be better if you could show me for just one character as example and where can i find the position to complete the rest then i can complete it. I am totally new to this.

[sh0umik]

@DavBfr this is what i found running the code above. Whats next ?

I/flutter (10254): SiyamRupali
I/flutter (10254): {EBDT: 393284, EBLC: 393460, GDEF: 399344, GPOS: 398924, GSUB: 393740, LTSH: 3756, OS/2: 504, VDMX: 4552, cmap: 399388, cvt : 25408, fpgm: 24280, gasp: 393268, glyf: 25416, hdmx: 6056, head: 380, hhea: 436, hmtx: 600, kern: 381088, loca: 377928, maxp: 472, name: 381112, post: 385900, prep: 25396}
I/flutter (10254): GSUB Version: 10000
I/flutter (10254): GSUB Offsets: 393750 393832 394122

[sh0umik]

Using this site called FontDrop

I can see the following...

Could it help implement the parser? If so then can you just guide me how to implement it ?

[DavBfr]

This site is really useful, thanks!

So the first step is to parse this GSUB table to find all the lookups.

Then I think when you have this:

{
  "ligGlyph": 237,
  "components": [
    102,
    86
  ]
}

if we want to draw the glyphs 102 and 86 next to eachother, we replace with 237

[DavBfr]

I think the tables to read for your issue is the Ligature Substitution Subtable:

https://docs.microsoft.com/en-us/typography/opentype/spec/gsub#lookuptype-4-ligature-substitution-subtable

Only one format, that should not be too difficult.

[sh0umik]

Thank you for your ans. I am starting to understand how this works but I am totally new to this and have no idea about the variables in the parser. Can you just write a function or a bloc of code just to parse this as an example

{
  "ligGlyph": 237,
  "components": [
    102,
    86
  ]
}

maybe after that i can follow that code and implement the rest ?

[DavBfr]

If you look at _parseGlyphs() in the same file, it will look the same:
read the binary file content using bytes.getInt16(offset); or getInt32 and friends.
Just follow the MS documentation to know what to read.

In the function parseGsub() your start offset is basePosition which gives you access to the offsets of the next tables : scriptList, featureList, lookupList.

The first to parse is the featureList described here:
https://docs.microsoft.com/en-us/typography/opentype/spec/chapter2#flTbl

and get all the features with the right tag, maybe https://docs.microsoft.com/en-us/typography/opentype/spec/features_ae#blws

Then the lookupList will give the right glyphs to replace.

[sh0umik]

Is there any chance you can add this feature soon ? I badly need this :( . Since i have no idea on the variable used in the parsing and how it works i think i wont be able to help much.

Eagerly waiting for your answer on this. Please, can find some of your spare time to add this feature ? Or Milestone ?

[sh0umik]

@DavBfr what functions to use for find and replace the glyphs

if we want to draw the glyphs 102 and 86 next to eachother, we replace with 237

I could not find any substitution function in ttf_parser ? is it in charToGlyphIndexMap map ?

[Peet-A]

how can i help you guys ?

[DavBfr]

I think all the needed info is on this ticket. I don't have time to work on this now. Unless some of you are willing to pay for the feature. Then I could take a look at crowdfunding. This also includes Arabic and other languages.

[DavBfr]

Please use this link:
https://www.bountysource.com/issues/86211023-bangla-unicode-font-ttf-gets-broken-during-pdf-generation

[ashutosh1211]

@DavBfr Hi, I'm able to prepare featureList , what to do next? should I prepare lookup table?

[DavBfr]

Yes, the lookup table is next.
I have some code already for that. Let me push it. to the branch arabic-fonts

[ashutosh1211]

Sure, :), I'm stuck at subtable parsing in lookup table.

[shofizone]

I am also facing same problem while generating PDF. Bangla Unicode Characters are broken.

[anandsubbu007]

I will suggest the shortcut (temporary solution) that I use to achieve the text in Tamil or whatever language.
Problem: For the Tamil language when க + ி = கி I didn't get this output. Because it fails when these two characters merge.
So, I did this trick as a temporary solution I downloaded Tamil font and with the help of a font editor, I replaced the unused symbols and replace it with the value (கி) that I need. And whenever that character comes in string i replace it with the symbol that I used.
That's it.
It is working fine for me. This is how I get Tamil font print.
This is only a temporary solution. i believe this will also work for other languages too.
😅

@anandsubbu007 Thanks for responding!. if possible, can you please share the code? i see the exact problem. It would be really appreciated if you could share the code as i could not find any other better than what you are suggesting for now. or if possible, can you please share the edited font for tamil.

https://github.com/anandsubbu007/anand-work/tree/BLOC/tamil_pdf

[techttiru]

I will suggest the shortcut (temporary solution) that I use to achieve the text in Tamil or whatever language.
Problem: For the Tamil language when க + ி = கி I didn't get this output. Because it fails when these two characters merge.
So, I did this trick as a temporary solution I downloaded Tamil font and with the help of a font editor, I replaced the unused symbols and replace it with the value (கி) that I need. And whenever that character comes in string i replace it with the symbol that I used.
That's it.
It is working fine for me. This is how I get Tamil font print.
This is only a temporary solution. i believe this will also work for other languages too.
😅

@anandsubbu007 Thanks for responding!. if possible, can you please share the code? i see the exact problem. It would be really appreciated if you could share the code as i could not find any other better than what you are suggesting for now. or if possible, can you please share the edited font for tamil.

https://github.com/anandsubbu007/anand-work/tree/BLOC/tamil_pdf

@anandsubbu007 - Thanks much Anand. This works nicely.. Really appreciate your response!.. I will do more testing and will share comments if any.

[iampopal]

@iampopal Sorry, me neither, someone else wrote this, I understand only Latin languages.

If possible to introduce the person to me who wrote this will be great, We really need the support for 6 Pashto special characters.

[DavBfr]

Use git blame on the source file ;)

[DabPb]

Please, check Gujarati font as well, I'm using google gujarati font(Noto Serif Gujarati)..

there is the string on-screen is લોગિન and when I print in pdf it becomes લોગનિ
Same for another ઉત્સવ becomes ઉત્ સવ(ignore space between words)!

[iampopal]

Use git blame on the source file ;)

How will this help me?

[AkamBarznji]

hello @DavBfr DavBfr Any new update to fix this issue ?!

and please let us know how can help you to fix it...

Thanks

[DavBfr]

@AkamBarznji I'm not working on this, I'll let native writers improve on this. I'll not try to fix something I don't understand.

[iampopal]

@DevBfr We really need a fix for this If its not possible to be fixed in Text Widget Please create a new widget and name it something like LocaleText which shall have support for all languages Or Have parameter for unicodes so developers can pass unicode of thier langiage as parameter

…

On Tue, Oct 11, 2022, 12:37 AM David PHAM-VAN ***@***.***> wrote: @AkamBarznji <https://github.com/AkamBarznji> I'm not working on this, I'll let native writers improve on this. I'll not try to fix something I don't understand. — Reply to this email directly, view it on GitHub <#198 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIUFLXGXHDXRY6AOZPXO2N3WCRZRVANCNFSM4KBO5LTQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[shafiquecbl]

Seemingly Hebrew has the range \u0590-\u05fe according to this nice JavaScript Unicode Regex generator:
https://apps.timwhitlock.info/js/regex#:~:text=0590%2D05FF,Hebrew

So we can use this to find and reverse Hebrew only from any given string.

Here is the perfect working example:

String reverseHebrew(String text) {
text = text.splitMapJoin(
RegExp(r'[\u0590-\u05FF]+'),
onMatch: (m) => m.group(0).split('').reversed.join(),
onNonMatch: (string) => string,
);
// reverse words in string
return text.split(' ').reversed.join(' ');
}

[niilx]

For printing Bangla font use this method:

1. Use font SiyamRupaliANSI
2. Use the bangla font and convert it to UNICODE TO BIJOY
   
3. Copy the converted code and paste it to your Text widget with fontfamily SiyamRupaliANSI
   CODE EXAMPLE:

final font = await rootBundle.load("fonts/SiyamRupaliANSI.ttf");
pw.Text( "e‡Kqv", style: pw.TextStyle(
                            fontWeight: FontWeight.bold,
                            fontSize: 14,
                            font: Font.ttf(font),
                    ),

4. And get this final output of pdf
   

[myselfuser1]

This will help https://www.youtube.com/watch?v=LLBoRBAQIw0

[kyawthet-naing]

same issue in myanmar unicode font

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[AkamBarznji]

Still same problem available in kurdish language also

[github-actions]

Closing this stale issue because it has no activity.

[sh0umik]

And this problem is yet not solved ?

[KozanMelan]

https://www.bountysource.com/issues/86211023-bangla-unicode-font-ttf-gets-broken-during-pdf-generation

how could you prepare feature list please, can you give some illustrations .. thanks

[khuzaifa0099]

how to use urdu font family

[khuzaifa0099]

i have a problem in my code when i use urdu language word then the dependency which is use has give me the whole world seperate this is my code pdfWid.Text(
"تاریخ",
style: pdfWid.TextStyle(
font: ttf,
fontSize: 15,
),
textDirection: pdfWid.TextDirection.rtl,
), here when i use this it show my word seperat like ت اری خ iwant it to show me like تاریخ

[khuzaifa0099]

    s = s
        .replaceAll('ټ', 'ت')
        .replaceAll('ډ', 'د')
        .replaceAll('ښ', 'ش')
        .replaceAll('ړ', 'ر')
        .replaceAll('ږ', 'ژ')
        .replaceAll('ڼ', 'ن')
        .replaceAll('ۍ', 'ی')
        .replaceAll('ۍ', 'ی')
        .replaceAll('ئ', 'ی')
        .replaceAll('ګ', 'گ');

Above is the list of characters that are PASHTO Language not supported by the pdf package, i am trying to replace them with nearly same characters

list of pashto not supported characters is: ټ ښ ډ ړ ږ ڼ ۍ ۍ ئ ګ

   Please let us know when will you guys be adding support for the above Pashto characters

i have a problem in my code when i use urdu language word then the dependency which is use has give me the whole world seperate this is my code pdfWid.Text(
"تاریخ",
style: pdfWid.TextStyle(
font: ttf,
fontSize: 15,
),
textDirection: pdfWid.TextDirection.rtl,
), here when i use this it show my word seperat like ت اری خ iwant it to show me like تاریخ

[rabadiaanurag]

As this issue still not resolved, Please report this issue to flutter so they can develop this.
flutter/flutter#144917

[AkamBarznji]

Use Vazirmatn font will help you

https://fonts.google.com/specimen/Vazirmatn

[rabadiaanurag]

hello @DavBfr When this issue will be fixed ?
and please let us know how can help you to fix it...
if you are not anymore working on this then why you started this package?
if you are not willing to develop this further then make someone else admin of this package and explain all the flow to him so someone can develop this further.
Thanks

[DavBfr]

@rabadiaanurag I won't be working on RTL and Script support. If someone wants to work on it, they can provide a PR. I'm only working on this package on my free time.
If you need this feature, you can implement it.