Setting rcParams ps.fonttype = 42 yields unreadable eps

[durack1]

I was just reading some notes in #5862, and it seems I'm hitting similar errors on MacOS/OS X (10.11.6) - in this case it doesn't appear that carriage returns are the issue. If I follow the guidance, and attempt to convert output to truetype fonts:

import matplotlib
matplotlib.rcParams['pdf.fonttype'] = 42 ; # pdf.fonttype : 42 # Output Type 3 (Type3) or Type 42 (TrueType)
matplotlib.rcParams['ps.fonttype'] = 42

My file size increases from 19Kb to 1.2Mb, but no software (MacOS Preview, or Adobe Photoshop 2017) can read the file. For those interested I have attached both *.eps output files in the zip archive.
matplotlib2-postscriptFontIssue.zip

The truetype file contains the following, and DejaVu Sans doesn't appear to be installed (checking the MacOS Font Book application):

...
%!PS-TrueTypeFont-1.0-2.22937
%%Title: DejaVu Sans
%%Copyright: Copyright (c) 2003 by Bitstream, Inc. All Rights Reserved. Copyright (c) 2006 by Tavmjong Bah. All Rights Reserved. DejaVu changes are in public domain 
%%Creator: Converted from TrueType to type 42 by PPR
...

A demo script to reproduce the problem is below:

import matplotlib
from matplotlib import pyplot as plt
fig = plt.figure(dpi=100, facecolor='w', edgecolor='w')
ax1 = plt.subplot2grid((2,2), (0, 0), rowspan=2, facecolor='w')
ax2 = plt.subplot2grid((2,2), (0, 1), facecolor='w')
ax3 = plt.subplot2grid((2,2), (1, 1), facecolor='w')
matplotlib.rcParams['ps.fonttype'] = 3
fig.savefig('demo-type3.eps') <- Opens fine
matplotlib.rcParams['ps.fonttype'] = 42
fig.savefig('demo-type42.eps') <- Errors on all applications I have tried

System config:

• Operating System: MacOS 10.11.6
• Matplotlib Version: 2.0.2-py27_1 (conda-forge)
• Python Version: 2.7.13-1 (conda-forge)
• Jupyter Version (if applicable): NA (1.0.0-py27_0 conda-forge)
• Other Libraries: NA (can provide if required)
• Installation path: conda (conda-forge channel)

[selasley]

I was able to view demo-type42.eps using ghostscript 9.21 and Inkscape 0.92 and the Affinity Designer and Affinity Photo apps under macOS 10.12.6. ghostscript's ps2pdf utility was able to convert the file to a pdf but Apple's pstopdf utility failed.

[durack1]

Thanks @selasley, believing this is a font issue I just tried

import matplotlib.font_manager
matplotlib.font_manager.findSystemFonts(fontpaths=None, fontext='ttf')

But that didn't seem to solve the problem

[WeatherGod]

Have you tried clearing your font cache?

…

On Wed, Aug 16, 2017 at 1:28 PM, Paul J. Durack ***@***.***> wrote: Thanks @selasley <https://github.com/selasley>, believing this is a font issue I just tried import matplotlib.font_manager matplotlib.font_manager.findSystemFonts(fontpaths=None, fontext='ttf') But that didn't seem to solve the problem — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#9044 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARy-HhIxfjjVFqDgLzZ-lzbUNsPlS9Sks5sYybSgaJpZM4O5HxS> .

[durack1]

@WeatherGod any tips as to how I would do that?

[selasley]

You can reboot with the shift key held down to clear various caches and boot into safe mode -https://support.apple.com/en-us/HT201262
Or you can use the command line utility atsutil

sudo atsutil databases -remove
atsutil server –shutdown
atsutil server –ping

demo-type42 and demo-type3 look identical in Affinity Designer. It's possible that ghostscript, Inkscape and the Affinity apps are substituting a different font for DejaVu when they open demo-type42. That may be the reason they work and Preview and pstopdf don't.

According to Font Book, Bitstream Vera Sans, Bitstream Vera Sans Mono, and Bitstream Vera Serif are installed on my mac, so I tried

matplotlib.rcParams['font.monospace'] = ['Bitstream Vera Sans Mono']
matplotlib.rcParams['font.sans-serif'] = ['Bitstream Vera Sans']
matplotlib.rcParams['font.serif'] = ['Bitstream Vera Serif']

before running your script. Now pstopdf fails because BitstreamVeraSans-Roman is missing

%%[ Error: invalidfont; OffendingCommand: definefont; ErrorInfo: .notdef --nostringval-- ]%%

Stack:
/Font
-dict-
/BitstreamVeraSans-Roman

%%[ Flushing: rest of job (to end-of-file) will be ignored ]%%
%%[ Warning: PostScript error. No PDF file produced. ] %%
pstopdf failed on file demo-type42.eps with error code -31000

I'm using python 3.6.2 and matplotlib 2.0.2

[durack1]

@selasley it seems to me like those rcParams commands are not sticking, I have confirmed that Courier is installed on the machine, and then try:

matplotlib.rcParams['font.monospace'] = ['Courier']
matplotlib.rcParams['font.san-serif'] = ['Courier']
matplotlib.rcParams['font.serif'] = ['Courier']

But the output (demo-type42.eps) still contains:

...
%!PS-TrueTypeFont-1.0-2.22937
%%Title: DejaVu Sans
%%Copyright: Copyright (c) 2003 by Bitstream, Inc. All Rights Reserved. Copyright (c) 2006 by Tavmjong Bah. All Rights Reserved. DejaVu changes are in public domain 
%%Creator: Converted from TrueType to type 42 by PPR
...

[WeatherGod]

No... gosh no. I was referring to matplotlib's font cache. I don't know where it is for macs, but for linux, it is in ~/.matplotlib.

…

On Wed, Aug 16, 2017 at 2:30 PM, Scott Lasley ***@***.***> wrote: You can reboot with the shift key held down to clear various caches and boot into safe mode -https://support.apple.com/en-us/HT201262 Or you can use the command line utility atsutil sudo atsutil databases -remove atsutil server –shutdown atsutil server –ping demo-type42 and demo-type3 look identical in Affinity Designer. It's possible that ghostscript, Inkscape and the Affinity apps are substituting a different font for DejaVu when they open demo-type42. That may be the reason they work and Preview and pstopdf don't. According to Font Book, Bitstream Vera Sans, Bitstream Vera Sans Mono, and Bitstream Vera Serif are installed on my mac, so I tried matplotlib.rcParams['font.monospace'] = ['Bitstream Vera Sans Mono'] matplotlib.rcParams['font.sans-serif'] = ['Bitstream Vera Sans'] matplotlib.rcParams['font.serif'] = ['Bitstream Vera Serif'] before running your script. Now pstopdf fails because BitstreamVeraSans-Roman is missing %%[ Error: invalidfont; OffendingCommand: definefont; ErrorInfo: .notdef --nostringval-- ]%% Stack: /Font -dict- /BitstreamVeraSans-Roman %%[ Flushing: rest of job (to end-of-file) will be ignored ]%% %%[ Warning: PostScript error. No PDF file produced. ] %% pstopdf failed on file demo-type42.eps with error code -31000 I'm using python 3.6.2 and matplotlib 2.0.2 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9044 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARy-O2a2TKDRFNt1QTQxbq6dCNiNm8Sks5sYzVEgaJpZM4O5HxS> .

[tacaswell]

Also see ~/.cache/matplotlib

[selasley]

The problem appears to be the definefont command in the eps files. I deleted the fontList cache files in my ~/.matplotlib directory and set the font.sans-serif rcParam to various fonts. In each case Preview was unable to open demo-type42.eps and the Apple utility pstopdf failed to convert it with the error

%%[ Error: invalidfont; OffendingCommand: definefont; ErrorInfo: .notdef --nostringval-- ]%%

The only place definefont appears in the eps files is in this line

FontName currentdict end definefont pop

Preview and pstopdf are able to open demo-type42.eps after changing the line to

FontName currentdict end

I don't know enough about postscript to know what ramifications removing "definefont pop" has, but it seems to work for Preview, MS Word, and a few other apps that failed to open the original eps file.

[tacaswell]

@selasley Thanks for you work tracking this down!

[tacaswell]

The only place that string shows up in the source is in extern/ttconv/ppdrv_tt.cpp which was last
touched in a14a4b7 and the string goes back to a55a4ad (in 2007) so if that command is a problem it is one we have had for a very long time.

[tacaswell]

The attached files work fine for me with ghostscript and okular on linux :(

[selasley]

It could be a macOS specific problem. pstopdf gives the same error under OS X 10.9 and 10.10. "unix-based" programs like ghoscscript, Inkscape and even TeXShop can open demo-type42.eps. "native" programs such as Preview, GraphicConverter, Microsoft Word, Intaglio and pstopdf have problems with it. Photoshop Elements 9 fails to open demo-type42 with the error message "Could not complete your request because the parser module cannot parse the file" Hopefully the OP can use one of the workarounds.

[durack1]

After perusing a few sites it seems that ghostscript when called with a nullpage device will report postscript errors, following this I ran:

% gs -sDevice=nullpage -dNOPAUSE -dBATCH /path/to/file/demo-type42.eps
GPL Ghostscript 9.19 (2016-03-23)
Copyright (C) 2016 Artifex Software, Inc.  All rights reserved.
This software comes with NO WARRANTY: see the file PUBLIC for details.
%%BoundingBox: 89 251 522 541
%%HiResBoundingBox: 89.999997 251.999992 521.999984 540.017984

So it seems at least ghostscript isn't hitting any obvious errors. I'm not sure if there is another way to validate ps/eps files for problems?

This discussion is also useful background

[durack1]

Ok and after removing the offending definefont pop (line 16968, thanks @selasley) I get:

% gs -sDevice=nullpage -dNOPAUSE -dBATCH /path/to/file/demo-type42-noDefinefont.eps 
GPL Ghostscript 9.19 (2016-03-23)
Copyright (C) 2016 Artifex Software, Inc.  All rights reserved.
This software comes with NO WARRANTY: see the file PUBLIC for details.
Querying operating system for font files...
Can't find (or can't open) font file %rom%Resource/Font/DejaVuSans.
Can't find (or can't open) font file DejaVuSans.
Didn't find this font on the system!
Substituting font Courier for DejaVuSans.
Loading NimbusMono-Regular font from %rom%Resource/Font/NimbusMono-Regular... 4128764 2787962 4963028 2541805 2 done.
%%BoundingBox: 89 251 522 541
%%HiResBoundingBox: 89.999997 251.999992 521.999984 540.017984

So it seems on MacOS, matplotlib 2.0.2 is not doing a good job of selecting a system installed default font - I would default to Courier as it's one of the most liberally distributed fonts

[tacaswell]

Can you check if this works with earlier version of Matplotlib?

[durack1]

@tacaswell is there a particular version that you'd recommend targeting? Your #9044 (comment) would suggest that it's been around since the dawn of matplotlib time

[tacaswell]

At least against 1.5 and maybe 1.4 (if it is easy to install).

[durack1]

@tacaswell looks like 1.5.0 is the earliest version available through conda

[selasley]

After some more digging it looks like the error message from pstopdf "ErrorInfo: .notdef --nostringval--" means that there is no definition of .notdef in the eps file. Removing definefont pop from demo-tye42.eps masks the problem because the embedded DejaVu is no longer defined so it is not used.

If I change this CharStrings block in demo-type42.eps located after the long block of /sfnts hex values from

/CharStrings 7 dict dup begin
/period 17 def
/zero 19 def
/one 20 def
/two 21 def
/four 23 def
/six 25 def
/eight 27 def
end readonly def

to

/CharStrings 8 dict dup begin
/.notdef 0 def
/period 17 def
/zero 19 def
/one 20 def
/two 21 def
/four 23 def
/six 25 def
/eight 27 def
end readonly def

Preview is able to open the modified demo-type42.eps file and does not complain about a missing DejaVu font. pstopdf converts the file without errors or messages about missing fonts.

I'm not sure how to add a definition of .notdef to the CharStrings block. It looks like the block is written by the ttfont_CharStrings routine in ppdrvr_tt.cpp where it loops through glyph_ids. The glyph_ids come from ps_renderer.used_characters in backend_ps.py That's about as far as I can trace it.

The BlueBook says .notdef is defined to print nothing. I found the "/.notdef 0 def" line in eps files from Inkscape so I presume defining .notdef as 0 is OK.

[dstansby]

Is this now fixed by #9059?

[anntzer]

I'll assume so in the absence of further activity. Feel free to request a reopen if necessary.

[durack1]

@anntzer I can confirm using the demo script as contained in #9044 (comment) I no longer have the eps read issue on macOS 10.14.6, using matplotlib 3.1.3.

There are an astounding number of differences in the output files between 2.0.2 and 3.1.3, I have included output files from each version for comparison in the attached archive
matplotlib2vs3-postscriptFontIssue.zip