why this script failed to generate ttf for oxygen-fonts? #1756
Comments
|
If you want to post a stacktrace, here's a guide: http://designwithfontforge.com/en-US/When_Things_Go_Wrong_With_Fontforge_Itself.html. |
|
Pickled data too sour? There were 70+ "PickledData:" sections in the SFD file. When those lines were experimentally removed, the SFD could be read and TTF file generated. Something's wrong with eating those pickles.... (I'm out of time to investigate more at the moment) |
|
|
On Mon, 29 Sep 2014, Parag Nemade wrote:
Your script requires two command-line parameters and you appear to have Matthew Skala |
|
It is interesting that this crashes using both Python 2.7.8 (as seen in pnemade's dump) and using Python 3.4.0 (as seen in my dump). Oh, and I put one I'm tempted to grab one of the pickled strings and see if I can unpickle in a test Python script. Hmmm... |
|
Hmm, no, the pickled data works just fine in isolated Python scripts. Tried with both Python 2.7.6 and 3.4.0: Using the UI I read the original font file and then wrote to a copy using "SaveAs...". The form of the pickled data did change a bit after passing through FontForge UI, but when using that new SFD copy with the scripted Open() it crashed again. |
|
SFDUnPickle calls nlgetc, but SFDUnPickle and nlgetc each attempt to remove one level of backslash quoting. Only one level was applied by SFDPickleMe, so the data will be corrupted if it literally contains anything that looks like backslash quoting. We ought to be using UTF7 for this instead of backslashes, as is done with FontLog values. Then each field value would fit on a single line, making scanning easier; and the file format would be more consistent. |
|
I am not familiar with fontforge coding. Can someone able to confirm if its a fontforge issue or font sfd issue? I created ttf->sfd using fontforge UI and then used script as given above and it worked!! Btw anyone knows if there is any scripting available that will convert ttf to sfd? Thanks. |
|
@pnemade, it is okay, some of the above messages are where we are discussing what is going on inside FontForge. It is a FontForge issue. Yes, when you wrote an .sfd file from FontForge, it did not contain any of that extra Later today I'll see if I can come up with a simple awk/sed/Perl script to remove the Converting any font file (like TTF) into the intermediate SFD file form can be done just like that script you were using, except instead of using the There are lots of things you can do with scripting and functions |
|
@pnemade, for now, use this on .sfd files that hit this bug. This removes just the |
|
@tshinnic thanks 👍 |
|
Finally remembered to retry with the 2012 packaged FF and that doesn't fail. Debugged some more, and for me (as above) it is actually dying trying to call Python API But... I use Python 3... I'm going to rebuild recent master and build using Python 2, and then retest. BTW: Tried to find the shortest simplest Python pickled string, and "(t." unpickles into an empty tuple. No magic characters or embedded control characters! |
|
So recent source dies built with Python 2 also, in Then brilliant idea! Change the native script into equivalent Python script. That way we know that Python has been fully initialized. And that script runs just fine. The Python initialization when not running Python scripts is broken somehow. Note mention in #1713 and almost direct description in #1687, which inspired the third test above, which has me rolling my eyes... |
We should add unit tests when fixing scripting bugs. |
|
Haha! I started weeks ago, got up to 3K+ lines on one small related set of But one area where I got bogged down was trying to figure out if the But for sure this has been a real deficiency - testing. Add in the Coming from the Perl world it is surprizing. It is almost a religion there
|
|
George Williams made FF as a fun retirement project, and writing tests wasn't fun for him. |
|
Segfaults are more enjoyable I reckon. ;) |
|
@davelab6, the starting nature of the project, and the generous motivation to provide something 'useful' to other people where there was a gap, has been understood. And appreciated! So much code and so many features! However, there might also have been a generational difference? I know that when I began coding, stuff either worked or it didn't. When it worked (at that moment) and processed the known data (of that moment) it was releasable. Over the decades, though, that stance became... less satisfactory. Nervousness over the likelihood that new work was breaking old was 'rewarded' by example after example of that happening. Own extra work to "avoid future problems" was rewarded differently, in positive ways. :) So it has become an article of belief for me that code should have tests of at least some small degree of thoroughness, giving you some degree of comfort upon changing code. And that the strange new data files that showed the need for new code or fixes should somehow be retained for those needed tests. And so perhaps also we see a change in needs simply because the project has broadened out away from the one man organization? Which I see as making it even more desirable that other people have some comfort that person XYZ wasn't just having a really bad day when they dropped some code in. Heck, I haven't submitted half the changes worked out that seem warranted by things like the Coverity reports, because I have no way to test those changes. When I found that mind-boggling PDF parsing table bug in the Coverity report, it took a lot of work to verify - to find suitable data files. And since there weren't "before results" to check against, I still don't know if correctly fixing that bug might still not break someone's prior usage! Anyway, I've already apologized privately to @adrientetar for the tone of the above. Perhaps I can ask for forbearance from all, when I start submitting some really ugly test code? I know I scared @frank-trampe months ago when I went Rambo trying to test the overlap/simplify code. :-) |
|
I am reaching the conclusion i should help make a release of FF and abandon |
|
No doubt FF python scripts have become very useful resource as they provide a way to build a binary font from sfd file. This is very useful for distributions like Fedora which like to promote to build a font package from source font file if its available in upstream tarball. |
|
@davelab6 For instance, take pdf.js from the Mozilla people, which I again used as reference material just lately. It's new, modern, developed with good methods and technologies, has all those test thingies I rant about :) , has both own development people and outside contributors, and is widely used and thus is well-exercised. It is marvelous! Unless you wanted to do more than view PDF files. If you want to interact with them, modify and rewrite PDF files, that's too bad. Out of scope. All that beauty, 'unusable'. Other packages can update PDF files, but they don't have the correctness of pdf.js. In my case Inkscape was similarly marvelous. It allowed me to explore the subject area and experiment with ideas. I familiarized myself with concepts and manipulations of shapes. It served well as a test platform. It would be enough for most people. But I found I needed more. I needed to mechanically transform hundreds of shapes. I had to export a few thousand SVG-generated glyphs into a variety of exported font files. And use ligatures/mark2base or some such on ~800 of those glyphs, which would be much more pleasurable if done using scripts. And Inkscape glyph/curve simplification stunk so badly. After checking out a number of packages (more than 5) and some web sites that purported to handle SVG to font translation/transformations, and submitting bug reports and fixes to a few of those, I gave up on those incomplete packages and landed at FF. Weight has advantages and disadvantages. FF has depth in a number of areas. However it is hard to pick up and move in new directions... Breadth has advantages and disadvantages. FF tries to provide so many possibilities it has something to offer to everyone. But pulling it out from the bushes/trees into the clear light needs a tractor... A large awkward aged source package resulting from many spurts of disconnected activity, trying to bridge several functional domains, is a sight to behold. But as an all-around tool for things font-related it seems the better fit for the audience. Small package, more lacks. Huge package, more cracks. But which is more likely to be of service to more people? (Oh, and one-line fix for this issue coming next) |
This change fixes the issue fontforge#1756 "why this script failed to generate ttf for oxygen-fonts?" There are a number of ways that embedded Python is used from FontForge, such as script files, source from STDIN, entered via UI dialog, and even via a socket (!?). And also it can be used as part of a convenient way to encode/decode complex object data through SFD files, in font and glyph `PickledData:` sections. When native scripts were used with FontForge, the embedded Python was not initialized for running, because it wasn't going to be used. :) Except, if the script was opening SFD files that happened to contain `PickledData:` sections, fontforge/sfd.c `SFD_GetFont()` and `SFDGetChar()` routines would call `SFDUnPickle()` and thence `PyFF_UnPickleMeToObjects()`. Trying to use the Python pickle/unpickle APIs without having initialized Python would then crash. This change simply adds a call to `FontForge_InitializeEmbeddedPython()` to the initialization of the FF pickle/unpickle routines. It does not address when the now needed Python cleanup at program end should happen. This change should logically also address part of issue fontforge#1687, but it seems that the problem there is not completely covered by this fix. Tested with both Python 2 and 3, with native and Python scripts, and using the UI.
This change fixes the issue fontforge#1756 "why this script failed to generate ttf for oxygen-fonts?" There are a number of ways that embedded Python is used from FontForge, such as script files, source from STDIN, entered via UI dialog, and even via a socket (!?). And also it can be used as part of a convenient way to encode/decode complex object data through SFD files, in font and glyph `PickledData:` sections. When native scripts were used with FontForge, the embedded Python was not initialized for running, because it wasn't going to be used. :) Except, if the script was opening SFD files that happened to contain `PickledData:` sections, fontforge/sfd.c `SFD_GetFont()` and `SFDGetChar()` routines would call `SFDUnPickle()` and thence `PyFF_UnPickleMeToObjects()`. Trying to use the Python pickle/unpickle APIs without having initialized Python would then crash. This change simply adds a call to `FontForge_InitializeEmbeddedPython()` to the initialization of the FF pickle/unpickle routines. It does not address when the now needed Python cleanup at program end should happen. This change should logically also address part of issue fontforge#1687, but it seems that the problem there is not completely covered by this fix. Tested with both Python 2 and 3, with native and Python scripts, and using the UI.
Noticed redundant #ifndef's during investigation of fontforge#1756. Since almost all code within fontforge/python.c is surrounded by `#ifndef _NO_PYTHON` it was strange to see individual bits of code surrounded by additional such `#ifndefs` Also, one copy-n-paste was one line too low, with an `exit(0)` before it. And just happened to notice an extra unneeded declaration in fontforgeexe/startui.c Do modern C compilers just drop those? And wow, Coverity didn't complain either.
|
Eloquent as always Thomas :) My point remains, that I think the lacks are
|
Please have a look at this repository
http://quickgit.kde.org/?p=oxygen-fonts.git&a=tree&hb=2be96834c4ef99b0470eb605fbe9ed0211b1d6a0&h=2092b587149fbdc6737d011b8cab6a7cf58c6b28
if above does not look correctly then I mean v5.0.2 version. The script generate-ttf.pe fails to generate sfd to ttf and segfaults.
Can someone look and provide information what can be wrong? Here is quick error on my machine
$ ./generate-ttf.pe src/OxygenMono-Regular.sfd
Copyright (c) 2000-2014 by George Williams. See AUTHORS for Contributors.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
with many parts BSD http://fontforge.org/license.html. Please read LICENSE.
Based on sources from 04:56 UTC 24-Sep-2014-ML-D.
Based on source from git with hash:
strippedname:/home/parag/oxygen-fonts/oxygen-fonts/mono-400/src/OxygenMono-Regular.sfd
Segmentation fault (core dumped)
The text was updated successfully, but these errors were encountered: