[CTS 14] Handling SVG? #89
Comments
|
Just a note, prompted by wkHTMLtoPDF problem with SVG being rasterized. SVG input should preferably not be rasterized, but should be transformed into PDF graphics primitives. The idea is that such graphics should cleanly scale to all sizes. I need to investigate whether PDF graphics are a subset or superset of SVG graphics, and whether there is anything in SVG that would be a problem to translate to PDF graphics. |
|
I just spent a number of hours digging through SVG, and it's huge — almost as big as PDF itself! Unless I can find a prewritten Perl library to parse arbitrary SVG code, it may just be too large a task to try to support all of SVG. SVG is supposed to be formatted as XML, so an XML-processing library may at least be able to do the task of parsing (and syntax checking) the SVG, leaving me to traverse the DOM and implement (at least) selected parts of it (outputting PDF graphics, text, and image primitives). The worst case would be to hand-code a parser for a small subset of SVG, to take care of the most common things. The challenge here is to later extend it (without breaking something) when someone says they desperately need the transparent bleems function… could I please tack that on? There's stuff that PDF may not implement (many of the filters), and all the interactive stuff (e.g., |
|
See CTS 10 (#99) for thoughts on using SVG instead of extending "regular" calls with relative coordinate versions. |
|
Pages 510-511 of the PDF 1.7 spec show a SlideShow with embedded SVG file. It seems to imply that it's possible that a PDF Reader can handle SVG natively. Be sure to look at that before putting any more effort into parsing an SVG and reproducing using PDF graphics and text primitives. Add: possible false alarm. This PDF "slideshow" is the only place SVG is mentioned, and it is starting to sound like a "slideshow" is using external utilities to display various formats, including SVG (possibly using a browser, as JavaScript is mentioned). It looks like the SVG file is merely carried along (embedded) to hand over to the browser, but I will check to make sure. |
|
@sciurius has requested SVG vector support on PDF::API2 (RT 134780). I'm not going to get SVG out in PDF::Builder 3.022 (out probably within a week or two), but since there's interest, maybe that will get me moving faster on SVG support (already partly implemented). I need to get a bit further along before I put it on GitHub, but if anyone has interest in joining in, I can get it out a bit earlier. |
|
Depending on how big the SVG code ends up being, I'm considering packaging it as its own CPAN module. It would in turn call three other packages (overall XML parser SVG::Parser, "path" decode Image::SVG::Path, and "transform" decode Image::SVG::Transform) and return an array of standardized, low level graphics and text calls, including all the applicable attributes at each call. The PDF SVG support would then output these primitives as PDF graphics and text calls. The new package would not be exclusively for PDF, but could be used in a wide range of applications. I'm still fairly early in the design and writing process, so this architecture could change considerably. I'd like to get feedback on what others think of the idea. I don't know if I can support 100% of the SVG spec, but I will try to get in all of the SVG into the returned array, and on the PDF side handle most of the non-interactive static visual content. PDF may have to output a single object rather than outputting to graphics and text objects, so that items are rendered in the desired order. The input to the SVG package would be either a file path or a string of SVG code (so you could create an image on the fly). |
|
My current prototype/experiment uses SVG::Parser, but that's not cast in stone. I've seen a number of SVG-parsing packages floating around that use all sorts of XML parsers, and I'm open at this point to changing over if you can give some good arguments to do so. I do want to avoid prereq'ing a huge number of other packages, as some of these modules do. I also want to have something very portable that doesn't require a (for example) Python or R installation, things which don't come with a Windows box. Keep it fairly lightweight. Whether or not I offer a Perl-based SVG parser as a separate module (described previously), internally it will use that structure to break down the SVG into a list of primitives and associated data and settings/CSS. That will be fed to something that generates the matching PDF primitives for output. If anyone has seen Perl-based support SVG parsers that largely or entirely handle this already, I'd appreciate hearing about them! |
|
Mon May 17 16:38:22 2021 PMPERRY@cpan.org - Correspondence added (to RT 134780) On Sun Mar 21 17:12:23 2021, JV wrote:
I expect to fairly soon (mid June?) be able to devote some serious time to this. Please read and start giving your thoughts on the GitHub issue (89) listed above. It would be nice to have input from people who have dived into SVG processing and are willing to share their experiences, while I'm still early in the architecting of this thing. Thanks! |
|
Closest I can come is a svg renderer written in python. It renders the SVG on the canvas. |
|
Well, if there are good algorithms for transcribing SVG into PDF primitives, I'd be happy to hear about them. The first stage is to parse an arbitrary SVG file into some level of primitives, and the second stage is to build PDF from that list. I'm currently using SVG::Parser, but someone (it might have been you) complained that it was a barely warmed-over XML Parser. Nevertheless, I'd like to avoid re-inventing the wheel, and use as much existing library code as possible to parse the SVG. If there's some library more up-to-date and better maintained, I would consider using it. I hope by mid to late June to have a first cut of the SVG parser on GitHub for you and others to play with. I'm going to wait until that's done to decide whether to make it a separate project and CPAN release. |
|
I finally found my preliminary development work on SVG conversion (I thought I had lost it when a PDF::Builder install overwrote the files, but I did turn out to have it on backups!). Release 0.001 is on GitHub under PhilterPaper/Perl-SVG-Reader. There will not be a CPAN release until 1.000 is ready (along with PDF::Builder SVG image support, using SVG::Reader). At this point, I am assuming that I will keep this code in that repository, and release a CPAN package at release 1.000. However, if it turns out that the code is lightweight and trivial enough, and no one is trying to use it for their own projects, I may choose to fold it into PDF::Builder. I don't know at this point -- I want to wait to find out how big a project this is. This first commit is very basic, and only starts the process. I still have a lot more work to do on it, but want people to see its progress. If I can spare time from other projects, I will work on it at a fairly steady pace. I hope by the end of July (or mid August) to have it largely "there". At some point, I need to write the code in PDF::Builder that actually makes use of this SVG::Reader. This package will not be released until it appears to be usable for PDF::Builder's SVG image support. Issues involving PDF::Builder's SVG support should be opened under this repository, but issues involving the new library should be under Perl-SVG-Reader. I look forward to comments and suggestions (and even code) from others! |
|
Good job! |
|
I ran into a mess with SVG::Parser, in which it seems to randomly pick which library (Expat or SAX) it uses, based on exactly how it's invoked, resulting in somewhat differently structured hashes. I think I can work through finding which one is produced, and properly digesting its data, but it's still a nuisance. My query to SVG::Parser's ticket system (https://rt.cpan.org/Public/Bug/Display.html?id=138495) is still unanswered, so I'm becoming concerned that this product is unsupported. Does anyone have a suggestion on a better parser to use for SVG, possibly a more generic XML parser? It should produce something similar to SVG::Parser, and be supported! |
|
Not SVG itself, but related: consider support for HP-GL pen plotter (vector) "image" input. Presumably pen plotters are still in use for large drawings, so some sort of viewport and/or scaling will likely be necessary. Rendering as vector graphics would permit unlimited zooming in to look at details. No one in their right mind is going to hand-write HP-GL diagrams, charts, etc., but there still may be plenty of programs that produce it as output, and it would be nice to be able to publish as PDF, even if you don't have a physical pen plotter. |
|
HP-GL is dead simple to implement, it is just a series of move and draw instructions. |
|
Looking at my HP-GL/2 Reference Guide, I see that the plotter language is very complex, approaching that of SVG and PDF. There's lots of commands for text (with downloadable fonts) and absolute/relative coordinate versions of almost everything. I don't think a full implementation of HP-GL would be worth the effort, but a healthy subset might be useful for someone who has utilities or programs that create HP-GL output. I'm thinking of using a common "generic vector graphics" routine to handle the output of an HP-GL reader, a barcode routine (see #48), and possibly even the SVG reader. It would then output PDF primitives. Something to think about, anyway. Initially, it would support library routines for barcodes and maybe a small subset of HP-GL, and would be expanded over time to handle the output of a reader for a reasonable subset of SVG. Something like that. If and when PDF::Builder users ask for more vector graphics capabilities, the appropriate reader(s) and the generic vector graphics routine/library could be enhanced. |
|
Oh yes, it is a lot of commands, but they all seem rather straightforward. |
|
Let me think about making HP-GL/2 (or at least, a large subset) the "generic" vector plot language. It ought to do nicely for barcodes, but I'm not sure about SVG. And of course, it would be directly usable for anyone who wants to use HP-GL as some sort of vector plotting language. I don't see trying to support HP's PJL, PCL, or RTL, unless there's a huge demand for it. I've got a ton of stuff on my plate right now, including SVG support for standalone graphics, MathJax equation support, and Gnuplot graphing support, in addition to extending |
|
How about a subset of SVG that can 'drive' HPGL? Last year I've been working on an SVG module and got reasonably far. Unfortunately the people that write SVG generating programs like to incorporate more and more CSS3 features, making a complete(r) implementation impossible. It would be like rewriting half of Firefox in Perl. I'm quite busy now, but I may take up the SVG module later this year. |
|
If you're asking about a translator from SVG to an intermediate form such as HPGL, and then directly "interpret" the HPGL (into PDF primitives), that's along the lines of what I was thinking of doing. It is probably not feasible to try to support the entire SVG definition, but I think it's possible to come up with a reasonable subset that would prove widely useful. If users ask for unimplemented features (such as embedded raster images or specific CSS), we can consider adding them piece-by-piece, so long as the original architecture was flexible enough to allow that. That's what I've tried to do with the The whole point is to have scalable vector graphics, rather than raster images, to embed into a PDF document. I'm not sure if the HPGL interpreter A few years ago I started playing with a Perl implementation to translate SVG into some intermediate form, but didn't get far before being distracted by more pressing matters. I would be happy to let someone take the lead with resuming work on SVG::Reader (outputting HPGL code) and even working on an HPGL-to-PDF "interpreter" to complete the job. I can retain ownership of SVG::Reader, or hand it over to someone else who wants it. The HPGL-to-PDF interpreter would be contributed to PDF::Builder. The idea behind having a Graphics::BarCode package would be to output a wide range of bar codes in a neutral format, here eventually rendered as PDF, while other uses might be other graphics formats such as GIF, GD, etc. There are already many bar code packages, but each directly outputs in some specific format not necessarily usable by something such as PDF::Builder. |
|
@sciurius , if you (or anyone else) wants to pick up the SVG processing task and/or the HPGL processing task, please let me know (even if it's just wanting to do design work on it at this point). I want to avoid duplication of effort and one of us being disappointed by having their hard work discarded. Not that I'm near ready to do active work on SVG and HPGL (which would need to be coordinated so that the HPGL processor can handle everything SVG does), but some time this year I hope to get back to it. By the way, regarding HPGL, there is a lot of stuff in there for handling text (fonts) for output by the LB label command. I may need to trim it down to a reasonable subset. Apparently, an HP plotter can handle two fonts at any given time -- a standard or primary font, and an alternate or secondary font. There are commands to switch between them, and you can use SO and SI to select fonts within a single LaBel string. There seems to be some limited multibyte capability, though I'm not sure if UTF-8 is supported (I think 8 and 16 bit characters are). Anyway, someone with a solid understanding of SVG will need to work with whoever is doing HPGL to coordinate things (and if necessary, do separate PDF output for SVG if HPGL is insufficient). Needless to say, both SVG and HPGL support are likely to be only subsets of the full languages for quite some time. We should endeavor to make the architectures flexible enough to be able to add additional features in the future, getting closer to full support of the definition. |
|
Three things:
|
|
There are several tools that claim to turn HPGL into SVG. I don't know any of them but it they do the job it is better for us to concentrate on SVG. I will definitely pick up the SVG module I've been working on last year. As I said earlier the graphics handling (drawing instructions) is pretty much functional but it is CSS3 that makes it hard. My personal goal is to be able to deal with the output of some SVG generating tools that I need for one of my projects. It may be of (more or less limited) general use but that will be a fortunate sideeffect. Establishing the bounding box without painting is trivial if the SVG has an accurate drawing box. If not, my approach would be to draw the image in an XObject so you have the liberty to move/scale/etc the result depending on its dimensions. As SVG is a graphics language, it is content agnostic. Breaking long (formula) lines is the task of the tool that formats the equation and produces SVG. I don;t think this can be done afterwards. |
|
Ok, the package is renamed I've added some 40 SVG sample files as regression tests, that all pass for PDF::Builder and PDF::API2. There is one test that fails with perl 5.30 and before, possibly due to rounding errors in the math library. I have to investigate. |
|
Sounding good. I look forward to the SVGPDF package on CPAN. No great rush -- I have a bunch of stuff to get into PDF::Builder 3.026 Real Soon Now (I hope!). Then, the decks will be clear to add lots of SVGPDF-based stuff. I appreciate your efforts. I support Perl 5.26 and up for PDF::Builder, so it would be nice to have that last example working properly. If it's really an edge or corner case, consider leaving it out for the initial release. Does it happen only in specific Perl builds (particularly with extended math libraries)? I have seen that kind of thing before, where t-tests failed due to different levels of available precision (more is not always better!). I ended up rounding some math results to "standard" double-precision (or even to single-precision) so they would pass the tests regardless of the Perl build. |
|
The lowest Perl version I support is 5.26 so no sweat. The problem has been located: This prints "-1.78" with 5.30 and up, and "-1.79" with 5.28 and lower. The difference plays no serious role, it is just that the debugging statements use |
|
Odd. I would say that's a fairly serious bug, unless pre-5.30 it wasn't rounding to specification and they fixed it in 5.30. When dealing with negative numbers, you've got truncation, round away from zero, and round towards zero. Not to mention what to do when the first digit to drop is a 5 (round to odd or round to even). For someone who wants consistent results across a range of Perl versions, that's nasty to change specs like that (if they did). |
|
You seem to be good at math... My math is extremely rusty. Do you know how to convert an SVG transformation matrix to a PDF matrix, given that in the PDF the y coordinate must be flipped. I.e., the SVG coordinate system runs from top left 0,0 to bottom right 100,100, the corresponding PDF coordinates are translated 0,100 so top left becomes 0,0 and bottom right is 100,-100. |
|
If your math is extremely rusty, mine has long since corroded away! Anyway, this might be better discussed by telling what sort of transformations are being done in SVG, that you need to transfer to PDF. If it's just a matter of flipping the coordinate system over (so 0,0 is at the top left instead of bottom left, an y grows downward instead of upward), I think you need to do two things: Y-scale of -1 (X-scale is +1), and Y-translate up by some amount less than media size (X-translate is 0). Presumably you're not doing skews or rotations (leave them 0). I'm not sure if you'll need to reverse (negate) a skew or rotation angle. Shouldn't your resulting Y at the bottom right be 100,0 rather than 100,-100 (so it will be visible on the page)? Or perhaps with some positive offset to get it to the top of the page? I think the PDF matrix for this would be I recall a few years ago dealing with some sort of PDF editors that left the coordinate system flipped over (and some oddball page height), resulting in anything additional written in PDF::Builder being upside down or something like that. The solution was to first add low level PDF commands to set the transform matrix to reverse and offset Y -- I think it's described in Content's POD under Advanced Methods. |
|
The initial transform depends on the viewBox. If the viewbox is 0,0,width,height then the transform is translate(0,height). 0,0 will be topleft and all subsequent y coordinates must be negated. The basics are simple: SVG (x,y) → PDF (x,-y) but the challenge is to process an arbitrary SVG transformation matrix. SVG matrix a,b,c,d,e,f → SVG matrix ???? Flipping the Y coordinate by using scale(1,-1) does not really work, since that also affects texts which will become upside down. However if there's no alternative I think I will have to use scale(1,-1) and apply another scale(1,-1) to the texts only. Doable but a lof of work. Changing selected pluses into minuses and vice versa all over the place is very error-prone. BTW: |
|
Update: Changing the pluses and minuses turned out to be less of a problem than I anticipated... And the matrix transform is now functional. Thanks a lot for the hint! |
Yea! BTW, the SVG and PDF specs both detail in what order transformations should be applied, at least if specified as separate calls. I'm not sure what the rules are if a transformation matrix is used. Anyway, be careful to check that you're not assuming a particular order of operations. Regarding the rounding issue, if Perl has specified all along the intended rounding behavior, and it wasn't implemented correctly until 5.30, then that's a bug in pre-5.30 implementations. Still, it's annoying that such important behavior is changed/fixed right in the middle of the 5.x release stream. It's a bug in 5.30+ if it met specs (if given) before and now doesn't, and the spec should not have been changed mid-stream. Note that Perl may or may not have met IEEE-754 floating point specs before (or now), and might have had its own f.p. spec (a bad idea... IEEE-754 has been around forever)*. Further note that there are many extended f.p. libraries around which add extra digits of precision at the cost of incompatibility with other extended libraries, and might even affect "standard" precision work. That's what I ran into several years ago -- some Perls are built with one or another extended f.p. library, which will produce varying results, although I have never seen rounding issues such as you describe. * Almost everyone has used IEEE-754 in hardware for quite some time, except very old architectures (e.g., IBM s/360 family). |
|
JV, I see in "samples.pdf" PDF output from the equation SVG's that appears to include vertical extents and a baseline (vertical). In other words, the descender and ascender values. I just want to confirm that I'll have this information available to me when your package returns a PDF xo. I will need it to vertically align the returned image with the text baseline when doing inline equations, and want to make sure this vital information will still be present. Thanks! |
|
The result depends on what the SVG provides. For the MathJax formulae: The baseline is at y=0, so there a ascender of 1749.5 and a descender of 1251.5. You can see it in this picture, generated with a debugging option. |
|
Inline equations via MathJax are the only place I can think of that would need vertical alignment on a baseline. |
|
Example of how to inline a MathJax SVG. |
|
Looks great! "Display" equations (centered, optional tag at right margin) will simply be treated as SVG images. Let me know how you make out with the recursive |
|
Progress... In this PDF I have manually tweaked the positioning, to make sure that the formula is correctly rendered and that nesting of SVGs works correctly. For example (outer SVG): The viewBox is 21707.5 -1749.5 1 2999. Min-width = 109.889ex and height=6.785ex. The specified width of the viewBox ( where the xx are the formula, and nn the label. |
|
You may be getting too elaborate with this. I presume that you noticed that your example has too narrow a page width, and the tag overlaps the equation. A PDF page is not dynamically resized, as an HTML page can be, so I think it should be the responsibility of whatever program is assembling the page to check the available column width against the returned SVG image's width, and decide what to do about it. Most likely, like any other image, if it's too wide it will simply be centered and the left and right edges will extend beyond the column bounds and possibly off-page, requiring the author to shrink or split up the equation, or even span two or more columns on the page. All that's required of you is to tell me the height and width (in points) of the returned XO, as well as any baseline position information (particularly for an inline equation). I can strip off the tag and handle it separately, if need be, and I can tell you whether or not this is an inline equation (if necessary for dealing with baseline/ascender/descender heights). Of course, if you enjoy the technical challenge of doing this, go right ahead! |
|
Well, I enjoyed the technical challenge for a while but now I'm giving up. Attached how close I got... The interaction of nested viewBox, width, height, bbox and preserveAspectRatio is, for me, now, unfathomable. So I'm advancing to the next step: using the renderer in production. |
|
If nested Here's a new equation to try: |
|
No problems with IL and DNT. |
OK, unless you tell me otherwise in the next few months, I will plan to strip the \tag{} off of equation input and handle it separately. A "display" equation (as SVG image) will be horizontally centered on the page. It will not dynamically reposition, since PDF pages are of fixed width. I think that should be adequate. By the way, I understand that some SVG's can include raster graphic images. Do you have any plans for those? I don't think it's all that important, at least for an initial release, but eventually someone will request it. |
|
I already support PNG and JPG as links and inline data |
|
At this point, assuming Johan's SVG-to-PDF library comes to fruition, I will drop any further efforts on my SVG::Reader library. I don't need SVG for any purpose other than PDF output, so there will be no need for an SVG-to-generic-graphics converter, nor support in PDF::Builder for this "generic" format. I may keep what I've done so far around for a while, hoping that someone else will pick it up if they need such a converter. If not, at some future point I may just delete it. |
|
What's brewing...? As of the next release, Text::Layout will support inline images. E.g. the marked up text string results in
Attributes can control appearance, e.g.
|
|
Hi Johan, How is SVGPDF coming along? I periodically check your GitHub respositories, and haven't seen anything. Are you still working on it? I just released PDF::Builder 3.026 and will be looking soon to add SVG support for 3.027. Please let me know if you have decided not to proceed with an SVG-to-PDF library, in which case I will have to resume work on my own :-( . The work you showed me looked very promising, doing everything I need (so far) to do for PDF::Builder, and I'm hoping to see it as a released Perl library! By the way, I know you do a lot of work with musical notation. I thought you might find this video of interest: https://www.youtube.com/watch?v=Eq3bUFgEcb4 . |
|
Hi Phil, The library is alive and kicking and in full use in ChordPro with very good results. I kept the repo private to avoid early forks and potential problems with changing the API. I hope the API is now stable, and I have made the repository No doubt there will be some wrinkles to iron out... |
|
Thanks for making SVGPDF visible to me. I copied it over OK, but when I try running it, it prereqs File::LoadLines. This package does not successfully install on Windows 10 (Strawberry Perl 5.32 or 5.38). The errors in t/13-nochomp.t appear to be the same ones as in the CPAN Testers Matrix for Windows. CPAN refused to let me open an RT ticket against File::LoadLines (403 Forbidden, when I tried to submit a new ticket). |
|
Sorry for the inconvenience... I use File::LoadLines extensively, also on windows, and it seems to work fine. I'll check the tests. You can safely install with Issues → https://github.com/sciurius/perl-File-LoadLines/issues |
|
FYI -- I've uploaded File::LoadLines 1.041 to CPAN. This should build find on Windows. Thanks for the 'Notation must die' link. I've enjoyed it. |
|
Another music-related video you might be interested in: https://www.youtube.com/watch?v=Qct6LKbneKQ . It is about the development process for MuseScore4, including notation/engraving, by the same guy who did the Notation video. MuseScore is not only printable notation (engraving) but also interactive editing and MIDI/VAST playback interface. |
PDF::Builder currently handles several image formats (GIF, JPEG, PNG, TIFF, etc.), any of which can be dynamically produced (on the fly) for a web page. Another increasingly popular format is SVG (Scalable Vector Graphics), which can be easily produced on the fly, and displayed by a browser. Would it be worthwhile to handle SVG graphics in the same manner as other graphics? The basic SVG commands are fairly simple to parse into PDF drawing commands, but SVG can also embed images within its codes, adding a layer of complication. The alternative could be to use an external utility to convert SVG content into an already-supported image format, but this would require that the user install some external program(s). Another choice could be to add hooks to permit invoking a user-supplied command line utility to convert SVG (or an arbitrary image format) into a supported image format.
Something like an existing file with SVG or image content (and SVG possibly embedded yet another image file) could either be read in or processed externally, while in-line SVG command text might be directly parsed by image_svg(). This could also pull in image files. Anyway, there are many combinations and possibilities for SVG support, that could be direct support (convert to PDF primitives) plus embedded images, or that SVG is externally converted to a supported image format. A program wishing to output PDF might also dynamically generate SVG for document graphics, in lieu of using PDF primitives directly, such as generating content suitable for browser (HTML) or PDF output.
The text was updated successfully, but these errors were encountered: