XPS documents in .NET Core can't be opened #51929
Comments
|
I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label. |
dotnet-issue-labeler
bot
added
the
untriaged
|
Tagging subscribers to this area: @carlossanlop Issue DetailsDescriptionAs adviced from .NET Framework 4.8 is able to open XPS files from printer driver containing many .piece files.
|
|
When porting System.IO.Packaging over to .NET Core, two features interrelated features were dropped: Streaming mode (which allows creation of interleaved parts), and support for reading/updating existing interleaved parts. The reason they were dropped was because the porter was under instruction to replatform the the code to use System.IO.Compression, instead of MS.Internal.IO.Zip. Since at the time, a major goal was still to keep the runtime size small by avoiding duplicated code, this made sense. However, part of the support for interleaved files (which is specific to Open Packaging Conventions) was implemented in the MS.Internal.IO.Zip namespace, and therefore was not ported, even though it could have been moved into to Packaging namespace where I would argue it should have been in the first place, and updated to be based on System.IO.Compression, just like the rest of the OPC specific code. This issue is basically requesting to restore the missing support for reading and updating interleaved files. The read and update support actually has no public API surface (unlike streaming), and its absence is a correctness issue, since support for reading interleaved OPC files is not optional in the specification. On the other hand, the ability to produce such files is optional, since interleaving is only meant as a possible performance enhancement, so restoring streaming mode is less important. It appears that all the needed code has been released under MIT, either as part of the initial commit of the code to this repository (where unfortunately, it had already been heavily modified from the original, so hopefully no needed code was removed, but it looks like most was just ifdefed out), or for the MS.Internal.IO.Zip code needed, was released as part of open sourcing WPF, although the code was deleted in later commits due to not being used by anything over there. |
|
ok @KevinCathcart thanks for explaining in detail. What does it mean for us now? Will it be added again to .NET Core /.NET 5? |
|
It is up to the area owners to decide if restoring such support is code they are willing to have and maintain. If it is then somebody needs to port the code, which while probably not hard, is also not completely trivial. Given that .NET 6 is already full in terms of planned projects, it would be unlikely to happen for that release unless a contributor creates a pull request. The chances of this being backported to older releases is pretty much zero, although as an OOB package, one ought to be able to reference a newer version from older net core based frameworks. Perhaps @carlossanlop or one of the other area owners can comment if a pull request fixing this regression vs .NET Framework (which is technically also a correctness issue) by porting the missing code might be accepted. As mentioned before there is no associated public API change. Instead, Zip-based OPC packages with interleaved parts would simply have those parts be visible again if opened with this package. Right now, such parts get silently ignored, which is not even easy for apps to detect is happening, unless they see a required part is missing or look in the zip file themselves. |
jozkee
added
needs-further-triage
|
@jozkee @carlossanlop @KevinCathcart @GrabYourPitchforks any updates? 🥺 |
|
🥺 🤞 😭 |
|
@carlossanlop @jozkee what's the status of this? |
|
Could this be achieved by using the code in a non Microsoft owned project? |
|
@danmoseley I hope I understand and answer your questions correctly. (I'm not a native English speaker) reproduction code can be found here: dotnet/wpf#3546 (comment) sample documents can be found here: dotnet/wpf#3546 (comment) the reason why it can't be opened can be found here: dotnet/wpf#3546 (comment) One of our products is based on/working with the Microsoft V4 XPS print driver. .NET Framework is able to open these XPS documents (print jobs). Why? Credits to @ThomasGoulet73 to figuring out that a part of .NET Framework code to support .piece files in System.IO.Packaging.* was not ported. So we have a product that worked fine in .NET Framework but doesn't work fine anymore in .NET (Core). I hope this answered the question "Could this be achieved by using the code in a non Microsoft owned project?". "Is there a way to get a sense of the level of interest in this feature, such that it would justify the cost of maintenance etc?" To begin with, everybody prints, but not everybody writes printing code or is in the printing industry. It is a niche, as you probably know. Personally: P.S.: As you have noticed we tried to wake up the WPF team at Microsoft and I believe the community has spoken, as you have seen in the latest discussions ( dotnet/wpf#6542 ). One of my personal reasons is that XPS is a good clean format, but as with WPF/XAML/XPS there is a lack of updates/fixes/improvements. I really hope when somebody looks at this XPS issue, additional improves to performance will be done also, for example: #51930 https://github.com/dotnet/runtime/issues?q=XPS I really hope Microsoft will spend some time on porting this .NET Framework code to .NET Core (and maybe some additional XPS fixes/improvements). Our customers and our team would be so thankful. |
|
It looks to me like “Piece” support is just a wrapper over top of the directory structure of the ZIP to create an aggregate stream that hides the underlying file parts from the zip: Here ZipFileInfo in netfx is analogous to ZipArchiveEntry in netcore. A null entry indicates that it might be a piece-file. Here’s the read path: I can’t see anything in the implementation of piece support that looks particularly hard but I haven’t dug too deep. I see it uses seekable zip entry streams which means it won’t work with our read-only archives (necessary for large zips) but that’s a limitation – not a blocker. I might be wrong but I bet we could fix that limitation as it looks like the seeking could even be redundant. I tend to agree that it looks like it was accidentally excluded because of the namespace. The devil may be in the details but I don’t expect it would be to hard to get to a prototype by adding in the missing code and mapping old concepts to new. I think we'd be willing to accept a contribution here that takes the InterleavedZipPartStream.cs and related classes from .NETFramework and ports them to System.IO.Packaging. The first step here would be to move that code over, porting from the WPF-Zip types to the IO.Compression types and see if there are any difficult dependencies. If not then I think we could accept a PR that added that along with tests. |
|
A mostly untested (I've got it to compile, but that is about it) prototype port of the interleaving feature can be found at: The adjustments to use ZipArchiveEntry were mostly straightforward. The biggest differences are the .Name for ZipFileInfo is .FullName in ZipArchiveEntry, and deletion is done via the entry, rather than the zip archive, and lastly the use of the ZipStreamManager. There were also a few small changes to account for nullability annotations, and some stylistic things that this repo has flagged as errors. I've not tried do any big changes like avoiding seeking for reads (I suspect the .NET Framework Version may simply have failed to handle zip files that big). In any case, multi gigabyte XPS files are probably a low priority to support, and XPS files are the only use of OPC that I am aware of that uses interleaving. I'm pretty sure the office open XML spec forbids interleaving, and obviously nuget's spec also forbids the use of interleaving (since the current nuget client cannot handle it). Hope this helps. |
|
Big up to @ericstj and @KevinCathcart looking forward to using this when it is available.
Regular (GDI) print jobs can grow to sevaral gigabytes due to the metafiles in them. XPS files tend to start smaller but can reach also several 100's of MB's or GB's easily. |
|
@KevinCathcart any idea when this will be added/released in .NET 6? |
danmoseley
added
help wanted
|
@ericstj this is marked 7.0, but we are now in stabilization mode for that release. from your comment above, I believe the status is that this isn't currently scheduled, but we would take a contribution as you described if offered? I need to post an announcement in this repo with the schedule. I'll do that now. |
Yes, at the moment this isn't something that the @dotnet/area-system-io-compression team is working on but a contribution would be considered. We are approaching a place in .NET 7.0 where we won't be able to take risky changes and are already past the place for new API. It is still possible to get no-API changes in |
|
any updates? Again being moved to "future" 😢 . .net core 1x --> .net core 2x --> .net core 3.x --> .net 5 --> .net 6 --> .net 7 --> "future" again... our frustrations are getting bigger again. 😠 |
|
Any updates? @jeffhandley @ericstj @danmoseley @KevinCathcart @carlossanlop |
|
I don't think there is a change from before. The issue is still open and marked as "help-wanted": we'd accept a PR for this, but it's not being actively worked on. Here's what I mentioned before.
|
|
Sorry @ericstj ... We use software developed by Microsoft. We therefore assume that you will make some efforts to fix bugs. We make efforts to use your products and ensure that our customers continue to use the Microsoft/.net eco-system. You can hardly expect us to also do the work for as a Microsoft employee. We already made an effort to alert you to the problem and write a (slow) workaround ourselves. Hundreds of our users would benefit from your fix. Now we keep saying that the problem lies with Microsoft and you guys won't fix it. (Believe me, at government agencies they don't like to hear this). Which of course is not a good advertisement for you guys either, but that's just the way it is. I really hope you guys can take up @KevinCathcart his proof-of-concept and fix it. He already made great efforts. (I am not able to spend days to get WPF/net running locally here, let alone weeks to learn to understand your entire internal code first before writing a fix. That knowledge is already with you guys, as @KevinCathcart proved). Thanks. (update: created a PR from Kevin his code so that you can pick it up: #78374 ) |
|
@ericstj what is the status? Can you make a push for .net 8 or do we have to wait another release? |
|
.NET 8 is done. |
|
😞 27-06-2016 - ... 2024? 2025? 7+ years have passed since .net core release without any fixes. fingers crossed...? |
|
what is the status of this for .net 9 ? |
|
I'm currently working on this, and hopefully there should be a PR ready for review "soon" - probably the next week or so, realistically. I'm adding test coverage for the new functionality as I go, but it doesn't do much to test the existing code. That particular gap should be triaged separately IMO. It's possible to control whether or not the XPS Document Writer printer triggers this behaviour, which makes it easier to test with real files. In the printer's advanced printing preferences, the interleaving option can be toggled off or on (the default behaviour is off.) Of course, this doesn't help particularly if you're not in control of the files you receive! |
|
@edwardneal you can find test documents here: In general, XPS still uses arraylists and code base could be improved. I often use this 900+ test document: imagine the performance boost if the STA thread requirement could be dropped. |
|
Thanks. I briefly tested with file 02 outside of the automatic tests, and was able to enumerate, read and write its package parts. There are definitely performance and functionality improvements to be made from ZipArchive upwards. Some of the largest improvements need an API review, but I think they're ultimately necessary. I'd personally like to see async support in ZipArchive, some memory usage reductions, and to move thumbnails and digital signatures from XpsDocument to Package - they're part of the base OPC specification, and Office documents (and possibly NuGet packages and VSIX extensions) use them. |
|
@edwardneal will these changes be backported to .net 6 and .net 7 (as some of our customers are stuck for months/years on .net 6/7....) |
|
I don't know about .NET 6, but I'd be surprised if any new functionality was backported to .NET 7. MS' support lifecycle puts .NET 7 in Maintenance support, in which only security fixes are made. The process for requesting a backport is found here. It reads to me like you'll need to wait for the current PR to be merged, then raise a second PR requesting its backport. |
|
I think it is unlikely for backports to be approved for this, but it also shouldn't be needed at least longer term, as if accepted it will become part of the System.IO.Packaging nuget that targets .NET Standard 2.0, so any impacted .NET 6 or .NET 7 or .NET 8 project should be able to just reference the 9.0 version of the package to add this functionality back. Unfortunately that NuGet probably won't contain builds targeting 6 or 7, so the .NET Standard 2.0 build would get used, I'd expect that would merely impact performance rather than functionality, and the resulting performance we are talking about here should still be at .NET Framework level or better, so quite unlikely to be a showstopper. |
|
.net 6 would be great but not necessary. Backport to .net 7 would be needed. The problem is that some clients (gov, fortune500, ..) cannot install newer (internally not approved) .net releases on some servers, and at least not .net 8 / .net 9 (some even require that a release needs to be on the market for 6months or 1 year already to be approved as "ok" or "stable") That is why I asked if this will be backported so that we could roll this out to some customers. Minor releases from an approved framework version are faster accepted. Appreciate already what you are doing! Let's make XPS great again 👍 |
|
@edwardneal would you be able to tackle this other EPIC issue with xps in .net? (dropping the STA requirement) dotnet/wpf#4000 (comment) 🤞 😃 can't wait to test the changes in production 👍 |
|
We can consider this fixed by #97898 unless someone reports otherwise. |
Description
As adviced from
/runtime/wpfI should open a ticket here for thedotnet/runtimeteam...Please see dotnet/wpf#3546 for full details, test documents and many more.
.NET Framework 4.8 is able to open XPS files from printer driver containing many .piece files.
.NET 5 ( / .NET Core) is unable to open XPS files from printer driver containing many .piece files.
The text was updated successfully, but these errors were encountered: