ZipFile.ExtractToDirectory throws IOException

[velocitysystems]

Steps to Reproduce

1. Use System.IO.Compression.ZipFile.ExtractToDirectory to extract a ZIP archive.

Expected Behavior

Method executes successfully.

Actual Behavior

Method fails and an IOException is thrown with the following stack trace:

System.IO.IOException: Invalid parameter
  at System.IO.File.SetLastWriteTime (System.String path, System.DateTime lastWriteTime) [0x0002a] in <f32579baafc1404fa37ba3ec1abdc0bd>:0 
  at System.IO.Compression.ZipFileExtensions.ExtractToFile (System.IO.Compression.ZipArchiveEntry source, System.String destinationFileName, System.Boolean overwrite) [0x00067] in <8d4e08735b1842d0842f236ab3e206a3>:0 
  at System.IO.Compression.ZipFileExtensions.ExtractToDirectory (System.IO.Compression.ZipArchive source, System.String destinationDirectoryName, System.Boolean overwrite) [0x0009d] in <8d4e08735b1842d0842f236ab3e206a3>:0 
  at System.IO.Compression.ZipFile.ExtractToDirectory (System.String sourceArchiveFileName, System.String destinationDirectoryName, System.Text.Encoding entryNameEncoding, System.Boolean overwrite) [0x00017] in <8d4e08735b1842d0842f236ab3e206a3>:0 
  at System.IO.Compression.ZipFile.ExtractToDirectory (System.String sourceArchiveFileName, System.String destinationDirectoryName, System.Text.Encoding entryNameEncoding) [0x00000] in <8d4e08735b1842d0842f236ab3e206a3>:0 
  at System.IO.Compression.ZipFile.ExtractToDirectory (System.String sourceArchiveFileName, System.String destinationDirectoryName) [0x00000] in <8d4e08735b1842d0842f236ab3e206a3>:0 

Note: Method executes successfully on Xamarin.iOS and Windows environments.

Version Information

Visual Studio Professional 2017 for Mac
Version 7.5.3 (build 7)
Installation UUID: 62b6ba72-e19d-446a-96a0-c688ca3c1161
Runtime:
Mono 5.10.1.57 (2017-12/ea8a24b1bbf) (64-bit)
GTK+ 2.24.23 (Raleigh theme)
Xamarin.Mac 4.4.1.178 (master / eeaeb7e6)

Package version: 510010057

NuGet
Version: 4.3.1.4445

.NET Core
Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
2.1.1
2.0.5
2.0.0
1.0.2
SDK: /usr/local/share/dotnet/sdk/2.1.301/Sdks
SDK Versions:
2.1.301
2.1.4
2.0.0
1.0.0-preview2-003148
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/5.10.1/lib/mono/msbuild/15.0/bin/Sdks

Xamarin.Profiler
Version: 1.6.2
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

Apple Developer Tools
Xcode 9.4.1 (14161)
Build 9F2000

Xamarin.Mac
Version: 4.4.1.193 (Visual Studio Professional)

Xamarin.iOS
Version: 11.12.0.4 (Visual Studio Professional)
Hash: 64fece5f
Branch: d15-7
Build date: 2018-05-29 20:00:44-0400

Xamarin.Android
Version: 8.3.3.2 (Visual Studio Professional)
Android SDK: /Users/mattrichnz/Library/Developer/Xamarin/android-sdk-macosx
Supported Android versions:
7.0 (API level 24)
7.1 (API level 25)
8.1 (API level 27)

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 27.0.1
SDK Build Tools Version: 27.0.3

Java SDK: /usr
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)

Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

Xamarin Inspector
Version: 1.4.0
Hash: b3f92f9
Branch: master
Build date: Fri, 19 Jan 2018 22:00:34 GMT
Client compatibility: 1

Build Information
Release ID: 705030007
Git revision: 13cecd02aceddf29a1ed57b86f81c02994df1483
Build date: 2018-06-14 15:48:08-04
Xamarin addins: 7065de97cf22c9038fdc39dd627f2c30790fd8af
Build lane: monodevelop-lion-d15-7

Operating System
Mac OS X 10.13.6
Darwin 17.7.0 Darwin Kernel Version 17.7.0
Thu Jun 21 22:53:14 PDT 2018
root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64

Enabled user installed addins
StyleCop Support 1.1.0.0
Internet of Things (IoT) development (Preview) 7.5

Workaround

using (ZipArchive archive = ZipFile.OpenRead(sourceArchive.Path))
{
    foreach (var entry in archive.Entries)
    {
        using (var entryStream = entry.Open())
        {
            // Write unzipped file using an IO file stream.
        }
    }
}

[1Cor125]

I am seeing this as well. The workaround seems to not perform as well so would really like this addressed soon.

[apwillies]

Be nice to see this fixed.

[velocitysystems]

@jonpryor Hi Jonathan. I was just wondering, does the XA team require any more information to investigate this issue?

[velocitysystems]

@jonathanpeppers @JonDouglas Just wondering if this issue can be investigated?
It is still present in the latest version. Xamarin iOS is fine.

[jonathanpeppers]

@mattrichnz this seems weird that it works on Xamarin.iOS. That System.IO.Compression code is in Mono’s BCL so should be the same on both platforms.

It Labor Day (holiday weekend) in the US, but I should be able to look at this next week.

[velocitysystems]

@jonathanpeppers Thanks Jonathan - We really appreciate that so much.
We've put together a repro project too which demonstrates the issue.
ExtractZipArchive.zip

[jonathanpeppers]

@mattrichnz this project (ExtractZipArchive.zip) worked for me on a Pixel 2 (running Android Pie) and a HAXM emulator (running Android Oreo).

I am also using VS 2017 15.8.2 (on Windows), which has Xamarin.Android 9.0.

As a side note, in your sample, I had to add this code:

if (Directory.Exists (destExtractedPath))
    Directory.Delete (destExtractedPath, true);

Or subsequent runs would fail saying the files already exist.

So the question is:

• When you posted this you were using Xamarin.Android 8.3.x, does Xamarin.Android 9.0 fix it?
• Is it a specific device with the problem?

[velocitysystems]

@jonathanpeppers Thanks for looking into this issue.
Just retested on VS 2017 15.8.2 (Windows) with Xamarin.Android 9.0.x and same crash.
The device is an ASUS P028 (running Android Nougat).

Note: I've also updated the repro project to include your code adjustment: ExtractZipArchiveV2.zip

[velocitysystems]

@jonathanpeppers See additional device test results below. Two passing and one failing device.

Motorola Moto G4 (Android 7.0)
No crash.

Motorola XT1032 (Android 5.1)
No crash.

Samsung GT-5110 (Android 4.4)
Crash, same exception as on the ASUS P028.

09-04 16:57:13.537 I/MonoDroid(24295): System.IO.IOException: Invalid parameter
09-04 16:57:13.537 I/MonoDroid(24295):   at System.IO.File.SetLastWriteTime (System.String path, System.DateTime lastWriteTime) [0x0002a] in <43dbbdc147f2482093d8409abb04c233>:0 
09-04 16:57:13.537 I/MonoDroid(24295):   at System.IO.Compression.ZipFileExtensions.ExtractToFile (System.IO.Compression.ZipArchiveEntry source, System.String destinationFileName, System.Boolean overwrite) [0x00067] in <46b93a3f54d44d41b7f6507929074c62>:0 
09-04 16:57:13.537 I/MonoDroid(24295):   at System.IO.Compression.ZipFileExtensions.ExtractToDirectory (System.IO.Compression.ZipArchive source, System.String destinationDirectoryName, System.Boolean overwrite) [0x0009d] in <46b93a3f54d44d41b7f6507929074c62>:0 
09-04 16:57:13.537 I/MonoDroid(24295):   at System.IO.Compression.ZipFile.ExtractToDirectory (System.String sourceArchiveFileName, System.String destinationDirectoryName, System.Text.Encoding entryNameEncoding, System.Boolean overwrite) [0x00017] in <46b93a3f54d44d41b7f6507929074c62>:0 
09-04 16:57:13.537 I/MonoDroid(24295):   at System.IO.Compression.ZipFile.ExtractToDirectory (System.String sourceArchiveFileName, System.String destinationDirectoryName, System.Text.Encoding entryNameEncoding) [0x00000] in <46b93a3f54d44d41b7f6507929074c62>:0 
09-04 16:57:13.537 I/MonoDroid(24295):   at System.IO.Compression.ZipFile.ExtractToDirectory (System.String sourceArchiveFileName, System.String destinationDirectoryName) [0x00000] in <46b93a3f54d44d41b7f6507929074c62>:0 
09-04 16:57:13.537 I/MonoDroid(24295):   at NetStandardLib.Zip.ExtractToDirectory (System.String sourceArchiveFilename, System.String destinationDirectoryName) [0x00001] in C:\Users\mattrichnz\Desktop\ExtractZipArchive\NetStandardLib\Zip.cs:9 
09-04 16:57:13.537 I/MonoDroid(24295):   at ExtractZipArchive.MainActivity.OnCreate (Android.OS.Bundle savedInstanceState) [0x0009a] in C:\Users\mattrichnz\Desktop\ExtractZipArchive\ExtractZipArchive\MainActivity.cs:41 
09-04 16:57:13.537 I/MonoDroid(24295):   at Android.App.Activity.n_OnCreate_Landroid_os_Bundle_ (System.IntPtr jnienv, System.IntPtr native__this, System.IntPtr native_savedInstanceState) [0x00011] in <a10f61e70eeb434e952fef884856c199>:0 
09-04 16:57:13.537 I/MonoDroid(24295):   at (wrapper dynamic-method) System.Object.3(intptr,intptr,intptr)

[jonathanpeppers]

@mattrichnz could this be an issue with your timezone? New Zealand, right?

If we look at this code, it uses ZipArchiveEntry.LastWriteTime.DateTime: https://github.com/mono/corefx/blob/ba2f6954028e8dc4a571b34a23a29fbe9e8a40d2/src/System.IO.Compression.ZipFile/src/System/IO/Compression/ZipFileExtensions.cs#L293

Can you add this code and see if any of the dates look fishy?

using (var zip = ZipFile.OpenRead (sourceArchiveFilename)) {
	foreach (var entry in zip.Entries) {
		Console.WriteLine ($"File: {entry.FullName}, {entry.LastWriteTime.DateTime}");
	}
}

Output:

09-04 12:06:20.898 I/mono-stdout(20303): File: Monkeys.jpg, 9/3/2018 9:44:24 AM
09-04 12:06:20.899 I/mono-stdout(20303): File: __MACOSX/, 9/3/2018 9:44:32 AM
09-04 12:06:20.899 I/mono-stdout(20303): File: __MACOSX/._Monkeys.jpg, 9/3/2018 9:44:24 AM

You might also try setting the timezone on the device to a US one and see what happens?

[velocitysystems]

@jonathanpeppers Thanks Jonathan.
Yes, I'm currently working outside the US (was in New Zealand, currently in the UK).

I've added the sample code you mentioned, with one minor change to show DateTimeOffset:
Console.WriteLine ($"File: {entry.FullName}, {entry.LastWriteTime.ToString()}");
ExtractZipArchiveV3.zip

Console output:

09-05 09:29:57.112 I/mono-stdout( 2906): File: Monkeys.jpg, 9/3/2018 9:44:24 AM +01:00
09-05 09:29:57.114 I/mono-stdout( 2906): File: __MACOSX/, 9/3/2018 9:44:32 AM +01:00
09-05 09:29:57.114 I/mono-stdout( 2906): File: __MACOSX/._Monkeys.jpg, 9/3/2018 9:44:24 AM +01:00

Everything looks as expected.
The DateTimeOffset dates look correct showing a +01:00 offset for the UK time zone.

I retested on all the devices I have here too and tried changing timezone etc.

ASUS P028 (Android 7.0)
Crash.
Device set to GMT+01:00 British Summer Time.
No effect changing to EDT/PDT.

Motorola Moto G4 (Android 7.0)
No crash.
Device set to GMT+01:00 British Summer Time.

Motorola XT1032 (Android 5.1)
No crash.
Device set to GMT+01:00 British Summer Time.

Samsung GT-5110 (Android 4.4)
Crash.
Device set to GMT+01:00 British Summer Time.
No effect changing to EDT/PDT.

[velocitysystems]

@jonathanpeppers Seems that setLastModified() is unreliable on Android, working on some devices and not others.

Android issue:
http://code.google.com/p/android/issues/detail?id=15480

Other references:

• https://bugzilla.xamarin.com/show_bug.cgi?id=3029
• https://stackoverflow.com/questions/18677438/android-set-last-modified-time-for-the-file
• [texto-to-speech] DataCache File.SetLastWriteTime throws IOException: Invalid parameter on Android watson-developer-cloud/unity-sdk#125

So, it seems this is not a Mono issue but an Android OS issue. Our workaround has been to put underlying code from Mono's BCL into a static extension method and remove the call to File.SetLastWriteTime(destinationFileName, source.LastWriteTime.DateTime);

Would you have any further recommendations?

[jonathanpeppers]

@mattrichnz I would definitely recommend your approach. Even if we got a change in Mono, it takes quite a while for that to make it to a Visual Studio release.

What concerns me though...

Does File.Copy also break in this way? It preserves the timestamp from the source file.

[velocitysystems]

@jonathanpeppers Good Q. I tested on the affected devices and doesn't break. Well, that's a relief 👍

Our final solution in the end was to put the call to ExtractToDirectory in a try-catch, and fall back to the manual implementation listed in the workaround at the beginning of the topic on affected devices.

Marking this issue as closed since not related to Xamarin.Android or Mono but to the FUSE filesystem in Android OS. Since FUSE has been replaced by SDCardFS in Android Oreo and above, this issue is effectively resolved in future versions of Android.