Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A sample of bz2 file that takes too long to decode (7mb -> 1min) #10

Closed
Lakr233 opened this issue Apr 26, 2020 · 5 comments
Closed

A sample of bz2 file that takes too long to decode (7mb -> 1min) #10

Lakr233 opened this issue Apr 26, 2020 · 5 comments

Comments

@Lakr233
Copy link

Lakr233 commented Apr 26, 2020

Sample File:
Packages.bz2.zip

Programme Behave:
截屏2020-04-26下午9 48 49

I have no idea why this is happening............ :/

@tsolomko
Copy link
Owner

I have slightly unrelated question: what have you used to create your ZIP file?

@tsolomko tsolomko changed the title A smaple of bz2 file that takes too long to decode (7mb -> 1min) A sample of bz2 file that takes too long to decode (7mb -> 1min) Apr 26, 2020
@Lakr233
Copy link
Author

Lakr233 commented Apr 26, 2020

I have slightly unrelated question: what have you used to create your ZIP file?

I used the Finder to create. (that right-click

btw just be clear, GitHub does not allow me to upload .bz2 file directly so I uploaded a zip of that bz2

@tsolomko
Copy link
Owner

Your ZIP file actually helped me uncover an interesting issue. Apparently, Finder, or whatever it is using under the hood, produces ZIP files which don't strictly conform to ZIP specifications (Extended Timestamp Extra Field in the Central Directory must only contain modification time, but Finder also puts there access time and creation time). What's even more interesting is that Finder in my version of macOS produces correct ZIP files.

Currently, SWCompression crashes when trying to open such incorrect ZIP files. While I think that this is, generally, an issue on Finder's part and should be fixed by Apple, at the same time SWCompression probably shouldn't crash when it encounters these ZIP files. I am going to fix this particular issue in the near future.


Anyway, back to your original question. I was unable to reproduce the problem you've encountered: on my pretty old machine (Macbook Pro Late 2011) with macOS 10.13 and Swift 5.0.3 it took SWCompression (built in Release mode) less than 2 seconds to decompress your BZip2 file. Which leads me to the following questions:

  1. Are you building SWCompression in Debug or Release mode? Please note, that the optimizations Swift compiler does in Release mode dramatically improve the performance of SWCompression.

  2. What Swift version are you using?

@Lakr233
Copy link
Author

Lakr233 commented Apr 26, 2020

Are you building SWCompression in Debug or Release mode?

Currently debut mode. I will pay attention to this tricky switch next time.

What Swift version are you using?

Swift 5.1? I used latest Xcode with it’s SDKs.

———
Currently, debug sessions are reporting decode times from 30s to 350s floating between them while other files with same type can do a pretty job. I still having no idea why this is happening. I’ll follow up this post when I got some idea.

———

Your ZIP file actually helped me uncover an interesting issue

The zip file format was badly played by coders around the world, fact is that command line such as unzip (usually from apt or home brew) cannot handle all of them. There are always zip files with extra info attached to or with illegal byte sequence. ! Been a headache over years 🤦‍♂️

@Lakr233
Copy link
Author

Lakr233 commented Apr 26, 2020

This bug is gone silently in release mode.

@Lakr233 Lakr233 closed this as completed Apr 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants