Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop support for RAR archives or replace Libarchive with different library #165

Closed
CirnoT opened this issue Aug 6, 2019 · 5 comments
Closed

Comments

@CirnoT
Copy link
Contributor

CirnoT commented Aug 6, 2019

LRR Version and OS
n/a

Bug Details
Libarchive support for RAR is severely lacking. 2/2 RAR archives I've tried did not work, instead generating files filled with zeroes despite being fully working using any other tool such as unrar, unrar-free or 7z. The issue can be traced here: libarchive/libarchive#373
The documentation for libarchive is misleading in that it supports RAR and RAR5. For reference, this was tested both with libarchive 3.2.2-3.1ubuntu0.3 as well as custom build directly from master branch of https://github.com/libarchive/libarchive/. This issue proposes to remove support from RAR from LRR or to change Libarchive to different library supporting RAR archives. As it stands now, there is a high chance that any RAR files dropped at LRR will end up with no thumbnail and files being extracted filled with zeroes.

Matching Logs

[LANraragi] [debug] Files found in archive:
 $VAR1 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/01.jpg';
$VAR2 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/02.jpg';
$VAR3 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/03.png';
$VAR4 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/04.png';
$VAR5 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/05.png';
$VAR6 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/06.png';
$VAR7 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/07.png';
$VAR8 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/08.png';
$VAR9 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/09.png';
$VAR10 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/10.png';
$VAR11 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/11.png';
$VAR12 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/12.png';
$VAR13 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/13.png';
$VAR14 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/14.png';
$VAR15 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/15.png';
$VAR16 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/16.png';
$VAR17 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/17.png';
$VAR18 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/18.png';
$VAR19 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/19.png';
$VAR20 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/20.png';
$VAR21 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/21.png';
$VAR22 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/22.png';
$VAR23 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/23.png';
$VAR24 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/24.png';
$VAR25 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/25.jpg';
$VAR26 = '/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/26.jpg';

[LANraragi] [debug] Thumbnail not found at /db/thumb/c0cfa338cdb7768cdad8595f1e32a587287296bf.jpg ! (force-thumb flag = 0)
[LANraragi] [debug] Regenerating from /opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/01.jpg
Image::Scale unknown file type (/opt/lanraragi/script/../public/temp/c0cfa338cdb7768cdad8595f1e32a587287296bf/01.jpg), first 8 bytes were: 00 00 00 00 00 00 00 00

as well as

[2019-08-06 19:21:53.98959] [10811] [error] Parsing filters is unsupported. at /usr/local/lib/x86_64-linux-gnu/perl/5.26.1/Archive/Peek/Libarchive.pm line 20.

Screenshots
n/a

@Difegue
Copy link
Owner

Difegue commented Aug 6, 2019

I'd seen this problem when I implemented libarchive; RAR5 seems rather uncommon so I don't think it's worth removing the entire filetype over it.

(Libarchive is also pretty ubiquitous for my usecase and I don't think there are better alternatives out there.)

I could use a proper error message encouraging users to switch the archive to .zip when the parsing filters stuff happens though, it's unique enough to be detected.

@CirnoT
Copy link
Contributor Author

CirnoT commented Aug 6, 2019

The archives in question were not RAR5 format however. I can provide archive for testing if necessary.

WinRAR_2019-08-07_08-43-41

Also worth of note is that new version of Libarchive that i compiled did not trigger that error but still silently produced empty files.

@Difegue
Copy link
Owner

Difegue commented Aug 8, 2019

Right, parsing filters are also present in RAR 3/4 iirc. (RAR5 is simply not supported at all) It's a coin toss as to whether archives will use those or not; Usually only WinRAR implements the RAR spec fully.

That last point is indeed way more annoying; An issue should be sent to the libarchive folks but considering how spotty their rar support is, I'm not sure it'll be answered. The issue might also be coming from the perl glue library not propagating the error properly.

Test files would be cool; I'll look this up when I have the time.

@CirnoT
Copy link
Contributor Author

CirnoT commented Aug 8, 2019

Usually only WinRAR implements the RAR spec fully.

And 99% of RARs downloaded are made by WinRAR. The glue library is fairly simple in this specific case, it fails at direct call to Peek.

I doubt libarchive folks are interested in doing anything with RAR support. That being said, LRR should most likely try to warn user about RAR archives in some way considering how delicate their support is right now.

@Difegue
Copy link
Owner

Difegue commented Sep 24, 2019

Took me a bit too long, but I added both a warning if the archive extracts without errors (to match the problem you encountered with empty files), and an explicit error if we get unsupported format on a .rar file.

I thought about adding separate functions for handling RARs that'd use the rarlabs nonfree extracter, but that'd be a bunch of extra code to write for little gain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants