XrdEc or XrdZip : can it handle more than 32k entries in a .zip ?

[wyang007]

While playing with XrdEc, I ran into an issue that when a file is really large, and require putting more than 32k chunks in backend .zip archives, I can't read them back. If this very large file itself is a non-zip file, xrdcp (with XrdEc config loaded) will get:

Run: [ERROR] Received corrupted data: End-of-central-directory signature not found. (source)

If this very large file is a .zip file, xrdcp will crash.

Furthermore, if I go to the backend data servers that host those corresponding zip archives, and trying to unzip them, unzip will complain:

End-of-central-directory signature not found.

[xrootd-dev]

The issue here is that the "directory" is assumed to be no more than 64K and that is all that is read. Obviously, in your case, it is bigger than that and will not be able to be processed. Michal may be able to correct this but given the odditities of zip files it may not be all that easy. Andy

…

On Tue, 5 Jul 2022, Wei Yang wrote: While playing with XrdEc, I ran into an issue that when a file is really large, and require putting more than 32k chunks in backend .zip archives, I can't read them back. If this very large file itself is a non-zip file, xrdcp (with XrdEc config loaded) will get: Run: [ERROR] Received corrupted data: End-of-central-directory signature not found. (source) If this very large file is a .zip file, xrdcp will crash. Furthermore, if I go to the backend data servers that host those corresponding zip archives, and trying to unzip them, unzip will complain: End-of-central-directory signature not found. -- Reply to this email directly or view it on GitHub: #1733 You are receiving this because you are subscribed to this thread. Message ID: ***@***.***> ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1

[abh3]

I am closing this ticket. Please reopen if necessary.