FreeCAD parts library is corrupted

[lfrnndz]

Hi, devs!

I downloaded the zip file, but I get this message upon extraction of the file.

I also can't get the parts library installed using the addon manager. I let it run for over two hours, but FreeCAD just seemed to hang. For comparison, I downloaded the zip file here in github for about 30 minutes.

[luzpaz]

We need to do something about the size of the repo it's enormous and GH maybe throttling it.

CC @yorikvanhavre @chennes

[chennes]

If the download worked fine direct from GitHub, it's probably the Python url lib request that's borking. A quick look at the code makes me think it's actually trying to keep the whole thing in RAM instead of streaming it to disk. That's a pretty easy change, and I'm working in AddonManager now anyway. I'll go ahead and implement that.

[bitfarmer]

Some more insights on this problem. I twice tried a direct git clone in a terminal session overnight, protected from logout by screen and on both occasions it aborted at some stage during the night. I then tried a direct zip download from GitHub and that is indicating, as above, to be corrupted: unexpected end of archive. I am now trying a shallow-clone to see if there is any hope of obtaining the library.

For what it is worth, I suspect that something is overflowing its number field width, either 32 bit (blocks) or 64 bit (bytes).
If the shallow clone works, I will report later

[bitfarmer]

That didn't work either. I suspect that the shallow clone is nearly as big as the normal one. In case anybody is analysing this, here is the terminal log:

Cloning into 'parts_library'...
remote: Enumerating objects: 11809, done.
remote: Counting objects: 100% (11809/11809), done.
remote: Compressing objects: 100% (8146/8146), done.
client_loop: send disconnect: Broken pipe0 GiB | 354.00 KiB/s
fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: fetch-pack: invalid index-pack output

The total downloaded amount was 4103GiB (4302741503 bytes) before aborting. And whats interesting is none of these numbers is near a 32 bit or 64 bit integer limit. Maybe something does overflow internally.

[chennes]

I would very much like to fix this "for real" and not just find a temporary workaround. I've started a discussion in the forums here: https://forum.freecadweb.org/viewtopic.php?f=8&t=63140

[ghost]

One of the best way to reduce the size of the repo is to squash all the commits messages. With a git client it freezzes when im seeing the commits