Repository size

[shadiakiki1986]

Because

• of my need to install pymssql from source due to (issue 432)
• the pip install was taking so long on my machine
• pip install does not support the depth parameter that git clone has (ref)

I was looking at the pymssql repository size. Downloading it shows that it's a mere ~700 KB, but cloning it shows that it's around 14MB.

The bash script here that lists large files in a repository shows the following

size  pack  SHA                                       location
3629  3592  53466ca5870402f2e54c5d500629808e0bfe3ea0  win32/freetds.zip
2374  827   87b3ac4d88046e3c1795f73c7c3b155e87db27c5  freetds/vs2008_32/lib/db-lib.lib
2118  638   93fda1d7f41f30dfe2fb91fffb59a832773ee6fa  freetds/nix_64/lib/libsybdb.a
1501  544   a2f009f11c35ff4bc7c7de223a8ba517c177a6f6  freetds/vs2008_32/lib/tds.lib
1492  612   ee526b3437773eb6f56b39540cdeee8bb0c96af8  freetds/nix_32/lib/libsybdb.a
1455  1435  a76e0390a2616331c751b4336bc40c5515b7e8c8  win32/freetds.zip
982   292   566720b7916fdee35005904984ef6a763502c452  freetds/nix_64/lib/libsybdb.a
903   235   39444f4ef8e41d6681f0a53556a9c43035d6ced4  freetds/nix_64/lib/libsybdb.a
701   230   eabab9a9891588c5995d3eecfa39668cca4385bc  freetds/nix_32/lib/libsybdb.a
701   258   d6874fc1cbb08f4388cecce433ec4f8d285800d5  freetds/nix_32/lib/libsybdb.a

It seems that your commit history contains copies of win32/freetds.zip of about 5 MB.
Removing those 2 files from the history could cut the pip install size by a third.

To help more, I tried to get the commit date of the first SHA with git log --oneline 53466ca5870402f2e54c5d500629808e0bfe3ea0, but that doesn't show anything. If I replace log with show, then I get a bunch of binary output, and no dates at all.
What are your thoughts on this? Do you think this is worth it?

[shadiakiki1986]

So I forked the pymssql repository, followed the instructions linked above (here again) and removed these files from the history (full steps here).
Then I ran a benchmarking test time pew new ENV_FORK -r requirements.txt -d where requirements.txt had one of the following

• git+https://github.com/shadiakiki1986/pymssql.git
• git+https://github.com/pymssql/pymssql.git

Testing from an AWS EC2 instance, it turns out there is no speedup from the deep cleaning.
Maybe the effect would be seen from machines with slower connections?
Not sure

[shadiakiki1986]

I retested this on a machine with a slower connection, and both the pip install ...pymssql/pymssql.git and pip install ...shadiakiki1986/pymssql.git take 2m 40s. So the fork and removal of the large files is basically useless. Sorry about all the noise in this issue. Closing.