Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize repo size #241

Closed
xavier506 opened this issue Jan 18, 2020 · 3 comments
Closed

Optimize repo size #241

xavier506 opened this issue Jan 18, 2020 · 3 comments
Assignees

Comments

@xavier506
Copy link
Member

Remove large unused git objects to reduce repo size.

The largest file is an idx called something like .git/objects/pack-f8... that has a large object of over 70mb thats making cloninfg the repo really slow.

we need to remove that object from the history también.

@rubenabix
Copy link
Contributor

@danazkari I made a fork and try with
https://rtyley.github.io/bfg-repo-cleaner/ and everything is ok but in the eosrate repo I found an error.

! [remote rejected] refs/pull/94/head -> refs/pull/94/head (deny updating a hidden ref)

rtyley/bfg-repo-cleaner#36

Maybe we can try
https://help.github.com/en/github/authenticating-to-github/removing-sensitive-data-from-a-repository

@rubenabix
Copy link
Contributor

`➜ eosrate java -jar /Users/rub/Downloads/bfg-1.13.0.jar --strip-blobs-bigger-than 60M eos-rate.git

Using repo : /Users/rub/Documents/git/EOSCR/eosrate/eos-rate.git

This repo has been processed by The BFG before! Will prune repo before proceeding - to avoid unnecessary cleaning work on unused objects...
Completed prune of old objects - will now proceed with the main job!

Scanning packfile for large blobs: 11000
Scanning packfile for large blobs completed in 65 ms.
Found 1 blob ids for large blobs - biggest=70592168 smallest=70592168
Total size (unpacked)=70592168
Found 178 objects to protect
Found 113 commit-pointing refs : HEAD, refs/heads/226, refs/heads/44, ...

Protected commits

These are your protected commits, and so their contents will NOT be altered:

  • commit 5fb448e (protected by 'HEAD')

Cleaning

Found 543 commits
Cleaning commits: 100% (543/543)
Cleaning commits completed in 397 ms.

Updating 69 Refs

Ref                   Before     After   
-----------------------------------------
refs/pull/100/head  | 0e884782 | 3a190fe7
refs/pull/101/head  | d7ffec86 | 5a8e9e15
refs/pull/102/head  | 8c299c6a | 468dc133
refs/pull/106/head  | 7095bdfc | 0ff5809e
refs/pull/109/head  | d6b8d1ac | 715b32d4
refs/pull/112/head  | 6030c71e | e454fb87
refs/pull/113/head  | 3c429ae7 | 0cf1dc41
refs/pull/114/head  | f19d5a96 | 9bcd8478
refs/pull/115/head  | c4e934fe | 532d5427
refs/pull/116/head  | 2ff11c29 | a70704f9
refs/pull/124/head  | 09120c4a | 70966489
refs/pull/131/head  | afddef5d | 0c255b4a
refs/pull/132/head  | e7e039a4 | df8722fb
refs/pull/138/head  | 3b122a8c | 1af96351
refs/pull/143/head  | dd2107db | c8c78112
...

Updating references: 100% (69/69)
...Ref update completed in 70 ms.

Commit Tree-Dirt History

Earliest                                              Latest
|                                                          |
...........DDDDDDDDDDmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm

D = dirty commits (file tree fixed)
m = modified commits (commit message or parents changed)
. = clean commits (no changes to file tree)

                        Before     After   
-------------------------------------------
First modified commit | e053022c | 1c183c62
Last dirty commit     | 35fab144 | 2c2fd08c

Deleted files

Filename                     Git id            
-----------------------------------------------
eosio.cdt-1.4.0.x86_64.deb | e140d558 (67.3 MB)

In total, 357 object ids were changed. Full details are logged here:

/Users/rub/Documents/git/EOSCR/eosrate/eos-rate.git.bfg-report/2020-01-18/11-33-46

BFG run is complete! When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressive`

eosio.cdt-1.4.0.x86_64.deb | e140d558 (67.3 MB)

@rubenabix
Copy link
Contributor

rubenabix commented Jan 20, 2020

I used, the repo size is fixed

git filter-branch --index-filter "git rm --cached --ignore-unmatch *.deb" --tag-name-filter cat -- --all

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants