Delete configlet binaries from history? #300
Comments
|
Thanks for this analysis and for making the action easier. I agree with your sentiment and think we should do this. I'm investigating the implications. |
|
The problem is, that the website actually keeps a not of the SHA to determine which version of an exercise has been solved by the student. Rewriting history will change all SHAs since the first removed/altered commit. I'm pretty sure it will affect the website if done blindly, @iHiD or @kytrinyx should give a go first. |
|
Yeah - we can't rewrite history. Basically. It'll break everything. |
|
To elaborate on at least one thing that I know will break: The exercism CLI uses git commit hashes to download and display the proper version of the test suite. So when a test suite is updated on a language track, the git commit hash on the master branch for that point in time is used as a key. I suppose the lesson here is that we shouldn't use externally provided IDs that have another primary purpose as keys in our data structures. :-P @iHiD: If we were to eventually want to prune our git histories, are there other places that depend on git commit hashes? The test suite version is one issue that can, for a small track like Ocaml, easier be addressed than on so many other tracks. |
|
It's not really using it as a key. It literally checks out the git repo with the sha and the displays the contents of it to the user. So if the sha isn't there, the checkout will fail and nothing will be displayed to the user. |
|
@sshine So the thing we could do here, would be to map between the old shas and the new shas during the rebase (maybe matching on commit strings if they're uniq?) and then update the database to change all old shas to new shas. And push the rebased version over the top? It's a lot of hassle, but it's a workable solution I think, if we ever actually need to do this. |
|
I went and sorted all track repos by size and inspected the 12 largest repos using @ee7's snippet, yielding
I don't know how to estimate the win of purging stale history from tracks other than On that account, I think @iHiD would have to call the shots, and I fully understand if this is a low priority of his resources. |
|
I suppose the fix that avoids dependency on GitHub commit hashes is to use a timestamp instead and then at any given point when one needs to find e.g. the test suite appropriate for a given point in time (this is the only use of git commit hashes that I know of), convert this timestamp to a commit hash via the GitHub API or similar. Given such a change, rewriting history would be less of an issue. But this, I suppose, does not change the priority for now. |
Issue
Try running
git clone https://github.com/exercism/ocaml.gitExpected behavior
It finishes instantly.
Actual behavior
It takes a long time.
Diagnosis
The repo download is about 24 MiB, which is roughly 100x larger than it should be.
Here is a useful shell script that finds large commits in git history.
The output:
It looks like #11 was not successful. The
configletbinaries were deleted in e615131, but they remain in the repo history.These commits are still in
master:8dd7d33
e662023
296040f
2f99459
e71f60c
ffff920
16d5a30
89e9f96
Solution
As mentioned in #11, you would need to rewrite the history. I don't know whether it's worth fixing now.
I didn't see any related discussion, so I'm just checking whether it's a known issue.
Other tracks
As far as I can tell, no other Exercism tracks still have the
configletbinaries in themasterhistory.If you're curious, here are the other tracks with large files:
Go
Javascript
PHP
Swift
The text was updated successfully, but these errors were encountered: