theme: reorganize repo to reduce size

[McShelby]

As mentioned in the official forum as well as in disscussion #686 (now deleted by @calalloo), the theme repo is quite big on your disc due to the fact that it contains the history of all the docs screenshots.

Outsourcing the docs is only part of the solution as it will not cause the cloned repo to shrink. So further actions are required:

• rewrite the repos history. This may results in new hashes for existing changesets. Implications for the users are currently unclear.
  https://stackoverflow.com/a/61544937
  https://stackoverflow.com/a/65543374
• replace the repos history. Pointed out by @cagix below.
• archive this repo and restart with a fresh one. This will cause the loss of GitHub stars as well as issue notes, discussions and the old history only available thru the archived repo

I will investigate the options in the above order.

[cagix]

honestly, i'm not sure if it's worth your time. if the complete history is too large (300 mb currently?) one can always get a shallow clone (or just use the release artefacts). this should be documented, probably.

that said, you could split the theme folder and it's history to into a new repo using git filter-branch. however, it is quite convenient to have one self contained repo instead of things sprawled over several repos imho (also you'd need to keep all repos in sync then).

also, i've seen somewhere a technique where you continue your work in a fresh repo and the old repo being git linked somehow to the new one. but that wouldn't solve the underlying issue - your new repo will grow, especially when you change screenshots and images a lot.

edit: rewriting history will annoy all users who rely on the commit id's.

edit: you could split the theme itself into a separate repo (git filter-branch) and include this new repo in your main repo as git submodule. this way there would be a small theme repo, and a complete main repo containing documentation and other artefacts (and the theme via git submodule), and you wouldn't need to worry about keeping everything in sync. this could work, but again, not sure if it's worth the hassle.

[McShelby]

Living in a 1st world country is a privilege but often we (I often do!) take it for granted. On a slow network this can quickly get annoying.

one can always get a shallow clone (or just use the release artefacts). this should be documented, probably.

I am not sure if this will work if the theme is included as a go submodule.

I have a subtle hope, that the commit id's will be changed in a compatible way by git-filter-repo. Having the old commit id's vanish is not acceptable. I then would probably go the second route.

The modified repo would still come with a very limited exampleSite. This also adds the opportunity to provide a config.toml that is configured to best practices / recommended settings while not changing default values in the theme itself.

Syncing the docs repo with the theme repo should be manageable automatically by repository_dispatch events as lined out in the second stackoverflow link above.

The good(?) thing is: The repo is already big for quite some time. So there is no immediate need to do it right now as long as I keep updates to images resonable.

This task may be something for a cold winter holiday. Let's hope it will get cold this winter ;-)

[cagix]

one can always get a shallow clone (or just use the release artefacts). this should be documented, probably.

I am not sure if this will work if the theme is included as a go submodule.

good point, but i'm failing to imagine a use case which wouldn't allow me to change the method of inclusion...

I have a subtle hope, that the commit id's will be changed in a compatible way by git-filter-repo. Having the old commit id's vanish is not acceptable. I then would probably go the second route.

well, in my experience you get to keep the relevant history (commits, dates, ...), but (nearly?) all commit id's will be rewritten when using filter-branch. i don't think there is a way of splitting a repo and keeping the original commit id's.

The modified repo would still come with a very limited exampleSite. This also adds the opportunity to provide a config.toml that is configured to best practices / recommended settings while not changing default values in the theme itself.

but isn't the example site changing quite often (containing images)? and isn't the example site a kind of living documentation and should be in the "documentation" repo?

wouldn't it be better for users using go submodules to have just the theme as separate small repo?

Syncing the docs repo with the theme repo should be manageable automatically by repository_dispatch events as lined out in the second stackoverflow link above.

hmmm, that's just handling synchronised releases using github workflows?

including the theme repo via git submodule into the main repo would allow users to clone just one repo instead of several repos.

[cagix]

@McShelby they claim they in the readme of the git-filter-repo

[Old commit references] Provide a way for users to use old commit IDs with the new repository (in particular via mapping from old to new hashes with refs/replace/ references).

if this really holds true, you could refactor the non-theme stuff away into a new repo and thus keeping your stars and the mapped commit id's in the smaller theme repo.

[McShelby]

The modified repo would still come with a very limited exampleSite. This also adds the opportunity to provide a config.toml that is configured to best practices / recommended settings while not changing default values in the theme itself.

but isn't the example site changing quite often (containing images)? and isn't the example site a kind of living documentation and should be in the "documentation" repo?

This new exampleSite would change quite infrequently as it would be more in the manner of hugo new site

if this really holds true, you could refactor the non-theme stuff away into a new repo and thus keeping your stars and the mapped commit id's in the smaller theme repo.

That's my straw. Sounds almost to good to be true.

[cagix]

@McShelby you could also consider "git replace" (https://git-scm.com/book/en/v2/Git-Tools-Replace) and retire the old history to a new repo and continue work in this repo with the last (few) commit(s). in the example the last commit in the old repo and the first one in the new repo are linked, so developers can get the complete history if needed. and it comes without rewriting history and it keeps stuff in a single repo :)

[McShelby]

Thanks @cagix. It's always good to have alternatives here.