Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support partial clones #4203

Closed
paulpach opened this issue Jul 21, 2020 · 6 comments
Closed

Support partial clones #4203

paulpach opened this issue Jul 21, 2020 · 6 comments
Labels
duplicate Similar issue or pull request already exists. enhancement Adding or requesting a new feature. undecided These features might not be implemented. Can be prioritized by sponsorship.

Comments

@paulpach
Copy link

paulpach commented Jul 21, 2020

Is your feature request related to a problem? Please describe.
I am making a game. My repo is 8GB
the translation files are just about 3 MB
Checking out my entire repo will quickly fill up my weblate storage with irrelevant data.

Describe the solution you'd like

I would like weblate to use git partial clones if available when creating a component. See here for a short guide:
https://docs.gitlab.com/ee/topics/git/partial_clone.html

I was able to configure this manually: I cloned my repo with the guide provided above, put it where weblate would clone it and then I configured it as a component in weblate. It works like a charm. The cloned repo is roughly 18MB in size and it only has the translation files.

Describe alternatives you've considered

I have also tried keeping the translation files in a git submodule, but this negatively impacts my workflow.

@nijel
Copy link
Member

nijel commented Jul 21, 2020

Weblate already uses shallow clones to reduce amount of the data. I don't think supporting sparse clones out of the box is doable in near future, here are issues I can see with it right now:

  • This all is described as experimental on both Gitaly and Git.
  • Filtering the files would break things like choosing files on repository clone or component discovery. This essentially means it would have to be an opt-in feature which would block other Weblate features.
  • Additional logic would be needed to handle situations when file added by Weblate is not part of the checkout (several addons can generate files or new language can be added).

Overall, it seems that it would require a lot of code (potentially fragile) for a feature that most people would not use.

@nijel nijel added enhancement Adding or requesting a new feature. undecided These features might not be implemented. Can be prioritized by sponsorship. labels Jul 21, 2020
@github-actions
Copy link

This issue has been put aside. Currently, it is unclear whether it will ever be implemented as it seems to cover too narrow of a use case or doesn't seem to fit into Weblate. Please try to clarify the use case or consider proposing something more generic to make it useful to more users.

@paulpach
Copy link
Author

paulpach commented Jul 21, 2020

To add a bit more details:

My folder structure looks like this:

Assets/
    ...       <-  lots of files and folders under Assets, such as images, models, scripts, etc.  Roughly 8GB 
    Resources/
          locales/         <- all files relevant to weblate are in this folder
               UI.pot
               es/
                    UI.po
               en/
                    UI.po
               ru/
                    UI.po

my .filterspec contains this:

# clone everything inside this folder
Assets/Resources/locales/

Adding, removing files, adding languages, discovering new components, all works fine as long as it happens within the locales folder.

Filtering the files would break things like choosing files on repository clone or component discovery

^ I don't get this point, I don't filter files inside my locales folder, it is the same as cloning the entire repo.

But I understand the position, not everyone would need it, and it is experimental.
At least I was able to do it manually, so I suppose my problem is solved.

@nijel
Copy link
Member

nijel commented Jul 21, 2020

Adding, removing files, adding languages, discovering new components, all works fine as long as it happens within the locales folder.

The only reason is that it works with your filter and repository structure. Doing this automatically by Weblate and make it work in all cases is not that easy.

@tipa
Copy link
Contributor

tipa commented Oct 23, 2023

I also very much support this feature request. I am developing multiple apps that each are available for four platforms. In my repository I have the apps code (small in size), app assets like app icons (medium in size) and then additional resources for marketing like images and videos (very large in size). I am using Weblate in a shared hosting environment which has rather strict limitations when it comes to available storage.
My resources for a specific component (translations for a specific platform) are located in one specific subfolder that doesn't have any subfolders with large assets itself - I think this is a pretty common scenario.
Right now when I want to add four components (for the four platforms) for my project/app then the same git repository with all the large Asset files is downloaded four entire times. And I have multiple multi-platform apps... It's a dealbreaker for the Git integration, at least for me.
I understand the complexities when adding such a feature but it would be a very appreciated addition.

@nijel nijel added the duplicate Similar issue or pull request already exists. label Oct 24, 2023
@nijel
Copy link
Member

nijel commented Oct 24, 2023

Duplicate of #1161

@nijel nijel marked this as a duplicate of #1161 Oct 24, 2023
@nijel nijel closed this as completed Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate Similar issue or pull request already exists. enhancement Adding or requesting a new feature. undecided These features might not be implemented. Can be prioritized by sponsorship.
Projects
None yet
Development

No branches or pull requests

4 participants
@nijel @paulpach @tipa and others