Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No longer recognizing untracked files #620

Open
simplyandyg opened this issue Apr 7, 2022 · 50 comments
Open

No longer recognizing untracked files #620

simplyandyg opened this issue Apr 7, 2022 · 50 comments
Assignees

Comments

@simplyandyg
Copy link

Describe the bug
MGit is no longer recognizing untracked files on Android. I know that there is a filesystem issue due to new permissions in Android 11+, but isn't there a "see all files" permission? I know that there are a few apps that have it, such as Solid Explorer.

I have even moved my repo and files to the Documents folder as I saw recommended in some of the Google Play Store review replies. That did not solve the issue either. I have multiple new files that I created with a different app, and I am not seeing them show up in MGit as untracked files like I would expect.

To Reproduce
Steps to reproduce the behavior:

  1. Create a file with an app such as Obsidian or Markor in a file within the repo
  2. Open MGit
  3. Check for changes - no changes found

Expected behavior
After creating new files they should be seen as untracked files in MGit

Screenshots
If applicable, add screenshots to help explain your problem.

Smartphone (please complete the following information):

  • Device: Google Pixel 6 Pro
  • OS: Android 12
  • MGit App Version 1.6.2
@aaronromeo
Copy link

I have been trying to work through this as well. I am able to track modifications on existing files, or adding files through MGit and then modifying them. However any new file is being ignored.

@simplyandyg
Copy link
Author

simplyandyg commented Apr 7, 2022

Yes, same. If create the file through MGit, it works.

And for some reason, I'm now noticing that if there was a file named "ABC" already in the directory, even if I rename the file to something else like "XYZ", MGit won't let me create a new "ABC" file. No error, just fails silently. Unlike When I try to create a file that I know is there, I do get the error that a file with that name already exists.

I suppose it could be a weird interaction with Obsidian, but I don't think that should cause any issues. It should just create the file

@tonydiep
Copy link

tonydiep commented Apr 8, 2022

Also experienced this on April 7, 2022
Android 11
Oxygen OS 11.0.4.BE83BA

@klutometis
Copy link

Seeing the same thing; creating a new file for a pre-existing tracked file doesn't seem to pick up external changes, either.

@dleeftink
Copy link

Also experiencing this on Android 11 with One UI 3.0.

@maks
Copy link
Owner

maks commented Apr 19, 2022

Thank you everyone for reporting this.
Unfortunately there is nothing I can do at the moment as I have applied for the "All File Permission" on Google Play for MGit and it was not granted. I have appealed that decision and am awaiting a result for that at the moment.

@simplyandyg
Copy link
Author

simplyandyg commented Apr 19, 2022

@maks I understand. Thank you for the update though. It's really going to be unfortunate if they don't grant you the permission, as your app is the best git tool I've found for Android.

@cognifloyd
Copy link

I'm happy to download an APK and install it outside of Google play. I'm already doing that with termux and a few other apps. Are you open to posting the built APK on GitHub somewhere? (Pretty please!)

@tonydiep
Copy link

I'm happy to download an APK and / or email someone at Google if it will help persuade the Play Store to grant the permission.

@simplyandyg
Copy link
Author

I was also wondering about an APK. I would do it that way as well. Google not giving you this permission has really messed with my workflow

@maks
Copy link
Owner

maks commented Apr 23, 2022

Thank you everyone for your support.
I currently am waiting to hear back from Google Play in regards to the appeal. I'm hesitant to distribute APK's myself but I will look at that option if it ends up not being possible to continue in Google Play.

@sandorex
Copy link

sandorex commented Apr 23, 2022

I'm hesitant to distribute APK's myself but I will look at that option if it ends up not being possible to continue in Google Play.

fdroid is an option, i wouldn't mind downloading directly from github either though

Edit: I didn't know the app was already on fdroid

@maks maks mentioned this issue May 1, 2022
@jfmcbrayer
Copy link

MGit is already on F-Droid, but even installed from there, it doesn't have the 'all files' permission it needs. I assume it needs to be added to the manifest, and then I guess a Play Store edition created without it.

@simplyandyg
Copy link
Author

simplyandyg commented May 9, 2022

I ended up setting up Termux and following an online guide to get command line git setup. It's annoying because Termux has its own permissions issues. But as long as you are okay with setting up your local repo in one of the places Termux does have access to then it's fine. It recognizes new files. You just have to be okay with doing things on the command line on a phone, which is its own special level of masochistic sometimes.

@jfmcbrayer
Copy link

Would it be possible to use the Storage Access Framework for setting the base directory for your repositories? That's supposed to give you access to all of the files and directories under that one...

https://developer.android.com/training/data-storage/shared/documents-files#grant-access-directory

@kb-1000
Copy link
Contributor

kb-1000 commented May 10, 2022

No, because JGit can't deal with that.
Unless there's something to be able to access it using java.io.File.

@maks
Copy link
Owner

maks commented May 10, 2022

@jfmcbrayer Unfortunately not. I've looked into using the SAF in the past but there are couple of showstoppers:

  1. JGit would need to be rewritten to use SAF APIs
  2. The SAF is far too low performance for the needs of Git, eg. being able to get high speed access to large numbers of files.

@jfmcbrayer
Copy link

Thanks, that's terribly unfortunate.

@daimoniuma
Copy link

daimoniuma commented May 19, 2022

You have access to Download (not tested, info from system log, peoples write it isn't usable) and Documents (tested) directories. Just place Github folder in Documents directory.

@simplyandyg
Copy link
Author

@daimoniuma that's interesting. I'll try that out

@yuttai
Copy link

yuttai commented Jul 16, 2022

Although I haven't added untracked files that often so I haven't tried, is it a possible work around to use the "New File" command in MGit to create a blank file and then replace the content? Even this works, it is awkward to write non-text content... Is it possible for MGit to add a function like "Add File", and call an Android file manager, like com.google.android.documentsui , to list untracked file and add the select file to git?

@prahal
Copy link

prahal commented Sep 9, 2022

@maks Maybe in your appeal you should point out to google that even though MGit is not aimed at modifying files from other application folders, other applications are to modify files in MGit folder. Thus MGit requires to be able to access files "from" other applications inside it own folder. (Ie also you could point them out that the editor is not part of MGit thus MGit requires to the ability to access files created by the editor application which is ... another application).
Other than them not understanding this point, I am out of guess why they would have refused MGit access to files created by other applications).

Also, there is the "access to all file types", you could point to them that in a git repository there are files that are not media files (especially with regards to #640 which I had me rename my ssh private key with a JPG extension to be able to import it in Android up to 11).

I am all for an FDroid repo workaround. But it is insane of Google to forbid VCS because if they do not give permission to MGit any VCS ought not to get any better luck.

@simplyandyg
Copy link
Author

simplyandyg commented Sep 9, 2022 via email

@prahal
Copy link

prahal commented Sep 9, 2022

I also pointed out the issue that MGit Google Play store version only allows "access to media files". There is the issue of not being able to access files that are not media files out of those MGit creates (the latter I suppose because I guess MGit is still allowed to create pdf, zip, etc., not only files with an extension of mp3, jpg, etc). But currently, you cannot import an ssh key file from MGit without renaming it with a JPG extension (then removing it via MGit UI after the import to a mgit subfolder succeed.
It is not an issue for me to have to create the MGit repo location under Documents. But being able to import only multimedia files is more of a problem. (I do not even think txt files are allowed to be imported).

@shaigluskin
Copy link

Actually, after updating to 1.6.3 (F-Droid), the permissions manager lets me set "Allow management of all files" ("media only" is the default), and repositories in Documents work correctly.

This solution worked for me!

For folks not so familiar with Android, the "permissions manager" is in the Android system settings and not in the app settings. Also, at least on my OnePlus running Android 12, when you go "permissions manager" you are NOT provided with a list of apps to choose from, rather you are presented with a list of permissions categories. At that point choose "Storage" and then from there it sorts by level of permission, starting with apps that have full permissions. Scroll down to the apps that have limited permission, find MGit, click on it and then select "All Files."

Now MGit will recognize new files on its own.

@maks
Copy link
Owner

maks commented Jan 1, 2023

I've made one last attempt at trying to make the app compliant with Google Play policies in regards to use of the "All Files access" permission and have resubmitted for review to Google Play.
At this point (as of #666) as far as I can see, MGit's use of the permission matches what VLC does and they have been approved on Google Play.
If after this latest round MGit is still rejected by Google Play staff I will sadly have to remove MGit from Google Play and only have it available via F-Droid and possibly other third party apps stores.

@maks
Copy link
Owner

maks commented Jan 3, 2023

Sadly Google Play has once again rejected the latest version so that's the end of the line for MGit on Google Play.

I will shortly publish a new version on Google Play that will remove support for Android 11 onwards and display a message to that effect, referring users to instead install the app from F-Droid.

To help users who would need to upgrade vs uninstalling and re-installing from F-Droid (because F-Droid signs APK's with their own certificate and not mine) I will also begin publishing APK's from CI soon, though I would like to do that only once I have a update check mechanism in place (#669)

@yuttai
Copy link

yuttai commented Jan 6, 2023

Besides of F-Droid and APK`s, is it possible to also make MGit available on Samsung's Galaxy Store? I control my phone by Google Family Link, which does not allow install apps by APK, but it allows me to install apps via preinstalled Galaxy Store...

@maks
Copy link
Owner

maks commented Jan 18, 2023

@yuttai I'll try to look into this as I'm not sure what the requirements are to be able to publish on the Galaxy Store.

@husnoo
Copy link

husnoo commented Jul 7, 2023

Mgit 1.7.0 from F-droid, on Android 13.0.0/Lineage 20 - "Clone failed, Operation not permitted" in both /storage/emulated/0/[Download|Documents].

@hgabreu
Copy link

hgabreu commented Jul 9, 2023

It is working fine here. Same version 1.7.0 from F-droid on Android 13. Just cloned a new repo on /storage/.../Documents to test a new clone, worked fine. Are you sure it's not you ssh key or some other issue?

@husnoo
Copy link

husnoo commented Jul 9, 2023 via email

@Alekssavin
Copy link

sorry my bad, emacs made a scratch file that I accidentally committed and for some reason isn't valid on Android. check out worked fine after I removed the file.

On Sun, Jul 9, 2023, 16:00 Henrique Abreu @.> wrote: It is working fine here. Same version 1.7.0 from F-droid on Android 13. Just cloned a new repo on /storage/.../Documents to test a new clone, worked fine. Are you sure it's not you ssh key or some other issue? — Reply to this email directly, view it on GitHub <#620 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWJWSWQWCUW6FNVPS3NMS3XPLBRRANCNFSM5SXVDHJA . You are receiving this because you commented.Message ID: @.>

Hi. It seems like I have the same issue. I wanted to clone my big repo to my phone, and got the same error message. However, I was able to successfully clone other repos, both public and my private.

How does one detect the file which makes Mgit to reject the whole downloaded repo? @maks

@Alekssavin
Copy link

sorry my bad, emacs made a scratch file that I accidentally committed and for some reason isn't valid on Android. check out worked fine after I removed the file.

On Sun, Jul 9, 2023, 16:00 Henrique Abreu @.> wrote: It is working fine here. Same version 1.7.0 from F-droid on Android 13. Just cloned a new repo on /storage/.../Documents to test a new clone, worked fine. Are you sure it's not you ssh key or some other issue? — Reply to this email directly, view it on GitHub <#620 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWJWSWQWCUW6FNVPS3NMS3XPLBRRANCNFSM5SXVDHJA . You are receiving this because you commented.Message ID: _@**.**_>

Hi. It seems like I have the same issue. I wanted to clone my big repo to my phone, and got the same error message. However, I was able to successfully clone other repos, both public and my private.

How does one detect the file which makes Mgit to reject the whole downloaded repo? @maks

It took me some time to figure it out.

Since the private repo I had the issues with is a collection of my notes, their names tend to contain a lot of special characters (in my case " and ?), which are invalid to use in filenames in the latest version of Android. I've just switched to a new phone, so I didn't know that.

The list of the invalid characters one can see here.

@prahal
Copy link

prahal commented Oct 1, 2023

sorry my bad, emacs made a scratch file that I accidentally committed and for some reason isn't valid on Android. check out worked fine after I removed the file.

On Sun, Jul 9, 2023, 16:00 Henrique Abreu @.> wrote: It is working fine here. Same version 1.7.0 from F-droid on Android 13. Just cloned a new repo on /storage/.../Documents to test a new clone, worked fine. Are you sure it's not you ssh key or some other issue? — Reply to this email directly, view it on GitHub <#620 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWJWSWQWCUW6FNVPS3NMS3XPLBRRANCNFSM5SXVDHJA . You are receiving this because you commented.Message ID: _@**.**_>

Hi. It seems like I have the same issue. I wanted to clone my big repo to my phone, and got the same error message. However, I was able to successfully clone other repos, both public and my private.
How does one detect the file which makes Mgit to reject the whole downloaded repo? @maks

It took me some time to figure it out.

Since the private repo I had the issues with is a collection of my notes, their names tend to contain a lot of special characters (in my case " and ?), which are invalid to use in filenames in the latest version of Android. I've just switched to a new phone, so I didn't know that.

The list of the invalid characters one can see here.

As the function name shows private static boolean isValidFatFilenameChar(char c) { these are invalid names to write to a FAT partition. These filesystems are used on Android on extension cards (SD).
If you write to the internal emmc these characters are OK.

That is why these are not MGit or even Android per see forbidden file characters. These characters are forbidden on Microsoft filesystems which are the only ones we can use on external media on Android (so that Windows users can read these external media).

@CodeMazeSolver
Copy link

Are there any updates or a work around for the situation regarding untracked files?

@prahal
Copy link

prahal commented Jun 10, 2024

@CodeMazeSolver as told in the preceding comment there is no MGit untracked file bug.
Only if you install MGit from Google Store it will tag MGit as not having the rights to check for files other than media files. Access to other files requires Google to accept to grant this right and they refused (it looks insane to me as they granted this right to the application where it was not critical and here it is critical).
The only way out is to install from FDroid or any other store where MGit has been uploaded.
This is google decision to not allow MGit to access files on the storage. You should bug them if you want them to change that.
Ie the permission is not shipped in the application code. This is tagged on the application when uploaded to the Google Store. Thus why the same code uploaded to other stores works.

But this was explained in this thread already. Only a few last comments talk about an unrelated issue that was due to Android using Microsoft FAT format for external storage (SD cards) to let users read the cards under Windows and ... well FAT not supporting quite a lot of characters. Thus MGit is unable to check out the git repository on FAT FS when such a character is in a filename in the git files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests