Using Buildozer to build .apk files

[DragonsBloodQ]

I'm posting here on the recommendation of a Stack Overflow user. The Kivy library for Python has a tool that works along side it known as Buildozer. It's mainly used to take Python files utilizing Kivy and turn them into .apk files for Android phones (it can also compile to other file formats). Buildozer behaves as such that it cannot be effectively run as a root user. This includes being run as non-root sudoer users. All attempts to access the files to the lxss set of folders in AppData have been unsuccessful, despite multiple attempts to copy the source code there.

Since I have to have root permissions to access files on the actual Windows file system, I've reached an impasse. There seems to be no way to profitably run BuildozerAs for the actual problem, the terminal just hangs. Buildozer will go through a few commands and then proceed to do nothing after about 5 seconds.

I don't know if anything can actually be done about the issue, I'm mainly posting for visibility on the off chance that something can be done.

[rodrymbo]

You say you must have root permissions to access files on the Windows file system. It sounds like you may mean that you have to "elevate" to administrator privileges. If you are looking at the lxss folder in your user profile %LOCALAPPDATA% via something like Explorer, your normal user should have permissions there, though bad things happen to Bash if you try to do more than delete files there, which you should do only as a last resort. If you can only access the files on the Windows side if elevated, something is probably messed up with Permissions on the Windows side.

Perhaps your statement makes it only sound like you are confusing using sudo/root in Bash.exe (which lets you install programs and do certain maintenance tasks but only in Bash.exe) versus running a Windows program like Bash.exe "elevated". There are a couple of tasks where you might want to run Bash.exe elevated, such as using the ping command (until that gets fixed). But otherwise it's not usually a good idea and not necessary. Elevated in Windows is not exactly the same as using root/sudo in Bash.exe, and it can sometimes get confusing when I try to understand a problem someone else is describing.

Anyway, if you are accessing files in the hidden lxss folder via Windows tools, you may very well have messed up permissions there, and the primary way to resolve that is to uninstall WSL/Bash.exe, check to make sure the hidden lxss folder is gone, then reinstall Bash.exe and vow never to look in that hidden lxss folder again, via the Windows side of things. Pretend you don't know it is there.

If you need to back up your home directory first before reinstalling Bash.exe, there's a simple script that can do that for you, if I haven't messed it up by posting in that forum.

Once you get WSL set up clean again, let us know at when you find yourself blocked and tempted to run something Elevated (or logged in as Administrator) on the Windows side, or tempted to access files in that hidden lxss folder. There is probably a simple workaround that will keep things accessible. This doesn't mean that Buildozer is working, just that it might work if we get past the need to be Elevated.

Or maybe I'm just confused and missing the point entirely.

[DragonsBloodQ]

@rodrymbo I think you may be slightly confused. I'm running Bash as a user (myself) with Admin permissions, but I am not running it "as administrator", which is what I believe you're referring to. Let me see if I can break this down a little more nicely

Attempt One: Run Buildozer straight up using Bash on Ubuntu on Windows
Result: Buildozer hangs

Buildozer, as noted by the creators of the utility both on their site, on Stack Overflow, and in the program itself, can not be run with root permissions.

Attempt Two: Run Buildozer using created Linux user "AppCreate" which does not have root permissions
Result: Security issue, unable to access target folder

So, naturally, in the interest of being thorough...

Attempt Three: As AppCreate, with directory set to the folder containing my script, 'sudo buildozer android debug'
Result: 'Warning: buildozer is running as root. This is not recommended and may cause issues in the future', [continue anyway] Buildozer hangs

At this point, I did some quick Google Fu and it was suggested that I move the files to be compiled somewhere that the AppCreate user would have access to.

Attempt Four: Moved Python files into %LOCALAPPDATA%\lxss, 'ls' in Bash
Result: No files are listed in response to ls command. Moved files to a few different subfolders, switched the directory, still no dice.

It was at this point that I more or less was out of options with Bash on Ubuntu on Windows and had to seek alternate solutions.

[fpqc]

@DragonsBloodQ You can't copy them into %localappdata%\lxss, which is sort of the "configuration directory" of WSL and contains some hidden database stuff and stores that keep track of the additional attributes on VolFS. You also can't untar them from Windows. If they exist on your windows HDD, use WSL and
cp -R /mnt/c/path/to/source/ ~/my/project/and/build/folder

I understand that this is incovenient, but right now the full Linux filesystem emulation is only implemented in that region. The WSL team is looking into ways of extending functionality to "DriveFS", but they can't do things that would break the system or break the WSL.

What the person probably meant is that you have to untar (from inside bash using tar -d) whatever your source tarball is inside of your user home and build it there. I don't know how buildozer works, but if it uses anything like inodes or whatever, it will fail.

It still may fail anyway, if buildozer makes use of un-implemented syscalls (chroot for example) or certain types of un-implemented IPC (like AF_UNIX sockets in Datagram mode, for example).

[rodrymbo]

Thanks for the followup.

Just to clarify: you have not logged in to Windows as different users, nor have you run anything "As Administrator" or "Elevated" In Windows (though you could have done, since you are a member of the Administrators group). All your access denied errors are inside the Bash.exe environment. Getting clearer.

There is not a lot of difference (semantically) between Elevated and Admin permissions, on the Windows side. On the Bash side, it's probably confusing to talk about Admin permissions; I'd reserve Admin for talking about Windows, and keep superuser or root or sudo for talking about things going on in Bash.exe (but that's just my way of keeping things separated).

If you do something as your "defaultuser" in Bash, and then you do something as root (which is what using sudo means, by default) then make a different "defaultuser" and do something with that, it is quite easy and expected that you would get access denied problems, at least if you are in the VolFS filesystem (anything that does not begin /mnt).

If you are getting access denied in parts of the filesystem that begin /mnt that you would expect to be able to access, that is something to concentrate on (apart from running a build process). (Obviously you'll get "Permission Denied" if you try to access, say, /mnt/c/Users/Administrator/. But if you can create a folder and add files to the folder before you start running programs, the whole WSL environment (no matter which user) is supposed to have access there.)

Then there is that "moved files to %LOCALAPPDATA%\lxss". It is expected, almost guaranteed. that VERY BAD THINGS happen if you do that; so don't do it. Your best bet, since you are seeing problems there and in other areas, is to delete and try moving the files while INSIDE the Bash.exe environment. Pretend you don't know about %LOCALAPPDATA%\lxss on the Windows side! (Files in lxss that are not created from within the Bash.exe environment are BY DESIGN not visible from within.)

In any case, I wouldn't say you are out of options, just that filesystem permissions within the Bash.exe environment are messed up and the %LOCALAPPDATA%\lxss environment is probably also messed up. The sure way to get past that is to uninstall, delete, and reinstall. If you are going to use the appcreate user in Bash.exe, do so from the beginning. Do everything inside the Bash.exe environment, or at least don't do anything in %LOCALAPPDATA%\lxss from the Windows side. Try doing your work in some place in /mnt and see what happens. If that doesn't work, do it someplace like /home/appcreate/build. Don't use sudo unless it is required to install some program. Realize that if something is in a path beginning with /mnt, file settings may be too permissive if copied directly to /home/appcreate/build. (Not guaranteed either way, just something to note.)

If you need programs set up, I'd put the source code in /usr/local/src and the binaries in something like /usr/local/bin (using sudo to copy them or for make install). Those are part of VolFS and permissions get created and maintained according to Linux expectations (or they are supposed to). Sometimes that is all you need in order to use those programs on files in /mnt. But (as @fpqc notes) if what you are working on expects a full Linux environment, then you need to put all the files someplace that does not begin /mnt, e.g. /home/appcreate/build, from the start.

Let us know how it goes.

[rodrymbo]

@fpqc - I suspect you mean tar --extract (-d is for "diff") ... but then I have to look the tar commands up myself in order to get them right. ;)

[fpqc]

@rodrymbo I wouldn't use /usr/local/src/whatever. Just compile everything in the user home ~/make/this/source/directory. To move stuff into /usr/local/src/whatever, you have to use sudo or perform it as root.

[rodrymbo]

Thanks. Sorry, I just realized that. /usr/local/src is a really bad example for not needing root, but good for stowing source code where one can find it. On trusted packages that should just install, I usually do the whole thing as root, and should break that habit. My point was that you'd use sudo to put the binaries where they belong, but if everything works as it is supposed to, you don't need to use sudo to actually run the program against files elsewhere on the system, because permissions and links and all will work.

[fpqc]

@rodrymbo Arch does not allow you to makepkg as root. It is just disabled entirely!

[rodrymbo]

@fpqc - another reason to break the habit. But I still like to have source code in /usr/local/src. Sigh.