Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Android Huavei mobile phones and Proot #87

Open
JanuszChmiel opened this issue Dec 21, 2019 · 47 comments
Open

Android Huavei mobile phones and Proot #87

JanuszChmiel opened this issue Dec 21, 2019 · 47 comments

Comments

@JanuszChmiel
Copy link

I have read The following debate related to Userland development.

CypherpunkArmory/UserLAnd#170
So I Am wondering, if non functioning Pokedata of Huavei Android kernels can not be also The cause of The problems with The file system extraction.
Programmers had to recode Proot source to overcome The issue with Pokedata, so Proot work much more faster. But what if this googly thinked work around can cause file system extractions issues on some devices.
There is one fact. Some Android kernels do not support Proot as it should. There are issues with file system case file sensitivity and other issues. The question is, if there is even The chance from The perspective of Proot developers to overcome those issues. Or if those issues are The effect of various ddevelopment strategies while developing The Android kernels for various devices.
What would happen, if would try older release of Proot, which have been originally created by Electron company? Or it was necessary to recode it to be compatible with Termux, so I can not simply recompile it from Termux and give it A try.
I want to find some work around to support Huavei 10 lite mate and android 7.0 kernel. I had A right. The most functioning distribution is Fedora. Its container is causing only a few issues while installing some specific packages such as Perl interpreter.
But Fedora have other important issues related to dbus. Debian is much more universal and Dbus is working even while using Debian chrooted container. No crashes, no issues. I have analysed many many articles on The Internet to find other routines which would work as a Proot, but I had no luck. Proot is very probably very improtant programmers invention and there is no similar technique. Qemu work very slowly if user do not installer rooted Android and if user will not install KQEMU kernel module for Android.
I Am still thinkink and may be, that I will have some success on future.

@michalbednarski
Copy link
Collaborator

POKEDATA issue doesn't appear to be present on your device (according to log you've posted in #84 (comment))

I still think you should be able to use Debian with modifications (as I've previously described in #84 (comment)), however for some packages there might be further configuration changes necessary; please let me know which packages have issues there.

Also, could you run following commands (outside proot) and share trace.txt so I could confirm/reject hypothesis that case folding behavior is triggered by tracing program:

pkg install -y gdb strace
mkdir casetest
cd casetest
strace -o trace.txt touch teststrace TESTSTRACE
gdb --batch -ex r --args touch testgdb TESTGDB &>> trace.txt
touch testdirect TESTDIRECT
proot -v 9 touch testproot TESTPROOT &>> trace.txt
PROOT_NO_SECCOMP=1 proot -v 9 touch testnproot TESTNPROOT &>> trace.txt
ls -l &>> trace.txt

@JanuszChmiel
Copy link
Author

Dear MR Bednarski,
Here is attached log file. I hope, that it will not be problems, that I have created it on download folder to easily move it from mobile device to The computer.
The most annoing fact is, that not only Debian have issues with packages. All Linux distributions. But working with Debian makes The biggest sense for Me. Because task mate-desktop configures Dbus so it can be started by using Marco window manager. Fedora is using much more different philosophy so Dbus configuration from container is rather impossible. The onlyone hack is to use very old DBUS from Fedora before version 27 with 31 and this is not good technique.
I will report Debian problems with packages here since my previous issue is closed.

trace.txt

Thank you for yours patience with Me. My second phone, Samsung Galaxy 5.1 with Android 6.1 have The same issue. So it is not only Huavei kernel problem, it seems to Me, that there are more manufacturers who have made this strange think with their kernels.

@michalbednarski
Copy link
Collaborator

Looks like trace.txt wasn't generated correctly because some newlines were removed while you've copy-pasted commands

Could you try doing that using following snippet which replaces newlines with semicolons and switches to home directory (as on storage system is always case insensitive by design; after running code you can cp trace.txt /storage/emulated/0/Download/trace2.txt in order to upload it)

mkdir ~/casetest ; cd ~/casetest ; strace -o trace.txt touch teststrace TESTSTRACE ; gdb --batch -ex r --args touch testgdb TESTGDB &>> trace.txt ; touch testdirect TESTDIRECT ; proot -v 9 touch testproot TESTPROOT &>> trace.txt ; PROOT_NO_SECCOMP=1 proot -v 9 touch testnproot TESTNPROOT &>> trace.txt ; ls -l &>> trace.txt

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Dec 22, 2019 via email

@JanuszChmiel
Copy link
Author

trace2.txt

Oh I have understood. My previous log have been generated incorrectly because I have used copy and paste commands to copy The command between Github WEB page window and Putty Window. May be, that NVDA screen reader places extra empty lines because it uses virtual area to present information about loaded WEB page. I will use select all and copy for The second time to use copy operation directly from The Browser.

@JanuszChmiel
Copy link
Author

OK, lI will rather return to The Debian, because The template
apt install task-mate-desktop
registers DBUS so marco window manager can correctly initiate ATSPI accessibility bus and Orca screen reader can perfectly work thanks to revolutionar and fully functioning Pulseaudio sound server which run directly from Termux.
So thanks to yours script path, I can correctly install Debian.
For a science validity and to prevent chaos from my side, here is The pathing script, if I have not corrupted some code inside it.

pather.txt

When you confirm that patching script is OK, I will install Debian and I will install mate-desktop
But there is so many many packages, that we have very probably very limited chance to success. If I would know, which steps apt is doing that dbus is working with Marco, I could only install gnome-orca and Marco. But I have tested it, it is not enough. Only task-mate-desktop configures something very strangly but it is working with Orca.
Mraco C source contain The needed code to initiate at-spi accessibility bus.

@michalbednarski
Copy link
Collaborator

Looking at log you've provided it looks like problem has occurred with proot (in both operation modes), but not with gdb nor strace, I've made some additional tests in C and I'd ask for running below command and sharing case-test-results.txt

pkg install -y clang ; wget https://gist.github.com/michalbednarski/438b058f77f43dba78aa8d2ddd08f629/raw/de21dee783274d114fe0ed7b930bfa9219052fb6/proot-case-tests.c ; clang proot-case-tests.c -o ~/proot-case-tests ; ~/proot-case-tests &> case-test-results.txt

Depending on results it might be possible to make proot avoid kernel bug, although no promises yet.

Here testing program source can be viewed as gist.


If you've successfully installed system using above script then it means it was pasted correctly; in form you've attached it to above post it fails because of bash having problem with file having Windows style line endings as well as script has markdown markers. I yet have to prepare Debian configuration, although I'd prefer to fix problem on proot side if it turns out to be possible.

@JanuszChmiel
Copy link
Author

Here is The bug when installing task-mate-desktop
I have used The following command

DEBIAN_FRONTEND=noninteractive apt install task-mate-desktop

Preparing to unpack .../19-python2.7_2.7.16-2+deb10u1_arm64.deb ...
Progress: [ 2%] [#.........................................................] Unpacking python2.7 (2.7.16-2+deb10u1) ...
Selecting previously unselected package libpython2-stdlib:arm64.
Preparing to unpack .../20-libpython2-stdlib_2.7.16-1_arm64.deb ...
Unpacking libpython2-stdlib:arm64 (2.7.16-1) ...
Selecting previously unselected package libpython-stdlib:arm64.
Preparing to unpack .../21-libpython-stdlib_2.7.16-1_arm64.deb ...
Unpacking libpython-stdlib:arm64 (2.7.16-1) ...
Errors were encountered while processing:
/tmp/apt-dpkg-install-tv2usz/06-perl-modules-5.28_5.28.1-6_all.deb
/tmp/apt-dpkg-install-tv2usz/08-libperl5.28_5.28.1-6_arm64.deb

E: Sub-process /usr/bin/dpkg returned an error code (1)
And I can only dream about The successfull installation.

To be 100 % sure, that commands are working, I have used The following commands before typing
./enter_deb
from The bin folder.
mkdir -p deboot_debian/etc/dpkg/dpkg.cfg.d

echo path-exclude /usr/share/man/man7/pam.7.gz > deboot_debian/etc/dpkg/dpkg.cfg.d/01_bad_case

@JanuszChmiel
Copy link
Author

I Am afraid, that my The onlyone hope is to make my own virtual HTTP WEB server, which will host all .deb packages for ARM64 architeczure for The Debian version which is being used by The script.
Rename all package names of .deb files so there will be only little letters, lover case never uppercase.
Modify all .deb files so all file names inside it will only contain lover case letters, no uppercase.
Recode Binary .so files and other so if it refer to uppercase .so file or similar file name so I would had to recode its call to lovercase letters.
But when I analyse it, it would engage one or more year process.

@JanuszChmiel
Copy link
Author

Here is The needed .txt file. What do you think, is there any chance to avoid kernel bug, or unfortunately no?

case-test-results.txt

I Am deeply appraciting you that you have The good will to invest yours time for my issue. Without yours knowledge, I would have The zero chance to success.

Thank you for yours thinkink, debugging and for yours kind access to Me.

@michalbednarski
Copy link
Collaborator

Test cases didn't trigger kernel bug while invoking most important calls used by proot, which suggests that it'd probably be possible to avoid the bug, but still I have no idea how as I still don't know what actually triggers bug. I'll yet have to think how to analyze this further.

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Dec 23, 2019 via email

@michalbednarski
Copy link
Collaborator

I've modified patched installation to apply fixes and install task-mate-desktop. This script should be run from Termux home directory. Note that this script will erase existing installation and install new system with task-mate-desktop package. Patched installation has updated list of path excludes for dpkg and also skips configuration of udisks2 which is separate issue (probably not to be fixed on proot side).
debian-mate-case.txt


To actually fix case sensitivity issue I think that I'd need interactive access to your phone, so in order to grant me remote access run following commands in Termux and post result from last one (which contains address to which I'll have to connect, but I still need to authenticate with my SSH key so it should be okay to post on public issue)

pkg install -y tmate ;
wget https://github.com/michalbednarski.keys ;
tmate -a michalbednarski.keys

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Dec 25, 2019 via email

@michalbednarski
Copy link
Collaborator

As far as i know in case of system hang you could reboot by long pressing (for about 10 seconds) power button, but I think it won't be necessary here.

Anyway, note that after running above commands you'll have to share connection address that is shown after running last command as otherwise I won't be able to connect nor see that you've ran them

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Dec 25, 2019 via email

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Dec 25, 2019 via email

@michalbednarski
Copy link
Collaborator

michalbednarski commented Dec 25, 2019 via email

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Dec 25, 2019 via email

@michalbednarski
Copy link
Collaborator

Thank you, I've just finished analysis and exited tmate, you might still need to release wakelock and/or exit Termux from Termux notification.

Tomorrow I'll start implementing workaround

Reduced code triggering bug and some technical notes are available at https://gist.github.com/michalbednarski/bef82aec3519d9f3225b72b9764a78ed

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Dec 25, 2019 via email

@michalbednarski
Copy link
Collaborator

I think I have a fix, you can install it with following commands (If you still have environment after my yesterday analysis you can skip first two lines)

pkg install -y proot git make clang ;
git clone https://github.com/termux/proot.git ;
cd proot/src ;
git fetch ;
git checkout -f origin/f2fs-case ;
make ;
make install ;

You can go back to normal proot version with apt install --reinstall proot

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Dec 26, 2019 via email

@michalbednarski
Copy link
Collaborator

I meant skipping git clone command as well (but if you keep it, it will just fail and remaining commands will continue using already cloned repo)

Update in apt repo is rebuild after dependency update and has no functional changes.

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Dec 26, 2019 via email

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Dec 26, 2019 via email

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Dec 26, 2019 via email

@michalbednarski
Copy link
Collaborator

Looks like I've got sign with errno wrong in initial bug probe code (which decides if workaround should be activated on device), could you try running installation with PROOT_F2FS_WORKAROUND=1 environment variable set

(Run export PROOT_F2FS_WORKAROUND=1 and then proceed with installation as usual)

If still won't work with that I'd ask for log from following installation commands (these commands erase existing Debian installation)

rm -rf deboot_debian debootstrap debootstrap-1.0.116
PROOT_F2FS_WORKAROUND=1 PROOT_VERBOSE=9 bash -x debian_on_termux.sh |& gzip -9 > log-setup.txt.gz

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Dec 26, 2019 via email

@JanuszChmiel
Copy link
Author

Here is The evidence, that Debian have been really installed successfully.

Debian successfully installed.txt

michalbednarski added a commit to michalbednarski/termux-packages that referenced this issue Dec 28, 2019
* Report /dev/shm as tmpfs (termux/proot#79)
* Avoid accessing nonexistent files on buggy f2fs file system (termux/proot#87)
* Change TCSAFLUSH flag to TCSANOW (termux/proot#88)
ghost pushed a commit to termux/termux-packages that referenced this issue Dec 28, 2019
* Report /dev/shm as tmpfs (termux/proot#79)
* Avoid accessing nonexistent files on buggy f2fs file system (termux/proot#87)
* Change TCSAFLUSH flag to TCSANOW (termux/proot#88)
@JanuszChmiel
Copy link
Author

JanuszChmiel commented Dec 29, 2019 via email

@michalbednarski
Copy link
Collaborator

That sounds like separate issue to be analyzed separately, but if you think it's due to workaround you can try turning workaround off by entering your Debian with:

PROOT_F2FS_WORKAROUND=0 bin/enter_deb

(Using proot with workaround disabled should be okay when you're not installing packages; PROOT_F2FS_WORKAROUND=0 disables workaround, PROOT_F2FS_WORKAROUND=1 enables workaround, anything else or lack of variable causes proot to automatically check if workaround should be activated)

Yesterday update for proot was released to apt repository; it also fixed use-after-free bug that was present in version that you've installed from GitHub for testing so you should update (preferably from apt but you can also re-run same commands to install from source).

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Dec 29, 2019 via email

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Jan 4, 2020 via email

@michalbednarski
Copy link
Collaborator

shmget is syscall used for System V shared memory, which is not supported on Android (nor handled by proot, however there were third-party workarounds based on Termux's libandroid-shmem, but I don't think you'll need them here)

You can pass -noshm switch to x11vnc and it will work without need for shared memory

When using XServer XSDL with external physical keyboard one workaround I've found is to remap menu key (the key between right alt and ctrl, you can use another key if you want) to function as alt key. This way handling of special characters is handled in guest system and not Android.

To remap key:

  • Start XServer XSDL
  • Click CHANGE DEVICE CONFIGURATION button at top of screen
  • Click Remap physical keys
  • Press key you'd want to use as alt on physical keyboard
  • Click Show all keycodes
  • Click RALT

Once you have key remapped you can configure keyboard layout in guest system and then use remapped key as alt when writing diactrics.

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Jan 5, 2020 via email

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Jan 15, 2020 via email

@michalbednarski
Copy link
Collaborator

Wasn't mate-session first command you use to get desktop environment with panel and window manager running? I think that once you're able to get to run dialog you shouldn't need to launch it.

As far as I see in glib source, the Error creating IO channel for /proc/mounts warning means that glib won't be able to receive notification about new filesystem being mounted, which won't happen inside proot anyway so probably that warning can be ignored.

proot error: POKEDATA workaround stub got signal 0 - previously I've seen in proot logs that POKEDATA workaround was not active on your device, have you got this message on different device than the one you've posted logs previously from?

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Jan 16, 2020 via email

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Jan 16, 2020 via email

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Jan 16, 2020 via email

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Jan 21, 2020 via email

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Jan 22, 2020 via email

@michalbednarski
Copy link
Collaborator

In regards to /proc/stat there already exists workaround to copy example contents of /proc/stat (for example from desktop Linux) and bind-mount it in guest (Using proot -b option, with this workaround task managers still won't be able to show CPU usage but usually won't crash).

I've made small script automating that task: patchdeb.sh.txt

(Contents of /proc/stat were borrowed from TermuxArch script, TermuxArch as far as I know was only distribution script to include /proc/stat workaround, unfortunately I haven't made proper documentation for distribution script authors and such hints on proot configuration are scattered in issues here)

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Jan 22, 2020 via email

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Jan 23, 2020 via email

@JanuszChmiel
Copy link
Author

JanuszChmiel commented Jan 23, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants