Package request: git-annex #420

Open
matthiasbeyer opened this Issue Aug 28, 2016 · 39 comments

Comments

Projects
None yet
@matthiasbeyer

Would be awesome to have git-annex in termux, so I can sync music and movies from my other devices with it.

@joeyh

This comment has been minimized.

Show comment
Hide comment
@joeyh

joeyh Jan 8, 2017

I am the developer of git-annex, and I would also like to see it in termux. Am willing to do work to make it happen.

I do have git-annex building for android. As a haskell program, it needs https://github.com/neurocyte/ghc-android to build.

joeyh commented Jan 8, 2017

I am the developer of git-annex, and I would also like to see it in termux. Am willing to do work to make it happen.

I do have git-annex building for android. As a haskell program, it needs https://github.com/neurocyte/ghc-android to build.

@joeyh

This comment has been minimized.

Show comment
Hide comment
@joeyh

joeyh Jan 8, 2017

@fornwall I see you have a disabled attempt to port ghc at https://github.com/termux/termux-packages/tree/master/disabled-packages/ghc -- I think ghc-android should contain all the patches you need, and I am somewhat familiar with it, if I can help.

That is, ghc-android gets you a ghc that can cross-compile for android. It doesn't get a ghc that runs natively on android, and I think that would be a lot of additional work if it's your goal. That's not needed for porting git-annex though.

joeyh commented Jan 8, 2017

@fornwall I see you have a disabled attempt to port ghc at https://github.com/termux/termux-packages/tree/master/disabled-packages/ghc -- I think ghc-android should contain all the patches you need, and I am somewhat familiar with it, if I can help.

That is, ghc-android gets you a ghc that can cross-compile for android. It doesn't get a ghc that runs natively on android, and I think that would be a lot of additional work if it's your goal. That's not needed for porting git-annex though.

@pjones

This comment has been minimized.

Show comment
Hide comment
@pjones

pjones Jul 6, 2017

I'd love to see this happen! If I can help please let me know.

pjones commented Jul 6, 2017

I'd love to see this happen! If I can help please let me know.

@jamesd

This comment has been minimized.

Show comment
Hide comment
@jamesd

jamesd Feb 9, 2018

So the author of git-annex offers to help make a termux package happen over a year ago and still nothing?

jamesd commented Feb 9, 2018

So the author of git-annex offers to help make a termux package happen over a year ago and still nothing?

@joeyh

This comment has been minimized.

Show comment
Hide comment
@joeyh

joeyh Feb 9, 2018

Well, we're all busy and this is probably somewhat low-priority for all of us and involves a complicated compiler port or cross build.

There has, however, been significant work on ghc for android in the past year, by zw3rk.

All cross-compiling so far, running ghc on android is not on the map that I know of.

joeyh commented Feb 9, 2018

Well, we're all busy and this is probably somewhat low-priority for all of us and involves a complicated compiler port or cross build.

There has, however, been significant work on ghc for android in the past year, by zw3rk.

All cross-compiling so far, running ghc on android is not on the map that I know of.

@jamesd

This comment has been minimized.

Show comment
Hide comment
@jamesd

jamesd Feb 9, 2018

@joeyh, I'm glad you're still involved. I was worried.

I don't follow Haskell development so I did not know the cross compiler is still in development. With the latest work by zw3rk hopefully we are not too far off from a git-annex package for termux.

jamesd commented Feb 9, 2018

@joeyh, I'm glad you're still involved. I was worried.

I don't follow Haskell development so I did not know the cross compiler is still in development. With the latest work by zw3rk hopefully we are not too far off from a git-annex package for termux.

@matthiasbeyer

This comment has been minimized.

Show comment
Hide comment
@matthiasbeyer

matthiasbeyer Feb 9, 2018

I'm really looking forward to git-annex on termux for my mobile device, though I have to note that git-annex might be really slow on the devices.

For example, I had the git-annex android app installed some time ago (I own a Oneplus One, which works perfectly well for todays apps still). Cloning my music collection (4347 directories, 52794 files) onto the phone and git annex syncing (not putting the data on my phone, simply syncing the repository) took hours! In the end I aborted it because it took too long.

Still, for smaller repositories/collections, it might be worth it!

matthiasbeyer commented Feb 9, 2018

I'm really looking forward to git-annex on termux for my mobile device, though I have to note that git-annex might be really slow on the devices.

For example, I had the git-annex android app installed some time ago (I own a Oneplus One, which works perfectly well for todays apps still). Cloning my music collection (4347 directories, 52794 files) onto the phone and git annex syncing (not putting the data on my phone, simply syncing the repository) took hours! In the end I aborted it because it took too long.

Still, for smaller repositories/collections, it might be worth it!

@Teyras

This comment has been minimized.

Show comment
Hide comment
@Teyras

Teyras Mar 25, 2018

Do I understand correctly that adding git-annex to Termux would allow starting git-annex when the device boots (using Termux::Boot) without having to open a terminal every time (which is the case with the current git-annex Android app)?

Teyras commented Mar 25, 2018

Do I understand correctly that adding git-annex to Termux would allow starting git-annex when the device boots (using Termux::Boot) without having to open a terminal every time (which is the case with the current git-annex Android app)?

@its-pointless

This comment has been minimized.

Show comment
Hide comment
@its-pointless

its-pointless Mar 25, 2018

Contributor

Yes

Contributor

its-pointless commented Mar 25, 2018

Yes

@joeyh

This comment has been minimized.

Show comment
Hide comment
@joeyh

joeyh Mar 25, 2018

joeyh commented Mar 25, 2018

@jan

This comment has been minimized.

Show comment
Hide comment
@jan

jan Mar 26, 2018

@jan summons me, @joeyh.

jan commented Mar 26, 2018

@jan summons me, @joeyh.

@joeyh

This comment has been minimized.

Show comment
Hide comment
@joeyh

joeyh Apr 24, 2018

Turns out that the arm tarball from http://git-annex.branchable.com/install/Linux_standalone/ just works in termux. Just untar inside your termux user's home directory, and cd git-annex.linux && ./runshell will add a working git-annex to your path.

joeyh commented Apr 24, 2018

Turns out that the arm tarball from http://git-annex.branchable.com/install/Linux_standalone/ just works in termux. Just untar inside your termux user's home directory, and cd git-annex.linux && ./runshell will add a working git-annex to your path.

@sorsasampo

This comment has been minimized.

Show comment
Hide comment
@sorsasampo

sorsasampo Apr 25, 2018

@joeyh

~ $ git-annex.linux/runshell
ERROR: ld.so: object '/data/data/com.termux/files/usr/lib/libtermux-exec.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.
Bad system call

@joeyh

~ $ git-annex.linux/runshell
ERROR: ld.so: object '/data/data/com.termux/files/usr/lib/libtermux-exec.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.
Bad system call
@tomty89

This comment has been minimized.

Show comment
Hide comment
@tomty89

tomty89 Apr 25, 2018

Contributor

unset LD_PRELOAD

Contributor

tomty89 commented Apr 25, 2018

unset LD_PRELOAD

@sorsasampo

This comment has been minimized.

Show comment
Hide comment
@sorsasampo

sorsasampo Apr 25, 2018

So the git-annex tarball is not compatible with termux-exec?

So the git-annex tarball is not compatible with termux-exec?

@joeyh

This comment has been minimized.

Show comment
Hide comment
@joeyh

joeyh Apr 25, 2018

joeyh commented Apr 25, 2018

@sorsasampo

This comment has been minimized.

Show comment
Hide comment
@sorsasampo

sorsasampo Apr 26, 2018

@joeyh, still having some issue:

~ $ unset LD_PRELOAD
~ $ git-annex.linux/runshell
\w $ git-annex
sh: 1: git-annex: not found
\w $ git annex
fatal: 'annex' appears to be a git command, but we were not
able to execute it. Maybe you need to 'pkg install perl'?
\w $ sh -x $(which git-annex)
+ GIT_ANNEX_PROGRAMPATH=/data/data/com.termux/files/home/git-annex.linux/bin/git-annex
+ export GIT_ANNEX_PROGRAMPATH
+ exec /data/data/com.termux/files/home/git-annex.linux/exe/git-annex --library-path /data/data/com.termux/files/home/git-annex.linux//usr/lib/arm-linux-gnueabi/gconv:/data/data/com.termux/files/home/git-annex.linux//usr/lib/arm-linux-gnueabi/audit:/data/data/com.termux/files/home/git-annex.linux//etc/ld.so.conf.d:/data/data/com.termux/files/home/git-annex.linux//lib:/data/data/com.termux/files/home/git-annex.linux//lib/arm-linux-gnueabi:/data/data/com.termux/files/home/git-annex.linux//usr/lib/arm-linux-gnueabi: /data/data/com.termux/files/home/git-annex.linux/shimmed/git-annex/git-annex
Bad system call

(and yes, perl is installed)

@joeyh, still having some issue:

~ $ unset LD_PRELOAD
~ $ git-annex.linux/runshell
\w $ git-annex
sh: 1: git-annex: not found
\w $ git annex
fatal: 'annex' appears to be a git command, but we were not
able to execute it. Maybe you need to 'pkg install perl'?
\w $ sh -x $(which git-annex)
+ GIT_ANNEX_PROGRAMPATH=/data/data/com.termux/files/home/git-annex.linux/bin/git-annex
+ export GIT_ANNEX_PROGRAMPATH
+ exec /data/data/com.termux/files/home/git-annex.linux/exe/git-annex --library-path /data/data/com.termux/files/home/git-annex.linux//usr/lib/arm-linux-gnueabi/gconv:/data/data/com.termux/files/home/git-annex.linux//usr/lib/arm-linux-gnueabi/audit:/data/data/com.termux/files/home/git-annex.linux//etc/ld.so.conf.d:/data/data/com.termux/files/home/git-annex.linux//lib:/data/data/com.termux/files/home/git-annex.linux//lib/arm-linux-gnueabi:/data/data/com.termux/files/home/git-annex.linux//usr/lib/arm-linux-gnueabi: /data/data/com.termux/files/home/git-annex.linux/shimmed/git-annex/git-annex
Bad system call

(and yes, perl is installed)

@Grimler91

This comment has been minimized.

Show comment
Hide comment
@Grimler91

Grimler91 Apr 26, 2018

Collaborator

@sorsasampo what device do you have?
What's the output of termux-info?

Collaborator

Grimler91 commented Apr 26, 2018

@sorsasampo what device do you have?
What's the output of termux-info?

@sorsasampo

This comment has been minimized.

Show comment
Hide comment
@sorsasampo

sorsasampo Apr 26, 2018

~ $ termux-info
Updatable packages:
All packages up to date
System information:
Linux localhost 4.4.13-13201289-QB17339056 #1 SMP PREEMPT Thu Mar 15 18:46:17 KST 2018 aarch64 Android
Termux-packages arch:
aarch64
Android version:
8.0.0
Device manufacturer:
samsung
Device model:
SM-G950F
~ $ termux-info
Updatable packages:
All packages up to date
System information:
Linux localhost 4.4.13-13201289-QB17339056 #1 SMP PREEMPT Thu Mar 15 18:46:17 KST 2018 aarch64 Android
Termux-packages arch:
aarch64
Android version:
8.0.0
Device manufacturer:
samsung
Device model:
SM-G950F
@Grimler91

This comment has been minimized.

Show comment
Hide comment
@Grimler91

Grimler91 Apr 26, 2018

Collaborator

Seems like you are trying to run a arm binary on a aarch64 system. Try a aarch64/armv8l binary instead if that is available.

Collaborator

Grimler91 commented Apr 26, 2018

Seems like you are trying to run a arm binary on a aarch64 system. Try a aarch64/armv8l binary instead if that is available.

@xeffyr

This comment has been minimized.

Show comment
Hide comment
@xeffyr

xeffyr Apr 26, 2018

Contributor

Seems like you are trying to run a arm binary on a aarch64 system.

@Grimler91 It is possible to run arm binary on aarch64 system.

Bad system call

@sorsasampo Your binary is blocked by seccomp (feature introduced in Android 8). Your binary is compiled against glibc which don't know about blacklisted system calls on Android. Tool strace can help to find out which call actually causes this problem.

I have downloaded the arm tarball of git-annex and executed your commands.
Output (only part of text as it was too long):

[termux]:~/test:$ git-annex.linux/runshell
ERROR: ld.so: object '/data/data/com.termux/files/usr/lib/libtermux-exec.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.
$ unset LD_PRELOAD
$ ls
git-annex.linux
$ git annex
fatal: 'annex' appears to be a git command, but we were not
able to execute it. Maybe you need to 'apt install perl'?
$ sh -x $(which git-annex)
+ GIT_ANNEX_PROGRAMPATH=/data/data/com.termux/files/home/test/git-annex.linux/bin/git-annex
+ export GIT_ANNEX_PROGRAMPATH
+ exec /data/data/com.termux/files/home/test/git-annex.linux/exe/git-annex --library-path /data/data/com.termux/files/home/test/git-annex.linux//usr/lib/arm-linux-gnueabi/gconv:/data/data/com.termux/files/home/test/git-annex.linux//usr/lib/arm-linux-gnueabi/audit:/data/data/com.termux/files/home/test/git-annex.linux//etc/ld.so.conf.d:/data/data/com.termux/files/home/test/git-annex.linux//lib:/data/data/com.termux/files/home/test/git-annex.linux//lib/arm-linux-gnueabi:/data/data/com.termux/files/home/test/git-annex.linux//usr/lib/arm-linux-gnueabi: /data/data/com.termux/files/home/test/git-annex.linux/shimmed/git-annex/git-annex
Missing: COMMAND

Usage: git-annex COMMAND
  git-annex - manage files with git, without checking their contents in
  
  Commonly used commands:
  
  add             PATH ...         add files to annex
  addurl          URL ...          add urls to annex
  assistant                        automatically sync changes
  copy            PATH ...         copy content of files to/from another repository
  drop            PATH ...         remove content of files from repository
  edit            PATH ...         same as unlock
  export          TREEISH          export content to a remote
  get             PATH ...         make content of annexed files available
  help            COMMAND          display help
  import          PATH ...         move and add files from outside git working copy
  importfeed      URL ...          import files from podcast feeds

I don't have bad system call as I'm on Android 7.0 aarch64.

Contributor

xeffyr commented Apr 26, 2018

Seems like you are trying to run a arm binary on a aarch64 system.

@Grimler91 It is possible to run arm binary on aarch64 system.

Bad system call

@sorsasampo Your binary is blocked by seccomp (feature introduced in Android 8). Your binary is compiled against glibc which don't know about blacklisted system calls on Android. Tool strace can help to find out which call actually causes this problem.

I have downloaded the arm tarball of git-annex and executed your commands.
Output (only part of text as it was too long):

[termux]:~/test:$ git-annex.linux/runshell
ERROR: ld.so: object '/data/data/com.termux/files/usr/lib/libtermux-exec.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.
$ unset LD_PRELOAD
$ ls
git-annex.linux
$ git annex
fatal: 'annex' appears to be a git command, but we were not
able to execute it. Maybe you need to 'apt install perl'?
$ sh -x $(which git-annex)
+ GIT_ANNEX_PROGRAMPATH=/data/data/com.termux/files/home/test/git-annex.linux/bin/git-annex
+ export GIT_ANNEX_PROGRAMPATH
+ exec /data/data/com.termux/files/home/test/git-annex.linux/exe/git-annex --library-path /data/data/com.termux/files/home/test/git-annex.linux//usr/lib/arm-linux-gnueabi/gconv:/data/data/com.termux/files/home/test/git-annex.linux//usr/lib/arm-linux-gnueabi/audit:/data/data/com.termux/files/home/test/git-annex.linux//etc/ld.so.conf.d:/data/data/com.termux/files/home/test/git-annex.linux//lib:/data/data/com.termux/files/home/test/git-annex.linux//lib/arm-linux-gnueabi:/data/data/com.termux/files/home/test/git-annex.linux//usr/lib/arm-linux-gnueabi: /data/data/com.termux/files/home/test/git-annex.linux/shimmed/git-annex/git-annex
Missing: COMMAND

Usage: git-annex COMMAND
  git-annex - manage files with git, without checking their contents in
  
  Commonly used commands:
  
  add             PATH ...         add files to annex
  addurl          URL ...          add urls to annex
  assistant                        automatically sync changes
  copy            PATH ...         copy content of files to/from another repository
  drop            PATH ...         remove content of files from repository
  edit            PATH ...         same as unlock
  export          TREEISH          export content to a remote
  get             PATH ...         make content of annexed files available
  help            COMMAND          display help
  import          PATH ...         move and add files from outside git working copy
  importfeed      URL ...          import files from podcast feeds

I don't have bad system call as I'm on Android 7.0 aarch64.

@Grimler91

This comment has been minimized.

Show comment
Hide comment
@Grimler91

Grimler91 Apr 26, 2018

Collaborator

@xeffyr I stand corrected. Is this true on all aarch64 devices?
It obviously won't work for dynamically linked binaries but I was also under the impression that the kernel must have been compiled with certain options, see this so question.

Maybe all android devices are though.

Collaborator

Grimler91 commented Apr 26, 2018

@xeffyr I stand corrected. Is this true on all aarch64 devices?
It obviously won't work for dynamically linked binaries but I was also under the impression that the kernel must have been compiled with certain options, see this so question.

Maybe all android devices are though.

@xeffyr

This comment has been minimized.

Show comment
Hide comment
@xeffyr

xeffyr Apr 26, 2018

Contributor

@Grimler91 There is a way to check this:

[termux]:~:$ cat /system/build.prop | grep abi
# ro.product.cpu.abi and ro.product.cpu.abi2 are obsolete,
# use ro.product.cpu.abilist instead.
ro.product.cpu.abi=arm64-v8a
ro.product.cpu.abilist=arm64-v8a,armeabi-v7a,armeabi
ro.product.cpu.abilist32=armeabi-v7a,armeabi
ro.product.cpu.abilist64=arm64-v8a

If list contains armeabi or armeabi-v7a, then 32-bit programs are supported in your OS.

Contributor

xeffyr commented Apr 26, 2018

@Grimler91 There is a way to check this:

[termux]:~:$ cat /system/build.prop | grep abi
# ro.product.cpu.abi and ro.product.cpu.abi2 are obsolete,
# use ro.product.cpu.abilist instead.
ro.product.cpu.abi=arm64-v8a
ro.product.cpu.abilist=arm64-v8a,armeabi-v7a,armeabi
ro.product.cpu.abilist32=armeabi-v7a,armeabi
ro.product.cpu.abilist64=arm64-v8a

If list contains armeabi or armeabi-v7a, then 32-bit programs are supported in your OS.

@joeyh

This comment has been minimized.

Show comment
Hide comment
@joeyh

joeyh Apr 27, 2018

joeyh commented Apr 27, 2018

@joeyh

This comment has been minimized.

Show comment
Hide comment
@joeyh

joeyh Apr 27, 2018

joeyh commented Apr 27, 2018

@joeyh

This comment has been minimized.

Show comment
Hide comment
@willsALMANJ

This comment has been minimized.

Show comment
Hide comment
@willsALMANJ

willsALMANJ Apr 28, 2018

A path to a working git-annex in Termux is great news. Is the git annex arm build system compatible with the Termux build system? In other words, does this mean git-annex could be packaged in the Termux repository? Or does that still require that the Haskell dependencies be packaged for Termux?

willsALMANJ commented Apr 28, 2018

A path to a working git-annex in Termux is great news. Is the git annex arm build system compatible with the Termux build system? In other words, does this mean git-annex could be packaged in the Termux repository? Or does that still require that the Haskell dependencies be packaged for Termux?

@xeffyr

This comment has been minimized.

Show comment
Hide comment
@xeffyr

xeffyr Apr 28, 2018

Contributor

@willsALMANJ
These builds are compiled against glibc:
2018-04-28 17-39-22

This means that:

  1. git-annex arm build system is not compatible with the Termux build system.
  2. git-annex will be available when it will be possible to compile it against Android's bionic c library.
  3. Haskell dependencies are still needed to build it.
Contributor

xeffyr commented Apr 28, 2018

@willsALMANJ
These builds are compiled against glibc:
2018-04-28 17-39-22

This means that:

  1. git-annex arm build system is not compatible with the Termux build system.
  2. git-annex will be available when it will be possible to compile it against Android's bionic c library.
  3. Haskell dependencies are still needed to build it.
@sorsasampo

This comment has been minimized.

Show comment
Hide comment
@sorsasampo

sorsasampo Apr 29, 2018

@joeyh

@sorsasampo I've prepared a test build for arm64: https://downloads.kitenet.net/git-annex/linux/test/git-annex-standalone-arm64.tar.gz

Here is output of strace with this arm64 tarball:

~ $ git-annex.linux/runshell
\w $ git-annex
/data/data/com.termux/files/home/git-annex.linux/shimmed/sh/sh: 1: git-annex: not found
\w $ git annex
fatal: 'annex' appears to be a git command, but we were not
able to execute it. Maybe you need to 'pkg install perl'?
\w $ strace -f bash -x $(which git-annex)
<clip>
mprotect(0x5991d22000, 4096, PROT_READ) = 0
set_tid_address(0x788e5c9a70) = 16947
set_robust_list(0x788e5c9a80, 24 <unfinished ...>
--- SIGSYS {si_signo=SIGSYS, si_code=SYS_SECCOMP, si_call_addr=0x7890921958, si_syscall=__NR_set_robust_list, si_arch=AUDIT_ARCH_AARCH64} ---
<... set_robust_list resumed>) = ?
+++ killed by SIGSYS +++
Bad system call

So there's verification for what @xeffyr wrote earlier:

@sorsasampo Your binary is blocked by seccomp (feature introduced in Android 8). Your binary is compiled against glibc which don't know about blacklisted system calls on Android. Tool strace can help to find out which call actually causes this problem.

@joeyh

@sorsasampo I've prepared a test build for arm64: https://downloads.kitenet.net/git-annex/linux/test/git-annex-standalone-arm64.tar.gz

Here is output of strace with this arm64 tarball:

~ $ git-annex.linux/runshell
\w $ git-annex
/data/data/com.termux/files/home/git-annex.linux/shimmed/sh/sh: 1: git-annex: not found
\w $ git annex
fatal: 'annex' appears to be a git command, but we were not
able to execute it. Maybe you need to 'pkg install perl'?
\w $ strace -f bash -x $(which git-annex)
<clip>
mprotect(0x5991d22000, 4096, PROT_READ) = 0
set_tid_address(0x788e5c9a70) = 16947
set_robust_list(0x788e5c9a80, 24 <unfinished ...>
--- SIGSYS {si_signo=SIGSYS, si_code=SYS_SECCOMP, si_call_addr=0x7890921958, si_syscall=__NR_set_robust_list, si_arch=AUDIT_ARCH_AARCH64} ---
<... set_robust_list resumed>) = ?
+++ killed by SIGSYS +++
Bad system call

So there's verification for what @xeffyr wrote earlier:

@sorsasampo Your binary is blocked by seccomp (feature introduced in Android 8). Your binary is compiled against glibc which don't know about blacklisted system calls on Android. Tool strace can help to find out which call actually causes this problem.

@xeffyr

This comment has been minimized.

Show comment
Hide comment
@xeffyr

xeffyr Apr 29, 2018

Contributor

--- SIGSYS {si_signo=SIGSYS, si_code=SYS_SECCOMP, si_call_addr=0x7890921958, si_syscall=__NR_set_robust_list, si_arch=AUDIT_ARCH_AARCH64} ---

Looks like syscall set_robust_list is not provided by bionic libc. At least it is not exist in: https://github.com/aosp-mirror/platform_bionic/blob/master/libc/SYSCALLS.TXT

Seccomp allows:

  1. all the syscalls exposed via bionic
  2. syscalls to allow Android to boot
  3. syscalls used by popular Android applications, as determined by running Google's full app compatibility suite

More info about seccomp in Android available here: https://android-developers.googleblog.com/2017/07/seccomp-filter-in-android-o.html

Contributor

xeffyr commented Apr 29, 2018

--- SIGSYS {si_signo=SIGSYS, si_code=SYS_SECCOMP, si_call_addr=0x7890921958, si_syscall=__NR_set_robust_list, si_arch=AUDIT_ARCH_AARCH64} ---

Looks like syscall set_robust_list is not provided by bionic libc. At least it is not exist in: https://github.com/aosp-mirror/platform_bionic/blob/master/libc/SYSCALLS.TXT

Seccomp allows:

  1. all the syscalls exposed via bionic
  2. syscalls to allow Android to boot
  3. syscalls used by popular Android applications, as determined by running Google's full app compatibility suite

More info about seccomp in Android available here: https://android-developers.googleblog.com/2017/07/seccomp-filter-in-android-o.html

@joeyh

This comment has been minimized.

Show comment
Hide comment
@joeyh

joeyh May 4, 2018

set_robust_list is called by glibc in eg, pthread_create. No easy way to disable it without recompiling; OTOH if the syscall were intercepted with ptrace and NOOPed, it would probably not harm the program (unless it caused it to deadlock when a thread died leaving a futex locked).

proot seems a likely way to get around this, and indeed I see they have dealt with just this in termux/proot@5ed0da1
and there are a few other workarounds for some other innocuous syscalls that this IMHO illfounded security policy of Android's blocks.

@sorasampo, you could try pkg install proot and then try proot git-annex.linux/git-annex

joeyh commented May 4, 2018

set_robust_list is called by glibc in eg, pthread_create. No easy way to disable it without recompiling; OTOH if the syscall were intercepted with ptrace and NOOPed, it would probably not harm the program (unless it caused it to deadlock when a thread died leaving a futex locked).

proot seems a likely way to get around this, and indeed I see they have dealt with just this in termux/proot@5ed0da1
and there are a few other workarounds for some other innocuous syscalls that this IMHO illfounded security policy of Android's blocks.

@sorasampo, you could try pkg install proot and then try proot git-annex.linux/git-annex

@joeyh

This comment has been minimized.

Show comment
Hide comment
@joeyh

joeyh May 4, 2018

@sorsasampo see my comment above for something to try

joeyh commented May 4, 2018

@sorsasampo see my comment above for something to try

@sorsasampo

This comment has been minimized.

Show comment
Hide comment
@sorsasampo

sorsasampo May 5, 2018

@joeyh nice, it indeed seems to work:

~ $ git-annex.linux/git-annex version
Bad system call
~ $ proot git-annex.linux/git-annex version |head -1
git-annex version: 6.20180426-g9926727ee

I'll try to give it a proper spin later.

Can the same workaround be used in git-annex, or do you plan on making the wrapper script wrap git-annex execution in a proot call?

@joeyh nice, it indeed seems to work:

~ $ git-annex.linux/git-annex version
Bad system call
~ $ proot git-annex.linux/git-annex version |head -1
git-annex version: 6.20180426-g9926727ee

I'll try to give it a proper spin later.

Can the same workaround be used in git-annex, or do you plan on making the wrapper script wrap git-annex execution in a proot call?

@joeyh

This comment has been minimized.

Show comment
Hide comment
@joeyh

joeyh May 5, 2018

@sorsasampo I'll have the wrapper script use proot, assuming this works well enough.

joeyh commented May 5, 2018

@sorsasampo I'll have the wrapper script use proot, assuming this works well enough.

@sorsasampo

This comment has been minimized.

Show comment
Hide comment
@sorsasampo

sorsasampo May 6, 2018

So far I have not run into any problems, apart from trying to figure out how to deal with a crippled filesystem (as git-annex kindly denounces it), and v6 origin.

@joeyh, is it possible for the git-annex wrapper to catch the signal and helpfully suggest installing proot? I'm assuming you are not going to require it for everyone, since some devices might not need it.

So far I have not run into any problems, apart from trying to figure out how to deal with a crippled filesystem (as git-annex kindly denounces it), and v6 origin.

@joeyh, is it possible for the git-annex wrapper to catch the signal and helpfully suggest installing proot? I'm assuming you are not going to require it for everyone, since some devices might not need it.

peti pushed a commit to peti/git-annex that referenced this issue May 9, 2018

use proot to support Android 8
runshell: Use proot when running on Android, to work around Android 8's
ill-advised seccomp filtering of system calls, including ones crucial for
reliable thread locking. (This will only work with termux's version of
proot.)

See termux/termux-packages#420 (comment)

This commit was sponsored by andrea rota.
@quincunx

This comment has been minimized.

Show comment
Hide comment
@quincunx

quincunx May 10, 2018

@joeyh I've tried to run git-annex according to install_on_Android_in_Termux.mdwn, but this error message is shown:

proot error: execve("/data/data/com.termux/files/home/git-annex.linux/bin/sh"): No such file or directory
proot info: possible causes:
* the program is a script but its interpreter (eg. /bin/sh) was not found;
* the program is an ELF but its interpreter (eg. ld-linux.so) was not found;
* the program is a foreign binary but qemu was not specified;
* qemu does not work correctly (if specified);
* the loader was not found or doesn't work.
fatal error: see `proot --help`. 

The mentionend file /data/data/com.termux/files/home/git-annex.linux/bin/sh does exist, though.

I'm using a Nokia 5 with Android 8.1.0.

@joeyh I've tried to run git-annex according to install_on_Android_in_Termux.mdwn, but this error message is shown:

proot error: execve("/data/data/com.termux/files/home/git-annex.linux/bin/sh"): No such file or directory
proot info: possible causes:
* the program is a script but its interpreter (eg. /bin/sh) was not found;
* the program is an ELF but its interpreter (eg. ld-linux.so) was not found;
* the program is a foreign binary but qemu was not specified;
* qemu does not work correctly (if specified);
* the loader was not found or doesn't work.
fatal error: see `proot --help`. 

The mentionend file /data/data/com.termux/files/home/git-annex.linux/bin/sh does exist, though.

I'm using a Nokia 5 with Android 8.1.0.

@chrysn chrysn referenced this issue in termux/termux-widget May 15, 2018

Open

[wishlist] Support .desktop files #21

@Chobbes

This comment has been minimized.

Show comment
Hide comment
@Chobbes

Chobbes May 16, 2018

I'm having the same problem as @quincunx. I am also using an arm64 phone. Could this be the issue?

Can we get arm64 into the build bot?

Chobbes commented May 16, 2018

I'm having the same problem as @quincunx. I am also using an arm64 phone. Could this be the issue?

Can we get arm64 into the build bot?

@chrysn

This comment has been minimized.

Show comment
Hide comment
@chrysn

chrysn May 17, 2018

I've done some debugging on my LineageOS 14.1 (Android 7.1.2) device and documented it in the git-annex wiki.

chrysn commented May 17, 2018

I've done some debugging on my LineageOS 14.1 (Android 7.1.2) device and documented it in the git-annex wiki.

@aspiers

This comment has been minimized.

Show comment
Hide comment
@aspiers

aspiers Jun 26, 2018

@quincunx @Chobbes I believe you need to run

termux-fix-shebang git-annex.linux/bin/*

as mentioned on install on Android in Termux.

However, if you have an aarch64 phone like me (or arm64, I'm not sure what the exact difference is) then after running that you'll get:

% ./git-annex.linux/runshell
proot info: vpid 1: terminated with signal 11

Some further digging suggests that this is due to the architecture mismatch between my OnePlus 5T which is aarch64, and the linker which is 32-bit:

% PROOT_VERBOSE=9 ./git-annex.linux/runshell |& tail -n3
proot info: vpid 1: translate("/" + "/data/data/com.termux/files/home/git-annex.linux/lib/ld-linux.so.3")
proot info: vpid 1:          -> "/data/data/com.termux/files/home/git-annex.linux/lib/ld-linux.so.3"
proot info: vpid 1: terminated with signal 11
% file /data/data/com.termux/files/home/git-annex.linux/lib/ld-linux.so.3
/data/data/com.termux/files/home/git-annex.linux/lib/ld-linux.so.3: ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=59819e8e8c922b822204d3778cd4d6c843422462, stripped
% termux-info
Updatable packages:
All packages up to date
System information:
Linux localhost 4.4.78-perf+ #1 SMP PREEMPT Wed Jun 13 17:23:58 CST 2018 aarch64 Android
Termux-packages arch:
aarch64
Android version:
8.1.0
Device manufacturer:
OnePlus
Device model:
ONEPLUS A5010

@joeyh What's the next step here?

aspiers commented Jun 26, 2018

@quincunx @Chobbes I believe you need to run

termux-fix-shebang git-annex.linux/bin/*

as mentioned on install on Android in Termux.

However, if you have an aarch64 phone like me (or arm64, I'm not sure what the exact difference is) then after running that you'll get:

% ./git-annex.linux/runshell
proot info: vpid 1: terminated with signal 11

Some further digging suggests that this is due to the architecture mismatch between my OnePlus 5T which is aarch64, and the linker which is 32-bit:

% PROOT_VERBOSE=9 ./git-annex.linux/runshell |& tail -n3
proot info: vpid 1: translate("/" + "/data/data/com.termux/files/home/git-annex.linux/lib/ld-linux.so.3")
proot info: vpid 1:          -> "/data/data/com.termux/files/home/git-annex.linux/lib/ld-linux.so.3"
proot info: vpid 1: terminated with signal 11
% file /data/data/com.termux/files/home/git-annex.linux/lib/ld-linux.so.3
/data/data/com.termux/files/home/git-annex.linux/lib/ld-linux.so.3: ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=59819e8e8c922b822204d3778cd4d6c843422462, stripped
% termux-info
Updatable packages:
All packages up to date
System information:
Linux localhost 4.4.78-perf+ #1 SMP PREEMPT Wed Jun 13 17:23:58 CST 2018 aarch64 Android
Termux-packages arch:
aarch64
Android version:
8.1.0
Device manufacturer:
OnePlus
Device model:
ONEPLUS A5010

@joeyh What's the next step here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment