Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems with Debootstrap #5932

Closed
ghost opened this issue Oct 12, 2020 · 30 comments
Closed

Problems with Debootstrap #5932

ghost opened this issue Oct 12, 2020 · 30 comments
Assignees
Labels
bug report Something is not working properly help wanted Help is wanted in order to solve the issue

Comments

@ghost
Copy link

ghost commented Oct 12, 2020

Problem description
Hi. i was using Debootstrap to create a minimal Debian 9/10 system using Termux to create a chroot environment it works fine there's a working apt package manager. but i tried to setup an Ubuntu Trusty and Debian jessie to see if it works. but after setting that up there's no APT package manager and the packages didn't Configure. just extracted

Steps to reproduce

  1. debootstrap --arch=arm64 trusty ubuntu http://archive.ubuntu.com/ubuntu
  2. Setup a proot environment with necessary mounts and environment variable.

Expected behavior
I was expecting to work just fine to test old programs in bootstrapped environment. just like setting up a Ubuntu 14.04 on my PC using Debootstrap

Additional information
After setting up a trusty base system. i was hoping to work just fine but after logging in. i think the bash shell is unconfigured than debian stable system and instead of
root@localhost~# it says bash x.x# at PS1 environment variable

and by typing apt to update packages. it says
bash: apt: command not found

Tried Debian 8 and 7 but it's seems it's the same problem that i was configuring trusty base system

Device Architecture: aarch64
Android Version: 9
Rooted: No

@ghost
Copy link

ghost commented Oct 13, 2020

Steps to reproduce
debootstrap --arch=arm64 trusty ubuntu http://archive.ubuntu.com/ubuntu

$ debootstrap --arch=arm64 trusty ubuntu http://archive.ubuntu.com/ubuntu
W: Cannot check Release signature; keyring file not available /data/data/com.termux/files/usr/share/keyrings/ubuntu-archive-keyring.gpg
I: Retrieving InRelease 
I: Retrieving Release 
I: Retrieving Packages 
I: Retrieving Packages 
I: Retrieving Packages 
E: Couldn't download http://archive.ubuntu.com/ubuntu/dists/trusty/main/binary-arm64/Packages

Wrong steps or something is wrong with Ubuntu repositories. At least I don't have access to ARM64 packages, only AMD64 and i386 are available. See below
Снимок экрана от 2020-10-13 13-15-22

@ghost
Copy link

ghost commented Oct 13, 2020

it says bash x.x# at PS1 environment variable

Have not read profile due to non-login shell.

and by typing apt to update packages. it says
bash: apt: command not found

Are you sure that PATH is correctly set? Should be

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

@ghost
Copy link
Author

ghost commented Oct 13, 2020

Are you sure that PATH is correctly set? Should be

Yes. added all the necessary paths including. /sbin /usr/local, paths

Wrong steps or something is wrong with Ubuntu repositories

Whoops. i mean. i used ports.ubuntu.com Repository. and it downloads the arm64 bootstrap packages:
Screenshot_2020-10-13-21-10-49-31.png

I don't exactly remember because there's too many Ubuntu repos. like old-releases, ports, and archive

But after downloading packages. it only extracts the packages but didn't go to configuration steps. leaving the /debootstrap directory left behind. and didn't specify the foregin argument so normally it should automatically configure it than performing a debootstrap --second-stage

@ghost
Copy link
Author

ghost commented Oct 14, 2020

FYI: Here's the log:

Terminal Transcript:
https://pastebin.com/sWzN3vdw

debootstrap.log file:
https://pastebin.com/WjEwQ4ma

@ZhymabekRoman
Copy link
Contributor

I confirm that I also have the same problems with debootstrap'ed Debian.

@ghost
Copy link

ghost commented Oct 15, 2020

Problem is here:

2020-10-14 13:32:46 URL:http://ports.ubuntu.com/ubuntu-ports/pool/main/z/zlib/zlib1g_1.2.8.dfsg-1ubuntu1_arm64.deb [52138/52138] -> "/data/data/com.termux/files/home/ubuntu/trusty//var/cache/apt/archives/partial/zlib1g_1%3a1.2.8.dfsg-1ubuntu1_arm64.deb" [1]
chown: changing ownership of '/data/data/com.termux/files/home/ubuntu/trusty/etc/fstab': Operation not permitted

Debootstrap shouldn't execute chown or should do that only under proot -0.

@ghost
Copy link
Author

ghost commented Oct 15, 2020

Does Debootstrap uses proot -0 when configuring packages?

@ghost
Copy link

ghost commented Oct 15, 2020

It is patched for

CHROOT_CMD="proot -w /home -b /dev -b /proc --link2symlink -0 -r $TARGET"

and few other places to use proot.

@ghost
Copy link

ghost commented Oct 15, 2020

It also has following patch for debian-common

@@ -172,6 +171,20 @@
 echo \"Warning: Fake start-stop-daemon called, doing nothing\"" > "$TARGET/sbin/start-stop-daemon"
 	chmod 755 "$TARGET/sbin/start-stop-daemon"
 
+	# Replace problematic binaries with a stub
+	echo "" > "$TARGET/bin/chown"
+	echo "" > "$TARGET/usr/sbin/groupadd"
+	echo "" > "$TARGET/usr/sbin/groupdel"
+	echo "" > "$TARGET/usr/sbin/groupmod"
+	echo "" > "$TARGET/usr/sbin/useradd"
+	echo "" > "$TARGET/usr/sbin/userdel"
+	echo "" > "$TARGET/usr/sbin/usermod"
+	echo "" > "$TARGET/usr/bin/chage"
+	echo "" > "$TARGET/usr/bin/chfn"
+	echo "" > "$TARGET/usr/bin/dpkg-statoverride"
+
+	echo "nameserver 8.8.8.8\nnameserver 8.8.4.4" > "$TARGET/etc/resolv.conf"
+
 	setup_dselect_method apt
 
 	smallyes '' |

chown is not proot'ed:

~/.../debootstrap/debootstrap $ grep chown scripts/trusty 
		chown 0:0 "$TARGET/etc/fstab"; chmod 644 "$TARGET/etc/fstab"

ghost pushed a commit that referenced this issue Oct 15, 2020
@ghost
Copy link

ghost commented Oct 15, 2020

@wmcb-tech pkg upgrade and try debootstrap again.

@ghost ghost added the bug report Something is not working properly label Oct 15, 2020
@ghost
Copy link
Author

ghost commented Oct 16, 2020

@xeffyr it does skips the configuration of fstab but the issue is that when symlink has changed. it gives an error with Debian Jessie (armhf) and Ubuntu Trusty:

Unpacking base-files (8+deb8u11) ... dpkg: error processing archive /var/cache/apt/archives/base-files_8+deb8u11_armhf.deb (--install): symbolic link '/etc/os-release' size has changed from 26 to 21 Errors were encountered while processing: /var/cache/apt/archives/base-files_8+deb8u11_armhf.deb

However i haven't tested this on Android x86 with SELinux disabled. only my phone

I was so close. maybe due to the Restrictions of my phone,

@ghost
Copy link

ghost commented Oct 16, 2020

symbolic link '/etc/os-release' size has changed from 26 to 21

Need to report this to https://github.com/termux/proot, maybe @michalbednarski knows what to do with symbolic link size changes.

Will try on my device.

@ghost
Copy link

ghost commented Oct 16, 2020

Got failure at this step:

Setting up udev (204-5ubuntu20) ...
initctl.REAL: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused
invoke-rc.d: policy-rc.d denied execution of restart.
Removing 'diversion of /bin/udevadm to /bin/udevadm.upgrade by fake-udev'
update-initramfs: deferring update (trigger activated)
Setting up initramfs-tools (0.103ubuntu4) ...
update-initramfs: deferring update (trigger activated)
Setting up procps (1:3.3.9-1ubuntu2) ...
update-alternatives: using /usr/bin/w.procps to provide /usr/bin/w (w) in auto mode
initctl.REAL: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused
invoke-rc.d: policy-rc.d denied execution of start.
Processing triggers for libc-bin (2.19-0ubuntu6) ...
Processing triggers for initramfs-tools (0.103ubuntu4) ...
Errors were encountered while processing:
 makedev

@ghost
Copy link
Author

ghost commented Oct 16, 2020

I guess proot is not stable at all on newer devices. but Debootstrap in termux is useful for just Bootstrapping a latest debian system (buster/stretch).

@ghost
Copy link

ghost commented Oct 16, 2020

In case of Debootstrap, problem is not with proot, in my case. It just tries to execute commands related to init system. I will look if we can overwrite faulty binaries (e.g. initctl.REAL) with no op script, so it won't fail when configuring packages.

Also I can make Debian rootfs for use with proot-distro.

@ghost
Copy link
Author

ghost commented Oct 16, 2020

Thanks @xeffyr this should make termux more useful.

@ghost
Copy link

ghost commented Oct 16, 2020

Added

echo "" > $TARGET/sbin/initctl.REAL
echo "" > $TARGET/sbin/initctl

just before the package configuration command in Debootstrap files and it seems doesn't work. Binaries are being restored and I still get

initctl.REAL: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused

So variant with getting sane rootfs of Ubuntu or Debian is only proot-distro or similar things (Andronix, AnLinux, etc).

@ghost ghost added the help wanted Help is wanted in order to solve the issue label Oct 16, 2020
@ghost ghost mentioned this issue Feb 20, 2021
@stale
Copy link

stale bot commented Nov 18, 2021

This issue/PR has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix Issue won't be fixed label Nov 18, 2021
@stale stale bot closed this as completed Dec 2, 2021
@nathaneltitane
Copy link

nathaneltitane commented Jan 2, 2022

issue still relevant. is anyone looking at this? ref #6424

ˋˋˋ
rm -rf $PREFIX/cnt-deb; mkdir -p $PREFIX/cnt-deb; debootstrap --variant=minbase --exclude=systemd --arch=arm64 testing $PREFIX/cnt-deb/ http://ftp.debian.org/debian
ˋˋˋ

returns:

ˋˋˋ
[...]
I: Configuring apt...
I: Configuring libc-bin...
W: Failure while configuring required packages.
W: See /data/data/com.termux/files/usr/cnt-deb/debootstrap/debootstrap.log for details (possibly the package dash is at fault)
ˋˋˋ

and checking the log then gives:

ˋˋˋ
[...]
[ termux ] >_ $ tail -n 5 /data/data/com.termux/files/usr/cnt-deb/debootstrap/debootstrap.log Setting up adduser (3.118) ...
Setting up apt (2.3.13) ...
Processing triggers for libc-bin (2.33-1) ...
Errors were encountered while processing:
dash
ˋˋˋ

then, excluding dash using --exclude returns:

ˋˋˋ
[...]
I: Extracting zlib1g...
I: Installing core packages...
W: Failure trying to run: proot -w /home -b /dev -b /proc --link2symlink -0 -r /data/data/com.termux/files/usr/cnt-deb dpkg --force-depends --install /var/cache/apt/archives/base-passwd_3.5.52_arm64.deb
W: See /data/data/com.termux/files/usr/cnt-deb/debootstrap/debootstrap.log for details
ˋˋˋ

checking the new log gives:

ˋˋˋ
[...]
dpkg: warning: parsing file '/var/lib/dpkg/status' near line 5 package 'dpkg':
missing 'Architecture' field
dpkg: warning: 'sh' not found in PATH or not executable
dpkg: error: 1 expected program not found in PATH or not executable
Note: root's PATH should usually contain /usr/local/sbin, /usr/sbin and /sbin
ˋˋˋ

@ghost ghost reopened this Jan 2, 2022
@stale stale bot removed the wontfix Issue won't be fixed label Jan 2, 2022
@nathaneltitane
Copy link

@xeffyr edited previous with feedback and log (tail -n 5)

@nathaneltitane
Copy link

nathaneltitane commented Jan 5, 2022

@xeffyr new development!!!

On a clean Termux setup (0.117/API 0.51)

ˋˋˋ
pkg install debootstrap
termux-setup-storage
cd $HOME/storage/shared/
rm -rf testing; debootstrap testing testing https://ftp.debian.org
ˋˋˋ

the debootstrap process returns as last line:

ˋˋˋ
[...]
I: Chosen extractor for .deb packages: dpkg-deb
ˋˋˋ

The process seems to complete (?) without any errors when the debootstrap directory location is NOT set under Termux's $PREFIX...

Perhaps this be due to permission scheme?

I have tested incrementally these arguments:

--exclude=systemd
--arch=arm64

both run properly and without any error on last line.

@ghost
Copy link

ghost commented Jan 6, 2022

debootstrap --variant=minbase --exclude=systemd --arch=arm64 testing testing http://ftp.debian.org/debian

Fails in both $HOME or $PREFIX:

Setting up libpam0g:arm64 (1.4.0-11) ...
Setting up util-linux (2.37.2-5) ...
update-alternatives: using /bin/more to provide /usr/bin/pager (pager) in auto mode
Setting up dash (0.5.11+git20210903+057cd650a4ed-3) ...
No diversion 'diversion of /bin/sh by bash', none removed.
Adding 'diversion of /bin/sh to /bin/sh.distrib by dash'
dpkg-divert: error: error creating new backup file '/var/lib/dpkg/diversions-old': Operation not permitted
dpkg: error processing package dash (--configure):
 installed dash package post-installation script subprocess returned error exit status 2
Setting up libssl1.1:arm64 (1.1.1m-1) ...
Setting up libpam-modules-bin (1.4.0-11) ...
Setting up libkrb5-3:arm64 (1.18.3-7) ...
Setting up libgssapi-krb5-2:arm64 (1.18.3-7) ...
Setting up libtirpc3:arm64 (1.3.2-2) ...
Setting up libnsl2:arm64 (1.3.0-2) ...
Setting up libpam-modules:arm64 (1.4.0-11) ...
Setting up passwd (1:4.8.1-2) ...
Shadow passwords are now on.
Setting up libpam-runtime (1.4.0-11) ...
Setting up login (1:4.8.1-2) ...
Setting up adduser (3.118) ...
Setting up apt (2.3.13) ...
Processing triggers for libc-bin (2.33-1) ...
Errors were encountered while processing:
 dash

@ghost
Copy link
Author

ghost commented Jan 6, 2022

dpkg-statoverride or dpkg-divert likely creates hard-link copies of files and diversions. proot already handles hardlinks but i think dpkg probably has different approach

@ghost
Copy link

ghost commented Jan 6, 2022

My fault. It is not possible to reproduce dpkg-divert issue when link2symlink is enabled.

~ $ proot --link2symlink -S testing /usr/bin/env -i DEBIAN_FRONTEND=noninteractive PATH=/usr/bin:/usr/sbin dpkg --configure dash
Setting up dash (0.5.11+git20210903+057cd650a4ed-3) ...
No diversion 'diversion of /usr/share/man/man1/sh.1.gz by bash', none removed.
Adding 'diversion of /usr/share/man/man1/sh.1.gz to /usr/share/man/man1/sh.distrib.1.gz by dash'

I will delete previous comments regarding that.

@nathaneltitane
Copy link

nathaneltitane commented Jan 6, 2022

debootstrap --variant=minbase --exclude=systemd --arch=arm64 testing testing http://ftp.debian.org/debian

Fails in both $HOME or $PREFIX:

Setting up libpam0g:arm64 (1.4.0-11) ...
Setting up util-linux (2.37.2-5) ...
update-alternatives: using /bin/more to provide /usr/bin/pager (pager) in auto mode
Setting up dash (0.5.11+git20210903+057cd650a4ed-3) ...
No diversion 'diversion of /bin/sh by bash', none removed.
Adding 'diversion of /bin/sh to /bin/sh.distrib by dash'
dpkg-divert: error: error creating new backup file '/var/lib/dpkg/diversions-old': Operation not permitted
dpkg: error processing package dash (--configure):
 installed dash package post-installation script subprocess returned error exit status 2
Setting up libssl1.1:arm64 (1.1.1m-1) ...
Setting up libpam-modules-bin (1.4.0-11) ...
Setting up libkrb5-3:arm64 (1.18.3-7) ...
Setting up libgssapi-krb5-2:arm64 (1.18.3-7) ...
Setting up libtirpc3:arm64 (1.3.2-2) ...
Setting up libnsl2:arm64 (1.3.0-2) ...
Setting up libpam-modules:arm64 (1.4.0-11) ...
Setting up passwd (1:4.8.1-2) ...
Shadow passwords are now on.
Setting up libpam-runtime (1.4.0-11) ...
Setting up login (1:4.8.1-2) ...
Setting up adduser (3.118) ...
Setting up apt (2.3.13) ...
Processing triggers for libc-bin (2.33-1) ...
Errors were encountered while processing:
 dash

@xeffyr yes, this is what I have come to notice. but not when the root dir is set in the internal storage i.e.: /storage/emulated/0/, hence the $HOME/storage/shared/testing - this is why I asked if it may have had anything to do with overall system permissions?

Another thing I've come to notice is that debootstrap does not seem to unpack anything? the only directories present under 'testing' were 'debootstrap' (of course) and 'var'....

Aside from the obvious bits pertaining to architecture and host system, this link makes me think that debootstrap unpacks the base system from the packages that are loaded as well and populates 'testing' with the usual directories?

@ghost
Copy link
Author

ghost commented Jan 7, 2022

My fault. It is not possible to reproduce dpkg-divert issue when link2symlink is enabled.

~ $ proot --link2symlink -S testing /usr/bin/env -i DEBIAN_FRONTEND=noninteractive PATH=/usr/bin:/usr/sbin dpkg --configure dash
Setting up dash (0.5.11+git20210903+057cd650a4ed-3) ...
No diversion 'diversion of /usr/share/man/man1/sh.1.gz by bash', none removed.
Adding 'diversion of /usr/share/man/man1/sh.1.gz to /usr/share/man/man1/sh.distrib.1.gz by dash'

I will delete previous comments regarding that.

@xeffyr Probably you spoke too soon about it. reproducing it with --link2symlink and still fails creating diversion backup

~ % proot --link2symlink -S debian-testing/ /usr/bin/env -i PATH=/usr/bin:/usr/sbin:/bin:/sbin dpkg --configure -a
Setting up dash (0.5.11+git20210903+057cd650a4ed-3) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/arm-linux-gnueabi/perl/5.32.1 /usr/local/share/perl/5.32.1 /usr/lib/arm-linux-gnueabi/perl5/5.32 /usr/share/perl5 /usr/lib/arm-linux-gnueabi/perl-base /usr/lib/arm-linux-gnueabi/perl/5.32 /usr/share/perl/5.32 /usr/local/lib/site_perl) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)
debconf: falling back to frontend: Teletype
No diversion 'diversion of /bin/sh by bash', none removed.
Adding 'diversion of /bin/sh to /bin/sh.distrib by dash'
dpkg-divert: error: error creating new backup file '/var/lib/dpkg/diversions-old': Operation not permitted
dpkg: error processing package dash (--configure):
 installed dash package post-installation script subprocess returned error exit status 2
Errors were encountered while processing:
 dash

this is probably proot bug on some devices

@michalbednarski
Copy link
Contributor

Should be fixed in proot 5.1.107-52 (Available now or soon in apt repo depending on mirror)

@ghost
Copy link

ghost commented Jan 7, 2022

Probably you spoke too soon about it. reproducing it with --link2symlink and still fails creating diversion backup

In my case I forgot about enabling link2symlink and it failed during linkat() syscall.

@stale
Copy link

stale bot commented Feb 21, 2022

This issue/PR has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix Issue won't be fixed label Feb 21, 2022
@stale stale bot closed this as completed Mar 7, 2022
@xtkoba xtkoba reopened this Dec 24, 2022
@xtkoba xtkoba removed the wontfix Issue won't be fixed label Dec 24, 2022
@sylirre sylirre self-assigned this Jan 5, 2023
@sylirre
Copy link
Member

sylirre commented Jan 5, 2023

Debootstrap appears to be working now (needs binutils though for some distributions).

Don't forget to use ports repository for ARM and AArch64 devices, as original is only for x86(64) CPUs.

debootstrap --arch=arm64 jammy ubuntu http://ports.ubuntu.com/

Login to the rootfs:

unset LD_PRELOAD
proot -S ./ubuntu /bin/su -l

/bin/su is easy way to properly configure environment variables, you can try using /usr/bin/env with parameters instead.

@sylirre sylirre closed this as completed Jan 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug report Something is not working properly help wanted Help is wanted in order to solve the issue
Projects
None yet
Development

No branches or pull requests

5 participants