Failed to configure LXC container with lxc 3.0

[Bertil-100]

** Please paste the result of anbox system-info below:**
anbox system-info
version: local-03f69c8
os:
name: Antergos Linux
version: 18.3-ISO-Rolling
snap-based: false
kernel:
version: Linux version 4.15.14-1-ARCH (builduser@heftig-6127) (gcc version 7.3.1 20180312 (GCC)) #1 SMP PREEMPT Wed Mar 28 17:34:29 UTC 2018
binder: true
ashmem: true
graphics:
egl:
vendor: Mesa Project
version: 1.4 (DRI2)
extensions:
- EGL_ANDROID_native_fence_sync
- EGL_CHROMIUM_sync_control
- EGL_EXT_buffer_age
- EGL_EXT_create_context_robustness
- EGL_EXT_image_dma_buf_import
- EGL_EXT_image_dma_buf_import_modifiers
- EGL_IMG_context_priority
- EGL_KHR_config_attribs
- EGL_KHR_create_context
- EGL_KHR_create_context_no_error
- EGL_KHR_fence_sync
- EGL_KHR_get_all_proc_addresses
- EGL_KHR_gl_renderbuffer_image
- EGL_KHR_gl_texture_2D_image
- EGL_KHR_gl_texture_3D_image
- EGL_KHR_gl_texture_cubemap_image
- EGL_KHR_image
- EGL_KHR_image_base
- EGL_KHR_image_pixmap
- EGL_KHR_no_config_context
- EGL_KHR_reusable_sync
- EGL_KHR_surfaceless_context
- EGL_KHR_wait_sync
- EGL_MESA_configless_context
- EGL_MESA_drm_image
- EGL_MESA_image_dma_buf_export
- EGL_NOK_texture_from_pixmap
- EGL_WL_bind_wayland_display
gles2:
vendor: Intel Open Source Technology Center
vendor: OpenGL ES-CM 1.1 Mesa 17.3.7
extensions:
- GL_EXT_blend_minmax
- GL_EXT_multi_draw_arrays
- GL_EXT_texture_filter_anisotropic
- GL_EXT_texture_lod_bias
- GL_OES_byte_coordinates
- GL_OES_fixed_point
- GL_OES_stencil_wrap
- GL_OES_compressed_paletted_texture
- GL_OES_query_matrix
- GL_OES_read_format
- GL_OES_single_precision
- GL_EXT_texture_compression_dxt1
- GL_OES_draw_texture
- GL_OES_point_size_array
- GL_OES_point_sprite
- GL_EXT_texture_format_BGRA8888
- GL_OES_compressed_ETC1_RGB8_texture
- GL_OES_depth24
- GL_OES_element_index_uint
- GL_OES_fbo_render_mipmap
- GL_OES_framebuffer_object
- GL_OES_mapbuffer
- GL_OES_rgb8_rgba8
- GL_OES_stencil8
- GL_OES_texture_env_crossbar
- GL_OES_texture_mirrored_repeat
- GL_OES_texture_npot
- GL_OES_EGL_image
- GL_OES_packed_depth_stencil
- GL_OES_texture_cube_map
- GL_APPLE_texture_max_level
- GL_EXT_discard_framebuffer
- GL_EXT_read_format_bgra
- GL_OES_blend_equation_separate
- GL_OES_blend_func_separate
- GL_OES_blend_subtract
- GL_OES_EGL_image_external
- GL_OES_EGL_sync
- GL_OES_vertex_array_object
- GL_ANGLE_texture_compression_dxt3
- GL_ANGLE_texture_compression_dxt5
- GL_EXT_map_buffer_range
- GL_KHR_debug
- GL_OES_required_internalformat
- GL_OES_surfaceless_context
- GL_EXT_compressed_ETC1_RGB8_sub_texture
- GL_EXT_polygon_offset_clamp
- GL_KHR_no_error

Please describe your problem:
$ anbox session-manager
[ 2018-04-03 09:21:43] [client.cpp:49@start] Failed to start container: Failed to start container: Failed to configure LXC container
[ 2018-04-03 09:21:43] [session_manager.cpp:162@operator()] Lost connection to container manager, terminating.
[ 2018-04-03 09:21:43] [daemon.cpp:58@Run] Container is not running
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injectorboost::log::v2_mt_posix::system_error >'
what(): Failed to set TLS value: Invalid argument
Aborted (core dumped)

If I enter this command:
anbox launch --package=org.anbox.appmgr --component=org.anbox.appmgr.AppViewActivity
[ 2018-04-03 09:20:59] [launch.cpp:134@operator()] Anbox session manager service isn't running, trying to start it.
[ 2018-04-03 09:21:04] [launch.cpp:134@operator()] Anbox session manager service isn't running, trying to start it.
[ 2018-04-03 09:21:09] [launch.cpp:134@operator()] Anbox session manager service isn't running, trying to start it.
[ 2018-04-03 09:21:14] [launch.cpp:206@operator()] Couldn't get a connection with the session manager

What were you expecting?:
Hint to get anbox working under newest arch linux.

Additional info:

THANK YOU !

[chang196700]

i have the same problem under the newest arch linux.

[zergling-man]

anbox.sh:

sudo systemctl start anbox-container-manager
sudo cp anbox.service /usr/lib/systemd/user/anbox-session-manager.service
systemctl --user daemon-reload
systemctl --user start anbox-session-manager

anbox.service:

[Unit]
Description=Anbox Session Manager

[Service]
ExecStart=/usr/bin/anbox session-manager --gles-driver=host

[Install]
WantedBy=default.target

I don't actually remember exactly what I was doing with the service, but this is the general direction you need to go to solve this issue. At least, I think it is. I ran into the generic "waited too long, something has gone wrong" error after doing this stuff. For some reason I later uninstalled it, and now can't install again (#637), so I can't verify or test any of this.

ED: #236 (comment) was heavily related to how I came up with the above.

[xomachine]

The same issue.
system info
backtrace

[garyvdm]

@zergling-man As far as I can see the only thing I can see you are doing that is different is running with the session-manager with --gles-driver=host. I tried this and it did not help.

[garyvdm]

I'm trying to debug this.

I've started by making a patch to show which set_config_item call is failing:

diff --git a/src/anbox/container/lxc_container.cpp b/src/anbox/container/lxc_container.cpp
index 962832b..221349c 100644
--- a/src/anbox/container/lxc_container.cpp
+++ b/src/anbox/container/lxc_container.cpp
@@ -285,8 +285,11 @@ void LxcContainer::stop() {
 
 void LxcContainer::set_config_item(const std::string &key,
                                    const std::string &value) {
-  if (!container_->set_config_item(container_, key.c_str(), value.c_str()))
-    BOOST_THROW_EXCEPTION(std::runtime_error("Failed to configure LXC container"));
+  if (!container_->set_config_item(container_, key.c_str(), value.c_str())) {
+    char buffer[250];
+    sprintf(buffer, "Failed to configure LXC container: %s=%s", key.c_str(), value.c_str());
+    BOOST_THROW_EXCEPTION(std::runtime_error(buffer));
+  }
 }

With this I get this error:

[ 2018-04-19 11:10:21] [client.cpp:49@start] Failed to start container: Failed to start container: Failed to configure LXC container: lxc.pts=1024

Right. So I checked out the lxc documention. lxc.pts was renamed to lxc.pty.max in lxc 2.1 and the backward compatibility option was removed in lxc 3.0.

So unless anbox supports lxc < 2.1, we can fix this by using lxc.pty.max instead of lxc.pts. That will support lxc >= 2.1. Unfortunately Ubuntu 16.04 only has lxc 2.0, with lxc 2.1 only added in Ubuntu 17.10.

However, according to man 5 lxc.container.conf, this property is 1) not implemented, and 2) defaults to 1024 anyway. So I think a better fix is to simply remove this set_config_item call.

I tried doing this, and just got a different error, which I'm yet to investigate. But I think a workaround for arch users in the mean time is to maybe downgrade lxc.

[garyvdm]

@Bertil-100 Please will you change the title of the bug to Failed to configure LXC container with lxc 3.0.

[garyvdm]

I was able to get anbox running with lxc 3.0 with this patch

diff --git a/src/anbox/container/lxc_container.cpp b/src/anbox/container/lxc_container.cpp
index 962832b..6c5d3b9 100644
--- a/src/anbox/container/lxc_container.cpp
+++ b/src/anbox/container/lxc_container.cpp
@@ -65,24 +65,24 @@ void LxcContainer::setup_id_maps() {
   const auto base_id = unprivileged_user_id;
   const auto max_id = 65536;
 
-  set_config_item("lxc.id_map",
+  set_config_item("lxc.idmap",
                   utils::string_format("u 0 %d %d", base_id, creds_.uid() - 1));
-  set_config_item("lxc.id_map",
+  set_config_item("lxc.idmap",
                   utils::string_format("g 0 %d %d", base_id, creds_.gid() - 1));
 
   // We need to bind the user id for the one running the client side
   // process as he is the owner of various socket files we bind mount
   // into the container.
-  set_config_item("lxc.id_map",
+  set_config_item("lxc.idmap",
                   utils::string_format("u %d %d 1", creds_.uid(), creds_.uid()));
-  set_config_item("lxc.id_map",
+  set_config_item("lxc.idmap",
                   utils::string_format("g %d %d 1", creds_.gid(), creds_.gid()));
 
-  set_config_item("lxc.id_map",
+  set_config_item("lxc.idmap",
                   utils::string_format("u %d %d %d", creds_.uid() + 1,
                                        base_id + creds_.uid() + 1,
                                        max_id - creds_.uid() - 1));
-  set_config_item("lxc.id_map",
+  set_config_item("lxc.idmap",
                   utils::string_format("g %d %d %d", creds_.uid() + 1,
                                        base_id + creds_.gid() + 1,
                                        max_id - creds_.gid() - 1));
@@ -188,42 +188,40 @@ void LxcContainer::start(const Configuration &configuration) {
   set_config_item("lxc.mount.auto", "proc:mixed sys:mixed cgroup:mixed");
 
   set_config_item("lxc.autodev", "1");
-  set_config_item("lxc.pts", "1024");
-  set_config_item("lxc.tty", "0");
-  set_config_item("lxc.utsname", "anbox");
+  set_config_item("lxc.tty.max", "0");
+  set_config_item("lxc.uts.name", "anbox");
 
   set_config_item("lxc.group.devices.deny", "");
   set_config_item("lxc.group.devices.allow", "");
 
   // We can't move bind-mounts, so don't use /dev/lxc/
-  set_config_item("lxc.devttydir", "");
+  set_config_item("lxc.tty.dir", "");
 
   set_config_item("lxc.environment",
                   "PATH=/system/bin:/system/sbin:/system/xbin");
 
-  set_config_item("lxc.init_cmd", "/anbox-init.sh");
-  set_config_item("lxc.rootfs.backend", "dir");
+  set_config_item("lxc.init.cmd", "/anbox-init.sh");
 
   const auto rootfs_path = SystemConfiguration::instance().rootfs_dir();
   DEBUG("Using rootfs path %s", rootfs_path);
-  set_config_item("lxc.rootfs", rootfs_path);
+  set_config_item("lxc.rootfs.path", rootfs_path);
 
-  set_config_item("lxc.loglevel", "0");
+  set_config_item("lxc.log.level", "0");
   const auto log_path = SystemConfiguration::instance().log_dir();
-  set_config_item("lxc.logfile", utils::string_format("%s/container.log", log_path).c_str());
+  set_config_item("lxc.log.file", utils::string_format("%s/container.log", log_path).c_str());
 
   setup_network();
 
 #if 0
     // Android uses namespaces as well so we have to allow nested namespaces for LXC
     // which are otherwise forbidden by AppArmor.
-    set_config_item("lxc.aa_profile", "lxc-container-default-with-nesting");
+    set_config_item("lxc.apparmor.profile", "lxc-container-default-with-nesting");
 #else
   // FIXME: when using the nested profile we still get various denials from
   // things Android tries to do but isn't allowed to. We need to look into
   // those and see how we can switch back to a confined way of running the
   // container.
-  set_config_item("lxc.aa_profile", "unconfined");
+  set_config_item("lxc.apparmor.profile", "unconfined");
 #endif
 
   if (!privileged_)

I would like to submit a pull request. @morphis Please can I get feed back which of the following options you would prefer:

• Use the patch from above. This would be the simplest code, but would drop support for lxc < 2.1.
• Have a runtime check on the lxc version, and use the appropriate config item keys based on that. This would need some code to parse the lxc version string so correctly do this decision making.
• Have #ifdef option so that one can choose which lxc versions your build will support.

[xiayesuifeng]

@garyvdm according to the lxc documentation, lxc.network should be changed to lxc.net.0

[Traneptora]

This patch does not work for me. I'm still getting the same error.

[tiannian]

@garyvdm I apply your patch on Archlinux, based on anbox-git.It seems to work. It no longer generate an error when I execute the command, and some process are running . I also can find device in adb. But I when i execute anbox launch --package=org.anbox.appmgr --component=org.anbox.appmgr.AppViewActivity , nothing to display. and I also can't connect device by adb.

[swayf]

The same for me.. it hangs in boot looping :(

[morphis]

@garyvdm The plan is to switch to LXC 3.x real soon when we migrate the snap to a 18.04 base as well. So I am happy to receive a PR doing the legwork for this and will merge it as soon as we're at that point. How does that sound?

[garyvdm]

Ok - I will submit a PR tonight.

…

On Mon, May 21, 2018 at 8:36 PM, Simon Fels ***@***.***> wrote: @garyvdm <https://github.com/garyvdm> The plan is to switch to LXC 3.x real soon when we migrate the snap to a 18.04 base as well. So I am happy to receive a PR doing the legwork for this and will merge it as soon as we're at that point. How does that sound? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#669 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAVfr-3hNFjKhsqubefor0XtWgDiR8Ezks5t0wk2gaJpZM4TEvu3> .

[mprogram]

Thanks to @garyvdm spotting and debugging the issue first. It's better, however, not to drop the lxc.pts (lxc.pty.max) line as it leads to unwanted entries in the log.

The below alternative approach clears the way to see an already reported error: No such file or directory - Failed to exec "/anbox-init.sh" (though it is in /var/lib/anbox/rootfs). Apparently, this secondary issue is unrelated to anbox, as it is an independed command and needs to be investigated further: sudo lxc-start --name=default --logfile=/dev/stdout --logpriority=DEBUG --lxcpath=/var/lib/anbox/containers --

The whole thing (patching src/anbox/container/lxc_container.cpp) for now and on could be done as per relevant LXC release anouncement. The following approach is universal and could be applied to any project using legacy LXC (<= 2.1.1) code:

sed 's/lxc.aa_profile/lxc.apparmor.profile/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.aa_allow_incomplete/lxc.apparmor.allow_incomplete/g' -i; src/anbox/container/lxc_container.cpp; sed 's/lxc.console/lxc.console.path/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.devttydir/lxc.tty.dir/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.haltsignal/lxc.signal.halt/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.id_map/lxc.idmap/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.init_cmd/lxc.init.cmd/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.init_gid/lxc.init.gid/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.init_uid/lxc.init.uid/g' -i src/anbox/container/lxc_container.cpp; sed '/lxc.kmsg/d' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.limit/lxc.prlimit/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.logfile/lxc.log.file/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.loglevel/lxc.log.level/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.mount/lxc.mount.fstab/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.mount.fstab.auto/lxc.mount.auto/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.mount.fstab.entry/lxc.mount.entry/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.network./lxc.net.0./g' -i src/anbox/container/lxc_container.cpp; sed '/lxc.pivotdir/d' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.pts/lxc.pty.max/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.rebootsignal/lxc.signal.reboot/g' -i src/anbox/container/lxc_container.cpp; sed '/lxc.rootfs.backend/d' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.rootfs/lxc.rootfs.path/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.rootfs.path.mount/lxc.rootfs.mount/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.rootfs.path.options/lxc.rootfs.options/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.se_context/lxc.selinux.context/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.seccomp/lxc.seccomp.profile/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.stopsignal/lxc.signal.stop/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.syslog/lxc.log.syslog/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.tty/lxc.tty.max/g' -i src/anbox/container/lxc_container.cp; sed 's/lxc.tty.max.dir/lxc.tty.max/g' -i src/anbox/container/lxc_container.cpp; sed 's/lxc.utsname/lxc.uts.name/g' -i src/anbox/container/lxc_container.cpp

[Tanner-Lin]

@mprogram You made a typo with the file name on one line. This seems to be correct:

sed -i  '/lxc.kmsg/d'                                              src/anbox/container/lxc_container.cpp
sed -i  '/lxc.pivotdir/d'                                          src/anbox/container/lxc_container.cpp
sed -i  '/lxc.rootfs.backend/d'                                    src/anbox/container/lxc_container.cpp
sed -i 's/lxc.aa_allow_incomplete/lxc.apparmor.allow_incomplete/g' src/anbox/container/lxc_container.cpp
sed -i 's/lxc.aa_profile/lxc.apparmor.profile/g'                   src/anbox/container/lxc_container.cpp
sed -i 's/lxc.console/lxc.console.path/g'                          src/anbox/container/lxc_container.cpp
sed -i 's/lxc.devttydir/lxc.tty.dir/g'                             src/anbox/container/lxc_container.cpp
sed -i 's/lxc.haltsignal/lxc.signal.halt/g'                        src/anbox/container/lxc_container.cpp
sed -i 's/lxc.id_map/lxc.idmap/g'                                  src/anbox/container/lxc_container.cpp
sed -i 's/lxc.init_cmd/lxc.init.cmd/g'                             src/anbox/container/lxc_container.cpp
sed -i 's/lxc.init_gid/lxc.init.gid/g'                             src/anbox/container/lxc_container.cpp
sed -i 's/lxc.init_uid/lxc.init.uid/g'                             src/anbox/container/lxc_container.cpp
sed -i 's/lxc.limit/lxc.prlimit/g'                                 src/anbox/container/lxc_container.cpp
sed -i 's/lxc.logfile/lxc.log.file/g'                              src/anbox/container/lxc_container.cpp
sed -i 's/lxc.loglevel/lxc.log.level/g'                            src/anbox/container/lxc_container.cpp
sed -i 's/lxc.mount.fstab.auto/lxc.mount.auto/g'                   src/anbox/container/lxc_container.cpp
sed -i 's/lxc.mount.fstab.entry/lxc.mount.entry/g'                 src/anbox/container/lxc_container.cpp
sed -i 's/lxc.mount/lxc.mount.fstab/g'                             src/anbox/container/lxc_container.cpp
sed -i 's/lxc.network./lxc.net.0./g'                               src/anbox/container/lxc_container.cpp
sed -i 's/lxc.pts/lxc.pty.max/g'                                   src/anbox/container/lxc_container.cpp
sed -i 's/lxc.rebootsignal/lxc.signal.reboot/g'                    src/anbox/container/lxc_container.cpp
sed -i 's/lxc.rootfs/lxc.rootfs.path/g'                            src/anbox/container/lxc_container.cpp
sed -i 's/lxc.rootfs.path.mount/lxc.rootfs.mount/g'                src/anbox/container/lxc_container.cpp
sed -i 's/lxc.rootfs.path.options/lxc.rootfs.options/g'            src/anbox/container/lxc_container.cpp
sed -i 's/lxc.seccomp/lxc.seccomp.profile/g'                       src/anbox/container/lxc_container.cpp
sed -i 's/lxc.se_context/lxc.selinux.context/g'                    src/anbox/container/lxc_container.cpp
sed -i 's/lxc.stopsignal/lxc.signal.stop/g'                        src/anbox/container/lxc_container.cpp
sed -i 's/lxc.syslog/lxc.log.syslog/g'                             src/anbox/container/lxc_container.cpp
sed -i 's/lxc.tty/lxc.tty.max/g'                                   src/anbox/container/lxc_container.cpp
sed -i 's/lxc.tty.max.dir/lxc.tty.max/g'                           src/anbox/container/lxc_container.cpp
sed -i 's/lxc.utsname/lxc.uts.name/g'                              src/anbox/container/lxc_container.cpp

It's not working for me on Arch though. Still getting the same error.

Edit: what's going on with that '/lxc.pivotdir/d' line?

[mprogram]

@Tanner-Lin, thanks for edits, indeed one typo near the end instead of cpp.

'/lxc.{kmsg,pivotdir,rootfs.backend}/d' delete the corresponding line(s) alltogether

It must work on Arch as I ported pacman to a bleeding edge Debian-system and succesfully cross-compiled it to run it there. On my computer I only pay attention to defferences /lib/systemd/system instead of /usr/lib/systemd/system on Arch and the likes.

You may want to merge @morphis' changes from his fork/branch of latest unmerged commits manually to advance anbox's code a little bit further. Additionally don't forget to switch usr.bin.lxc AppArmor profile into complain mode with sudo aa-complain usr.bin.lxc-start, and copy/import Anbox's AppArmor profile with something like sudo /sbin/apparmor_parser -r /etc/apparmor.d/anbox-container.aa.

Let's concentrate on why it doesn't start with lxc, it truly puzzles me.

[mprogram]

Update: I have successfully compiled and also started the latest code, double-checked with lxc-start. Thanks @morphis for all the work.

[okias]

can be closed then?

[mprogram]

Surely.