iocage import fails

[2otherjobs]

Make sure to follow and check these boxes before submitting an issue! Thank you.

• [iocage version 1.2RC ] Supply iocage --version

iocage import mail
Importing dataset: mail
Importing dataset: mail/root
cannot receive: failed to read from stream
Killed

Exported from iocage on another machine, then sent to this machine using scp. Sending
seems successful.
Expected successful import.

[richardbowden]

Had the same issue where I was importing a large jail, as the current version decompresses the exported image in to memory, before sending to zfs recv. Larges images can fail due to running out or memory.

the function linked would need tobe re worked to decompress the image to say /tmp, then import from there, in a streaming manner, but not sure how that would work with popen.

iocage/iocage_lib/ioc_image.py

Line 286 in e448521

data = f.open(name).read()

I will try and make a PR over the xmas holidays to help fix this issue

as a work around you can
you may need todo zfs destroy -r zroot/iocage/jails/plex to clean up a failed import
adjust the paths for your setup

1. unzip the exported jail
2. zfs recv -F zroot/iocage/jails/plex < plex_2019-10-30
zfs recv -F zroot/iocage/jails/plex/root < plex_2019-10-30_root

[2otherjobs]

Thanks for the reply. I used znapzend to simplify the send receive process. It’s a great utility. I wanted to export a jail from one machine to another. I started by enabling passwordless root transfer over ssh. Then I created an empty zfs filesystem on the target machine within iocage: zfs create zpool/iocage/jails/plex Then I sent the pool from the source machine using znapzend. on the target machine, iocage list included the new jail. It started right up. znapzend just simplifiés the send receive process. Handles options you might forget or not know. Tom

…

Sent from my iPhone

On Nov 3, 2019, at 2:11 AM, Richard Bowden ***@***.***> wrote:  Had the same issue where I was importing a large jail, as the current version decompresses the exported image in to memory, before sending to zfs recv. Larges images can fail due to running out or memory. the function linked would need tobe re worked to decompress the image to say /tmp, then import from there. https://github.com/iocage/iocage/blob/e44852191374cd0f5a7efbd4c78ce4860c9dfe3d/iocage_lib/ioc_image.py#L286 I will try and make a PR over the xmas holidays to help fix this issue as a work around you can you may need todo zfs destroy -r zroot/iocage/jails/plex to clean up a failed import adjust the paths for your setup unzip the exported jail zfs recv -F zroot/iocage/jails/plex < plex_2019-10-30 zfs recv -F zroot/iocage/jails/plex/root < plex_2019-10-30_root — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[richardbowden]

Thanks for the reply. I used znapzend to simplify the send receive process. It’s a great utility. I wanted to export a jail from one machine to another. I started by enabling passwordless root transfer over ssh. Then I created an empty zfs filesystem on the target machine within iocage: zfs create zpool/iocage/jails/plex Then I sent the pool from the source machine using znapzend. on the target machine, iocage list included the new jail. It started right up. znapzend just simplifiés the send receive process. Handles options you might forget or not know. Tom
…
@2otherjobs
no worries, thanks for the tool tip, not heard about znapzend until you mentioned it, I now use it a lot. its great :)

[jhfoo]

@2otherjobs the zfs recv commands worked. Have no idea how to get started with znapzend. If the steps are minimal can you paste them here?

[2otherjobs]

@jhfoo
I will assume your use case is that you have an iocage jail (jail1) on server1 (s1) you want to send to server2 (s2) on FreeBSD, one time only.

1. Install znapzend on both machines using ports or pkg. Znapzend exists on GitHub and has excellent man pages. Read them. Setup per man pages.

2. Make sure root can log on using ssh from s1 to s2. I have done this using only s2's local ip addresses so root access remains otherwise very restricted. See /etc/ssh/sshd_config

3. Create a zfs filesystem on s2. I had activated iocage on s1pool and s2pool so iocage is set up on both s1 and s2:
   On s1:
   zfs list -r s1pool. gives:
   ...
   s1pool/iocage/jails/jail1
   s1pool/iocage/jails/jail1/root
   ...
   so on s2, do (if these filesets do not already exist) do:
   zfs create s2pool/iocage/jails/jail1
   zfs create s2pool/iocage/jails/jail1/root

4. Tricky part is to set up the appropriate instructions for znapzend to use. It has a subordinate utility called znapzendzetup that is used to actually create the instructions for the zfs send/receive that znapzend will perform. znapzend is actually designed to send snapshots periodically of a zfs file set like an iocage jail so I give znapzend instructions, as if the send/receive will run periodically, run it once, then delete the znapzendzetup instruction list.

5)znapzendzetup example:

znapzendzetup create --recursive
--mbuffer=/usr/local/bin/mbuffer \ <-- used by znapzend and pulled in by pkg at the beginning.
SRC '1week=>1day' s1pool/iocage/jails/jail1 . <-- SRC=source (s1) zfs fileset
DST:0 '1week=>1day' root@192.168.0.2:s2pool/iocage/jails/jail1 <--DST=destination zfs fileset

znapzendzetup will show errors if you have errors in your setup. The first part of the DST line gives the schedule for snapshot creation for repeated use, but since we plan only to send/receive once, the exact schedule does not matter. If you want to send snapshots periodically, see the man pages for details.

6. run znapzend -d --runonce=s1pool/iocage/jails/jail1 --autoCreation
   -d gives debug info.
   --runonce makes znapzend run immediately, not wait for scheduled times. Note that the name of the fileset being sent is the same as the SRC listed in the znapzendzetup command. This fileset names is the source fileset and serves as well as the name of the znapzendzetup instruction set.
   --autoCreation enables the creation of child zfs filesets on s2 necessary to complete the zfs receive. This avoids having to create them manually.

7. Since this use case is for a one-time zfs send/receive, now delete the znapzendzetup instruction:
   znapzendzetup delete s1pool/iocage/jails/jail1.

I have created this instruction set in haste. Use it at your own risk.
2otherjobs

[jhfoo]

Published the above steps here: http://ihazcodez.kungfoo.info/2020/TipsNTricks1.html

[jaredjennings]

Searching left me here instead of at #1072. This ticket appears to spring from the same issue, except that in this case we run out of memory, and in the other case we merely dip into a lot of swap.
This issue also should be fixed by #1218, which .... appears to have happened a full year after the latest release, 1.2