Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

existing ezjail conversion process omits many symlink - let's fix that #730

Open
dlangille opened this Issue Dec 17, 2018 · 27 comments

Comments

Projects
None yet
2 participants
@dlangille
Copy link

dlangille commented Dec 17, 2018

Following https://iocage.readthedocs.io/en/latest/install.html#migrating-from-ezjail-to-iocage the resulting jail works, but it is missing many symlinks. I think I know a way to improve the process.

At present, this issue exists just as a placeholder / discussion point.

My findings:

No action is requested.

Comments and suggestions on the approach are welcome.

Current status: I've identified a possible two step solution and will test it soon:

  • copy everything and use --exclude-from=FILE to ignore basejail symlinks
  • copy only the symlink directories using --include-from=FILE and --copy-links to populate the basejail directories (creating real directories, not symlinks)

Make sure to follow and check these boxes before submitting an issue! Thank you.

@skarekrow

This comment has been minimized.

Copy link
Member

skarekrow commented Dec 17, 2018

Interesting, I think we can do this simpler.

Here's my suggestions:
1 - Create a basejail with iocage of specified RELEASE
2 - Create the same name jail as the target (step 1 can be skipped if shared) as a basejail and start it
3 - Use that list from rsync and replace /usr/jails with /the/new/jail/mount/root
4 - Profit??

I think that could cover the same usecase with a lot less overhead. Thoughts?

@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 17, 2018

Is this what you mean?

1 - iocage create --basejail -r 12.0-RELEASE -n MyBaseJail
2 - iocage create --basejail -r 12.0-RELEASE -n snapshots (the ezjail jail being converted)
3 - The list from rsync being the list of missing symlinks?

@skarekrow

This comment has been minimized.

Copy link
Member

skarekrow commented Dec 17, 2018

Yes

@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 17, 2018

This assumes we have already completed the steps in "Migrating from Ezjail to Iocage"?

@skarekrow

This comment has been minimized.

Copy link
Member

skarekrow commented Dec 17, 2018

How so? This is all preparatory. You would rsync from the ezjail jail to the iocage jail. Running the iocage jail in step 2 is to make sure the basejail targets for the symlinks are mounted.

@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 17, 2018

I see. It wasn't clear this was all prep.

@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 17, 2018

OK, let's try it:

[dan@knew:~] $ sudo iocage create --basejail -r 12.0-RELEASE -n MyBaseJail
Fetching: 12.0-RELEASE

Downloading : MANIFEST [####################] 100% 0Mbit/s
Downloading : base.txz [####################] 100%  21.9Mbit/ss
Downloading : lib32.txz [####################] 100%  22.05Mbit/s
 22.05Mbit/sTraceback (most recent call last):
  File "/usr/local/bin/iocage", line 10, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/iocage/cli/create.py", line 143, in cli
    basejail=basejail, empty=empty)
  File "/usr/local/lib/python3.6/site-packages/iocage/lib/iocage.py", line 583, in create
    exit_on_error=self.exit_on_error).fetch_release()
  File "/usr/local/lib/python3.6/site-packages/iocage/lib/ioc_fetch.py", line 234, in fetch_release
    rel = self.fetch_http_release(eol, _list=_list)
  File "/usr/local/lib/python3.6/site-packages/iocage/lib/ioc_fetch.py", line 515, in fetch_http_release
    self.fetch_download(self.files)
  File "/usr/local/lib/python3.6/site-packages/iocage/lib/ioc_fetch.py", line 720, in fetch_download
    r.raise_for_status()
  File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://download.freebsd.org/ftp/releases/amd64/12.0-RELEASE/doc.txz

Ahh, yes, let's try devel:

[dan@knew:~] $ sudo iocage ge create --basejail -r 12.0-RELEASE -n MyBaseJail

Host: 11.2-RELEASE is not greater than target: 12.0-RELEASE
This is unsupported.

Yes, that's better, but I was wrong.

[dan@knew:~] $ sudo iocage create --basejail -r 11.2-RELEASE -n MyBaseJail
Fetching: 11.2-RELEASE

Downloading : MANIFEST [####################] 100% 
Downloading : base.txz [####################] 100% 
Downloading : lib32.txz [####################] 100% 
Downloading : doc.txz [####################] 100% 
Downloading : src.txz [####################] 100% 
Extracting: base.txz... 
Extracting: lib32.txz... 
Extracting: doc.txz... 
Extracting: src.txz... 

* Updating 11.2-RELEASE to the latest patch level... 
Traceback (most recent call last):
  File "/usr/local/bin/iocage", line 10, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/iocage_cli/create.py", line 169, in cli
    thickconfig=thickconfig)
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/iocage.py", line 551, in create
    silent=self.silent
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_fetch.py", line 231, in fetch_release
    rel = self.fetch_http_release(eol, _list=_list)
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_fetch.py", line 516, in fetch_http_release
    self.fetch_update()
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_fetch.py", line 916, in fetch_update
    su_env=fetch_env
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_exec.py", line 104, in __enter__
    close_fds=True, bufsize=0, env=self.su_env
  File "/usr/local/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/usr/local/lib/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: '/tmp/tmpx3ow361l'
@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 17, 2018

Fixed it:

[dan@knew:~] $ sudo rm /tmp/tmpx3ow361l

[dan@knew:~] $ iocage list
You need to be root to convert the configurations to the new format!
+-----+-----------+---------+---------+-----+
| JID |   NAME    |  STATE  | RELEASE | IP4 |
+=====+===========+=========+=========+=====+
| -   | snapshots | CORRUPT | N/A     | N/A |
+-----+-----------+---------+---------+-----+

[dan@knew:~] $ sudo iocage ge create --basejail -r 11.2-RELEASE -n MyBaseJail
Missing default rc.conf, creating it
MyBaseJail successfully created!



[dan@knew:~] $ iocage list
You need to be root to convert the configurations to the new format!
+-----+------------+---------+--------------+-----+
| JID |    NAME    |  STATE  |   RELEASE    | IP4 |
+=====+============+=========+==============+=====+
| -   | MyBaseJail | down    | 11.2-RELEASE | -   |
+-----+------------+---------+--------------+-----+
| -   | snapshots  | CORRUPT | N/A          | N/A |
+-----+------------+---------+--------------+-----+
[dan@knew:~] $ 

@skarekrow

This comment has been minimized.

Copy link
Member

skarekrow commented Dec 17, 2018

devel must be older then master, I'm not the ports committer :P

root@freenas[~]# /root/iocage/iocage clean -af
Cleaning iocage/templates
Cleaning iocage/releases
Cleaning iocage/log
Cleaning iocage/jails
Cleaning iocage/images
Cleaning iocage/download
Cleaning iocage
All iocage datasets have been destroyed.
root@freenas[~]# /root/iocage/iocage create --basejail -r 11.2-RELEASE -n foo
Creating tank/iocage/download
Creating tank/iocage/images
Creating tank/iocage/jails
Creating tank/iocage/log
Creating tank/iocage/releases
Creating tank/iocage/templates
Fetching: 11.2-RELEASE

Downloading: MANIFEST [####################] 100% 
Downloading: base.txz [####################] 100% 
Downloading: lib32.txz [####################] 100% 
Downloading: src.txz [####################] 100% 
Extracting: base.txz... 
Extracting: lib32.txz... 
Extracting: src.txz... 
'/tmp': not a ZFS filesystem

* Updating 11.2-RELEASE to the latest patch level... 
Looking up update.FreeBSD.org mirrors... 3 mirrors found.
Fetching public key from update4.FreeBSD.org... done.
Fetching metadata signature for 11.2-RELEASE from update4.FreeBSD.org... done.
Fetching metadata index... done.
Fetching 2 metadata files... done.
Inspecting system... done.
Preparing to download files... done.
Fetching 81 patches.....10....20....30....40....50....60....70....80 done.
Applying patches... done.

/usr/share/zoneinfo/Asia/Shanghai
/usr/share/zoneinfo/Asia/Tokyo
/usr/share/zoneinfo/Europe/Volgograd
/usr/share/zoneinfo/Pacific/Easter
/usr/share/zoneinfo/Pacific/Fiji
/usr/share/zoneinfo/Pacific/Honolulu
/usr/share/zoneinfo/zone.tab
/usr/src/contrib/tzdata/CONTRIBUTING
/usr/src/contrib/tzdata/Makefile
/usr/src/contrib/tzdata/NEWS
-- snip --
/usr/src/sys/netinet6/in6_proto.c
/usr/src/sys/netinet6/ip6_var.h
/usr/src/sys/netinet6/sctp6_usrreq.c
/usr/src/sys/netinet6/udp6_usrreq.c
/usr/src/sys/x86/include/specialreg.h
/usr/src/usr.sbin/bhyve/fwctl.c
Installing updates... done.
foo successfully created!

Do you have /tmp in a dataset? Or have strict permissions? That file is freebsd-update, just named differently.

EDIT: Ignore the couple bugs in the above output, they will be fixed

@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 17, 2018

Step 2:

[dan@knew:~] $ sudo iocage create --basejail -r 11.2-RELEASE -n snapshots2
snapshots2 successfully created!

[dan@knew:~] $ iocage list
You need to be root to convert the configurations to the new format!
+-----+------------+---------+--------------+-----+
| JID |    NAME    |  STATE  |   RELEASE    | IP4 |
+=====+============+=========+==============+=====+
| -   | MyBaseJail | down    | 11.2-RELEASE | -   |
+-----+------------+---------+--------------+-----+
| -   | snapshots  | CORRUPT | N/A          | N/A |
+-----+------------+---------+--------------+-----+
| -   | snapshots2 | down    | 11.2-RELEASE | -   |
+-----+------------+---------+--------------+-----+
[dan@knew:~] $ 

@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 17, 2018

So this step, unspecified, is not right:

[dan@knew:~] $ sudo rsync -a --copy-links /usr/jails/snapshots/ /iocage/jails/snapshots2/root/
symlink has no referent: "/usr/jails/snapshots/sys"
symlink has no referent: "/usr/jails/snapshots/basejail/usr/include/c++/v1/tr1/__tuple_03"
symlink has no referent: "/usr/jails/snapshots/basejail/usr/lib/libreadline.so"
symlink has no referent: "/usr/jails/snapshots/basejail/usr/lib/libkvm.so"
symlink has no referent: "/usr/jails/snapshots/basejail/usr/lib/libcrypto.so"
symlink has no referent: "/usr/jails/snapshots/basejail/usr/lib/libipx.so"
rsync: readlink_stat("/usr/jails/snapshots/basejail/usr/lib/libcam.so") failed: Too many levels of symbolic links (62)
symlink has no referent: "/usr/jails/snapshots/basejail/usr/libexec/ld-elf32.so.1"
symlink has no referent: "/usr/jails/snapshots/basejail/var/db/etcupdate/current/sys"
symlink has no referent: "/usr/jails/snapshots/basejail/var/db/etcupdate/current/etc/rmt"
symlink has no referent: "/usr/jails/snapshots/basejail/var/db/etcupdate/current/etc/unbound"
...

@skarekrow

This comment has been minimized.

Copy link
Member

skarekrow commented Dec 17, 2018

Neither jail is running in your list, which would be a requirement. I didn't say to run rsync directly, I said to take the list from it and sed out the paths. Perhaps do a dry rsync to sed and use cp or something. Not sure

@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 17, 2018

This list will vary from jail to jail and will be dependent upon what is installed in the jail.

The list is generated from the rsync. Thus, it must be run by each user, correct? For each jail, right?

@skarekrow

This comment has been minimized.

Copy link
Member

skarekrow commented Dec 17, 2018

I would assume so, that's why I suggested a dryrun of rsync.

@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 17, 2018

This is simpler? ;)

@skarekrow

This comment has been minimized.

Copy link
Member

skarekrow commented Dec 17, 2018

Hahaah perhaps not!

@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 20, 2018

I think this is progress.

The following is the difference in symlinks between the original ezjail jail and the copied iocage jail.

Let's explain some of those away:

  • some are symlink from newjail, and are no longer symlinks because it is now a thick jail
  • the /dev/ entries are present only in a running system
  • /etc/unbound did not exist in the old jail, but the new jail has it (unmerged 11.2 changes?)
  • /var/db/etcupdate was not present in the old jail (new feature?)
  • /usr/tests seems to be new
[dan@knew:~/tmp] $ diff -ruN  snapshots.iocage.sorted snapshots.ezjail.sorted.without.basejail 
--- snapshots.iocage.sorted	2018-12-20 22:25:04.112550000 +0000
+++ snapshots.ezjail.sorted.without.basejail	2018-12-20 22:27:17.497066000 +0000
@@ -1,11 +1,25 @@
+/bin
+/dev/stderr
+/dev/stdin
+/dev/stdout
+/dev/urandom
 /etc/aliases
 /etc/namedb
 /etc/rmt
 /etc/ssl/cert.pem
 /etc/termcap
-/etc/unbound
 /home
+/lib
+/libexec
+/rescue
+/sbin
 /sys
+/usr/bin
+/usr/include
+/usr/lib
+/usr/lib32
+/usr/libdata
+/usr/libexec
 /usr/local/bin/2to3
 /usr/local/bin/2to3-2
 /usr/local/bin/idle
@@ -1452,9 +1466,9 @@
 /usr/local/man/man8/pkg-unlock.8.gz
 /usr/local/man/man8/sudoedit.8.gz
 /usr/local/openssl/cert.pem.sample
-/usr/tests/lib/libc/tls/libh_tls_dynamic.so
-/usr/tests/lib/libthr/dlopen/h_pthread_dlopen.so
-/usr/tests/libexec/rtld-elf/libpythagoras.so
+/usr/ports
+/usr/sbin
+/usr/share
 /var/cache/pkg/aspell-0.60.6.1_5.txz
 /var/cache/pkg/aspell-0.60.6.1_6.txz
 /var/cache/pkg/aspell-0.60.6.1_7.txz
@@ -1719,35 +1733,6 @@
 /var/cache/pkg/tmux-2.6.txz
 /var/cache/pkg/tmux-2.6_1.txz
 /var/cache/pkg/tmux-2.7.txz
-/var/db/etcupdate/current/etc/aliases
-/var/db/etcupdate/current/etc/rmt
-/var/db/etcupdate/current/etc/termcap
-/var/db/etcupdate/current/etc/unbound
-/var/db/etcupdate/current/sys
-/var/db/etcupdate/current/usr/share/man/en.ISO8859-1/man1
-/var/db/etcupdate/current/usr/share/man/en.ISO8859-1/man2
-/var/db/etcupdate/current/usr/share/man/en.ISO8859-1/man3
-/var/db/etcupdate/current/usr/share/man/en.ISO8859-1/man4
-/var/db/etcupdate/current/usr/share/man/en.ISO8859-1/man5
-/var/db/etcupdate/current/usr/share/man/en.ISO8859-1/man6
-/var/db/etcupdate/current/usr/share/man/en.ISO8859-1/man7
-/var/db/etcupdate/current/usr/share/man/en.ISO8859-1/man8
-/var/db/etcupdate/current/usr/share/man/en.ISO8859-1/man9
-/var/db/etcupdate/current/usr/share/man/en.ISO8859-15
-/var/db/etcupdate/current/usr/share/man/en.UTF-8/man1
-/var/db/etcupdate/current/usr/share/man/en.UTF-8/man2
-/var/db/etcupdate/current/usr/share/man/en.UTF-8/man3
-/var/db/etcupdate/current/usr/share/man/en.UTF-8/man4
-/var/db/etcupdate/current/usr/share/man/en.UTF-8/man5
-/var/db/etcupdate/current/usr/share/man/en.UTF-8/man6
-/var/db/etcupdate/current/usr/share/man/en.UTF-8/man7
-/var/db/etcupdate/current/usr/share/man/en.UTF-8/man8
-/var/db/etcupdate/current/usr/share/man/en.UTF-8/man9
-/var/db/etcupdate/current/usr/share/nls/POSIX
-/var/db/etcupdate/current/usr/share/nls/en_US.US-ASCII
-/var/db/etcupdate/current/usr/share/openssl/man/en.ISO8859-1/man1
-/var/db/etcupdate/current/usr/share/openssl/man/en.ISO8859-1/man3
-/var/db/etcupdate/current/usr/share/openssl/man/en.ISO8859-15
 /var/db/etcupdate/old/etc/aliases
 /var/db/etcupdate/old/etc/namedb
 /var/db/etcupdate/old/etc/rmt
[dan@knew:~/tmp] $ 

The process was:

  1. remove the symlink /basejajil in the host, created as part of the existing conversion instructions
  2. iocage create --thick -r 11.2-RELEASE -n snapshots
  3. sudo rsync -a --links --exclude-from=/usr/home/dan/tmp/jails-newjail-symlinks.src.chrooted /usr/jails/snapshots/ /iocage/jails/snapshots2/root/

Where the list of things to not rsync is:

[dan@knew:~/tmp] $ cat /usr/home/dan/tmp/jails-newjail-symlinks.src.chrooted
/basejail
/usr/ports
/usr/libexec
/usr/lib
/usr/libdata
/usr/sbin
/usr/bin
/usr/share
/usr/include
/usr/lib32
/basejail
/bin
/sbin
/rescue
/lib
/libexec
/boot

This list is obtained with this command:

sudo find /usr/jails/newjail -type l -exec ls -d {} \; > ~/tmp/jails-newjail-symlinks

Then the prefix '/usr/jails/newjail' is removed from each of those lines.

@skarekrow

This comment has been minimized.

Copy link
Member

skarekrow commented Dec 20, 2018

Sounds like a good script idea hint ;)

@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 20, 2018

I have 66 jails on 6 hosts to convert.

Yes, there will be a script of some type.

@skarekrow

This comment has been minimized.

Copy link
Member

skarekrow commented Dec 20, 2018

Heh, make sure you host it for other users!

@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 21, 2018

FYI:

$ sudo iocage create --thick -r 11.2-RELEASE -n snapshots3
Error: no such option: --thick  (Possible options: --thickconfig, --thickjail)

There is no reference to thinkconfig in man iocage for Version 1.0 RC 2018/11

@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 21, 2018

Existing concerns about the script:

  • should be be upgrading the iocage release to the same patch level as the ezjail jail?
  • e.g. 11.2-RELEASE-p6 vs 11.2-RELEASE

Please review and test: https://github.com/dlangille/thin_to_thick

@skarekrow

This comment has been minimized.

Copy link
Member

skarekrow commented Dec 21, 2018

Nice! In your example that's important, as that's not longer a patch level difference but a world difference. I think it's a fair assumption that the users will create the proper RELEASE targets (if it's their first time with iocage then those will be on the latest patch level too) for each of their targets.

@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 21, 2018

I have updated my example, I meant for the release numbers to be the same.

However, will the destination jail have issues if the patch levels differ?

@skarekrow

This comment has been minimized.

Copy link
Member

skarekrow commented Dec 21, 2018

That's a pretty wide net to cast an opinion on, unfortunately. I would think it would be pretty minor differences as FreeBSD would typically delay major things for a world change. I wouldn't lose much sleep over it. But if you do, you can always issue an iocage update ALL at the end of your script (with 1.0, commit is here #748) the hope is to release 1.0 today, so it may be worth adding that to the script.

@dlangille

This comment has been minimized.

Copy link
Author

dlangille commented Dec 21, 2018

$ sudo iocage set ip4_addr="10.55.0.70" snapshots  
Property: ip4_addr has been updated to 10.55.0.70

Have you considered the sysrc output syntax?

It shows before and after.

 $ sudo sysrc things=stuff
things:  -> stuff

[dan@knew:/iocage/jails] $ sudo sysrc things=more 
things: stuff -> more
[dan@knew:/iocage/jails] $ sudo sysrc -x things     
[dan@knew:/iocage/jails] $ 

Why? So we know what it was before and that's often useful when adding/updating.

I'll raise an issue if you like.

@skarekrow

This comment has been minimized.

Copy link
Member

skarekrow commented Dec 22, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.