Sprint 52 (ending 2018-03-06) Blog Post Draft

[gilsonsouza]

Sprint 52

• This is just a draft for collecting the text for the team blog.
• Whenever you finish something worth mentioning at the blog just add a new entry
• The text will be reviewed and polished before publishing, this is just to collect
  the input from the developers. So be focused on the technical side, the wording,
  grammar, etc... can be improved later 😉

Thank you for your contribution! 👍

[joseivanlopez]

Configuring the Expert Partitioner

The Expert Partitioner recovers the "Settings" section where some general configurations can be done to influence how the Partitioner works. The first control available again is the configuration for the mount by options. Now, you can select the default value to use for mounting devices, for example, mount by path, UUID, label, etc.

And, moreover, the Partitioner now performs more checks over the mount options. For example, now the user will be warned if he is trying to mount a device by label, but the given file system label is not valid, for example because there is another file system using that label.

In future, more general configuration will be available in this "Settings" section to make the Partitioner to work just as you want.

Comment from Ancor: I believe there is an important information missing here. You can use this to change the default mount-by and then re-execute the Guided Setup in order to get a new auto-proposed layout, but now with your desired mount-by. In short, that's not only a Partitioner setting, it's a yast2-storage-ng setting that will affect upcoming operations. Somebody (Iván?) should integrate that information into the text above.

(mvidner): the popup in the screenshot has a typo, "volumem", we may need to correct it in the code.
(ivan): the type is already corrected in the code. I have updated the screenshot.

[aschnell]

Temporary mount and unmount File Systems for Resize

Unfortunately most file systems support resizing only while the file system is
mounted or while it is unmounted. Whereas doing a temporary mount is easy, a
temporary unmount is often not possible since file systems are usually is use.

libstorage-ng now inserts unmount and mount action when needed for the
specific file system. It also provides function to immediately mount or
unmount a file system so that the UI the give feedback to the user if
unmounting failed.

[gilsonsouza]

YaPI is deprecated and shouldn't be used anymore

As new features are coming with SLE 15, some modules and tools are leaving. And this time, YaPI is the tool being dropped from YaST.

YaPI was designed to sharing functionalities from YaST, but as it is not so much used and due to the complexity to maintain several interfaces always when a new feature is created, we decided that we will not maintain YaPI anymore. Although its code was not completely removed from YaST, YaPI is no longer a stable API to be used by external resources.

[wfeldt]

Take btrfs subvolume hierarchy into account when creating subvolumes

btrfs subvolumes are rather delicate beasts and great care has to be taken to do things right.

For one, subvolumes are organized like a directory hierarchy. And you cannot create new subvolumes just anywhere but only in free 'leaf' positions. That is, neither another subvolume with the same name nor a directory with this name must already exists.

In other words, if (for example) a subvolume foo/bar/xxx already exists you can no longer create a subvolume foo/bar.

See what happens:

# you must mount the btrfs file system first
~ mount /dev/sdb5 /mnt

~ btrfs subvolume list -tap /mnt
ID      gen     parent  top level       path
--      ---     ------  ---------       ----
257     8       5       5               foo
258     8       257     257             <FS_TREE>/foo/bar/xxx

~ btrfs subvolume create /mnt/foo/bar
ERROR: target path already exists: /mnt/foo/bar

That's because there's already a directory foo/bar. You have two options at this point:

• delete subvolume foo/bar/xxx and the directory foo/bar

or, if you don't want to loose the data of this subvolume

• rename temporarily and move it back later

Let's move the existing subvolume out of the way (yes, with the regular mv command) and then create the new one:

~ mv /mnt/foo/bar /mnt/foo/old_bar
~ btrfs subvolume create /mnt/foo/bar
Create subvolume '/mnt/foo/bar'
~ mv /mnt/foo/old_bar/xxx /mnt/foo/bar/xxx
~ rmdir /mnt/foo/old_bar

~ btrfs subvolume list -tap /mnt
ID      gen     parent  top level       path
--      ---     ------  ---------       ----
257     9       5       5               foo
258     8       259     259             <FS_TREE>/foo/bar/xxx
259     9       257     257             <FS_TREE>/foo/bar

That's rather a complicated stunt and you even might have valuable data in the old foo/bar directory that could get lost in the process. So the YaSTpartitioner refuses to do all this.

Instead, it recognizes the situation and shows a hint to the user:

Of course it's all different if you are about to format the btrfs filesystem anyway. Then the YaST partitioner will simply create the subvolumes in the correct order and everything is fine. You won't get the warning dialog in this case.

[ancorgs]

Better handling of unformatted DASD and other unavailable devices

Direct-access storage devices (DASD) are the most common storage devices in s390 mainframes. When compared to common hard disks, they are special in several ways. One of them is that they need to be formatted at low-level (to not be confused with the usual meaning of "format" related to creating a file system) in order to be used by the operating system.

We got a bug report for the pre-release of SLE15 because the installer was crashing when trying to use an unformatted DASD as part of the automatic partitioning proposal (i.e. the Guided Setup). So we instructed YaST to ignore such devices, not only in the automatic proposal but also in the Partitioner. It makes very little sense to list devices in the Partitioner that cannot be manipulated in any way. That's consistent with the SLE15 approach for other "untouchable" devices, like the hard disks that are part of a BIOS-defined RAID or the individual connections of a multipath device.

In other words, the "Hard Disks" section of the Partitioner only shows disks that can be indeed used as disks (e.g. can be partitioned) and, unlike previous SLE15 pre-releases, now unformatted DASDs are not longer in that list. As always, the user can still go a couple of steps back in the installation process and perform a low-level format the DASD in order to make them appear.

[joseivanlopez]

Partitioner: Cloning disks

Cloning a disk was one of that kind of surprising features the pre storage-ng Partitioner offered. Imagine you have a disk with some partitions and you would like to have the same partition scheme in another disk (or even in several disks). This is very common, for example, when you are creating a MD RAID setup, where it is useful to have the disks partitioned in the same way. To avoid the annoying work of creating all disk partitions manually, the new Expert Partitioner allows again to clone a disk by using the Expert button.

When you are cloning a disk, you can select all the devices where you want to clone the partition scheme of the current disk. Only suitable devices for cloning will be offered, that is, you will not see in the list disks without enough size or different topology.

After selecting in which disks you want to clone, a confirmation message will be presented and you will be warned about all devices that will be deleted before performing the disk cloning. In case you accept, you will have exactly the same partitions in all selected devices, including gaps between partitions.

[schubi2]

AutoYaST Upgrade

For SLES15 RC1 upgrading already installed old versions (SLES11,SLES12) via AutoYaST is now available.

[lslezak]

Offline Upgrade from SLE11 Products

In the previous post we announced support for the offline migration from SLE12 to SLE15 via SCC. In this sprint we focused on upgrading from SLE11 products to SLE15.

The main difference (from the registration point of view) between SLE11 and SLE12/SLE15 is that SLE11 uses the old Novell Customer Center (NCC) for registration, the newer products use the SUSE Customer Center (SCC).

However, the SCC server knows the systems registered in NCC (there is some kind of automatic synchronization between the systems) so it is possible to only use the SCC server during migration without any interaction with the old NCC. That makes the transition much easier for us and you do not have to register the system again.

The changes in YaST were mainly related to handling the different configuration files and there were some small improvements and adjustments. The most difficult part was testing as we found some issues on the SCC side and there are some difficulties regarding the synchronization between NCC and SCC, see below.

In the end we could migrate a SLES11 system to SLES15 via SCC. There were some issues related to package dependencies but these are not related to YaST or SCC, probably some RPM packages will need to be fixed.

SLE11 Migration Notes

When migrating from SLE11 system there some specific issues you should be aware:

• The SLE11 systems registered using the internal SUSE registration keys (belonging to the Novell Inc organization) are synchronized every 24 hours. That means it might take up to 24 hours after registering a new SLE11 system to make it upgradable to SLE15 via SCC. If the registration has not been synchronized from NCC yet you will see the "Invalid credentials" error during migration.
• For the customers there is a trick to log into the SCC Web UI, then the registrations should be sychronized in few minutes. But this does not work for the SUSE keys.
• Migration from SLE12 is not affected by this issue because SLE12 already uses SCC and the data synchronization from NCC is not involved in the migration process.

Generic Notes

These notes apply when upgrading from both SLE11 or SLE12:

• Because SLE15 is in Beta stage your registration key needs to be entitled for the Beta products, if it is not you will see "No migation found" error in YaST. Either join the SUSE Beta program or just wait until the SLE15 product is released and the migration is enabled.
• Once the system is migrated it cannot be migrated again (even if you rollback the migrated system back using btrfs snapshot or a virtual machine snapshot), the SCC server will still contain the new SLE15 registration.

[lslezak]

Package Installation

We got a bug report that when installing system from a NFS repository then the installed system cannot access it later.

The problem is that for accessing the NFS repositories (and Samba/CIFS as well) you need an additional package which can mount such file systems. And these packages are not included in the minimal installation.

That means you cannot access the repository later which is a quite unfortunate situation. You get into the chicken/egg problem - you need to install a package to access the repository but to access the repository you need to have the package already installed...

During debugging it turned out that the problem was caused by using a dropped value from the /etc/install.inf file. In the past linuxrc wrote the type of the installation repository there, but that's not written anymore.

The fix was to evaluate all used repositories and check if any of them is located on an NFS or Samba/CIFS system. For direct repositories this is simple, the repository URL starts with nfs:// or cifs://. It's a bit more complicated when using an ISO image, it starts with iso:// and the base location is set in the ?url= query parameter.

With this enhancement we also fixed the issue when installing from DVD but using an additional repository located on NFS.

[lslezak]

Fixing Text Domains in YaST

Currently we are getting quite a lot of bugs about missing translations in YaST. It turned out that some bugs were caused by a missing textdomain statement in the YaST code. When it is missing YaST does not extracts the translatable strings from that file. Then obviously the translators cannot translate the texts.

The YaST script printed a warning in that case, but that you can easily overlook. The reason for a warning is that the check is not perfect and it reports quite a lot of false positives. (It complains about a missing textdomain but it is not required in reality.)

Fixing the script would be too difficult so we took the other approach, we added the harmless textdomain texts also to files which strictly did not require it. Then we could change the warning to error and stop the script with a failure.

Now the more strict check is enabled in the continuous integration (Travis) so we should get a failure earlier, before the change actually hits the build service, avoiding bug reports later.

[teclator]

Preserve the DHCP client-id after installation

Since SLE15, wicked will use the rfc4361 client-id. The installer has been adapted copying the created identifies (duid + daid) from the ins-sys to the new installed system obtaining the same IP (when it's possible) used during installation after the first reboot.