Merge remote-tracking branch 'origin/master' into dynamic_profile_doc

README.md

@@ -122,6 +122,7 @@ Additional Documentation
 ========================
 
 * {file:doc/profile_handling.md Profile Handling}
+* {file:doc/profile_fetching.md Profile Fetching}
 * {file:doc/validation.md Profile Validation}
 * {file:doc/error_reporting.md Error Reporting}
 * {file:doc/network_scenarios.md Network Scenarios}

doc/profile_fetching.md

@@ -0,0 +1,64 @@
+# Profile Fetching
+
+Fetching the AutoYaST profile is not a trivial process. Most of the complexity comes from the
+rules/classes mechanism that, suprisingly, will come into play even if no rules or classes has been
+defined.
+
+Before reading the rest of this document, it is recommended to have a look to the [Rules and
+Classes](https://documentation.suse.com/sles/15-SP2/single-html/SLES-autoyast/#rulesandclass)
+chapter in the AutoYaST guide.
+
+## The Process 
+
+The {Yast::ProfileLocationClass#Process ProfileLocationClass#Process} method drives the fetching process,
+although it cooperates with other modules like {Yast::AutoinstConfigClass AutoinstConfig} or
+{Yast::AutoInstallRulesClass AutoInstallRules}.
+
+As a first step, {Yast::ProfileLocationClass#Process ProfileLocationClass#Process} reads the
+profile's location from the {Yast::AutoinstConfigClass AutoinstConfig} module and depending on
+whether the URL points to a file or a directory, it behaves in a slightly different way.
+
+* If it is a file, it just downloads the file and instructs the {Yast::AutoInstallRulesClass
+  AutoInstallRules} module to use only that file. See {Yast::AutoInstallRulesClass#CreateFile
+  AutoInstallRulesClass#CreateFile}.
+* If it is a directory, it asks {Yast::AutoInstallRulesClass AutoInstallRules} to process the rules
+  that are supposed to live in the `rules/rules.xml`, under the given directory. See
+  {Yast::AutoInstallRulesClass#Read}.
+
+In both cases, the {Yast::AutoInstallRulesClass AutoInstallRules} module comes into play and
+generates the final `autoinst.xml`. This logic is implemented in the {Yast::AutoInstallRulesClass
+AutoInstallRulesClass#Process} method, which takes care of:
+
+* Merging the files according to the rules to generate the profile, even if just one file was given.
+* Processing the classes. If any class definition is found, their definitions will be loaded and
+  merged into the final profile.
+
+## Parsing AutoYaST URLs
+
+The {Yast::AutoinstConfigClass AutoinstConfig} module is the responsible, among other things, for
+holding the location of the profile. Once the module is imported, it reads the URL from the
+`install.inf` directory and extracts the relevant information into a set of separated components
+(`scheme`, `host`, etc.).
+
+The URL of the profile can point to a file (like `http://example.net/tumbleweed.xml`) or to a
+directory (`http://example.net/profiles/`). The trailing slash is important. 
+
+## Encrypted Profiles
+
+AutoYaST supports PGP-encrypted profiles. Such a scenario is handled in the
+{Yast::ProfileLocationClass#Process ProfileLocationClass#Process} method. However, it looks like the
+{Yast::ProfileClass#ReadXML ProfileClass#ReadXML} implements its own logic to decrypt profiles, so
+extracting this logic to a new class might be the right thing to do.
+
+## Default Rules
+
+In case that not suitable rules are found (because the `rules/rules.xml` file does not exist),
+AutoYaST uses a set of default rules. These rules are based in the host ID (the IP in hex format)
+and the MAC address of the system. Check the {Yast::AutoInstallRulesClass#CreateDefault
+AutoinstallRulesClass#CreateDefault} method for further details.
+
+## Additional Notes
+
+Almost the {Yast::AutoinstConfigClass AutoinstConfig} is responsible for parsing the AutoYaST URL,
+some further processing is performed in the {Yast::ProfileLocationClass ProfileLocation} method when
+dealing with `relurl` and `file` schemas.

package/autoyast2.changes

@@ -1,3 +1,24 @@
+-------------------------------------------------------------------
+Wed Sep  2 15:07:52 UTC 2020 - Imobach Gonzalez Sosa <igonzalezsosa@suse.com>
+
+- Recognize installed_product and installed_product_version as
+  legal elements of rules.xml files (boo#1176089).
+- 4.3.43
+
+-------------------------------------------------------------------
+Wed Sep  2 14:58:25 UTC 2020 - Josef Reidinger <jreidinger@suse.com>
+
+- Add to erb templates more helpers (bsc#1175735)
+
+-------------------------------------------------------------------
+Tue Sep  1 11:32:48 UTC 2020 - Imobach Gonzalez Sosa <igonzalezsosa@suse.com>
+
+- Use <script> elements instead of <listentry> when exporting the
+  <postpartitioning-scripts> section (related to bsc#1175714).
+- Saving log files of postpartitioning-scripts (bsc#1145269)
+  (schubi@suse.de).
+- 4.3.42
+
 -------------------------------------------------------------------
 Thu Aug 27 13:20:00 UTC 2020 - Imobach Gonzalez Sosa <igonzalezsosa@suse.com>
 

package/autoyast2.spec

@@ -22,7 +22,7 @@
 %endif
 
 Name:           autoyast2
-Version:        4.3.41
+Version:        4.3.43
 Release:        0
 Summary:        YaST2 - Automated Installation
 License:        GPL-2.0-only

src/autoyast-rnc/rules.rnc

@@ -50,6 +50,8 @@ y2_match_to =
     | hostid
     | karch
     | linux
+    | installed_product
+    | installed_product_version
     | mac
     | memsize
     | network
@@ -59,29 +61,30 @@ y2_match_to =
     | totaldisk
     | xserver
 
-arch           = element arch            { MAP, (match_text & match_type?) }
-board          = element board           { MAP, (match_text & match_type?) }
-board_vendor   = element board_vendor    { MAP, (match_text & match_type?) }
-custom1        = element custom1         { MAP, (match_text & match_type? & script) }
-custom2        = element custom2         { MAP, (match_text & match_type? & script) }
-custom3        = element custom3         { MAP, (match_text & match_type? & script) }
-custom4        = element custom4         { MAP, (match_text & match_type? & script) }
-custom5        = element custom5         { MAP, (match_text & match_type? & script) }
-disksize       = element disksize        { MAP, (match_text & match_type?) }
-domain         = element domain          { MAP, (match_text & match_type?) }
-haspcmica      = element haspcmica       { MAP, (match_text & match_type?) }
-hostaddress    = element hostaddress     { MAP, (match_text & match_type?) }
-hostid         = element hostid          { MAP, (match_text & match_type?) }
-karch          = element karch           { MAP, (match_text & match_type?) }
-linux          = element linux           { MAP, (match_text & match_type?) }
-mac            = element mac             { MAP, (match_text & match_type?) }
-memsize        = element memsize         { MAP, (match_text & match_type?) }
-network        = element network         { MAP, (match_text & match_type?) }
-others         = element others          { MAP, (match_text & match_type?) }
-product        = element product         { MAP, (match_text & match_type?) }
-product_vendor = element product_vendor  { MAP, (match_text & match_type?) }
-totaldisk      = element totaldisk       { MAP, (match_text & match_type?) }
-xserver        = element xserver         { MAP, (match_text & match_type?) }
+arch                      = element arch                      { MAP, (match_text & match_type?) }
+board                     = element board                     { MAP, (match_text & match_type?) }
+board_vendor              = element board_vendor              { MAP, (match_text & match_type?) }
+custom1                   = element custom1                   { MAP, (match_text & match_type? & script) }
+custom2                   = element custom2                   { MAP, (match_text & match_type? & script) }
+custom3                   = element custom3                   { MAP, (match_text & match_type? & script) }
+custom4                   = element custom4                   { MAP, (match_text & match_type? & script) }
+custom5                   = element custom5                   { MAP, (match_text & match_type? & script) }
+disksize                  = element disksize                  { MAP, (match_text & match_type?) }
+domain                    = element domain                    { MAP, (match_text & match_type?) }
+hostaddress               = element hostaddress               { MAP, (match_text & match_type?) }
+hostid                    = element hostid                    { MAP, (match_text & match_type?) }
+installed_product         = element installed_product         { MAP, (match_text & match_type?) }
+installed_product_version = element installed_product_version { MAP, (match_text & match_type?) }
+karch                     = element karch                     { MAP, (match_text & match_type?) }
+linux                     = element linux                     { MAP, (match_text & match_type?) }
+mac                       = element mac                       { MAP, (match_text & match_type?) }
+memsize                   = element memsize                   { MAP, (match_text & match_type?) }
+network                   = element network                   { MAP, (match_text & match_type?) }
+others                    = element others                    { MAP, (match_text & match_type?) }
+product                   = element product                   { MAP, (match_text & match_type?) }
+product_vendor            = element product_vendor            { MAP, (match_text & match_type?) }
+totaldisk                 = element totaldisk                 { MAP, (match_text & match_type?) }
+xserver                   = element xserver                   { MAP, (match_text & match_type?) }
 
 match_type =
 element match_type { STRING_ATTR, ("greater" | "exact" | "lower" | "range" | "regex") }

src/include/autoinstall/xml.rb

@@ -51,6 +51,7 @@ def profileSetup
         "post-scripts"             => "script",
         "chroot-scripts"           => "script",
         "init-scripts"             => "script",
+        "postpartitioning-scripts" => "script",
         "local_domains"            => "domains",
         "masquerade_other_domains" => "domain",
         "masquerade_users"         => "masquerade_user",

src/lib/autoinstall/y2erb.rb

@@ -6,18 +6,114 @@ class Y2ERB
     def self.render(path)
       env = TemplateEnvironment.new
       template = ERB.new(File.read(path))
-      template.result(env.public_binding)
+      template.result(env.public_bindings) # intentional send as it is private method
     end
 
     class TemplateEnvironment
+      include Yast::Logger
+
       def hardware
         @hardware ||= Yast::SCR.Read(Yast::Path.new(".probe"))
       end
 
-      # expose method bindings
-      def public_binding
+      # @return [Array<Hash>] list of info about disks. Info contain:
+      #   `:vendor` of disk
+      #   `:device` kernel name of device
+      #   `:udev_names` list of udev names for given disk
+      #   `:model` model name from sysfs
+      #   `:serial` serial number of disk
+      #   `:size` disk size in bytes [Integer]
+      def disks
+        return @disks if @disks
+
+        @disks = []
+        hardware["disk"].each do |disk|
+          dev_name = ::File.basename(disk["dev_name"])
+          result = {
+            vendor:     disk["vendor"],
+            device:     dev_name,
+            udev_names: disk["dev_names"]
+          }
+          result[:model] = sys_block_value(dev_name, "device/model") || "Unknown"
+          result[:serial] = sys_block_value(dev_name, "device/serial") || "Unknown"
+          result[:size] = (sys_block_value(dev_name, "device/size") || "-1").to_i
+
+          @disks << result
+        end
+
+        @disks
+      end
+
+      # @return [Array<Hash>] list of info about network cards. Info contain:
+      #   `:vendor` of card
+      #   `:device` name of device
+      #   `:mac` mac address of card
+      #   `:active` if card io is active [Boolean]
+      #   `:link` if card link is up [Boolean]
+      def network_cards
+        return @network_cards if @network_cards
+
+        @network_cards = []
+        hardware["netcard"].each do |card|
+          resource = card["resource"]
+          mac = begin
+                  resource["hwaddr"].first["addr"]
+                rescue StandardError
+                  ""
+                end
+          active = begin
+                     resource["io"].first["active"]
+                   rescue StandardError
+                     false
+                   end
+          link = begin
+                   resource["link"].first["state"]
+                 rescue StandardError
+                   false
+                 end
+          result = {
+            vendor: card["vendor"],
+            device: card["dev_name"],
+            mac:    mac,
+            active: active,
+            link:   link
+          }
+
+          @network_cards << result
+        end
+
+        @network_cards
+      end
+
+      # @return [Hash] list of info about OS release. Info contain:
+      #   `:name` human readable name of OS like `"openSUSE Tumbleweed"` or `"SLES"`
+      #   `:version` of release like `"20200727"` or `"12.5"`
+      #   `:id` id of OS like `"opensuse-tumbleweed"` or `"sles"`
+      def os_release
+        return @os_release if @os_release
+
+        Yast.import "OSRelease"
+        @os_release = {
+          name:    Yast::OSRelease.ReleaseName,
+          version: Yast::OSRelease.ReleaseVersion,
+          id:      Yast::OSRelease.id
+        }
+      end
+
+      # allow to use env bindings
+      def public_bindings
         binding
       end
+
+    private
+
+      def sys_block_value(device, path)
+        sys_path = "/sys/block/#{device}/"
+        ::File.read(sys_path + path).strip
+      rescue StandardError => e
+        log.warn "read of #{sys_path + path}  failed with #{e}"
+        nil
+      end
     end
   end
 end

src/modules/AutoInstall.rb

@@ -265,6 +265,19 @@ def Finish(destdir)
         )
       )
 
+      # Saving postpartitioning-scripts scripts/logs
+      postpart_dir = AutoinstConfig.tmpDir + "/postpartitioning-scripts"
+      if Yast::FileUtils.Exists(postpart_dir)
+        SCR.Execute(
+          path(".target.bash"),
+          "/bin/cp #{postpart_dir}/* #{destdir}#{AutoinstConfig.scripts_dir}"
+        )
+        SCR.Execute(
+          path(".target.bash"),
+          "/bin/cp #{postpart_dir}/logs/* #{destdir}#{AutoinstConfig.logs_dir}"
+        )
+      end
+
       nil
     end