Merge pull request #708 from srl-labs/srl-overlay-cfg

SR Linux `startup-config` in CLI format

docs/manual/kinds/srl.md

@@ -3,7 +3,7 @@
 [Nokia SR Linux](https://www.nokia.com/networks/products/service-router-linux-NOS/) NOS is identified with `srl` kind in the [topology file](../topo-def-file.md). A kind defines a supported feature set and a startup procedure of a node.
 
 ## Managing SR Linux nodes
-There are many ways to manage SR Linux nodes, ranging from classic CLI management all the way up to the gNMI programming. Here is a short summary on how to access those interfaces:
+There are many ways to manage SR Linux nodes, ranging from classic CLI management all the way up to the gNMI programming.
 
 === "bash"
     to connect to a `bash` shell of a running SR Linux container:
@@ -25,7 +25,7 @@ There are many ways to manage SR Linux nodes, ranging from classic CLI managemen
     get --path /system/name/host-name
     ```
 === "JSON-RPC"
-    SR Linux has a JSON-RPC interface, that is enabled on port 80/443 for HTTP/HTTPS schemas accordingly.
+    SR Linux has a JSON-RPC interface over port 80/443 for HTTP/HTTPS schemas accordingly.
 
     HTTPS server uses the same TLS certificate as gNMI server.
 
@@ -41,7 +41,7 @@ With that naming convention in mind:
 * `e1-2` - second interface on a line card #1
 * `e2-1` - first interface on a line card #1
 
-These are the names of the interfaces that are seen in the linux shell, however, when configuring the interfaces via SR Linux CLI, the interfaces are named as `ethernet-X/Y` where `X/Y` is the `linecard/port` combination.
+These interface names are seen in the Linux shell; however, when configuring the interfaces via SR Linux CLI, the interfaces should be named as `ethernet-X/Y` where `X/Y` is the `linecard/port` combination.
 
 Interfaces can be defined in a non-sequential way, for example:
 
@@ -54,7 +54,7 @@ Interfaces can be defined in a non-sequential way, for example:
 ### Breakout interfaces
 If the interface is intended to be configured as a breakout interface, its name must be changed accordingly.
 
-The breakout interfaces will have the name in the form of `eX-Y-Z`, where `Z` is the breakout port number. For example, if interface `ethernet-1/3` on a IXR-D3 system is intended to be configured as a breakout 100Gb to 4x25Gb then the interfaces in the topology must take this into account and use the following naming:
+The breakout interfaces will have the name `eX-Y-Z` where `Z` is the breakout port number. For example, if interface `ethernet-1/3` on an IXR-D3 system is meant to act as a breakout 100Gb to 4x25Gb, then the interfaces in the topology must use the following naming:
 
 ```yaml
   links:
@@ -66,13 +66,13 @@ The breakout interfaces will have the name in the form of `eX-Y-Z`, where `Z` is
 ### Types
 For SR Linux nodes [`type`](../nodes.md#type) defines the hardware variant that this node will emulate.
 
-The available type values are: `ixr6`, `ixr10`, `ixrd1`, `ixrd2`, `ixrd3`, `ixrh2` and `ixrh3` which correspond to a hardware variant of Nokia 7250/7220 IXR chassis.
+The available type values are: `ixr6`, `ixr10`, `ixrd1`, `ixrd2`, `ixrd3`, `ixrh2` and `ixrh3`, which correspond to a hardware variant of Nokia 7250/7220 IXR chassis.
 
 By default, `ixrd2` type will be used by containerlab.
 
-Based on the provided type, containerlab will generate the topology file that will be mounted to SR Linux container and make it boot in a chosen HW variant.
+Based on the provided type, containerlab will generate the topology file that will be mounted to the SR Linux container and make it boot in a chosen HW variant.
 ### Node configuration
-SR Linux uses a `/etc/opt/srlinux/config.json` file to persist its configuration. By default containerlab starts nodes of `srl` kind with a basic "default" config, and with the `startup-config` parameter it is possible to provide a custom config file that will be used as a startup one.
+SR Linux uses a `/etc/opt/srlinux/config.json` file to persist its configuration. By default, containerlab starts nodes of `srl` kind with a basic "default" config, and with the `startup-config` parameter, it is possible to provide a custom config file that will be used as a startup one.
 #### Default node configuration
 When a node is defined without the `startup-config` statement present, containerlab will make [additional configurations](https://github.com/srl-labs/containerlab/blob/master/nodes/srl/srl.go#L38) on top of the factory config:
 
@@ -91,7 +91,48 @@ topology:
 The generated config will be saved by the path `clab-<lab_name>/<node-name>/config/config.json`. Using the example topology presented above, the exact path to the config will be `clab-srl_lab/srl1/config/config.json`.
 
 #### User defined startup config
-It is possible to make SR Linux nodes to boot up with a user-defined config instead of a built-in one. With a [`startup-config`](../nodes.md#startup-config) property of the node/kind a user sets the path to the local config file that will be mounted to a container:
+It is possible to make SR Linux nodes boot up with a user-defined config instead of a built-in one. With a [`startup-config`](../nodes.md#startup-config) property of the node/kind a user sets the path to the local config file that will be used as a startup config.
+
+The startup configuration file can be provided in two formats:
+
+* full SR Linux config in JSON format
+* partial config in SR Linux CLI format
+
+##### CLI
+A typical lab scenario is to make nodes to boot with a pre-configured use case. The easiest way to do that is to capture the intended changes as CLI commands.
+
+On SR Linux, users can configure the use case and output the changes in the form of `set` instructions with the `info flat` command. These CLI commands can be saved in a file and used as a startup configuration.
+
+???info "flat config CLI example"
+    this can be `myconfig.cli` referred below
+    ```bash
+    set / network-instance default protocols bgp admin-state enable
+    set / network-instance default protocols bgp router-id 10.10.10.1
+    set / network-instance default protocols bgp autonomous-system 65001
+    set / network-instance default protocols bgp group ibgp ipv4-unicast admin-state enable
+    set / network-instance default protocols bgp group ibgp export-policy export-lo
+    set / network-instance default protocols bgp neighbor 192.168.1.2 admin-state enable
+    set / network-instance default protocols bgp neighbor 192.168.1.2 peer-group ibgp
+    set / network-instance default protocols bgp neighbor 192.168.1.2 peer-as 65001
+    ```
+
+
+```yaml
+name: srl_lab
+topology:
+  nodes:
+    srl1:
+      kind: srl
+      type: ixr6
+      image: ghcr.io/nokia/srlinux
+      # a path to the partial config in CLI format relative to the current working directory
+      startup-config: myconfig.cli
+```
+
+In that case, SR Linux will first boot with the default configuration, and then the CLI commands from the `myconfig.cli` will be applied.
+
+##### JSON
+SR Linux persists its configuration as a JSON file that can be found by the `/etc/opt/srlinux/config.json` path. Users can use this file as a startup configuration like that:
 
 ```yaml
 name: srl_lab
@@ -101,17 +142,18 @@ topology:
       kind: srl
       type: ixr6
       image: ghcr.io/nokia/srlinux
-      startup-config: myconfig.json # a path relative to the current working directory
+      # a path to the full config in JSON format relative to the current working directory
+      startup-config: myconfig.json
 ```
 
-With such topology file containerlab is instructed to take a file `myconfig.json` from the current working directory, copy it to the lab directory for that specific node under the `config.json` name and mount that directory to the container. This will result in this config to act as a startup config for the node.
+Containerlab will take the `myconfig.json` file, copy it to the lab directory for that specific node under the `config.json` name, and mount that directory to the container. This will result in this config acting as a startup-config for the node.
 
 #### Saving configuration
-As was explained in the [Node configuration](#node-configuration) section, SR Linux containers can make their config persistent, because config files are provided to the containers from the host via the bind mount.
+As was explained in the [Node configuration](#node-configuration) section, SR Linux containers can make their config persistent because config files are provided to the containers from the host via the bind mount.
 
-When a user configures SR Linux node the changes are saved into the running configuration stored in memory. To save the running configuration as a startup configuration the user needs to execute the `tools system configuration save` CLI command. This will write the config to the `/etc/opt/srlinux/config.json` file that holds the startup config and is exposed to the host.
+When a user configures the SR Linux node, the changes are saved into the running configuration stored in memory. To save the running configuration as a startup configuration, the user needs to execute the `tools system configuration save` CLI command. This command will write the config to the `/etc/opt/srlinux/config.json` file that holds the startup-config and is exposed to the host.
 
-SR Linux node also supports the [`containerlab save -t <topo-file>`](../../cmd/save.md) command which will execute the command to save the running config on all the lab nodes. For SR Linux node the `tools system configuration save` will be executed:
+SR Linux node also supports the [`containerlab save -t <topo-file>`](../../cmd/save.md) command, which will execute the command to save the running-config on all lab nodes. For SR Linux node, the `tools system configuration save` will be executed:
 
 ```
 ❯ containerlab save -t quickstart.clab.yml
@@ -142,7 +184,7 @@ topology:
 ```
 
 ### TLS
-By default containerlab will generate TLS certificates and keys for each SR Linux node of a lab. The TLS related files that containerlab creates are located in the so-called CA directory which can be located by the `<lab-directory>/ca/` path. Here is a list of files that containerlab creates relative to the CA directory
+By default, containerlab will generate TLS certificates and keys for each SR Linux node of a lab. The TLS related files that containerlab creates are located in the so-called CA directory, which can be found by the `<lab-directory>/ca/` path. Here is a list of files that containerlab creates relative to the CA directory
 
 1. Root CA certificate - `root/root-ca.pem`
 2. Root CA private key - `root/root-ca-key.pem`

nodes/srl/srl.go

@@ -92,6 +92,8 @@ func init() {
 type srl struct {
 	cfg     *types.NodeConfig
 	runtime runtime.ContainerRuntime
+	// startup-config passed as a path to a file with CLI instructions will be read into this byte slice
+	startupCliCfg []byte
 }
 
 func (s *srl) Init(cfg *types.NodeConfig, opts ...nodes.NodeOption) error {
@@ -192,7 +194,7 @@ func (s *srl) PreDeploy(configName, labCADir, labCARoot string) error {
 		}
 	}
 
-	return createSRLFiles(s.cfg)
+	return s.createSRLFiles()
 }
 
 func (s *srl) Deploy(ctx context.Context) error {
@@ -201,14 +203,20 @@ func (s *srl) Deploy(ctx context.Context) error {
 }
 
 func (s *srl) PostDeploy(ctx context.Context, _ map[string]nodes.Node) error {
-	// only perform postdeploy additional config provisioning if there is not startup nor existing config
-	if s.cfg.StartupConfig != "" || utils.FileExists(filepath.Join(s.cfg.LabDir, "config", "config.json")) {
+	// do not enter in the postdeploy stage if config file is found in the lab directory.
+	// This can be either if the startup-config has been mounted by that path
+	// or the config has been previously generated and saved
+	if utils.FileExists(filepath.Join(s.cfg.LabDir, "config", "config.json")) {
 		return nil
 	}
 
 	log.Infof("Running postdeploy actions for Nokia SR Linux '%s' node", s.cfg.ShortName)
 
-	return s.addDefaultConfig(ctx)
+	if err := s.addDefaultConfig(ctx); err != nil {
+		return err
+	}
+
+	return s.addOverlayCLIConfig(ctx)
 }
 
 func (s *srl) GetImages() map[string]string {
@@ -294,55 +302,71 @@ func (s *srl) Ready(ctx context.Context) error {
 	}
 }
 
-//
-
-func createSRLFiles(nodeCfg *types.NodeConfig) error {
-	log.Debugf("Creating directory structure for SRL container: %s", nodeCfg.ShortName)
+func (s *srl) createSRLFiles() error {
+	log.Debugf("Creating directory structure for SRL container: %s", s.cfg.ShortName)
 	var src string
 	var dst string
 
-	if nodeCfg.License != "" {
+	if s.cfg.License != "" {
 		// copy license file to node specific directory in lab
-		src = nodeCfg.License
-		dst = filepath.Join(nodeCfg.LabDir, "license.key")
+		src = s.cfg.License
+		dst = filepath.Join(s.cfg.LabDir, "license.key")
 		if err := utils.CopyFile(src, dst, 0644); err != nil {
 			return fmt.Errorf("CopyFile src %s -> dst %s failed %v", src, dst, err)
 		}
 		log.Debugf("CopyFile src %s -> dst %s succeeded", src, dst)
 	}
 
 	// generate SRL topology file
-	err := generateSRLTopologyFile(nodeCfg.NodeType, nodeCfg.LabDir, nodeCfg.Index)
+	err := generateSRLTopologyFile(s.cfg.NodeType, s.cfg.LabDir, s.cfg.Index)
 	if err != nil {
 		return err
 	}
 
-	utils.CreateDirectory(path.Join(nodeCfg.LabDir, "config"), 0777)
+	utils.CreateDirectory(path.Join(s.cfg.LabDir, "config"), 0777)
 
 	// generate a startup config file
 	// if the node has a `startup-config:` statement, the file specified in that section
 	// will be used as a template in GenerateConfig()
-	if nodeCfg.StartupConfig != "" {
-		dst = filepath.Join(nodeCfg.LabDir, "config", "config.json")
+	if s.cfg.StartupConfig != "" {
+		dst = filepath.Join(s.cfg.LabDir, "config", "config.json")
 
-		log.Debugf("Reading startup-config %s", nodeCfg.StartupConfig)
+		log.Debugf("Reading startup-config %s", s.cfg.StartupConfig)
 
-		c, err := os.ReadFile(nodeCfg.StartupConfig)
+		c, err := os.ReadFile(s.cfg.StartupConfig)
 		if err != nil {
 			return err
 		}
 
+		// Determine if startup-config is a JSON file
+		// Get slice of data with optional leading whitespace removed.
+		// See RFC 7159, Section 2 for the definition of JSON whitespace.
+		x := bytes.TrimLeft(c, " \t\r\n")
+		isJSON := len(x) > 0 && x[0] == '{'
+		if !isJSON {
+			log.Debugf("startup-config passed to %s is in the CLI format. Will apply it in post-deploy stage",
+				s.cfg.ShortName)
+
+			s.startupCliCfg = c
+
+			// no need to generate and mount startup-config passed in a CLI format
+			// as we will apply it over the top of a default config in the post deploy stage
+			return nil
+		}
+
 		cfgTemplate := string(c)
 
-		err = nodeCfg.GenerateConfig(dst, cfgTemplate)
+		err = s.cfg.GenerateConfig(dst, cfgTemplate)
 		if err != nil {
-			log.Errorf("node=%s, failed to generate config: %v", nodeCfg.ShortName, err)
+			log.Errorf("node=%s, failed to generate config: %v", s.cfg.ShortName, err)
 		}
 	}
 
 	return err
 }
 
+//
+
 type mac struct {
 	MAC string
 }
@@ -414,3 +438,38 @@ func (s *srl) addDefaultConfig(ctx context.Context) error {
 
 	return nil
 }
+
+// addOverlayCLIConfig adds CLI formatted config that is read out of a file provided via startup-config directive
+func (s *srl) addOverlayCLIConfig(ctx context.Context) error {
+	// start waiting for initial commit and mgmt server ready
+	if err := s.Ready(ctx); err != nil {
+		return err
+	}
+
+	cfgStr := string(s.startupCliCfg)
+
+	log.Debugf("Node %q additional config from startup-config file %s:\n%s", s.cfg.ShortName, s.cfg.StartupConfig, cfgStr)
+	_, _, err := s.runtime.Exec(ctx, s.cfg.LongName, []string{
+		"bash",
+		"-c",
+		fmt.Sprintf("echo '%s' > /tmp/clab-config", cfgStr),
+	})
+
+	if err != nil {
+		return err
+	}
+
+	stdout, stderr, err := s.runtime.Exec(ctx, s.cfg.LongName, []string{
+		"bash",
+		"-c",
+		"sr_cli -ed --post 'commit save' < tmp/clab-config",
+	})
+
+	if err != nil {
+		return err
+	}
+
+	log.Debugf("node %s. stdout: %s, stderr: %s", s.cfg.ShortName, stdout, stderr)
+
+	return nil
+}

tests/02-basic-srl/01-two-srls.robot

@@ -15,9 +15,6 @@ Deploy ${lab-name} lab
     Log    ${output}
     Should Be Equal As Integers    ${rc}    0
 
-Wait 5 seconds for srl to boot
-    Sleep    5s
-
 Verify links in node srl1
     ${rc}    ${output} =    Run And Return Rc And Output
     ...    sudo containerlab --runtime ${runtime} exec -t ${CURDIR}/${lab-file-name} --label clab-node-name\=srl1 --cmd "ip link show e1-1"
@@ -32,6 +29,13 @@ Verify links in node srl2
     Should Be Equal As Integers    ${rc}    0
     Should Contain    ${output}    state UP
 
+Verify srl2 accepted user-provided CLI config
+    ${rc}    ${output} =    Run And Return Rc And Output
+    ...    sudo containerlab --runtime ${runtime} exec -t ${CURDIR}/${lab-file-name} --label clab-node-name\=srl2 --cmd "sr_cli 'info /system information location'"
+    Log    ${output}
+    Should Be Equal As Integers    ${rc}    0
+    Should Contain    ${output}    test123
+
 Verify saving config
     ${rc}    ${output} =    Run And Return Rc And Output
     ...    sudo containerlab --runtime ${runtime} save -t ${CURDIR}/${lab-file-name}

tests/02-basic-srl/02-srl02.clab.yml

@@ -13,6 +13,7 @@ topology:
       kind: srl
     srl2:
       kind: srl
+      startup-config: ./tests/02-basic-srl/srl2-startup.cli
 
   links:
     - endpoints: ["srl1:e1-1", "srl2:e1-1"]

tests/02-basic-srl/srl2-startup.cli

@@ -0,0 +1 @@
+set / system information location test123