Example: NVMe oF Target Creation Tear Down

Example NVMe oF Target Creation Tear Down on Redhat Linux using NVMe-oF RDMA transport.

Authors: Phil Cayton (Intel), Rajalaxmi Angadi (Intel)

Prerequisite: Load modules (configfs, nvme, nvmet, nvmet_rdma) on NVMe-oF Target with an RDMA adapter up with an address of 192.168.11.11

Summary: to configure and enable an Exported Logical NVM Subsystem, restrict access to specific Hosts and grant a specific Host access

1. Create an Exported Logical NVM Subsystem
2. Configure an Exported Logical Namespace (create an Exported Logical Namespace ID, associate it with an Underlying Namespace ID, and enable the Exported Logical Namespace).
   • Note: In this example the Exported Logical NVM Subsystem is restricted so ‘attr_allow_any_host’ is set to 0
3. Configure the Exported Logical Port (create an Exported Logical Port associate it with an Underlying Port, and assign a Transport Service ID and optional attributes)
4. Link the Exported Logical Port and the Exported Logical NVM Subsystem
5. Add an entry to the Known Host List
6. Grant Host Access by linking an entry from the Known Host List to the Exported Logical NVM Subsystem’s ‘allowed hosts’

Detailed steps to configure and enable an Exported Logical NVM Subsystem as summarized above:

1. Create the Exported Logical NVM Subsystem

     mkdir /sys/kernel/config/nvmet/subsystems/subsys1
   

2. Configure the Exported Logical Namespace

    mkdir /sys/kernel/config/nvmet/subsystems/subsys1/namespaces/1
    echo -n /dev/nvme0n1 > /sys/kernel/config/nvmet/subsystems/subsys1/namespaces/1/device_path
    echo -n 0 > /sys/kernel/config/nvmet/subsystems/subsys1/attr_allow_any_host
    echo -n 1 > /sys/kernel/config/nvmet/subsystems/subsys1/namespaces/1/enable
   

3. Configure the Exported Logical Port

   mkdir /sys/kernel/config/nvmet/ports/9
   echo -n 192.168.11.11 > /sys/kernel/config/nvmet/ports/9/addr_traddr
   echo -n ipv4 > /sys/kernel/config/nvmet/ports/9/addr_adrfam
   echo -n rdma > /sys/kernel/config/nvmet/ports/9/addr_trtype
   echo -n not required >/sys/kernel/config/nvmet/ports/9/addr_treq
   echo -n 4420 > /sys/kernel/config/nvmet/ports/9/addr_trsvcid
   

4. Link the Exported Logical Port and Exported Logical NVM Subsystem

   ln -s /sys/kernel/config/nvmet/subsystems/subsys1 /sys/kernel/config/nvmet/ports/9/subsystems/subsys1
   

5. Add a Known Host List Entry

   mkdir /sys/kernel/config/nvmet/hosts/h1_nqn
   

6. Grant Host Access to this Exported Logical NVM Subsystem

   ln –s /sys/kernel/config/nvmet/hosts/h1_nqn /sys/kernel/config/nvmet/subsystems/subsys1/allowed_hosts/h1_nqn
   

Summary: To tear down an Exported Logical NVM Subsystem: (Provided it is not connected by any host/initaiator)*

1. Revoke Host Access to the Exported Logical NVM Subsystem
2. Optional: Remove the Known Host List Entry, provided the Host NQN to be removed is:
   • not granted access to ANY Exported Logical NVM Subsystems on this Target
   • not going to be used to grant access to any Exported Logical Subsystems on this Target
3. Unlink the Exported Logical Port from the Exported Logical NVM Subsystem
4. Delete the Exported Logical Port
5. Delete the Exported Logical Namespace
6. Delete the Exported Logical NVM Subsystem

Steps to tear down an Exported Logical NVM Subsystem with restricted access as summarized above:

1. Revoke Host Access to the Exported Logical NVM Subsystem

   rm /sys/kernel/config/nvmet/subsystems/subsys1/allowed_hosts/h1_nqn
   

2. Remove the Known Host List Entry

   rmdir /sys/kernel/config/nvmet/hosts/h1_nqn
   

3. Unlink Exported Logical Port and Exported Logical NVM Subsystem

   rm /sys/kernel/config/nvmet/ports/9/subsystems/subsys1
   

4. Delete the Exported Logical Port

   rmdir /sys/kernel/config/nvmet/ports/9
   

5. Delete the Exported Logical Namespace

   echo -n 0 > /sys/kernel/config/nvmet/subsystems/subsys1/namespaces/1/enable 
   rmdir /sys/kernel/config/nvmet/subsystems/subsys1/namespaces/1
   

6. Delete the Exported Logical NVM Subsystem

   rmdir /sys/kernel/config/nvmet/subsystems/subsys1