iscsi storage plugin only logs out of primary portal

[tobad357]

Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see http://kubernetes.io/docs/troubleshooting/.): No

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): iscsi

---

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG

Kubernetes version (use kubectl version): v1.6.0

Environment:

• Cloud provider or hardware configuration: Bare metal
• OS (e.g. from /etc/os-release): Centos 7.3
• Kernel (e.g. uname -a): Linux srv02 3.10.0-514.2.2.el7.x86_64 Unit test coverage in Kubelet is lousy. (~30%) #1 SMP Tue Dec 6 23:06:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
• Install tools: kubeadm
• Others:

What happened:
When using iscsi storage with multiple target portal (TP) addresses + multipathing the volume manager
logs on to the IQN for all portal addresses, but when a pod gets destroyed the volume manager only logs out for the primary TP. This leaves a session dangling and the pod using the volume can't start as the SAN doesn't allow multiple hosts to connect to the same IQN

What you expected to happen:
The DetachDisk method in iscsi_utils.go should do the same as the AttachDisk which is to
iterate over all portals and log out from all of them.
Alternatively omit the -p command line to iscsiadm to log out all references to the IQN

How to reproduce it (as minimally and precisely as possible):

1. Create a pod with iscsi storage and list two portals in the portals property
2. Let the pod start
3. Delete the pod and make sure it comes up on a new node
4. You should now see a session still active on the old node against the second TP

Anything else we need to know:

[tobad357]

@humblec Want me to fix this or is there other work in the pipeline?

[mleklund]

I am also seeing this issue. Although in my case I am not using the multiple target addresses, but my iscsi equipment (equallogic) load balances you to another portal ip.

 iscsiadm -m session -P 1
Target: iqn.2001-05.com.equallogic:0-1cb196-6bc1ed52b-fa27a36c2af591cb-kubetesting (non-flash)
	Current Portal: X.X.X.10:3260,1
	Persistent Portal: X.X.X.2:3260,1
		**********
		Interface:
		**********
		Iface Name: eno3
		Iface Transport: tcp
		Iface Initiatorname: iqn.1993-08.org.debian:01:922ec832967e
		Iface IPaddress: X.X.X.115
		Iface HWaddress: <empty>
		Iface Netdev: eno3
		SID: 17
		iSCSI Connection State: LOGGED IN
		iSCSI Session State: LOGGED_IN
		Internal iscsid Session State: NO CHANGE

[mtanino]

I also hit this problem. I have a plan to post a fix.

[mtanino]

/cc @msau42 @jingxu97
/cc @kubernetes/sig-storage-bugs

Could you add sig/storage and king/bug labels to this issue?
Thanks,

[k8s-ci-robot]

@mtanino: Reiterating the mentions to trigger a notification:
@kubernetes/sig-storage-bugs.

In response to this:

/cc @msau42 @jingxu97
/cc @kubernetes/sig-storage-bugs

Could you add sig/storage and king/bug labels to this issue?
Thanks,

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[jsafrane]

@rootfs, PTAL

[rootfs]

@tobad357 Will review #46239 once it is ready.

/cc @humblec

[mtanino]

@rootfs

Thanks. I'll add test cases into #46239, then I will remove the WIP.