cannot attach iscsi persistent volume

[sbezverk]

pod does not start because kubelet cannot mount iscsi persistent storage. The host where pod/kubelet runs shows success in building iscsi session and it creates corresponding disk device in /dev

Here is the error, but more details are provided in the attached log file:

Dec 22 10:22:36 kube-1 kubelet: E1222 10:22:36.691419 781 iscsi.go:214] iscsi: failed to setup
Dec 22 10:22:36 kube-1 kubelet: E1222 10:22:36.691474 781 nestedpendingoperations.go:262] Operation for ""kubernetes.io/iscsi/a272d8c6-c859-11e6-a647-525400ebb7ee-maraidb" ("a272d8c6-c859-11e6-a647-525400ebb7ee")" failed. No retries permitted until 2016-12-22 10:24:36.691449237 -0500 EST (durationBeforeRetry 2m0s). Error: MountVolume.SetUp failed for volume "kubernetes.io/iscsi/a272d8c6-c859-11e6-a647-525400ebb7ee-maraidb" (spec.Name: "maraidb") pod "a272d8c6-c859-11e6-a647-525400ebb7ee" (UID: "a272d8c6-c859-11e6-a647-525400ebb7ee") with: Could not attach disk: Timeout after 10s

Mounting manually on kubelet host iscsi storage works without any problems.

iscsi-pv-pvc.txt

[sbezverk]

Forgot to mentione, it is kubernetes 1.5.1 and baremetal all in one installation.

[sbezverk]

the issue was related to the fact that attached partition has not been preped by mkfs.xfs. When kubelet was trying to run it automatically it failed because it required -f flag. After attaching manually target and then mkfs.xfs -f, kubelet managed to attach iscsi volume to the pod

[Sanjary]

@sbezverk Hey Hi, can you express a bit more on how you overcame the issue, I'm facing the exact same issue with openshift origin v1.4.1+3f9807a which comes with kubernetes v1.4.0+776c994.
After logging in to my iscsi target, I have an active session on my host, then, I formatted the device /dev/sdb (as it was assigned) with mkfs.xfs -f, then I deployed a pv, a pvc and a pod that uses it on the same host using node selector which talks to the iqn which I'm logged in to from the host. Still, no luck and the same problem with "MountVolume.SetUp failed" remains. So, can you please tell a bit more to help me out?