Re-authenticate in case of token expiry fails for Glance

[loewenstein]

Refreshing of expired tokens seems to be broken.

1. Fog::Image::OpenStack::V1::Real#initialize calls authenticate from included core module. Then @path is set to a supported version by https://github.com/fog/fog-openstack/blob/master/lib/fog/openstack/image_v1.rb#L119-L124.
2. Fog::OpenStack::Common#request rescues from Excon::Errors::Unauthorized with authenticate and retry (

   fog-openstack/lib/fog/openstack/common.rb

   Lines 22 to 28 in 9034252

   	rescue Excon::Errors::Unauthorized => error
   	raise if retried
   	retried = true
   	@openstack_must_reauthenticate = true
   	authenticate
   	retry

   )
3. Fog::OpenStack::Core#authenticate sets the path to keystone auth path in

   fog-openstack/lib/fog/openstack/core.rb

   Line 102 in 9034252

   @path = @openstack_management_uri.path

4. The retry of Fog::OpenStack::Common#request no longer has a valid Glance version as part of the path. Hence Glance responds with HTTP 300 and a json of supported versions.

You can see a good overall log of the consequences in the BOSH OpenStack CPI issue cloudfoundry/bosh-openstack-cpi-release#40.
I.e. 300 multiple choices while actively waiting for the image to be ready to use (repeated image.reload & check state) in the fog-openstack consumer log.
A 401 followed by a 300 in the Glance HTTP log.

I could imagine this is a more general problem that just with Fog::Image::OpenStack. The root cause seems to be in common.rb and core.rb.

[voelzmo]

@dhague Have you seen something like this before?

[voelzmo]

Any opinions on how the token caching works from one of the maintainers? Are @loewenstein's thoughts from above correct?

[Carthaca]

I don't think so, retry has been refactored to set the api path

fog-openstack/lib/fog/openstack/core.rb

Lines 85 to 92 in f72d2fc

	rescue Excon::Errors::Unauthorized => error
	# token expiration and token renewal possible
	if error.response.body != 'Bad username or password' && @openstack_can_reauthenticate && !retried
	@openstack_must_reauthenticate = true
	authenticate
	set_api_path
	retried = true
	retry

which gets done in the glance parts

fog-openstack/lib/fog/image/openstack/v2.rb

Lines 129 to 135 in 4217d76

	def set_api_path
	unless @path.match(SUPPORTED_VERSIONS)
	@path = Fog::OpenStack.get_supported_version_path(SUPPORTED_VERSIONS,
	@openstack_management_uri,
	@auth_token,
	@connection_options)
	end

fog-openstack/lib/fog/image/openstack/v1.rb

Lines 121 to 127 in 4217d76

	def set_api_path
	unless @path.match(SUPPORTED_VERSIONS)
	@path = Fog::OpenStack.get_supported_version_path(SUPPORTED_VERSIONS,
	@openstack_management_uri,
	@auth_token,
	@connection_options)
	end

and thus should work