Can't get it to work with auth token given in config

[depesz]

So, I need to get some data from consul. To get consul key I'm using vault.

I tested my problem on diplomat 2.5.1 and 2.6.4.

This part is trivial-ish:

require 'vault'
vault_client_profile='some-profile'
Vault.address = File.read("/run/vault-client/contexts/#{vault_client_profile}/addr")
Vault.token = File.read("/run/vault-client/contexts/#{vault_client_profile}/token")
consul_key = Vault.logical.read("consul/creds/some-role").data[:token]

Now, if I'd after this code, do:

ENV['CONSUL_TOKEN'] = consul_key
ENV['CONSUL_HTTP_ADDR']="consul:8500"
require 'diplomat'
pp Diplomat::Kv.get("global/some/key" )

it works beautifully. But I don't want require mixed in code.

And If I'd move require to top of script (the rest stays the same), I'm getting:

Traceback (most recent call last):                             
...
/usr/lib/ruby/2.7.0/net/http.rb:960:in `initialize': Failed to open TCP connection to localhost:8500 (Connection refused - connect(2) for "localhost" port 8500) (Faraday::ConnectionFailed)

So I tried to use confdiguration. Require at the top, and after getting token from vault I do:

Diplomat.configure do |config|                                                                                                                                                                                                                                                             
  config.url = "http://consul:8500"                                                                                                                                                                                                                                                        
  config.acl_token = consul_key                                                                                                                                                                                                                                                            
end                                                                                                                                                                                                                                                                                        

This leads to:

Traceback (most recent call last):
        2: from ./z.rb:24:in `<main>'
        1: from /var/lib/gems/2.7.0/gems/diplomat-2.5.1/lib/diplomat/rest_client.rb:57:in `method_missing'
/var/lib/gems/2.7.0/gems/diplomat-2.5.1/lib/diplomat/kv.rb:78:in `get': status 403: rpc error making call: Permission denied (Diplomat::UnknownStatus)

So, can you help me figure out what is the problem? I would HATE to have to configure consul by using env vars, and be forced to do require in the middle of code, but at the moment it looks that in no other way I can make it work.

[pierresouchay]

Hello @depesz,

I tested, this works for me.

Here are my reproduction steps...

Steps

Configure basic Consul with tokens:

Create file consul-token.hcl:

acl {
  enabled        = true
  default_policy = "deny"
  down_policy    = "extend-cache"
}

Start Consul with this file

consul agent -dev -config-file=consul-token.hcl

Configure ACLs

consul acl bootstrap

=>

AccessorID:       646b41dc-b508-0e74-fcbb-67c7c7d7741c
SecretID:         7f5ebb8b-a57c-9354-76ad-e2b2e29f56dd
Description:      Bootstrap Token (Global Management)
Local:            false
Create Time:      2022-08-26 23:53:10.374542 +0200 CEST
Policies:
   00000000-0000-0000-0000-000000000001 - global-management

=> Retrieve the token value displayed as SecretID => in this example 7f5ebb8b-a57c-9354-76ad-e2b2e29f56dd

Create the test script

#!/usr/bin/env ruby
# frozen_string_literal: true

require 'diplomat'

consul_token = ENV["MY_TOKEN"] || nil

Diplomat.configure do |config|
  config.acl_token = consul_token
end if consul_token

puts Diplomat::Health.service('consul', { 'x-consul-index': 42, 'wait': '5s' })
puts Diplomat::Datacenter::get()
puts Diplomat::Kv.put("global/some/key", "random_value")
puts Diplomat::Kv.get("global/some/key" )

And make it executable:

chmod a+x test.rb

Now, let's validate this works:

Without token:

./test.rb
dc1
Traceback (most recent call last):
	[...]
   <REDACTED>/faraday-2.5.2/lib/faraday/response/raise_error.rb:20:in `on_complete': the server responded with status 403 (Faraday::ForbiddenError)

With token

MY_TOKEN=7f5ebb8b-a57c-9354-76ad-e2b2e29f56dd ./test.rb
#<OpenStruct Node={"ID"=>"8f4feaae-9d98-252f-06dc-ad94548b3472", "Node"=>"mynode", "Address"=>"127.0.0.1", "Datacenter"=>"dc1", "TaggedAddresses"=>nil, "Meta"=>nil, "CreateIndex"=>13, "ModifyIndex"=>13}, Service={"ID"=>"consul", "Service"=>"consul", "Tags"=>[], "Address"=>"", "Meta"=>{"non_voter"=>"false", "raft_version"=>"3", "read_replica"=>"false", "serf_protocol_current"=>"2", "serf_protocol_max"=>"5", "serf_protocol_min"=>"1", "version"=>"1.10.1"}, "Port"=>8300, "Weights"=>{"Passing"=>1, "Warning"=>1}, "EnableTagOverride"=>false, "Proxy"=>{"Mode"=>"", "MeshGateway"=>{}, "Expose"=>{}}, "Connect"=>{}, "CreateIndex"=>13, "ModifyIndex"=>13}, Checks=[{"Node"=>"b188lfC02DJ1HZMD6M", "CheckID"=>"serfHealth", "Name"=>"Serf Health Status", "Status"=>"passing", "Notes"=>"", "Output"=>"Agent alive and reachable", "ServiceID"=>"", "ServiceName"=>"", "ServiceTags"=>[], "Type"=>"", "ExposedPort"=>0, "Definition"=>{}, "CreateIndex"=>13, "ModifyIndex"=>13}]>
dc1
true
random_value

With a wrong token

MY_TOKEN=some_wrong_token ./test.rb
Traceback (most recent call last):
	[...]
       <REDACTED>/faraday-2.5.2/lib/faraday/response/raise_error.rb:20:in `on_complete': the server responded with status 403 (Faraday::ForbiddenError)
	[...]
       <REDACTED>/diplomat-2.6.4/lib/diplomat/rest_client.rb:262:in `rescue in send_get_request': the server responded with status 403 (Diplomat::AclNotFound)

So, most probably there is a little mistake or some weird environment variable somewhere. Can you try with my steps and confirm this works?

Alternative

Most methods of diplomat also support per request token.

On my side, this is my preferred method, because Diplomat::configure has side effects, so I try to avoid it as much as possible (it changes the token for the whole instance of library).

So it is perfectly possible to write this code:

#!/usr/bin/env ruby
# frozen_string_literal: true

require 'diplomat'

consul_token = ENV["MY_TOKEN"] || nil

puts Diplomat::Kv.get("global/some/key", {:token=>consul_token} )

=> which works the same way, but with more flexibility (because you can use several tokens in the same diplomat instance)

$ MY_TOKEN=7f5ebb8b-a57c-9354-76ad-e2b2e29f56dd ./test.rb
random_value

$ ./test.rb
Traceback (most recent call last):
	2: from ./test.rb:9:in `<main>'
	1: from <REDACTED>/diplomat-2.6.4/lib/diplomat/rest_client.rb:57:in `method_missing'
<REDACTED>/diplomat-2.6.4/lib/diplomat/kv.rb:78:in `get': status 403: Permission denied (Diplomat::UnknownStatus)

Please if this works, can you close the issue?
Otherwise, could you specify your:

• Consul version
• Faraday version
• Diplomat version
• OS

Kind Regards

[depesz]

Sooo. I can't do any tests, because appranently admins changed environment so that I can now query consul without providing token at all.

So, sorry about notice, this can be close (I'm closing the ticket), but right now all works with just:

require 'diplomat'
puts Diplomat::Kv.get("some/key", { :http_addr => "http://consul:8500" } )

[pierresouchay]

@depesz still the steps I gave you let you test on your machine on a Consul agent in dev mode... (so, no need for infrastructure)

[depesz]

While I understand, I have no idea how to setup consul. what it needs, what is dev mode and so on. I just have "consul" in prod environmetn, and need to get data out of it. which didn't work, but now it apparently does :) So, everything is well for me.

[pierresouchay]

@depesz Just download it for your architecture and you are done: https://www.consul.io/downloads