Verk should be able to start even if Redis connection is not available

[keyan]

Several GenServers include operations in their init() functions that require a connection to Redis, for example:

verk/lib/verk/schedule_manager.ex

Lines 31 to 34 in e39293e

	def init(_) do
	{:ok, redis_url} = Application.fetch_env(:verk, :redis_url)
	{:ok, redis} = Redix.start_link(redis_url)
	Verk.Scripts.load(redis)

verk/lib/verk/queue_manager.ex

Lines 84 to 88 in e39293e

	def init([queue_name]) do
	node_id = Application.get_env(:verk, :node_id, "1")
	{:ok, redis_url} = Application.fetch_env(:verk, :redis_url)
	{:ok, redis} = Redix.start_link(redis_url)
	Verk.Scripts.load(redis)

This results in these GenServers failing to start when the Redis connection is not available. The effect is that if an application adds Verk.Supervisor to their supervision tree, the application will not be able to start if Redis is not available.

This doesn't seem to be what we would want in most cases. Instead I'd expect my application to be able to handle Redis connection retries in case there was a temporary connection disruption during startup.

Possible solution

We could move function calls that rely on the Redis connection to be present out of init() and instead do that work in a callback. This would allow for the work to fail and the GenServer to still start.

For example Verk.QueueManager.init would look something like:

  def init([queue_name]) do
    node_id = Application.get_env(:verk, :node_id, "1")
    Process.send_after(self(), :startup, 0)

    state = %State{queue_name: queue_name, redis: nil, node_id: node_id}

    {:ok, state}
  end

  def handle_info(:startup, state) do
    {:ok, redis_url} = Application.fetch_env(:verk, :redis_url)
    {:ok, redis} = Redix.start_link(redis_url)
    Verk.Scripts.load(redis)

    state.redis = redis
    Logger.info "Queue Manager started for queue #{state.queue_name}"
    {:noreply, state}
  end

[edgurgel]

Yeah, I agree that Verk should be able to start without Redis and keep retrying to reconnect etc. Your solution seems to be the way to go 👍

[lpil]

I'm unsure about this one. Given the init callback is about making guarantees about the readiness of the system should we consider Werk ready if it does not have a connected Redis instance? It seems like a fairly fundamental dependancy.

[keyan]

@lpil I understand your point and perhaps we should just leave it at that.

It might help to explain where I hit this issue at first. I have a webserver which has added Verk to its supervision tree but only uses Verk to enqueue tasks to Redis in the Resque/Verk format. There is a separate processor application that is responsible for dequeueing and processing those tasks. While I understand why it makes sense for the processor to crash if it cannot cannot to Redis, I would rather the webserver be able to start and serve traffic while it continues to retry connecting.

Maybe the premise is flawed here? It might be non-standard usage of the library and thus it doesn't make sense to build around support for the use case. I am not sure.

[lpil]

It seems that Verk doesn't require the application to be started in order to enqueue a job.

verk/lib/verk.ex

Lines 53 to 59 in d0f07c1

	def enqueue(%Job{jid: jid, queue: queue} = job, redis) do
	job = %Job{job | enqueued_at: Time.now |> DateTime.to_unix}
	case Redix.command(redis, ["LPUSH", "queue:#{queue}", Poison.encode!(job)]) do
	{:ok, _} -> {:ok, jid}
	{:error, reason} -> {:error, reason}
	end
	end

You could just call that function and pass in the pid/atom of your own Redis instance. :)

[edgurgel]

@keyan for your specific use case you could pass a connection to Verk.enqueue. So simply handling the connection on your side would do the job.

def enqueue(job, redis \ Verk.Redis)

Ok so Verk supports losing connectivity with Redis after it has started but it won't start without it.

I'm not sure if this "inconsistency" is bad or good...

[lpil]

I've been doing some thinking and reading and I think that now I agree with you. Werk can reasonably guarantee that it is in a state where it could accept jobs, though not that it has somewhere to accept them from. :)