Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parallel http requests are responded in the same order as the cassette #127

Open
sescobb27 opened this issue Jan 26, 2018 · 1 comment
Open

Comments

@sescobb27
Copy link

Hi, i'm getting errors when doing parallel http request that are responded from a cassette.
I'm fetching some records from an API and then fetching details for each record from a different API using Tasks, but it seems ExVcr is using the cassette to respond in the same order the requests were first made, but, because requests are run in parallel i'm getting answers mixed up

defmodule PlaceService do
  def get_places(city, lat, lon) do
    case HTTPoison.get("places_url") do
      {:ok, %HTTPoison.Response{status_code: 200, body: body}} ->
        %{"results" => results} = Poison.decode!(body)
        places = results
                |> enrich_places_with_details()
                |> Enum.filter(&(&1 != %{}))
        {:ok, places}
      {:ok, %HTTPoison.Response{status_code: _code, body: _body}} ->
        {:error, []}
    end
  end

  defp enrich_places_with_details(places) do
    Enum.map(places, fn place ->
      Task.Supervisor.async_nolink(Twd.TaskSupervisor, fn ->
        fetch_place_details(place)
      end)
    end)
    |> Task.yield_many()
    |> Enum.map(fn {task, result} ->
      case result do
        {:ok, details} -> details
        {:exit, _} -> %{}
        nil ->
          Logger.info("Timeout fetching details")
          Task.shutdown(task, :brutal_kill)
          %{}
      end
    end)
  end

  defp fetch_place_details(place) do
    url = "details_for_place_url"
    response = HTTPoison.get(url, [])
    {:ok, %HTTPoison.Response{status_code: 200, body: body}} = response
    case Poison.decode!(body) do
      %{"result" => result} -> Map.merge(place, result)
    end
  end
end
@masterkain
Copy link

masterkain commented Apr 4, 2021

I have a similar problem, in the latest iteration of the code this is shown even with a simple async stream:

    Symbol.supported()
    |> Task.async_stream(fn symbol ->
      HTTPoison.get(url, [], params: %{"indicator" => indicator, "exchange" => "binance", "symbol" => symbol, "interval" => interval, "candlesCount" => candles})
    end)
    |> Enum.into([], fn {:ok, {:ok, %HTTPoison.Response{body: body, request: %HTTPoison.Request{params: %{"symbol" => symbol}}}}} -> %{symbol => Poison.decode!(body)} end)

the cassette gets registrered properly, however the test doesn't pass:

defmodule Pandamex.IndicatorTest do
  use ExUnit.Case, async: true
  use ExVCR.Mock, adapter: ExVCR.Adapter.Hackney, options: [clear_mock: true]

  alias Pandamex.Indicator

  setup_all do
    HTTPoison.start()
  end

  describe "Pandamex.Indicator.get/4" do
    @tag timeout: :infinity
    test "works" do
      use_cassette "indicators/get_indicators" do
        assert Indicator.get() == [
                 %{"BTC/USDT" => %{"value" => 45.67296250448912}},
                 %{"ETH/USDT" => %{"value" => 44.17848150108721}},
                 %{"XRP/USDT" => %{"value" => 47.02039812884717}},
                 %{"LTC/USDT" => %{"value" => 52.7422974568094}},
                 %{"XMR/USDT" => %{"value" => 54.31603149404031}}
               ]
      end
    end
  end
end
  1) test Pandamex.Indicator.get/4 works (Pandamex.IndicatorTest)
     test/pandamex/indicator_test.exs:15
     Assertion with == failed
     code:  assert Indicator.get() == [%{"BTC/USDT" => %{"value" => 45.67296250448912}}, %{"ETH/USDT" => %{"value" => 44.17848150108721}}, %{"XRP/USDT" => %{"value" => 47.02039812884717}}, %{"LTC/USDT" => %{"value" => 52.7422974568094}}, %{"XMR/USDT" => %{"value" => 54.31603149404031}}]
     left:  [%{"BTC/USDT" => %{"value" => 54.31603149404031}}, %{"ETH/USDT" => %{"value" => 54.31603149404031}}, %{"XRP/USDT" => %{"value" => 54.31603149404031}}, %{"LTC/USDT" => %{"value" => 54.31603149404031}}, %{"XMR/USDT" => %{"value" => 54.31603149404031}}]
     right: [%{"BTC/USDT" => %{"value" => 45.67296250448912}}, %{"ETH/USDT" => %{"value" => 44.17848150108721}}, %{"XRP/USDT" => %{"value" => 47.02039812884717}}, %{"LTC/USDT" => %{"value" => 52.7422974568094}}, %{"XMR/USDT" => %{"value" => 54.31603149404031}}]
     stacktrace:
       test/pandamex/indicator_test.exs:17: (test)

as you can see it returns the value of the first entry in the cassette 54.31603149404031 for each symbol
removing exvcr from the spec and doing the actual http call works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants