http proxy with Elixir. wait request with multi port and forward to each URIs
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config arrange test codes Feb 5, 2017
hooks Add elixir 180 (#66) Jan 14, 2019
lib Add elixir 180 (#66) Jan 14, 2019
test fix credo Nov 20, 2018
.formatter.exs fix miss-format Feb 25, 2018
.gitignore add some paths Dec 5, 2015
.travis.yml Add elixir 180 (#66) Jan 14, 2019
CHANGELOG.md bump and update changelog Oct 21, 2018
LICENSE Create LICENSE Nov 4, 2015
README.md Add elixir 180 (#66) Jan 14, 2019
mix.exs Bump hackney from 1.6.5 to 1.15.0 Jan 8, 2019
mix.lock Bump ex_doc from 0.19.2 to 0.19.3 Jan 21, 2019

README.md

HttpProxy

Build Status

codecov

Simple multi HTTP Proxy using Plug. And support record/play requests.

MY GOAL

  • Record/Play proxied requests
    • http_proxy support multi port and multi urls on one execution command mix proxy.
  • Support VCR

architecture

           http_proxy
Client  (server  client)  proxied_server
  |            |            |
  | 1.request  |            |
  |  ------>   | 2.request  |
  |            |  ------>   |
  |            |            |
  |            | 3.response |
  | 4.response |  <------   |
  |  <------   |            |
  |            |            |
  1. The client sends a request to http_proxy, then the http_proxy works as a proxy server.
  2. When the http_proxy receives the request from the client, then the http_proxy sends the request to a proxied server, e.g. http://google.com, as a client.
  3. The http_proxy receives responses from the proxied_server, then the http_proxy sets the response into its response to the client.
  4. The Client receives responses from the http_proxy.

Quick use as http proxy

requirement

  • Elixir over 1.6

set application and deps

  • mix.exs
    • :logger is option.
    • :http_proxy is not need if you run http_proxy with HttpProxy.start/0 or HttpProxy.stop/0 manually.
def application do
  [applications: [:logger, :http_proxy]]
end

...

defp deps do
  [
    {:http_proxy, "~> 1.4.0"}
  ]
end

set configuration

  • config/config.exs
use Mix.Config

config :http_proxy,
  proxies: [
             %{port: 8080,
               to:   "http://google.com"},
             %{port: 8081,
               to:   "http://yahoo.com"}
            ]
  • To manage logger, you should define logger settings like the following.
config :logger, :console,
  level: :info

solve deps and run a server

$ mix deps.get
$ mix clean
$ mix run --no-halt # start proxy server

If you would like to start production mode, you should run with MIX_ENV=prod like the following command.

$ MIX_ENV=prod mix run --no-halt

launch browser

Launch browser and open http://localhost:8080 or http://localhost:8081. Then, http://localhost:8080 redirect to http://google.com and http://localhost:8081 do to http://yahoo.com.

Development

  • Copy pre-commit hook
    • cp hooks/pre-commit ./git/hooks/pre-commit

Configuration

Customize proxy port

  • You can customize a proxy port. For example, if you change a waiting port from 8080 to 4000, then you can access to http://google.com via http://localhost:4000.
use Mix.Config

config :http_proxy,
  proxies: [
             %{port: 4000,
               to:   "http://google.com"},
             %{port: 8081,
               to:   "http://yahoo.com"}
            ]

Add proxy

  • You can add a waiting ports in configuration file. For example, the following setting allow you to access to http://apple.com via http://localhost:8082 in addition.
use Mix.Config

config :http_proxy,
  proxies: [
             %{port: 8080,
               to:   "http://google.com"},
             %{port: 8081,
               to:   "http://yahoo.com"},
             %{port: 8082,
               to:   "http://apple.com"}
            ]

Play and Record mode

  • When :record and :play are false, then the http_proxy works just multi port proxy.
  • When :record is true, then the http_proxy works to record request which is proxied.
  • When :play is true, then the http_proxy works to play request between this the http_proxy and clients.
use Mix.Config

config :http_proxy,
  proxies: [                   # MUST
             %{port: 8080,     # proxy all request even play or record
               to:   "http://google.com"},
             %{port: 8081,
               to:   "http://yahoo.com"}
            ]
  timeout: 20_000,             # Option, ms to wait http request.
  record: false,               # Option, true: record requests. false: don't record.
  play: false,                 # Option, true: play stored requests. false: don't play.
  export_path: "test/example", # Option, path to export recorded files.
  play_path: "test/data"       # Option, path to read json files as response to.

Example

Record request as the following

{
  "request": {
    "headers": [],
    "method": "GET",
    "options": {
      "aspect": "query_params"
    },
    "remote": "127.0.0.1",
    "request_body": "",
    "url": "http://localhost:8080/hoge/inu?email=neko&pass=123"
  },
  "response": {
    "body_file": "path/to/body_file.json",
    "cookies": {},
    "headers": {
      "Cache-Control": "public, max-age=2592000",
      "Content-Length": "251",
      "Content-Type": "text/html; charset=UTF-8",
      "Date": "Sat, 21 Nov 2015 00:37:38 GMT",
      "Expires": "Mon, 21 Dec 2015 00:37:38 GMT",
      "Location": "http://www.google.com/hoge/inu?email=neko&pass=123",
      "Server": "sffe",
      "X-Content-Type-Options": "nosniff",
      "X-XSS-Protection": "1; mode=block"
    },
    "status_code": 301
  }
}

Response body will save in "path/to/body_file.json".

Play request with the following JSON data

  • Example is https://github.com/KazuCocoa/http_proxy/tree/master/test/data/mappings
  • You can set path or path_pattern as attribute under request.
    • If path, the http_proxy check requests are matched completely.
    • If path_pattern, the http_proxy check requests are matched with Regex.
  • You can set body or body_file as attribute under response.
    • If body, the http_proxy send the body string.
    • If body_file, the http_proxy send the body_file binary as response.

path and body case

{
  "request": {
    "path": "/request/path",
    "port": 8080,
    "method": "GET"
  },
  "response": {
    "body": "<html>hello world</html>",
    "cookies": {},
    "headers": {
      "Content-Type": "text/html; charset=UTF-8",
      "Server": "GFE/2.0"
    },
    "status_code": 200
  }
}

path_pattern and body_file case

  • Pattern match with Regex.match?(Regex.compile!("\A/request.*neko\z"), request_path)
  • File.read/2 via file/to/path.json and respond the binary
{
  "request": {
    "path_pattern": "\A/request.*neko\z",
    "port": 8080,
    "method": "GET"
  },
  "response": {
    "body_file": "file/to/path.json",
    "cookies": {},
    "headers": {
      "Content-Type": "text/html; charset=UTF-8",
      "Server": "GFE/2.0"
    },
    "status_code": 200
  }
}

dependencies

$ mix xref graph
lib/http_proxy.ex
└── lib/http_proxy/supervisor.ex
    ├── lib/http_proxy/agent.ex
    │   ├── lib/http_proxy/play/data.ex
    │   │   ├── lib/http_proxy/agent.ex
    │   │   └── lib/http_proxy/play/response.ex
    │   │       ├── lib/http_proxy/play/data.ex
    │   │       └── lib/http_proxy/utils/file.ex
    │   └── lib/http_proxy/play/paths.ex
    │       ├── lib/http_proxy/agent.ex
    │       └── lib/http_proxy/play/response.ex
    └── lib/http_proxy/handle.ex
        ├── lib/http_proxy/play/body.ex
        ├── lib/http_proxy/play/data.ex
        ├── lib/http_proxy/play/paths.ex
        ├── lib/http_proxy/play/response.ex
        └── lib/http_proxy/record/response.ex
            ├── lib/http_proxy/format.ex
            │   └── lib/http_proxy/data.ex (compile)
            └── lib/http_proxy/utils/file.ex

TODO

styleguide

http://elixir.community/styleguide

LICENSE

MIT. Please read LICENSE.