potential use question

[Outstep]

Hello,

I am working on a small proxy-type project which I have been coding up in C++, but it is taking a long time and does not support the features that I need so I started to investigate Golang multiplexers/routers, and Proxy servers.

What I am looking for is to develop a type of "middleware" that receives a REST API call from a frontend server and then re-writes the request so that it can be sent out like a proxy server to a backend REST API data server that will return a JSON result.

That JSON result will be re-written into a different JSON format & structure which will be sent back to the original caller.

I am wondering if your "GoProxy" (Proxy Networks) could be made to work along these lines, so I wanted to ask and investigate the potential.

Any ideas, or suggestion would be greatly appreciated.
Thanks in advance.

[Outstep]

Sorry for the message, I just found your group on Gitter and Telegram

[snail007]

your purpose is easily achieved by golang,the fllowing is an example for you,

package main

import (
    "fmt"
    "io"
    "net/http"
    "time"
)

var verbose = false;

var passthruRequestHeaderKeys = [...]string{
    "Accept",
    "Accept-Encoding",
    "Accept-Language",
    "Cache-Control",
    "Cookie",
    "Referer",
    "User-Agent",
}

var passthruResponseHeaderKeys = [...]string{
    "Content-Encoding",
    "Content-Language",
    "Content-Type",
    "Cache-Control", // TODO: Is this valid in a response?
    "Date",
    "Etag",
    "Expires",
    "Last-Modified",
    "Location",
    "Server",
    "Vary",
}

func main() {
    handler := http.DefaultServeMux
    
    handler.HandleFunc("/", handleFunc)
    
    s := &http.Server{
        Addr:           ":8080",
        Handler:        handler,
        ReadTimeout:    10 * time.Second,
        WriteTimeout:   10 * time.Second,
        MaxHeaderBytes: 1 << 20,
    }
    
    s.ListenAndServe()
}

func handleFunc(w http.ResponseWriter, r *http.Request) {
    fmt.Printf("--> %v %v\n", r.Method, r.URL)
    
    // Construct filtered header to send to origin server
    hh := http.Header{}
    for _, hk := range passthruRequestHeaderKeys {
        if hv, ok := r.Header[hk]; ok {
            hh[hk] = hv
        }
    }
    
    // Construct request to send to origin server
    rr := http.Request{
        Method: r.Method,
        URL: r.URL,
        Header: hh,
        Body: r.Body,
        // TODO: Is this correct for a 0 value?
        //       Perhaps a 0 may need to be reinterpreted as -1?
        ContentLength: r.ContentLength,
        Close: r.Close,
    }
    
    // Forward request to origin server
    resp, err := http.DefaultTransport.RoundTrip(&rr)
    if err != nil {
        // TODO: Passthru more error information
        http.Error(w, "Could not reach origin server", 500)
        return
    }
    defer resp.Body.Close()
    
    if (verbose) {
        fmt.Printf("<-- %v %+v\n", resp.Status, resp.Header)
    } else {
        fmt.Printf("<-- %v\n", resp.Status)
    }
    
    // Transfer filtered header from origin server -> client
    respH := w.Header()
    for _, hk := range passthruResponseHeaderKeys {
        if hv, ok := resp.Header[hk]; ok {
            respH[hk] = hv
        }
    }
    w.WriteHeader(resp.StatusCode)
    
    // Transfer response from origin server -> client
    if resp.ContentLength > 0 {
        // (Ignore I/O errors, since there's nothing we can do)
        io.CopyN(w, resp.Body, resp.ContentLength)
    } else if (resp.Close) { // TODO: Is this condition right?
        // Copy until EOF or some other error occurs
        for {
            if _, err := io.Copy(w, resp.Body); err != nil {
                break
            }
        }
    }
}

[Outstep]

Thank you so much for getting back to me and for sending over the code. It looks like I may have to spend some time to get up to speed on the power of Golang as it seems to have some advantages over my old experiences with C/C++. For the project that I was discussing I was thinking about trying to combine Goproxy (https://github.com/snail007/goproxy) with the routing/multiplexer abilities of Goji (https://goji.io/) or a similar router library so that you could establish channels in the re-writing JSON proxy so that it could talk to different JSON (or possibly XML) backend servers to collect returns and send them forward to the frontend web server. This might have potential as a MetaSearch engine middleware that gets data from different existing search engines and aggregates all of returns into a single collection that returns a JSON to the frontend server. That's the basic idea that I am shooting for, but with a simpler start of just re-writing a single JSON source to test things out. Cheers and have a great day :)

…

On Sat, Apr 28, 2018 at 12:32 AM, snail007 ***@***.***> wrote: your purpose is easyly achieved by golang,the fllowing is an example for you, package main import ( "fmt" "io" "net/http" "time" ) var verbose = false; var passthruRequestHeaderKeys = [...]string{ "Accept", "Accept-Encoding", "Accept-Language", "Cache-Control", "Cookie", "Referer", "User-Agent", } var passthruResponseHeaderKeys = [...]string{ "Content-Encoding", "Content-Language", "Content-Type", "Cache-Control", // TODO: Is this valid in a response? "Date", "Etag", "Expires", "Last-Modified", "Location", "Server", "Vary", } func main() { handler := http.DefaultServeMux handler.HandleFunc("/", handleFunc) s := &http.Server{ Addr: ":8080", Handler: handler, ReadTimeout: 10 * time.Second, WriteTimeout: 10 * time.Second, MaxHeaderBytes: 1 << 20, } s.ListenAndServe() } func handleFunc(w http.ResponseWriter, r *http.Request) { fmt.Printf("--> %v %v\n", r.Method, r.URL) // Construct filtered header to send to origin server hh := http.Header{} for _, hk := range passthruRequestHeaderKeys { if hv, ok := r.Header[hk]; ok { hh[hk] = hv } } // Construct request to send to origin server rr := http.Request{ Method: r.Method, URL: r.URL, Header: hh, Body: r.Body, // TODO: Is this correct for a 0 value? // Perhaps a 0 may need to be reinterpreted as -1? ContentLength: r.ContentLength, Close: r.Close, } // Forward request to origin server resp, err := http.DefaultTransport.RoundTrip(&rr) if err != nil { // TODO: Passthru more error information http.Error(w, "Could not reach origin server", 500) return } defer resp.Body.Close() if (verbose) { fmt.Printf("<-- %v %+v\n", resp.Status, resp.Header) } else { fmt.Printf("<-- %v\n", resp.Status) } // Transfer filtered header from origin server -> client respH := w.Header() for _, hk := range passthruResponseHeaderKeys { if hv, ok := resp.Header[hk]; ok { respH[hk] = hv } } w.WriteHeader(resp.StatusCode) // Transfer response from origin server -> client if resp.ContentLength > 0 { // (Ignore I/O errors, since there's nothing we can do) io.CopyN(w, resp.Body, resp.ContentLength) } else if (resp.Close) { // TODO: Is this condition right? // Copy until EOF or some other error occurs for { if _, err := io.Copy(w, resp.Body); err != nil { break } } } } — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#64 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AXkxHv4_e6X183MZSa7EkB5hkoTiTTnDks5ts_DfgaJpZM4Tqevt> .

[snail007]

You are beginner of golang, and also http proxies; Through you say above, i have some suggestions for you with below:
1.a powerful http server library in golang, https://github.com/valyala/fasthttp
2.a powerful router of fasthttp https://github.com/buaazp/fasthttprouter
3.json encode/decode example in golang, https://golang.org/src/encoding/json/example_test.go
4.http request library,https://github.com/parnurzeal/gorequest