Skip to content

Commit

Permalink
(#1554) Add support for all common compressions (write and read)
Browse files Browse the repository at this point in the history
- Remove the context.Context interface and export the *context, the iris.Context now points to the pointer\nSupport compression and rate limiting in the FileServer\nBit of code organisation


Former-commit-id: ad1c61bf968059510c6be9e7f2cceec7da70ba17
  • Loading branch information
kataras committed Jul 10, 2020
1 parent 645da2b commit 0f113df
Show file tree
Hide file tree
Showing 112 changed files with 2,113 additions and 3,384 deletions.
18 changes: 13 additions & 5 deletions HISTORY.md
Expand Up @@ -431,18 +431,18 @@ New Package-level Variables:

- `iris.DirListRich` to override the default look and feel if the `DirOptions.ShowList` was set to true, can be passed to `DirOptions.DirList` field.
- `iris.DirListRichOptions` to pass on `iris.DirListRich` method.
- `iris.ErrGzipNotSupported` to export the `context.ErrGzipNotSupported` when trying to write gzip but client does not support.
- `iris.GzipReader` middleware to decode gzip requests on next read actions.
- `iris.Compress` and `iris.CompressReader` middleware to compress responses and decode compressed request data respectfully.
- `iris.B, KB, MB, GB, TB, PB, EB` for byte units.
- `TLSNoRedirect` to disable automatic "http://" to "https://" redirections (see below)
- `CookieAllowReclaim`, `CookieAllowSubdomains`, `CookieSameSite`, `CookieSecure` and `CookieEncoding` to bring previously sessions-only features to all cookies in the request.

New Context Methods:

- `Context.Compress(bool) error` and `Context.CompressReader(bool) error`
- `Context.Clone() Context` returns a copy of the Context.
- `Context.IsCanceled() bool` reports whether the request has been canceled by the client.
- `Context.IsSSL() bool` reports whether the request is under HTTPS SSL (New `Configuration.SSLProxyHeaders` and `HostProxyHeaders` fields too).
- `Context.GzipReader(enable bool)` method and `iris.GzipReader` middleware to enable future request read body calls to decompress data using gzip, [example](_examples/request-body/read-gzip).
- `Context.CompressReader(enable bool)` method and `iris.CompressReader` middleware to enable future request read body calls to decompress data, [example](_examples/compression/main.go).
- `Context.RegisterDependency(v interface{})` and `Context.UnregisterDependency(typ reflect.Type)` to register/remove struct dependencies on serve-time through a middleware.
- `Context.SetID(id interface{})` and `Context.GetID() interface{}` added to register a custom unique indetifier to the Context, if necessary.
- `Context.GetDomain() string` returns the domain.
Expand All @@ -469,23 +469,31 @@ New Context Methods:

Breaking Changes:

- `ctx.Gzip(boolean)` replaced with `ctx.Compress(boolean) error`.
- `ctx.GzipReader(boolean) error` replaced with `ctx.CompressReader(boolean) error`.
- `iris.Gzip` replaced with `iris.Compress` (middleware).
- `iris.GzipReader` replaced with `iris.CompressReader` (middleware).
- `ctx.ClientSupportsGzip() bool` replaced with `ctx.ClientSupportsEncoding("gzip", "br" ...) bool`.
- `ctx.GzipResponseWriter()` is **removed**.
- `Party.HandleDir` now returns a list of `[]*Route` (GET and HEAD) instead of GET only.
- `Context.OnClose` and `Context.OnCloseConnection` now both accept an `iris.Handler` instead of a simple `func()` as their callback.
- `Context.StreamWriter(writer func(w io.Writer) bool)` changed to `StreamWriter(writer func(w io.Writer) error) error` and it's now the `Context.Request().Context().Done()` channel that is used to receive any close connection/manual cancel signals, instead of the deprecated `ResponseWriter().CloseNotify()` one. Same for the `Context.OnClose` and `Context.OnCloseConnection` methods.
- Fixed handler's error response not be respected when response recorder or gzip writer was used instead of the common writer. Fixes [#1531](https://github.com/kataras/iris/issues/1531). It contains a **BREAKING CHANGE** of: the new `Configuration.ResetOnFireErrorCode` field should be set **to true** in order to behave as it used before this update (to reset the contents on recorder or gzip writer).
- Fixed handler's error response not be respected when response recorder was used instead of the common writer. Fixes [#1531](https://github.com/kataras/iris/issues/1531). It contains a **BREAKING CHANGE** of: the new `Configuration.ResetOnFireErrorCode` field should be set **to true** in order to behave as it used before this update (to reset the contents on recorder).
- `Context.String()` (rarely used by end-developers) it does not return a unique string anymore, to achieve the old representation you must call the new `Context.SetID` method first.
- `iris.CookieEncode` and `CookieDecode` are replaced with the `iris.CookieEncoding`.
- `sessions#Config.Encode` and `Decode` are removed in favor of (the existing) `Encoding` field.
- `versioning.GetVersion` now returns an empty string if version wasn't found.
- Change the MIME type of `Javascript .js` and `JSONP` as the HTML specification now recommends to `"text/javascript"` instead of the obselete `"application/javascript"`. This change was pushed to the `Go` language itself as well. See <https://go-review.googlesource.com/c/go/+/186927/>.
- Remove the last input argument of `enableGzipCompression` in `Context.ServeContent`, `ServeFile` methods. This was deprecated a few versions ago. A middleware (`app.Use(iris.Gzip)`) or a prior call to `Context.Gzip(true)` will enable gzip compression. Also these two methods and `Context.SendFile` one now support `Content-Range` and `Accept-Ranges` correctly out of the box (`net/http` had a bug, which is now fixed).
- Remove the last input argument of `enableGzipCompression` in `Context.ServeContent`, `ServeFile` methods. This was deprecated a few versions ago. A middleware (`app.Use(iris.Compress)`) or a prior call to `Context.Compress(true)` will enable compression. Also these two methods and `Context.SendFile` one now support `Content-Range` and `Accept-Ranges` correctly out of the box (`net/http` had a bug, which is now fixed).
- `Context.ServeContent` no longer returns an error, see `ServeContentWithRate`, `ServeFileWithRate` and `SendFileWithRate` new methods too.
- `route.Trace() string` changed to `route.Trace(w io.Writer)`, to achieve the same result just pass a `bytes.Buffer`
- `var mvc.AutoBinding` removed as the default behavior now resolves such dependencies automatically (see [[FEATURE REQUEST] MVC serving gRPC-compatible controller](https://github.com/kataras/iris/issues/1449)).
- `mvc#Application.SortByNumMethods()` removed as the default behavior now binds the "thinnest" empty `interface{}` automatically (see [MVC: service injecting fails](https://github.com/kataras/iris/issues/1343)).
- `mvc#BeforeActivation.Dependencies().Add` should be replaced with `mvc#BeforeActivation.Dependencies().Register` instead
- **REMOVE** the `kataras/iris/v12/typescript` package in favor of the new [iris-cli](https://github.com/kataras/iris-cli). Also, the alm typescript online editor was removed as it is deprecated by its author, please consider using the [designtsx](https://designtsx.com/) instead.

There is a breaking change on the type alias of `iris.Context` which now points to the `*context.Context` instead of the `context.Context` interface. The **interface has been removed** and the ability to **override** the Context **is not** available any more. When we added the ability from end-developers to override the Context years ago, we have never imagine that we will ever had such a featured Context with more than 4000 lines of code. As of Iris 2020, it is difficult and un-productive from an end-developer to override the Iris Context, and as far as we know, nobody uses this feature anymore because of that exact reason. Beside the overriding feature support end, if you still use the `context.Context` instead of `iris.Context`, it's the time to do it: please find-and-replace to `iris.Context` as wikis, book and all examples shows for the past 3 years. For the 99.9% of the users there is no a single breaking change, you already using `iris.Context` so you are in the "safe zone".

# Su, 16 February 2020 | v12.1.8

New Features:
Expand Down
7 changes: 5 additions & 2 deletions NOTICE
Expand Up @@ -22,7 +22,10 @@ Revision ID: d1c07411df0bb21f6b21f5b5d9325fac6f29c911
a16f63faca
bluemonday 0a75d7616912ab9 https://github.com/microcosm-cc/
beb9cc6f7283ec1 bluemonday
917c61b135
917c61b135
brotli c3da72aa01ed78f https://github.com/andybalholm/brotli
164593b9624fd91
d25082d2d2
closestmatch 1fbe626be92eb4c https://github.com/schollz/closestmatch
347d182cae9f8f0
0a046bf2f4
Expand Down Expand Up @@ -97,4 +100,4 @@ Revision ID: d1c07411df0bb21f6b21f5b5d9325fac6f29c911
f5c1929ae8
uuid cb32006e483f2a2 https://github.com/google/uuid
3230e24209cf185
c65b477dbf
c65b477dbf
9 changes: 4 additions & 5 deletions _examples/README.md
Expand Up @@ -54,9 +54,6 @@
* [Reverse Routing](routing/reverse/main.go)
* [Router Wrapper](routing/custom-wrapper/main.go)
* [Custom Router](routing/custom-router/main.go)
* Custom Context
* [Method Overriding](routing/custom-context/method-overriding/main.go)
* [New Implementation](routing/custom-context/new-implementation/main.go)
* Subdomains
* [Single](routing/subdomains/single/main.go)
* [Multi](routing/subdomains/multi/main.go)
Expand Down Expand Up @@ -129,12 +126,10 @@
* [Bind Custom per type](request-body/read-custom-per-type/main.go)
* [Bind Custom via Unmarshaler](request-body/read-custom-via-unmarshaler/main.go)
* [Bind Many times](request-body/read-many/main.go)
* [Read/Bind Gzip compressed data](request-body/read-gzip/main.go)
* Response Writer
* [Content Negotiation](response-writer/content-negotiation)
* [Text, Markdown, YAML, HTML, JSON, JSONP, Msgpack, XML and Binary](response-writer/write-rest/main.go)
* [Protocol Buffers](response-writer/protobuf/main.go)
* [Write Gzip](response-writer/write-gzip/main.go)
* [HTTP/2 Server Push](response-writer/http2push/main.go)
* [Stream Writer](response-writer/stream-writer/main.go)
* [Transactions](response-writer/transactions/main.go)
Expand All @@ -143,6 +138,10 @@
* Cache
* [Simple](response-writer/cache/simple/main.go)
* [Client-Side (304)](response-writer/cache/client-side/main.go)
* Compression
* [Server-Side](compression/main.go)
* [Client-Side](compression/client/main.go)
* [Client-Side (using Iris)](compress/client-using-iris/main.go)
* Localization and Internationalization
* [i18n](i18n/main.go)
* Authentication, Authorization & Bot Detection
Expand Down
112 changes: 112 additions & 0 deletions _examples/compression/client-using-iris/main.go
@@ -0,0 +1,112 @@
package main

import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"

"github.com/kataras/iris/v12/context"
)

const baseURL = "http://localhost:8080"

// Available options:
// - "gzip",
// - "deflate",
// - "br" (for brotli),
// - "snappy" and
// - "s2"
const encoding = context.BROTLI

var client = http.DefaultClient

func main() {
fmt.Printf("Running client example on: %s\n", baseURL)

getExample()
postExample()
}

func getExample() {
endpoint := baseURL + "/"
req, err := http.NewRequest(http.MethodGet, endpoint, nil)
if err != nil {
panic(err)
}
// Required to receive server's compressed data.
req.Header.Set("Accept-Encoding", encoding)

resp, err := client.Do(req)
if err != nil {
panic(err)
}
defer resp.Body.Close()

// decompress server's compressed reply.
cr, err := context.NewCompressReader(resp.Body, encoding)
if err != nil {
panic(err)
}
defer cr.Close()

body, err := ioutil.ReadAll(cr)
if err != nil {
panic(err)
}

fmt.Printf("Received from server: %s", string(body))
}

type payload struct {
Username string `json:"username"`
}

func postExample() {
buf := new(bytes.Buffer)

// Compress client's data.
cw, err := context.NewCompressWriter(buf, encoding, -1)
if err != nil {
panic(err)
}

json.NewEncoder(cw).Encode(payload{Username: "Edward"})

// `Close` or `Flush` required before `NewRequest` call.
cw.Close()

endpoint := baseURL + "/"

req, err := http.NewRequest(http.MethodPost, endpoint, buf)
if err != nil {
panic(err)
}
req.Header.Set("Content-Type", "application/json")

// Required to send gzip compressed data to the server.
req.Header.Set("Content-Encoding", encoding)
// Required to receive server's compressed data.
req.Header.Set("Accept-Encoding", encoding)

resp, err := client.Do(req)
if err != nil {
panic(err)
}
defer resp.Body.Close()

// Decompress server's compressed reply.
cr, err := context.NewCompressReader(resp.Body, encoding)
if err != nil {
panic(err)
}
defer cr.Close() // Closes the request body too.

body, err := ioutil.ReadAll(cr)
if err != nil {
panic(err)
}

fmt.Printf("Server replied with: %s", string(body))
}
102 changes: 102 additions & 0 deletions _examples/compression/client/main.go
@@ -0,0 +1,102 @@
package main

import (
"bytes"
"compress/gzip"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
)

var client = http.DefaultClient

const baseURL = "http://localhost:8080"

func main() {
fmt.Printf("Running client example on: %s\n", baseURL)

getExample()
postExample()
}

func getExample() {
endpoint := baseURL + "/"
req, err := http.NewRequest(http.MethodGet, endpoint, nil)
if err != nil {
panic(err)
}
// Required to receive server's compressed data.
req.Header.Set("Accept-Encoding", "gzip")

resp, err := client.Do(req)
if err != nil {
panic(err)
}
defer resp.Body.Close()

// decompress server's compressed reply.
r, err := gzip.NewReader(resp.Body)
if err != nil {
panic(err)
}
defer r.Close()

body, err := ioutil.ReadAll(r)
if err != nil {
panic(err)
}

fmt.Printf("Received from server: %s", string(body))
}

type payload struct {
Username string `json:"username"`
}

func postExample() {
buf := new(bytes.Buffer)

// Compress client's data.
w := gzip.NewWriter(buf)

b, err := json.Marshal(payload{Username: "Edward"})
if err != nil {
panic(err)
}
w.Write(b)
w.Close()

endpoint := baseURL + "/"

req, err := http.NewRequest(http.MethodPost, endpoint, buf)
if err != nil {
panic(err)
}
req.Header.Set("Content-Type", "application/json")

// Required to send gzip compressed data to the server.
req.Header.Set("Content-Encoding", "gzip")
// Required to receive server's compressed data.
req.Header.Set("Accept-Encoding", "gzip")

resp, err := client.Do(req)
if err != nil {
panic(err)
}
defer resp.Body.Close()

// Decompress server's compressed reply.
r, err := gzip.NewReader(resp.Body)
if err != nil {
panic(err)
}
defer r.Close()

body, err := ioutil.ReadAll(r)
if err != nil {
panic(err)
}

fmt.Printf("Server replied with: %s", string(body))
}
64 changes: 64 additions & 0 deletions _examples/compression/main.go
@@ -0,0 +1,64 @@
package main

import "github.com/kataras/iris/v12"

func main() {
app := newApp()
app.Logger().SetLevel("debug")
app.Listen(":8080")
}

func newApp() *iris.Application {
app := iris.New()
// HERE and you are ready to GO:
app.Use(iris.Compress, iris.CompressReader)

app.Get("/", send)
app.Post("/", receive)

return app
}

type payload struct {
Username string `json:"username"`
}

func send(ctx iris.Context) {
ctx.JSON(payload{
Username: "Makis",
})
}

func receive(ctx iris.Context) {
var p payload
if err := ctx.ReadJSON(&p); err != nil {
ctx.Application().Logger().Debugf("ReadJSON: %v", err)
}

ctx.WriteString(p.Username)
}

/* Manually:
func enableCompression(ctx iris.Context) {
// Enable writing using compression (deflate, gzip, brotli, snappy, s2):
err := ctx.Compress(true)
if err != nil {
ctx.Application().Logger().Debugf("writer: %v", err)
// if you REQUIRE server to SEND compressed data then `return` here.
// return
}
// Enable reading and binding request's compressed data:
err = ctx.CompressReader(true)
if err != nil &&
// on GET we don't expect writing with gzip from client
ctx.Method() != iris.MethodGet {
ctx.Application().Logger().Debugf("reader: %v", err)
// if you REQUIRE server to RECEIVE only
// compressed data then `return` here.
// return
}
ctx.Next()
}
*/

0 comments on commit 0f113df

Please sign in to comment.