Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimization(all): Decrease allocs and responses time #228

Merged
merged 10 commits into from
Jul 10, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 43 additions & 1 deletion .github/workflows/plugins.yml
Original file line number Diff line number Diff line change
Expand Up @@ -373,6 +373,48 @@ jobs:
folder: Goyave
reporters: cli
delayRequest: 5000
build-kratos-validator:
name: Check that Souin build as Kratos middleware
runs-on: ubuntu-latest
steps:
-
name: Add domain.com host to /etc/hosts
run: |
sudo echo "127.0.0.1 domain.com" | sudo tee -a /etc/hosts
-
name: Install Go
uses: actions/setup-go@v2
with:
go-version: 1.18
-
name: Checkout code
uses: actions/checkout@v2
-
name: Build Souin as Kratos plugin
run: make build-and-run-kratos
-
name: Wait for Souin is really loaded inside Kratos as middleware
uses: jakejarvis/wait-action@master
with:
time: 50s
-
name: Set Kratos logs configuration result as environment variable
run: cd plugins/kratos && echo "$(make load-checker)" >> $GITHUB_ENV
-
name: Check if the configuration is loaded to define if Souin is loaded too
uses: nick-invision/assert-action@v1
with:
expected: '"Souin configuration is now loaded."'
actual: ${{ env.MIDDLEWARE_RESULT }}
comparison: contains
-
name: Run Kratos E2E tests
uses: anthonyvscode/newman-action@v1
with:
collection: "docs/e2e/Souin E2E.postman_collection.json"
folder: Kratos
reporters: cli
delayRequest: 5000
build-skipper-validator:
name: Check that Souin build as Skipper middleware
runs-on: ubuntu-latest
Expand All @@ -396,7 +438,7 @@ jobs:
name: Wait for Souin is really loaded inside Skipper as middleware
uses: jakejarvis/wait-action@master
with:
time: 50s
time: 60s
-
name: Set Skipper logs configuration result as environment variable
run: cd plugins/skipper && echo "$(make load-checker)" >> $GITHUB_ENV
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/workflow_plugins_generator.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
#!/bin/bash

plugins=("beego" "chi" "dotweb" "echo" "fiber" "gin" "go-zero" "goyave" "skipper" "souin" "traefik" "tyk" "webgo")
durations=("35" "30" "30" "30" "45" "30" "50" "35" "50" "40" "30" "30" "30")
versions=("16" "16" "16" "16" "16" "16" "16" "16" "18" "16" "16" "16" "16")
plugins=("beego" "chi" "dotweb" "echo" "fiber" "gin" "go-zero" "goyave" "kratos" "skipper" "souin" "traefik" "tyk" "webgo")
durations=("35" "30" "30" "30" "45" "30" "50" "35" "50" "60" "40" "30" "30" "30")
versions=("16" "16" "16" "16" "16" "16" "16" "16" "18" "18" "16" "16" "16" "16")

IFS= read -r -d '' tpl <<EOF
name: Build and validate Souin as plugins
Expand Down
4 changes: 3 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
DC=docker-compose
DC_BUILD=$(DC) build
DC_EXEC=$(DC) exec
PLUGINS_LIST=beego caddy chi dotweb echo fiber skipper gin go-zero goyave traefik tyk webgo souin
PLUGINS_LIST=beego caddy chi dotweb echo fiber gin go-zero goyave kratos skipper traefik tyk webgo souin

base-build-and-run-%:
cd plugins/$* && $(MAKE) prepare
Expand All @@ -29,6 +29,8 @@ build-and-run-echo: base-build-and-run-echo ## Run Echo with Souin as plugin

build-and-run-fiber: base-build-and-run-fiber ## Run Fiber with Souin as plugin

build-and-run-kratos: base-build-and-run-kratos ## Run Kratos with Souin as plugin

build-and-run-skipper: base-build-and-run-skipper ## Run Skipper with Souin as plugin

build-and-run-souin: base-build-and-run-souin ## Run Souin as plugin
Expand Down
74 changes: 67 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,13 @@
8.7. [Gin middleware](#gin-middleware)
8.8. [Go-zero middleware](#go-zero-middleware)
8.9. [Goyave middleware](#goyave-middleware)
8.10. [Skipper filter](#skipper-filter)
8.11. [Træfik plugin](#træfik-plugin)
8.12. [Tyk plugin](#tyk-plugin)
8.13. [Webgo middleware](#webgo-middleware)
8.14. [Prestashop plugin](#prestashop-plugin)
8.15. [Wordpress plugin](#wordpress-plugin)
8.10. [Kratos filter](#kratos-filter)
8.11. [Skipper filter](#skipper-filter)
8.12. [Træfik plugin](#træfik-plugin)
8.13. [Tyk plugin](#tyk-plugin)
8.14. [Webgo middleware](#webgo-middleware)
8.15. [Prestashop plugin](#prestashop-plugin)
8.16. [Wordpress plugin](#wordpress-plugin)
9. [Credits](#credits)

# Souin HTTP cache
Expand Down Expand Up @@ -726,6 +727,65 @@ func main() {
}
```

### Kratos filter
To use Souin as Kratos filter, you can refer to the [Kratos plugin integration folder](https://github.com/darkweak/souin/tree/master/plugins/kratos) to discover how to configure it.
You just have to start the Kratos HTTP server with the Souin filter like below:
```go
import (
httpcache "github.com/darkweak/souin/plugins/kratos"
kratos_http "github.com/go-kratos/kratos/v2/transport/http"
)

func main() {
kratos_http.NewServer(
kratos_http.Filter(
httpcache.NewHTTPCacheFilter(httpcache.DevDefaultConfiguration),
),
)
}
```

You can also use the configuration file to configuration the HTTP cache. Refer to the code block below:
```
server: #...
data: #...
# HTTP cache part
httpcache:
api:
souin: {}
default_cache:
regex:
exclude: /excluded
ttl: 5s
log_level: debug
```
After that you have to edit your server instanciation to use the HTTP cache configuration parser
```go
import (
httpcache "github.com/darkweak/souin/plugins/kratos"
kratos_http "github.com/go-kratos/kratos/v2/transport/http"
)

func main() {
c := config.New(
config.WithSource(file.NewSource("examples/configuration.yml")),
config.WithDecoder(func(kv *config.KeyValue, v map[string]interface{}) error {
return yaml.Unmarshal(kv.Value, v)
}),
)
if err := c.Load(); err != nil {
panic(err)
}

server := kratos_http.NewServer(
kratos_http.Filter(
httpcache.NewHTTPCacheFilter(httpcache.ParseConfiguration(c)),
),
)
// ...
}
```

### Skipper filter
To use Souin as skipper filter, you can refer to the [Skipper plugin integration folder](https://github.com/darkweak/souin/tree/master/plugins/skipper) to discover how to configure it.
You just have to add to your Skipper instance the Souin filter like below:
Expand Down Expand Up @@ -763,7 +823,7 @@ experimental:
plugins:
souin:
moduleName: github.com/darkweak/souin
version:
version: v1.6.12
```
After that you can declare either the whole configuration at once in the middleware block or by service. See the examples below.
```yaml
Expand Down
4 changes: 3 additions & 1 deletion api/prometheus/prometheus.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,9 @@ var registered map[string]interface{}

// Increment will increment the counter.
func Increment(name string) {
registered[name].(prometheus.Counter).Inc()
if _, ok := registered[name]; ok {
registered[name].(prometheus.Counter).Inc()
}
}

// Increment will add the referred value the counter.
Expand Down
11 changes: 11 additions & 0 deletions docker-compose.yml.test
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,17 @@ services:
- "2380:2380"
command: sh -c 'apk update; apk add curl;etcd -listen-client-urls http://0.0.0.0:2379 -advertise-client-urls http://etcd:2379'
<<: *networks

traefik:
image: traefik:latest
command: --providers.docker
volumes:
- /var/run/docker.sock:/var/run/docker.sock

whoami:
image: traefik/whoami
labels:
- traefik.http.routers.whoami.rule=Host(`domain.com`)
networks:
your_network:
external: true
162 changes: 161 additions & 1 deletion docs/e2e/Souin E2E.postman_collection.json
Original file line number Diff line number Diff line change
Expand Up @@ -1794,6 +1794,161 @@
}
]
},
{
"name": "Kratos",
"item": [
{
"name": "Default",
"event": [
{
"listen": "test",
"script": {
"exec": [
"utils.baseEndpoint(pm, `${utils.getVar(pm, 'app_url')}/default`, '', true)"
],
"type": "text/javascript"
}
}
],
"request": {
"method": "GET",
"header": [
{
"key": "Cache-Control",
"value": "",
"type": "text"
}
],
"url": {
"raw": "{{app_url}}/default",
"host": [
"{{app_url}}"
],
"path": [
"default"
]
}
},
"response": []
},
{
"name": "Default no cache",
"event": [
{
"listen": "test",
"script": {
"exec": [
"utils.baseEndpoint(pm, `${utils.getVar(pm, 'app_url')}/default`, 'no-cache')"
],
"type": "text/javascript"
}
}
],
"request": {
"method": "GET",
"header": [],
"url": {
"raw": "{{app_url}}/default",
"host": [
"{{app_url}}"
],
"path": [
"default"
]
}
},
"response": []
},
{
"name": "Default no store",
"event": [
{
"listen": "test",
"script": {
"exec": [
"utils.baseEndpoint(pm, `${utils.getVar(pm, 'app_url')}/default`, 'no-store')"
],
"type": "text/javascript"
}
}
],
"request": {
"method": "GET",
"header": [
{
"key": "Cache-Control",
"value": "no-store",
"type": "text"
}
],
"url": {
"raw": "{{app_url}}/default",
"host": [
"{{app_url}}"
],
"path": [
"default"
]
}
},
"response": []
},
{
"name": "Exclude regex",
"event": [
{
"listen": "test",
"script": {
"exec": [
"utils.baseEndpoint(pm, `${utils.getVar(pm, 'app_url')}/excluded`, '', false, true)"
],
"type": "text/javascript"
}
}
],
"request": {
"method": "GET",
"header": [
{
"key": "Cache-Control",
"value": "",
"type": "text"
}
],
"url": {
"raw": "{{app_url}}/excluded",
"host": [
"{{app_url}}"
],
"path": [
"excluded"
]
}
},
"response": []
}
],
"event": [
{
"listen": "prerequest",
"script": {
"type": "text/javascript",
"exec": [
"pm.collectionVariables.set('app_url', pm.collectionVariables.get('kratos_url'))"
]
}
},
{
"listen": "test",
"script": {
"type": "text/javascript",
"exec": [
""
]
}
}
]
},
{
"name": "Skipper",
"item": [
Expand Down Expand Up @@ -2781,6 +2936,11 @@
"key": "beego_url",
"value": "http://domain.com",
"type": "string"
},
{
"key": "kratos_url",
"value": "http://domain.com",
"type": "string"
}
]
}
}
Loading