Proposal to use Souin as cache system core to tyk

[darkweak]

What does Souin support ?

It's RFC compatible, supporting Vary, request coalescing and other specifications related to the RFC-7234.
It also supports the Cache-Status HTTP response header.
And the Ykeys cache invalidation support like Varnish will be merge this week.
It supports Olric as embedded mode and client mode to be able to store the cache through a distributed system. And for those who don't need a distributed storage system, it comes with the Badger storage system support (once the ykeys will be merge on master).
At the moment it works out of the box, as caddy module and as træfik unofficial plugin.

Finally I'll maintain Souin with the help of existing contributors, and I'm in contact with some pentesters to make an audit on this cache system to ensure this is safe for production.

I think it won't take more than 1 week to write a PoC and submit a review for this if you think it could be a good feature.

Open to discuss with you about it.

[buger]

Hi!

Happy to discuss it!
Can you give an idea of what it worth to do the integration with Souin?

[darkweak]

Hi @buger thanks for taking time to discuss about that.

I found the Tyk caching section here but it doesn't seem to be configurable in depth. With Souin you can configure the cache to store XX seconds per URL regex x headers combo.
It comes with an API to invalidate the cache by plain key, regex and ykeys instead of fully invalidate the cache.
And as it supports the distributed storage, it will ensure the high availability.

To talk about the technical implementation, I just have to write the plugin file (such as an adapter). You can find examples inside the Souin plugins directory. And we won't have to duplicate any code. By implementing your interface, I should be able to link Souin as Tyk plugin/middleware.

Do you think is it worth it to invest time to develop a poc?

[jenaye]

Hi @buger, I can confirm that an audit is planned for the Souin application soon.
We can of course give you our report at the end of it if you are interested

[buger]

I'm not quite sure then what is expected from our side here :)

If you're going to write some Tyk plugin which makes it talk with Souin will be great (we have support for Go plugins, so I guess smth like Caddy have should work https://tyk.io/docs/plugins/supported-languages/golang/). And since it does not touch main Tyk codebase, I do not see any blockers here.

After we for sure can add it to our documentation, as one of the options for caching. And maybe do some mutual marcoms campaign. If you want to chat more in details, feel free to reach me at leo@tyk.io

[darkweak]

Thanks for your feedback. It sounds great, I'll try to write a poc asap.
I won't touch the Tyk codebase and it would be great to work with the tyk team to integrate it in the documentation in the future :)
Thank you for linking resources and your contact email.

[darkweak]

@buger the first implementation of Souin as tyk middleware is available here : https://github.com/Darkweak/Souin/tree/master/plugins/tyk 🎉

[andyo-tyk]

Hi @darkweak,
I had a look at your repo and can see some recent commits.
I'd be more than happy to collaborate with you to look at how we can integrate Souin to the Tyk documentation and get you in touch with the right people here in Tyk.
Is this something you'd be interested in picking up?

[darkweak]

Hello @andyo-tyk, that would be awesome.

[darkweak]

@andyo-tyk Is it possible to discuss with the Tyk developer team about some plugin loading/behavior issues?