-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fulfil the open-source checklist #1
Comments
A quick comparison with libraries like https://github.com/zackehh/cachex ? |
Is cachex distributed though? I'd like to see (or plan to setup) some performance benchmarks. edit: you should also post on https://elixirforum.com. I found Nebulex from the Elixir Radar newletter |
exactly. comparison :) |
@princemaple yes, good point, I'll include this as part of the open source list (maybe write a blog post about it). @cohawk performance benchmark is actually one the requirements (highest priority) before to launch the v1.0.0, and I think it will come very very soon! And it should be included in this list as well, thanks! BTW, I'll post it on elixir forum as well :) !! |
@cohawk I just added a simple benchmarks, using benchfella, but those are very simple/naive tests, but I think at least it is a good starting point. These tests runs either for the local generational adapter and the distributed adapter, spawning three nodes, and setting up 1000 cache entries (this applies either for the local and distributed cache). My goal is to do some load/stress tests as well, in order to have a better and more real idea about the throughput and latency under heavy load, but I'll open a separated issue for that. Meanwhile you can run the simple benchmarks:
Overall, IMHO I think results looks pretty pretty good :), but as it is recommended: "you should do and run your own performance and/or load tests". Please try to run the bench and let me know your thoughts! On other hand, remember one of the main features of Nebulex is its extensibility, you are able to add your own adapters, for example an adapter for Redis, or an adapter for Cachex in order to have other option for local cache and use the distributed adapter on top of it, and actually this adapter is in the roadmap, the idea is not to compete with Cachex, the idea is they can complement each other. Anyways, this is where the @princemaple suggestion comes in – blog post coming! |
@princemaple Cachex no longer support distributed cache |
@cabol I'm curious why an issue was never opened on the Cachex repository to introduce your ideas in the existing codebase, rather than jumping to starting new project? Especially seeing reimplementation of existing Cachex features inside this project (hooks, stats, etc). I still have intentions of bringing distribution back as a first class feature in Cachex, and I'm curious whether we can merge ideas somehow to maintain the Cachex feature set whilst providing the distribution aspects. @WolfDan & @cohawk: to clarify on Cachex's "support" a little; distribution is only no longer a first class feature. If you were trying to replace Redis (or whatever), you could just have a master node in your cluster which hosts a Cachex cache in memory and have the other nodes RPC over to it to retrieve their data - that's not something which has to be explicitly "supported", so to speak. |
@whitfin first of all thanks for your thoughts!
That is a good question, my plan with Nebulex wasn't only to provide another Cache, but provide a Caching framework or wrapper that allows you to implement different types of Cache, different eviction algorithms, caching topologies, etc. For that reason, I took Ecto as inspiration, the API and its pluggable design based on adapters. Based on these ideas:
In terms of additional and/or complementary features (Nice-To-Have features) like hooks, stats, etc., you are right, these things are very similar. But, in terms of core features in a cache, like backend, eviction, generational cache, distribution, topologies, etc., Nebulex is not re-implementing Cachex at all, in that sense I think Nebulex is different.
That sounds great, I'm open to discuss this alternative in details. As I mentioned before, one of the ideas I have in mind, is to implement an adapter for Cachex, for me this can be a very nice feature, because we would be combining the best of both, Cachex as a local cache backend plus the distribution features of Nebulex, be able to implement the same caching topologies but having Cachex as a backend. Of course, if you have other ideas we can discuss them, and l'll glad to help :) !! Let's try to have a more detailed talk so we can cross our ideas and maybe come up with something nicer!!! Thanks! |
That's fair (and it would have).
Cachex is a little more than LRU (in fact, technically it's not LRU at all). Eviction is driven by expiration on keys, but size bounds can be provided using policies (
Sure, and even if that was your intention, competition isn't a bad thing :) I'd be interested in assisting you writing an adapter, if it comes to it.
The next step with Cachex now that v3 is out of the way, is actually a v3.1 which uses a sharding algorithm to shard across nodes in a cluster. This would be opt in and totally up to the user to control their node connections (since that's too much outside of the scope of Cachex), but essentially you would pass It'll be a basic algorithm which controls what goes where, no load balancing etc, but it provides the basic interaction a Cachex cache would need in a distributed state. Perhaps this is where we could focus the Cachex adapter pieces for Nebulex, as it can control the node management and then delegate the caching to Cachex. |
@whitfin I like the idea, actually Cachex might be the default cache backend, I like that, I'll start thinking on it, the simplest approach would be start writing the adapter mapping its callbacks to Cachex functions, but I'd like to review Cachex more in details because it has a lot of features we cannot afford to loose, so this mapping should be designed very carefully. Of course, the other option is to use the Cachex API, but I think it migh be a bit more tricky, in that case it mighr be better to take the distributed part of Nebulex and make the necessary fixes to integrate it within Cachex, or maybe in a separated repo. Anyways, I think we should consider to implement both, just to give more flexibility to the users, using the Cachex or the Nebulex API. So, I'll start working on the Cachex adapter for Nebulex and of course I'll be requesting your assistance :) – BTW let me know if you have something in mind already or some work already done that we can reuse as base. Thanks and let's keep in touch!! |
General Items
Features for First Version
Exhibition
Examples
Publish
The text was updated successfully, but these errors were encountered: