-
Notifications
You must be signed in to change notification settings - Fork 11
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
d032ce4
commit 9737c84
Showing
2 changed files
with
5 additions
and
6 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,6 @@ | ||
| { | ||
| "name": "Rede", | ||
| "tagline": "A Redis dehydrator module", | ||
| "body": "# ReDe - A Redis Element Dehydration Module\r\n\r\n:rocket:**TL;DR - A Dehydrator is a fancy delayed queue, see what commands this module provides [HERE](Commands.md)**\r\n\r\n\r\n**ReDe** /'redɪ/ *n.* a simple Redis Module for data dehydration. This is a pretty straightforward implementation of the dehydration system depicted in the article \"[Fast Data](https://goo.gl/DDFFPO)\". The Goal of this module is to solve the *Contextual Completeness* and *Emergent Relevancy* problems by adding the ability to postpone incoming elements to a later time in which we will have a complete information for these elements. Effectively acting as a snooze button to any element.\r\n\r\nFrom the article:\r\n> Dehydrators are simplistic time machines. They transport data elements that arrived prematurely in terms of their context right to the future where they might be needed, without loading the system while waiting. This concept is achieved by attaching a time-indexed data store to a clock, storing elements as they arrive to the dehydrator and re-introducing them as inputs to the system once a predetermined time period has passed.\r\n\r\nUsing this system it is also possible to craft a self cleaning \"claims check\", to minimize load on transportation and manipulation nodes of a pipeline architecture.\r\n\r\n*You can read further on the algorithm behind this module [here](Algorithm.md).*\r\n\r\nThe module works by adding a new type to Redis -`DehydratorType`. It will be ceated automatically when you call a push command on it, and it can be deleted using the `DEL` command like any other key.\r\n\r\n\r\n\r\n### Common Use Cases\r\n\r\n* **Stream Coordination** - Make data from one stream wait for the corresponding data from another (preferebly using sliding-window style timing).\r\n* **Event Rate Limitation** - Delay any event beyond current max throughput to the next available time slot, while preserving order.\r\n* **Self Cleaning Claims-Check** - Store data for a well known period, without the need to search for it when it is expired or clear it from the data-store yourself.\r\n* **Task Timer** - Assign actions and their respective payloads to a specific point in time.\r\n\r\n-----------------------------------------------------\r\n\r\n## Usage\r\n\r\nThe dehydrator is an effective 'snooze button' for events, you push an event into it along with an id (for future referance) and in how many seconds you want it back, and poll whenever you want the elements back. only expired elements would pop out.\r\n\r\n**It includes 4 main commands:**\r\n\r\n* [`REDE.PUSH`](Commands.md/#push) - Push an element, it will need an id, the element itself and dehydration time in seconds.\r\n* [`REDE.PULL`](Commands.md/#pull) - Pull the element with the appropriate id before it expires.\r\n* [`REDE.POLL`](Commands.md/#poll) - Pull and return all the expired elements.\r\n* [`REDE.LOOK`](Commands.md/#look) - Search the dehydrator for an element with the given id and if found return it's payload (without pulling).\r\n* [`REDE.UPDATE`](Commands.md/#update) - Set the element represented by a given id, the current element will be returned, and the new element will inherit the current expiration.\r\n* [`REDE.TTN`](Commands.md/#ttn) - Return the minimal time between now and the first expiration\r\n\r\n**The module also includes a test command:**\r\n* `REDE.TEST` - a set of unit tests of the above commands. **NOTE!** This command is running in fixed time (~15 seconds) as it uses `sleep` (dios mio, No! ✞✞✞).\r\n\r\n*see more about the commands in [Commands.md](Commands.md)*\r\n\r\n## Quick Start Guide\r\n\r\nHere's what you need to do to build this module:\r\n\r\n0. Build Redis in a build supporting modules.\r\n1. Build the module: `make`\r\n3. Run Redis loading the module: `/path/to/redis-server --loadmodule ./example/module.so`\r\n\r\nNow run `redis-cli` and try the commands:\r\n\r\n```\r\n127.0.0.1:9979> REDE.PUSH some_dehy id1 world 15\r\nOK\r\n127.0.0.1:9979> REDE.PUSH some_dehy id2 hello 1\r\nOK\r\n127.0.0.1:9979> REDE.PUSH some_dehy id3 goodbye 2\r\nOK\r\n127.0.0.1:9979> REDE.PULL some_dehy id3\r\n\"goodbye\"\r\n127.0.0.1:9979> REDE.POLL some_dehy\r\n1) \"hello\"\r\n127.0.0.1:9979> REDE.POLL some_dehy\r\n(empty list or set)\r\n127.0.0.1:6379> REDE.LOOK some_dehy id2\r\n(nil)\r\n127.0.0.1:6379> REDE.LOOK some_dehy id1\r\n\"world\"\r\n127.0.0.1:6379> REDE.PULL some_dehy id2\r\n(nil)\r\n127.0.0.1:6379> REDE.TTN some_dehy\r\n8\r\n```\r\n\r\nThis `(empty list or set)` reply from `REDE.POLL` means that the there are no more items to pull right now, so we'll have to wait until enough time passes for our next element to expire. using REDE.TTN we can see this will be in 8 seconds (in this example we waited a bit between commands). Once 8 seconds will pass we can run:\r\n\r\n```\r\n127.0.0.1:9979> REDE.POLL some_dehy\r\n1) \"world\"\r\n127.0.0.1:9979> REDE.TEST\r\nPASS\r\n(15.00s)\r\n127.0.0.1:9979> DEL some_dehy\r\nOK\r\n```\r\n\r\nEnjoy!\r\n\r\n\r\n## About This Module\r\n\r\nThis module is based off a python version of the same concepts developed in Tamar Labs by Adam Lev-Libfeld and Alexander Margolin and designed by Adam Lev-Libfeld in mid 2015.\r\n\r\nThe Redis module was created by Adam Lev-Libfeld during the RedisModulesHackathon in late 2016, and is maintained by him solely.\r\n", | ||
| "tagline": "A Redis Element Dehydration Module", | ||
| "body": "\r\n:rocket:**TL;DR - A Dehydrator is a fancy delayed queue, see the commands this module provide [HERE](Commands.md)**\r\n\r\n-----\r\n\r\n**ReDe** /'redɪ/ *n.* a simple Redis Module for data dehydration. This is a pretty straightforward implementation of the dehydration system depicted in the article \"[Fast Data](https://goo.gl/DDFFPO)\". The Goal of this module is to solve the *Contextual Completeness* and *Emergent Relevancy* problems by adding the ability to postpone incoming elements to a later time in which we will have a complete information for these elements. Effectively acting as a snooze button to any element.\r\n\r\nFrom the article:\r\n> Dehydrators are simplistic time machines. They transport data elements that arrived prematurely in terms of their context right to the future where they might be needed, without loading the system while waiting. This concept is achieved by attaching a time-indexed data store to a clock, storing elements as they arrive to the dehydrator and re-introducing them as inputs to the system once a predetermined time period has passed.\r\n\r\nUsing this system it is also possible to craft a self cleaning \"claims check\", to minimize load on transportation and manipulation nodes of a pipeline architecture.\r\n\r\n*You can read further on the algorithm behind this module [here](Algorithm.md).*\r\n\r\nThe module works by adding a new type to Redis -`DehydratorType`. It will be ceated automatically when you call a push command on it, and it can be deleted using the `DEL` command like any other key.\r\n\r\n\r\n\r\n### Common Use Cases\r\n\r\n* **Stream Coordination** - Make data from one stream wait for the corresponding data from another (preferebly using sliding-window style timing).\r\n* **Event Rate Limitation** - Delay any event beyond current max throughput to the next available time slot, while preserving order.\r\n* **Self Cleaning Claims-Check** - Store data for a well known period, without the need to search for it when it is expired or clear it from the data-store yourself.\r\n* **Task Timer** - Assign actions and their respective payloads to a specific point in time.\r\n\r\n-----------------------------------------------------\r\n\r\n## Usage\r\n\r\nThe dehydrator is an effective 'snooze button' for events, you push an event into it along with an id (for future referance) and in how many seconds you want it back, and poll whenever you want the elements back. only expired elements would pop out.\r\n\r\n**It includes 4 main commands:**\r\n\r\n* [`REDE.PUSH`](Commands.md/#push) - Push an element, it will need an id, the element itself and dehydration time in seconds.\r\n* [`REDE.PULL`](Commands.md/#pull) - Pull the element with the appropriate id before it expires.\r\n* [`REDE.POLL`](Commands.md/#poll) - Pull and return all the expired elements.\r\n* [`REDE.LOOK`](Commands.md/#look) - Search the dehydrator for an element with the given id and if found return it's payload (without pulling).\r\n* [`REDE.UPDATE`](Commands.md/#update) - Set the element represented by a given id, the current element will be returned, and the new element will inherit the current expiration.\r\n* [`REDE.TTN`](Commands.md/#ttn) - Return the minimal time between now and the first expiration\r\n\r\n**The module also includes a test command:**\r\n* `REDE.TEST` - a set of unit tests of the above commands. **NOTE!** This command is running in fixed time (~15 seconds) as it uses `sleep` (dios mio, No! ✞✞✞).\r\n\r\n*see more about the commands in [Commands.md](Commands.md)*\r\n\r\n## Quick Start Guide\r\n\r\nHere's what you need to do to build this module:\r\n\r\n0. Build Redis in a build supporting modules.\r\n1. Build the module: `make`\r\n3. Run Redis loading the module: `/path/to/redis-server --loadmodule ./example/module.so`\r\n\r\nNow run `redis-cli` and try the commands:\r\n\r\n```\r\n127.0.0.1:9979> REDE.PUSH some_dehy id1 world 15\r\nOK\r\n127.0.0.1:9979> REDE.PUSH some_dehy id2 hello 1\r\nOK\r\n127.0.0.1:9979> REDE.PUSH some_dehy id3 goodbye 2\r\nOK\r\n127.0.0.1:9979> REDE.PULL some_dehy id3\r\n\"goodbye\"\r\n127.0.0.1:9979> REDE.POLL some_dehy\r\n1) \"hello\"\r\n127.0.0.1:9979> REDE.POLL some_dehy\r\n(empty list or set)\r\n127.0.0.1:6379> REDE.LOOK some_dehy id2\r\n(nil)\r\n127.0.0.1:6379> REDE.LOOK some_dehy id1\r\n\"world\"\r\n127.0.0.1:6379> REDE.PULL some_dehy id2\r\n(nil)\r\n127.0.0.1:6379> REDE.TTN some_dehy\r\n8\r\n```\r\n\r\nThis `(empty list or set)` reply from `REDE.POLL` means that the there are no more items to pull right now, so we'll have to wait until enough time passes for our next element to expire. using REDE.TTN we can see this will be in 8 seconds (in this example we waited a bit between commands). Once 8 seconds will pass we can run:\r\n\r\n```\r\n127.0.0.1:9979> REDE.POLL some_dehy\r\n1) \"world\"\r\n127.0.0.1:9979> REDE.TEST\r\nPASS\r\n(15.00s)\r\n127.0.0.1:9979> DEL some_dehy\r\nOK\r\n```\r\n\r\nEnjoy!\r\n\r\n\r\n## About This Module\r\n\r\nThis module is based off a python version of the same concepts developed in Tamar Labs by Adam Lev-Libfeld and Alexander Margolin and designed by Adam Lev-Libfeld in mid 2015.\r\n\r\nThe Redis module was created by Adam Lev-Libfeld during the RedisModulesHackathon in late 2016, and is maintained by him solely.\r\n", | ||
| "note": "Don't delete this file! It's used internally to help with page regeneration." | ||
| } |