Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

plugin for fetching cache dump from any node after joining legion #221

Merged
merged 2 commits into from

2 participants

@prymitive
Collaborator

This plugin allows to fetch copy of cache content from any node joined to legion.
It allows for example to hot-add new SSL frontend server with already populated session cache (you just need to set lower frequency for legion so it joins faster).

Usage:

[uwsgi]
[...]
# define cache
cache2 = name=cache1,items=1000,store=/tmp/lc1.ucache,udp=225.1.1.1:1500,node=225.1.1.1:1500

# define legion as usual
legion = legion1 225.1.1.1:19678 101 bf-cbc:abc
legion-node = legion1 225.1.1.1:19678
legion-join = legion1 cmd:echo legion_joined_hook

# connect legion to cache
legion-cache = legion1 cache1
# set socket address advertised to other nodes using legion scroll
legion-cache-dump-socket = 127.0.0.1:3001

Please review.

@unbit
Owner

what about simplyfing it for being only a legion action ?

[uwsgi]
[...]
# define cache
cache2 = name=cache1,items=1000,store=/tmp/lc1.ucache,udp=225.1.1.1:1500,node=225.1.1.1:1500

# define legion as usual
legion = legion1 225.1.1.1:19678 101 bf-cbc:abc
legion-node = legion1 225.1.1.1:19678
legion-join = legion1 cmd:echo legion_joined_hook
# run the action when joining a legion (i think no options are needed, maybe timeout ?)
legion-join = legion1 legion-cache:
# add a plugin-friendly scroll
legion-scroll = legion1 legion-cache:dump-socket=127.0.0.1:3001,cache=cache1

Even the legion-cache name seems a bit too generic, maybe legion-cachedump ?

@prymitive
Collaborator

sure, if there are no objections to the code I can move it into core/legion.c as an action

I not good at naming things so yes, a better name would be nice ;)

  • legion-cache-dump
  • legion-cache-fetch
  • legion-cache-sync (?)

I would love if scroll code be more structured, simple key:value would be enough. So we would have:

legion-scroll = legion1 legion-cache dump-socket=127.0.0.1:3001,cache=cache1

and action would fetch "legion-cache" entry from scroll.
But this can be handled inside action without much trouble, so it's more of a design choice rather than technical issue

@unbit
Owner

it can remain an external plugin (it will be good even as a reference for cache and legions usage in plugins) Just remove the options in favour of rawly setting the scroll

@prymitive
Collaborator

ok, I'll push update later, is legion-cache-fetch as a name for action and legion_cache_fetch as a name for plugin ok?

@unbit
Owner

yes it's ok

@prymitive
Collaborator

Pushed stripped version of plugin, now it's just an action for legion subsystem.

Full example:

lc1.ini

[uwsgi]

plugin = legion_cache_fetch

master = true
http = :8081
socket = 127.0.0.1:3001
stats = :2101
wsgi-file = tests/staticfile.py

cache2 = name=cache1,items=1000,store=/tmp/lc1.ucache,udp=225.1.1.1:1500,node=225.1.1.1:1500

legion = legion1 225.1.1.1:19678 101 bf-cbc:abc
legion-node = legion1 225.1.1.1:19678
legion-join = legion1 legion-cache-fetch:cache1
legion-scroll = legion1 dump-socket=127.0.0.1:3001

lc2.ini

[uwsgi]

plugin = legion_cache_fetch

master = true
http = :8082
socket = 127.0.0.1:3002
stats = :2102
wsgi-file = tests/staticfile.py

cache2 = name=cache1,items=1000,store=/tmp/lc2.ucache,udp=225.1.1.1:1500,node=225.1.1.1:1500

legion = legion1 225.1.1.1:19678 101 bf-cbc:abc
legion-node = legion1 225.1.1.1:19678
legion-join = legion1 legion-cache-fetch:cache1
legion-scroll = legion1 dump-socket=127.0.0.1:3002
@unbit
Owner

is not enough to use the lord scroll directly ? (without iterating all the nodes)

If we have a lord its up by definition

@prymitive
Collaborator

each node has different data in its own scroll, every node puts its own ip:port address of uwsgi socket it has spawned, so I iterate so that cache-sync will try to fetch dump from all nodes.
if node A is lord and it fails right after node B joins, than node B would only make single unsuccessful attempt to fetch cache dump, if I first make a list of all node and pass it to node B, then even if lord manages to fail before or during cache-sync, sync handler will be able to try fetching it from another node.

@unbit
Owner

ok, seems fine but you should free the memory allocated by the kvlist parser for the cachedump item

@prymitive
Collaborator

Do you mean char *dump_socket ?
It seems that this is not copy but rather reference, freeing it will corrupt items on uc->sync_nodes;
I will be free if I need to update uc->sync_nodes with new list.

@unbit
Owner

effectively the action is run only one time per startup, i can merge as is, maybe in the future we can improve it to not leak memory (the string_list too is not freed)

@unbit unbit merged commit 82aa232 into unbit:master
@prymitive prymitive deleted the prymitive:legion_cache_plugin2 branch
@prymitive
Collaborator

I'll fix leaks today/tomorrow morning

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
View
62 plugins/legion_cache_fetch/legion_cache_fetch.c
@@ -0,0 +1,62 @@
+#include <uwsgi.h>
+
+
+static int legion_action_cache_fetch_from_legion(struct uwsgi_legion *ul, char *arg) {
+ uwsgi_log("[legion-cache-fetch] getting cache '%s' dump from legion '%s' nodes\n", arg, ul->legion);
+
+ struct uwsgi_cache *uc = uwsgi_cache_by_name(arg);
+ if (!uc) {
+ uwsgi_log("[legion-cache-fetch] cannot sync, cache '%s' not found\n", arg);
+ return 1;
+ }
+
+ struct uwsgi_string_list *dump_from_nodes = NULL;
+
+ struct uwsgi_legion_node *legion_nodes = ul->nodes_head;
+ while (legion_nodes) {
+ char *dump_socket = NULL;
+ if (uwsgi_kvlist_parse(legion_nodes->scroll, legion_nodes->scroll_len, ',', '=',
+ "dump-socket", &dump_socket,
+ NULL)) {
+ uwsgi_log("[legion-cache-fetch] cannot sync from %.*s, cache socket address not found in legion scroll: %.*s\n",
+ legion_nodes->name_len, legion_nodes->name, legion_nodes->scroll_len, legion_nodes->scroll);
+ }
+ else {
+ if (dump_socket) {
+ uwsgi_string_new_list(&dump_from_nodes, dump_socket);
+ }
+ else {
+ uwsgi_log("[legion-cache-fetch] cannot sync from %.*s, cache socket address not found in legion scroll: %.*s\n",
+ legion_nodes->name_len, legion_nodes->name, legion_nodes->scroll_len, legion_nodes->scroll);
+ }
+ }
+ legion_nodes = legion_nodes->next;
+ }
+
+ // update uc->sync_nodes list
+ struct uwsgi_string_list *usl = uc->sync_nodes;
+ struct uwsgi_string_list *next;
+ while (usl) {
+ next = usl->next;
+ free(usl);
+ usl = next;
+ }
+ uc->sync_nodes = dump_from_nodes;
+
+ // call sync
+ uwsgi_cache_sync_from_nodes(uc);
+
+ return 0;
+}
+
+
+static void legion_cache_register() {
+ uwsgi_legion_action_register("legion-cache-fetch", legion_action_cache_fetch_from_legion);
+}
+
+
+struct uwsgi_plugin legion_cache_fetch_plugin = {
+ .name = "legion_cache_fetch",
+ .on_load = legion_cache_register,
+};
+
View
8 plugins/legion_cache_fetch/uwsgiplugin.py
@@ -0,0 +1,8 @@
+
+NAME = 'legion_cache_fetch'
+CFLAGS = []
+LDFLAGS = []
+LIBS = []
+
+GCC_LIST = ['legion_cache_fetch']
+
Something went wrong with that request. Please try again.