Skip to content

Commit

Permalink
add "parallel detach" syntax
Browse files Browse the repository at this point in the history
which starts all children in the detached state
  • Loading branch information
alandekok committed Aug 11, 2019
1 parent 4ff8950 commit e8cdea4
Show file tree
Hide file tree
Showing 8 changed files with 201 additions and 65 deletions.
100 changes: 78 additions & 22 deletions doc/unlang/parallel.adoc
Expand Up @@ -3,15 +3,16 @@

[source,unlang]
----
parallel [ empty ] {
parallel [ empty | detach ] {
[ statements ]
}
----

The `parallel` section runs subsections in parallel. Once all of the
statements in the `parallel` section have finished execution, the
parallel section exits. The return code of the parallel section is
the same as if each subsection was run sequentially.
The `parallel` section runs multiple child subsections in parallel.
Once all of the statements in the `parallel` section have finished
execution, the parallel section exits. The return code of the
parallel section is the same as if each subsection was run
sequentially.

The `parallel` section is most useful for running modules which call
external systems and wait for a reply. When modules are run in
Expand All @@ -20,6 +21,10 @@ second and subsequent modules be executed. With parallel, the first
module is run, and then as soon as that module yeilds in order to wait
for an event, the second module can immediately run.

Each child subsection is executed using a
link:subrequest.adoc[subrequest]. Please see the next subsection for
more information about subrequests.

Despite the "parallel" name, the section will only run one module at a
time. This limitation ensures that there are no issues with
multi-threaded race conditions, locking, etc. The "parallel" name is
Expand Down Expand Up @@ -50,13 +55,18 @@ parallel {

=== Child Requests are Independent

Each module or subsection runs as a new child request which is an
identical copy of the parent request. Policies in the child can
update the original parent by using `update parent.request`, or
`update parent.reply`.
Each module or subsection runs as a new child request, i.e. a
link:subrequest.adoc[subrequest]. Each child request is an identical
copy of the parent request. Policies in the child can update the
original parent by using `update parent.request`, or `update
parent.reply`. Please see the link:list.adoc[list] syntax for a
more complete description of how to refer to parent requests.

The child requests are required because each subsection is run
independently, with independent events, timers, etc.
independently, with independent events, timers, etc. That is, each
child request has its own state, independent of any other request.
This independence is required in order to fully implement the
"parallel" nature of this keyword.

Once the child requests are finished, they are freed. Any information
stored in them is discarded. Information needed after the `parallel`
Expand Down Expand Up @@ -91,22 +101,30 @@ parallel {
}
----

=== Empty Child Requests
=== Subrequests are Synchronous

It is possible to create child requests which are not clones of the
parent, by using the `parallel empty { ... }` syntax. When the
`empty` syntax is used, child requests are still created, but they
contain no attributes.
Execution of the parent request is paused while each child request is
running. The parent request continues execution once all of the the
child requests have finished.

The `empty` syntax is most useful when it is necessary to control
which attributes go into a child request.
Unlike the link:subrequest.adoc[subrequest] keyword, the child
requests of a `parallel` section cannot be made asynchronous via the
link:detach.adoc[detach] keyword.

.Example
=== parallel empty

The `parallel empty { ... }` syntax creates empty child requests.
These requests contain no attributes. Attributes in the child request
must be added manually via an link:update.adoc[update] statement.

The contents of each child request can be manually controlled by using
the `parallel empty` syntax. In that case, the attributes in the
child request must be added manually via an link:update.adoc[update]
statement.
The `empty` keyword is most useful when it is necessary to manually
determine which attributes go into a child request.

The `empty` keyword cannot be used with the `detach` keyword. There
is no purpose to creating independent child requests which contain
nothing.

.Example

In this example, the `radius1` module sees a `User-Name` which is
modified from the parent `User-Name`, and a `User-Password` which is
Expand Down Expand Up @@ -134,6 +152,44 @@ parallel empty {
}
----

=== parallel detach

The `parallel detach { ... }` syntax creates child requests which are
immediately detached from the parent request. Each child request
contains copies of all attributes in the parent request.

The parent request continues immediately, and independently of the
child requests.

The `detach` keyword cannot be used with the `empty` keyword. There
is no purpose to creating independent child requests which contain
nothing.

The return code from the `parallel detach { ... }` section is `noop`.
Since all of the child requests are independent of the parent, they
cannot convey any information back to the parent. Therefore, the
return code is always `noop`.

The `detach` keyword is most useful when it is necessary to send
packets to multiple destinations, but where there is no need to wait
for an answer.

.Example

In this example, the `parallel detach { ... }` syntax is used to send
packets to four different destinations at the same time, in a "fire
and forget" manner.

[source,unlang]
----
parallel detach {
radius1
radius2
radius3
radius4
}
----

=== Exiting Early from a Parallel Section

In some situations, it may be useful to exit early from a parallel
Expand Down
12 changes: 9 additions & 3 deletions src/lib/unlang/compile.c
Expand Up @@ -2933,7 +2933,7 @@ static unlang_t *compile_parallel(unlang_t *parent, unlang_compile_t *unlang_ctx
char const *name2;
unlang_group_t *g;
bool clone = true;

bool detach = false;

/*
* No children? Die!
Expand All @@ -2951,19 +2951,25 @@ static unlang_t *compile_parallel(unlang_t *parent, unlang_compile_t *unlang_ctx
*/
name2 = cf_section_name2(cs);
if (name2) {
if (strcmp(name2, "empty") != 0) {
if (strcmp(name2, "empty") == 0) {
clone = false;

} else if (strcmp(name2, "detach") == 0) {
detach = true;

} else {
cf_log_err(cs, "Invalid argument '%s'", name2);
return NULL;
}

clone = false;
}

c = compile_group(parent, unlang_ctx, cs, group_type, parentgroup_type, mod_type);
if (!c) return NULL;

g = unlang_generic_to_group(c);
g->clone = clone;
g->detach = detach;

c->name = c->debug_name = unlang_ops[c->type].name;

Expand Down
2 changes: 1 addition & 1 deletion src/lib/unlang/interpret.c
Expand Up @@ -527,7 +527,7 @@ static inline unlang_frame_action_t frame_eval(REQUEST *request, unlang_stack_fr
*result = RLM_MODULE_YIELD; /* Fixup rcode */
yield:
/*
* Detach is magic. The parent "create" function
* Detach is magic. The parent "subrequest" function
* takes care of bumping the instruction
* pointer...
*/
Expand Down
116 changes: 80 additions & 36 deletions src/lib/unlang/parallel.c
Expand Up @@ -26,6 +26,7 @@ RCSID("$Id$")

#include "unlang_priv.h"
#include "parallel_priv.h"
#include "subrequest_priv.h"
#include "module_priv.h"

/** Run one or more sub-sections from the parallel section.
Expand All @@ -36,9 +37,19 @@ static rlm_rcode_t unlang_parallel_run(REQUEST *request, unlang_parallel_t *stat
int i, priority;
rlm_rcode_t result;
unlang_parallel_child_state_t done = CHILD_DONE; /* hope that we're done */
REQUEST *child;

// @todo - rdebug running the request.

/*
* The children are created all detached. We just return
* "noop".
*/
if (state->g->detach) {
state->priority = 0;
state->result = RLM_MODULE_NOOP;
}

/*
* Loop over all the children.
*
Expand All @@ -47,59 +58,72 @@ static rlm_rcode_t unlang_parallel_run(REQUEST *request, unlang_parallel_t *stat
*/
for (i = 0; i < state->num_children; i++) {
switch (state->children[i].state) {
/*
* Not ready to run.
*/
case CHILD_YIELDED:
RDEBUG3("parallel child %d is already YIELDED", i + 1);
rad_assert(state->children[i].child != NULL);
rad_assert(state->children[i].instruction != NULL);
done = CHILD_YIELDED;
continue;

/*
* Don't need to call this any more.
*/
case CHILD_DONE:
RDEBUG3("parallel child %d is already DONE", i + 1);
rad_assert(state->children[i].child == NULL);
rad_assert(state->children[i].instruction == NULL);
continue;

/*
* Create the child and then run it.
*/
case CHILD_INIT:
RDEBUG3("parallel child %d is INIT", i + 1);
rad_assert(state->children[i].instruction != NULL);
state->children[i].child = unlang_io_subrequest_alloc(request,
request->dict, UNLANG_NORMAL_CHILD);
state->children[i].state = CHILD_RUNNABLE;
state->children[i].child->packet->code = request->packet->code;

/*
* Push first instruction for child to execute
*/
unlang_interpret_push(state->children[i].child,
state->children[i].instruction, RLM_MODULE_FAIL,
UNLANG_NEXT_STOP, UNLANG_TOP_FRAME);
child = unlang_io_subrequest_alloc(request,
request->dict, state->g->detach);
child->packet->code = request->packet->code;

if (state->g->clone) {
if ((fr_pair_list_copy(state->children[i].child->packet,
&state->children[i].child->packet->vps,
/*
* Note that we do NOT copy the
* Session-State list! That
* contains state information for
* the parent.
*/
if ((fr_pair_list_copy(child->packet,
&child->packet->vps,
request->packet->vps) < 0) ||
(fr_pair_list_copy(state->children[i].child->reply,
&state->children[i].child->reply->vps,
(fr_pair_list_copy(child->reply,
&child->reply->vps,
request->reply->vps) < 0) ||
(fr_pair_list_copy(state->children[i].child,
&state->children[i].child->control,
(fr_pair_list_copy(child,
&child->control,
request->control) < 0)) {
REDEBUG("failed copying lists to clone");
for (i = 0; i < state->num_children; i++) TALLOC_FREE(state->children[i].child);
return RLM_MODULE_FAIL;
}
}

/*
* Push first instruction for child to execute
*/
unlang_interpret_push(child,
state->children[i].instruction, RLM_MODULE_FAIL,
UNLANG_NEXT_STOP, UNLANG_TOP_FRAME);

/*
* It is often useful to create detached
* children in parallel.
*/
if (state->g->detach) {
state->children[i].state = CHILD_DONE;
state->children[i].instruction = NULL;

/*
* Detach the child, and insert
* it into the backlog.
*/
if (unlang_detach(child, &result, &priority) == UNLANG_ACTION_CALCULATE_RESULT) {
return UNLANG_ACTION_STOP_PROCESSING;
}

if (fr_heap_insert(child->backlog, child) < 0) {
RPERROR("Failed inserting child into backlog");
return UNLANG_ACTION_STOP_PROCESSING;
}

continue;
}

state->children[i].child = child;
state->children[i].state = CHILD_RUNNABLE;

/* FALL-THROUGH */

/*
Expand Down Expand Up @@ -179,6 +203,26 @@ static rlm_rcode_t unlang_parallel_run(REQUEST *request, unlang_parallel_t *stat

rad_assert(done == CHILD_DONE);
break;

/*
* Not ready to run.
*/
case CHILD_YIELDED:
RDEBUG3("parallel child %d is already YIELDED", i + 1);
rad_assert(state->children[i].child != NULL);
rad_assert(state->children[i].instruction != NULL);
done = CHILD_YIELDED;
continue;

/*
* Don't need to call this any more.
*/
case CHILD_DONE:
RDEBUG3("parallel child %d is already DONE", i + 1);
rad_assert(state->children[i].child == NULL);
rad_assert(state->children[i].instruction == NULL);
continue;

}
}

Expand Down
13 changes: 11 additions & 2 deletions src/lib/unlang/subrequest.c
Expand Up @@ -300,15 +300,24 @@ static unlang_action_t unlang_subrequest(REQUEST *request,
return UNLANG_ACTION_YIELD;
}

static unlang_action_t unlang_detach(REQUEST *request,
unlang_action_t unlang_detach(REQUEST *request,
rlm_rcode_t *presult, int *priority)
{
VALUE_PAIR *vp;
unlang_stack_t *stack = request->stack;
unlang_stack_frame_t *frame = &stack->frame[stack->depth];
unlang_t *instruction = frame->instruction;

RDEBUG2("%s", unlang_ops[instruction->type].name);
/*
* The "parallel" command calls us in order to detach a
* child which hasn't done anything yet.
*
* Therefore, we print out that "detach" is running ONLY
* when it's running with a parent stack frame.
*/
if (stack->depth > 1) {
RDEBUG2("%s", unlang_ops[instruction->type].name);
}

rad_assert(request->parent != NULL);

Expand Down
4 changes: 4 additions & 0 deletions src/lib/unlang/subrequest_priv.h
Expand Up @@ -40,6 +40,10 @@ typedef struct {
void unlang_subrequest_free(REQUEST **child);

void unlang_subrequest_push(rlm_rcode_t *out, REQUEST *child, bool top_frame);

unlang_action_t unlang_detach(REQUEST *request,
rlm_rcode_t *presult, int *priority);

#ifdef __cplusplus
}
#endif

0 comments on commit e8cdea4

Please sign in to comment.