VIP28: designated vmod object methods

This proposal is the outcome of collaboration between Dridi and slink.

See below for phk's comments.

Synopsis

We propose to add designated methods to objects for casting them to/from VCL native types such that objects can be used wherever any other native VCL type can be used such as:

# vcl_init
new mydirector = directors.round-robin();
new intone = variable.integer()
new inttwo = variable.integer()

# other subs
if (myint == 4) {
  set inttwo = myint;
  set myint += 7;
}
set bereq.backend = mydirector;

Why

The motivation is the discussion around the introduction of native variables to VCL: We follow the idea that rather than adding support for variables to VCL, we might as well extend the object interface to allow use of objects in a way equally simple to how variables would be used, but also

• adding more value to other object implementations
• simplifying VCL syntax

How

We propose to extend the varnish vcl object interface by a $Cast declaration which defines the type to which objects of an instance gets cast when used in various contexts:

$Object <class>(...)
$Cast <type> ([get, set, unset, inc, dec, mul, div])

The designated methods are called when a <myobj> instance of <class> is used in the following contexts:

• get: used in any expression like:

  if (<myobj> != "foo") { ... }
  some.function(<myobj>);
  set whatever = <myobj>;
  

• set:

  set <myobj> = <expr>;
  

• unset:

  unset <myobj>;
  

• inc/dec/mul/div:

  set <myobj> += <expr>;
  set <myobj> -= <expr>;
  set <myobj> *= <expr>;
  set <myobj> /= <expr>;
  

If an object is used in a context where the designated method is missing, a VCL compile time error is raised.

The vmodtool-generated C-interface uses fixed method names and signatures for the designated methods:

<ctype> <vmod>_<class>_get(VRT_CTX, VPFX(<vmod>_<class>))
void    <vmod>_<class>_set(VRT_CTX, VPFX(<vmod>_<class>), <ctype>)
void    <vmod>_<class>_unset(VRT_CTX, VPFX(<vmod>_<class>))
void    <vmod>_<class>_inc(VRT_CTX, VPFX(<vmod>_<class>), <ctype>)
void    <vmod>_<class>_dec(VRT_CTX, VPFX(<vmod>_<class>), <ctype>)
void    <vmod>_<class>_mul(VRT_CTX, VPFX(<vmod>_<class>), <ctype>)
void    <vmod>_<class>_div(VRT_CTX, VPFX(<vmod>_<class>), <ctype>)

Examples

• Objects as variables

  • vcc:

    $Object integer;
    $Cast INT (get, set, unset, inc, dec, mul, div)
    

  • vcl:

    new myint = vmod.integer();
    
    unset myint;
    set myint = 4;
    some.func(myint);
    set myint += 7;
    

• Constant

  • vcc:

    $Object string_const(STRING)
    $Cast STRING (get)
    

  • vcl:

    new myconst = vmod.string_const("foo");
    
    some.func(myconst);
    
    # VCL compile time error:
    unset myconst;
    set myconst = "blah";
    set myconst += "brz";
    

• Directors

  • vcc:

    $Object round_robin()
    $Cast BACKEND (get, inc, dec)
    

  • vcl:

    # set bereq.backend = mydirector.backend()
    # ->
    set bereq.backend = mydirector;
    
    # mydirector.add_backend(otherdirector.backend());
    # ->
    set mydirector += otherdirector;
    

PHK's comments

This proposal's implementation (but not its goal!) is an architectural dead-end, which will cause VMOD source compatibility issues down the road and it is therefore a bad idea, and my vote is a firm no on this implementation.

If we want to reach this particular goal, we have to make vcc_expr.c more object oriented first.

The first step of this is to get all the type-specific knowledge out of vcc_expr.c and into methods hung from vcc_type_t.

To take an example: vcc_expr_add() will not know all the stuff about what you can add or subtract to/from the different types, it will simply call:

if ((*e)->fmt->expr_add == NULL) {
    … error handling: "You cannot add to type %s", (*e)->fmt->name); …
}
e = (*e)->fmt->expr_add(e, tl);

(This is an argument for doing all this work: vcc_expr.c will become easier to understand. One argument against is that it will take more source lines to do the exact same thing.)

In a new sourcefile, vcc_type_duration.c, where const struct type DURATION[1] will move from vcc_types.c, that function vcc_duration_expr_add() will do the stuff currently happening in vcc_expr_add() as far as it pertains to DURATION.

Similar for the rest of the "native" types.

Notice that ->expr_add() and friends is called at compile time, and manipulates struct expr - the same way vcc_expr.c does to day. It is not called at runtime to do that actual += operation, but it could stick a call to a (VRT-)function into struct expr if that is how to go about that operation, just line vcc_expr.c does today.

Second step then is to change VCC so set bla += expr becomes a call into vcc_expr to evaluate bla->fmt->expr_iadd(expr). This is mostly text-processing, but there are some wrinkles, mostly relating to producing good error messages.

There may be some tricky corners on the way to this point.

We initially started out with "type-driven parsing" and while this have been eliminated most places, I'm not sure we're done eliminating it. (This shows up in the final vcc_type_t fmt arg in many functions in vcc_expr.c, indicating what we would like to get back.)

The third step is to invite VMODs into this, and here have the limitation that we cannot and will not execute code in the VMOD at compile time (Many reasons: Exporting&Supporting VCC's internal namespaces, VMOD Static Constructors and General Security).

Therefore the VMOD object must communicate its abilities by exporting functions in the VMOD symbol table:

FOOBAR .expr_iadd(FOOBAR, BACKEND)
FOOBAR .expr_isub(FOOBAR, BACKEND)
FOOBAR .expr_add(FOOBAR, FOOBAR)

In FOOBAR's struct type we use a generic and common to all vmod-objects expr_iadd "meta-method" which goes hunting in the symbol table to see what it can do, and then does the VCC work to do that, emitting a struct expr which calls the VMODs expr_iadd() symbol with appropriate arguments at runtime.

Pythons __$something__ 'magic methods' is the general inspiration here, but in the above I have used the naming convention expr_$something, as I'm not sure it is wise to adopt pythons __$something__ in C-code context, but the precise naming is not important, as long as it is consistent.

This is relatively much work to do, but it moves us forward, not sideways, and as we expand the repertoire we want to allow VMOD's, its simply a matter of recognizing new .expr_$something methods, which means that existing VMODs will be both source and binary compatible automatically.