Skip to content
This repository

Parse transform can't handle `Attr' as a variable #54

Merged
merged 2 commits into from over 1 year ago

4 participants

Olle Törnström Andrew Thompson Likhanov Alexander Sean Cribbs
Olle Törnström
olle commented

Log statements with Attr bound to a variable, causes the parse transform to fail with a function_clause. To reproduce, try out the following:

Attr = [{a, alpha}, {b, beta}],
lager:info(Attr, "Hello ~s", ["world!"]).

Crashes during compile with something like the following (cropped).

{function_clause, [
  {lager_transform, concat_lists, [
    {var,24,'Attr'}, ...

The cause is clear, it is expected that the DefaultAttrs can be appended to the passed property list of attributes, so there's no clause for a var in concat_lists.

But - It's already noted in lager_transform.erl that transformation of logging statements is ambiguous for calls with arity 3 (2 arguments). Sadly I think it's not only that, but actually also broken for arity 2 if Attr is a variable - the match for a property list doesn't work. Actually, Is is at all possible to walk the AST in a parse transform, when the variable is passed from outside the module? To me it seems that this could be the real bringer of bad news.

I like the general idea of lager, having log statements transformed in order to add implicit meta-data, but I think that the way the API is designed right now, it's very hard to support a more free way of using, and abusing, it.

To be on the safe side I guess the API has to use stricter ordering for optional parameters i.e. attributes are always arity 3 and always on the end of the call. Another option could be to use tagged parameters.

Perhaps a solution for lager, and this API version, is to simply amend the documentation and more clearly state that the attributes, must be defined as an inline property list.

Or does someone else have a solution for this?

With kind regards
/O

Andrew Thompson

How does this look? Sorry it took forever to look into this...

Likhanov Alexander

Hi! Voting this too.
But I think we can do a little better, because Attrs can be not only variable, but a function call, maybe record field(why not?)
My solution is here vegayours#1
It is almost the same (I have found this pull request after fixing Attrs as variable issue)
If you are going to accept this pull request we can merge our work

Andrew Thompson

Yes, you can merge your work with mine, and add some tests to support function call arguments or whatever (list comprehensions, whatever).

Likhanov Alexander
Vagabond added some commits
Andrew Thompson Vagabond Support variables as arguments to lager:info and friends
As long as *one* of the arguments is a literal, lager can figure out
what you're trying to do and (re)arrange the arguments as necessary.
95fdf93
Andrew Thompson Vagabond Also allow list comps, function calls and record fields as lager argu…
…ments
0ec0d90
Andrew Thompson

Parse transform now handles variables, list comprehensions, function calls and record fields. Anything else? Begin/End blocks seem kind of stupid to support, as do clauses/ifs.

Sean Cribbs
Collaborator

Nothing in this PR concerns me except these EQC failures on seemingly unrelated code. Can you confirm?

Exception: badarg
FmtStr: []
Args:   []
Failed! After 1 tests.
{[],3}
trunc_io_eqc:42: eqc_test_...*failed*
in function trunc_io_eqc:'-eqc_test_/0-fun-1-'/1 (test/trunc_io_eqc.erl, line 42)
**error:{assertEqual_failed,[{module,trunc_io_eqc},
                     {line,42},
                     {expression,"eqc : quickcheck ( eqc : testing_time ( 14 , ? QC_OUT ( prop_format ( ) ) ) )"},
                     {expected,true},
                     {value,false}]}
  output:<<"Starting Quviq QuickCheck version 1.27.7
   (compiled at {{2012,11,20},{14,59,9}})
Licence for Basho reserved until {{2013,2,26},{10,54,59}}
">>

Failed! Reason: 
{'EXIT',{badarg,[{lager_trunc_io,format,
                                 [[],[]],
                                 [{file,"src/lager_trunc_io.erl"},{line,66}]},
                 {trunc_io_eqc,'-prop_equivalence/0-fun-2-',1,
                               [{file,"test/trunc_io_eqc.erl"},{line,197}]}]}}
After 1 tests.
[]
trunc_io_eqc:43: eqc_test_...*failed*
in function trunc_io_eqc:'-eqc_test_/0-fun-4-'/1 (test/trunc_io_eqc.erl, line 43)
**error:{assertEqual_failed,[{module,trunc_io_eqc},
                     {line,43},
                     {expression,"eqc : quickcheck ( eqc : testing_time ( 14 , ? QC_OUT ( prop_equivalence ( ) ) ) )"},
                     {expected,true},
                     {value,false}]}
Sean Cribbs
Collaborator

It seems @engelsanchez brought this up on #108.

Andrew Thompson

I think this is a different EQC problem, caused by this change. I'll investigate.

Andrew Thompson

Oh, so that EQC failure looks to be fixed over on the iolist branch:

https://github.com/basho/lager/pull/108/files#L0R32

If you rebase this branch on top of that one, the tests pass.

Sean Cribbs
Collaborator

:+1: then :smile:

Andrew Thompson Vagabond merged commit 2a8706b into from
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 2 unique commits by 1 author.

Feb 15, 2013
Andrew Thompson Vagabond Support variables as arguments to lager:info and friends
As long as *one* of the arguments is a literal, lager can figure out
what you're trying to do and (re)arrange the arguments as necessary.
95fdf93
Andrew Thompson Vagabond Also allow list comps, function calls and record fields as lager argu…
…ments
0ec0d90
This page is out of date. Refresh to see the latest.

Showing 2 changed files with 153 additions and 2 deletions. Show diff stats Hide diff stats

  1. +33 2 src/lager_transform.erl
  2. +120 0 test/lager_test_backend.erl
35 src/lager_transform.erl
@@ -107,10 +107,25 @@ transform_statement({call, Line, {remote, _Line1, {atom, _Line2, lager},
107 107 %% [Format, Args] or [Attr, Format].
108 108 %% The trace attributes will be a list of tuples, so check
109 109 %% for that.
110   - case Arg1 of
111   - {cons, _, {tuple, _, _}, _} ->
  110 + case {element(1, Arg1), Arg1} of
  111 + {_, {cons, _, {tuple, _, _}, _}} ->
112 112 {concat_lists(Arg1, DefaultAttrs),
113 113 Arg2, {atom, Line, none}};
  114 + {Type, _} when Type == var;
  115 + Type == lc;
  116 + Type == call;
  117 + Type == record_field ->
  118 + %% crap, its not a literal. look at the second
  119 + %% argument to see if it is a string
  120 + case Arg2 of
  121 + {string, _, _} ->
  122 + {concat_lists(Arg1, DefaultAttrs),
  123 + Arg2, {atom, Line, none}};
  124 + _ ->
  125 + %% not a string, going to have to guess
  126 + %% it's the argument list
  127 + {DefaultAttrs, Arg1, Arg2}
  128 + end;
114 129 _ ->
115 130 {DefaultAttrs, Arg1, Arg2}
116 131 end;
@@ -146,6 +161,22 @@ transform_statement(Stmt) ->
146 161 Stmt.
147 162
148 163 %% concat 2 list ASTs by replacing the terminating [] in A with the contents of B
  164 +concat_lists({var, Line, _Name}=Var, B) ->
  165 + %% concatenating a var with a cons
  166 + {call, Line, {remote, Line, {atom, Line, lists},{atom, Line, flatten}},
  167 + [{cons, Line, Var, B}]};
  168 +concat_lists({lc, Line, _Body, _Generator} = LC, B) ->
  169 + %% concatenating a LC with a cons
  170 + {call, Line, {remote, Line, {atom, Line, lists},{atom, Line, flatten}},
  171 + [{cons, Line, LC, B}]};
  172 +concat_lists({call, Line, _Function, _Args} = Call, B) ->
  173 + %% concatenating a call with a cons
  174 + {call, Line, {remote, Line, {atom, Line, lists},{atom, Line, flatten}},
  175 + [{cons, Line, Call, B}]};
  176 +concat_lists({record_field, Line, _Var, _Record, _Field} = Rec, B) ->
  177 + %% concatenating a record_field with a cons
  178 + {call, Line, {remote, Line, {atom, Line, lists},{atom, Line, flatten}},
  179 + [{cons, Line, Rec, B}]};
149 180 concat_lists({nil, _Line}, B) ->
150 181 B;
151 182 concat_lists({cons, Line, Element, Tail}, B) ->
120 test/lager_test_backend.erl
@@ -24,6 +24,7 @@
24 24 code_change/3]).
25 25
26 26 -record(state, {level, buffer, ignored}).
  27 +-record(test, {attrs, format, args}).
27 28 -compile([{parse_transform, lager_transform}]).
28 29
29 30 -ifdef(TEST).
@@ -180,6 +181,125 @@ lager_test_() ->
180 181 ok
181 182 end
182 183 },
  184 + {"variables inplace of literals in logging statements work",
  185 + fun() ->
  186 + ?assertEqual(0, count()),
  187 + Attr = [{a, alpha}, {b, beta}],
  188 + Fmt = "format ~p",
  189 + Args = [world],
  190 + lager:info(Attr, "hello"),
  191 + lager:info(Attr, "hello ~p", [world]),
  192 + lager:info(Fmt, [world]),
  193 + lager:info("hello ~p", Args),
  194 + lager:info(Attr, "hello ~p", Args),
  195 + lager:info([{d, delta}, {g, gamma}], Fmt, Args),
  196 + ?assertEqual(6, count()),
  197 + {_Level, _Time, Message, Metadata} = pop(),
  198 + ?assertMatch([{a, alpha}, {b, beta}|_], Metadata),
  199 + ?assertEqual("hello", lists:flatten(Message)),
  200 + {_Level, _Time2, Message2, _Metadata2} = pop(),
  201 + ?assertEqual("hello world", lists:flatten(Message2)),
  202 + {_Level, _Time3, Message3, _Metadata3} = pop(),
  203 + ?assertEqual("format world", lists:flatten(Message3)),
  204 + {_Level, _Time4, Message4, _Metadata4} = pop(),
  205 + ?assertEqual("hello world", lists:flatten(Message4)),
  206 + {_Level, _Time5, Message5, _Metadata5} = pop(),
  207 + ?assertEqual("hello world", lists:flatten(Message5)),
  208 + {_Level, _Time6, Message6, Metadata6} = pop(),
  209 + ?assertMatch([{d, delta}, {g, gamma}|_], Metadata6),
  210 + ?assertEqual("format world", lists:flatten(Message6)),
  211 + ok
  212 + end
  213 + },
  214 + {"list comprehension inplace of literals in logging statements work",
  215 + fun() ->
  216 + ?assertEqual(0, count()),
  217 + Attr = [{a, alpha}, {b, beta}],
  218 + Fmt = "format ~p",
  219 + Args = [world],
  220 + lager:info([{K, atom_to_list(V)} || {K, V} <- Attr], "hello"),
  221 + lager:info([{K, atom_to_list(V)} || {K, V} <- Attr], "hello ~p", [{atom, X} || X <- Args]),
  222 + lager:info([X || X <- Fmt], [world]),
  223 + lager:info("hello ~p", [{atom, X} || X <- Args]),
  224 + lager:info([{K, atom_to_list(V)} || {K, V} <- Attr], "hello ~p", [{atom, X} || X <- Args]),
  225 + lager:info([{d, delta}, {g, gamma}], Fmt, [{atom, X} || X <- Args]),
  226 + ?assertEqual(6, count()),
  227 + {_Level, _Time, Message, Metadata} = pop(),
  228 + ?assertMatch([{a, "alpha"}, {b, "beta"}|_], Metadata),
  229 + ?assertEqual("hello", lists:flatten(Message)),
  230 + {_Level, _Time2, Message2, _Metadata2} = pop(),
  231 + ?assertEqual("hello {atom,world}", lists:flatten(Message2)),
  232 + {_Level, _Time3, Message3, _Metadata3} = pop(),
  233 + ?assertEqual("format world", lists:flatten(Message3)),
  234 + {_Level, _Time4, Message4, _Metadata4} = pop(),
  235 + ?assertEqual("hello {atom,world}", lists:flatten(Message4)),
  236 + {_Level, _Time5, Message5, _Metadata5} = pop(),
  237 + ?assertEqual("hello {atom,world}", lists:flatten(Message5)),
  238 + {_Level, _Time6, Message6, Metadata6} = pop(),
  239 + ?assertMatch([{d, delta}, {g, gamma}|_], Metadata6),
  240 + ?assertEqual("format {atom,world}", lists:flatten(Message6)),
  241 + ok
  242 + end
  243 + },
  244 + {"function calls inplace of literals in logging statements work",
  245 + fun() ->
  246 + ?assertEqual(0, count()),
  247 + put(attrs, [{a, alpha}, {b, beta}]),
  248 + put(format, "format ~p"),
  249 + put(args, [world]),
  250 + lager:info(get(attrs), "hello"),
  251 + lager:info(get(attrs), "hello ~p", get(args)),
  252 + lager:info(get(format), [world]),
  253 + lager:info("hello ~p", erlang:get(args)),
  254 + lager:info(fun() -> get(attrs) end(), "hello ~p", get(args)),
  255 + lager:info([{d, delta}, {g, gamma}], get(format), get(args)),
  256 + ?assertEqual(6, count()),
  257 + {_Level, _Time, Message, Metadata} = pop(),
  258 + ?assertMatch([{a, alpha}, {b, beta}|_], Metadata),
  259 + ?assertEqual("hello", lists:flatten(Message)),
  260 + {_Level, _Time2, Message2, _Metadata2} = pop(),
  261 + ?assertEqual("hello world", lists:flatten(Message2)),
  262 + {_Level, _Time3, Message3, _Metadata3} = pop(),
  263 + ?assertEqual("format world", lists:flatten(Message3)),
  264 + {_Level, _Time4, Message4, _Metadata4} = pop(),
  265 + ?assertEqual("hello world", lists:flatten(Message4)),
  266 + {_Level, _Time5, Message5, _Metadata5} = pop(),
  267 + ?assertEqual("hello world", lists:flatten(Message5)),
  268 + {_Level, _Time6, Message6, Metadata6} = pop(),
  269 + ?assertMatch([{d, delta}, {g, gamma}|_], Metadata6),
  270 + ?assertEqual("format world", lists:flatten(Message6)),
  271 + ok
  272 + end
  273 + },
  274 + {"record fields inplace of literals in logging statements work",
  275 + fun() ->
  276 + ?assertEqual(0, count()),
  277 + Test = #test{attrs=[{a, alpha}, {b, beta}], format="format ~p", args=[world]},
  278 + lager:info(Test#test.attrs, "hello"),
  279 + lager:info(Test#test.attrs, "hello ~p", Test#test.args),
  280 + lager:info(Test#test.format, [world]),
  281 + lager:info("hello ~p", Test#test.args),
  282 + lager:info(Test#test.attrs, "hello ~p", Test#test.args),
  283 + lager:info([{d, delta}, {g, gamma}], Test#test.format, Test#test.args),
  284 + ?assertEqual(6, count()),
  285 + {_Level, _Time, Message, Metadata} = pop(),
  286 + ?assertMatch([{a, alpha}, {b, beta}|_], Metadata),
  287 + ?assertEqual("hello", lists:flatten(Message)),
  288 + {_Level, _Time2, Message2, _Metadata2} = pop(),
  289 + ?assertEqual("hello world", lists:flatten(Message2)),
  290 + {_Level, _Time3, Message3, _Metadata3} = pop(),
  291 + ?assertEqual("format world", lists:flatten(Message3)),
  292 + {_Level, _Time4, Message4, _Metadata4} = pop(),
  293 + ?assertEqual("hello world", lists:flatten(Message4)),
  294 + {_Level, _Time5, Message5, _Metadata5} = pop(),
  295 + ?assertEqual("hello world", lists:flatten(Message5)),
  296 + {_Level, _Time6, Message6, Metadata6} = pop(),
  297 + ?assertMatch([{d, delta}, {g, gamma}|_], Metadata6),
  298 + ?assertEqual("format world", lists:flatten(Message6)),
  299 + ok
  300 + end
  301 + },
  302 +
183 303 {"log messages below the threshold are ignored",
184 304 fun() ->
185 305 ?assertEqual(0, count()),

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.