cannot access variable by path in an inner context #100

Closed
jbcpollak opened this Issue Aug 7, 2012 · 43 comments

Comments

Projects
None yet
6 participants

The main dust.js documentation says:

To avoid brittle and confusing references, paths never backtrack up
the context stack. If you need to drill into a key available within
the parent context, pass the key as a parameter.

I take that to mean that given data like this:

{i18n : { firstname : "First Name" },
 people : [
    {"name" : "Moe"},
    {"name" : "Curly"}
 ]}

A template like this should work, but it does not:

{#people i18n=i18n}
    {i18n.firstname}: {name}
{/people}

How do you expose pathed variables to inner contexts?

Contributor

jairodemorais commented Aug 7, 2012

hi! @jbcpollak, I will take a look to this tomm :)

Contributor

vybs commented Aug 7, 2012

We do this ..

{#people fname=i18n.firstname}

{fname}

{/people}

How about doing this ...

{#people}

{#i18n}

{firstname}

{/i18n}

{/people}

vybs:

The first solution works as a one-off, but isn't useful if you have lots of parameters to index from the i18n object (in this case for example, populating a list of tags with i18n'd strings)

I will try the second option. that is of course a lot more verbose though than accessing via the path.

Its possible I'm using dust wrong too, in which case maybe a cookbook would be useful. :)

Is there a dust support group?

Contributor

rragan commented Aug 7, 2012

The two mechanisms cited by vybs are pretty much the main options. Once you have set a context with a section like {#people}, any pathed reference {xxx.yyy}, e.g using a dotted notation is restricted to only address down from the current context. A simple non-pathed section reference like {#i18n} is allowed to look up the stack and set a new context to that point. Then within that context you can address down with path notation.

That's how the dust language was defined by the creator. He seemed to think that upward dotted path references were brittle and confusing. I would agree if he had happened to be looking at Handlebars which allows this but only using a ../../xxx notation which is brittle and confusing.

However, making {#i18n} reachable but {i18n.firstname} not reachable seems quite arbitrary and annoying. It causes a need to pass many params or hack a section upward to make a new temporary context to get at some values.

I did develop some changes to the dust core that would make upward dotted references just work. The rules, as best I recall right now, would have been:

  1. if the first element in the dotted path is in the current context, try going down and return empty if not found (just like now)
  2. if the first element is not found in the current context, try going upward and if found attempt the downward reference from that point and return the value.

The change worked fine but feeling seemed to be to leave the original language resolving rules alone.

I suppose all of that makes sense, but the documentation says:

If you need to drill into a key available within the parent context, pass the key as a parameter.

The way I read that, it says to me that if you pass the key as a parameter, you can then drill down into it. It doesn't say "pass the value as a parameter". Accordingly, the following should work:

{#childContext key=parentKey}
{key.foo}
{/childContext}

Is there a dustjs mailing list / Google Group / etc for discussion? I couldn't find one. I think it would be helpful.

Contributor

rragan commented Aug 7, 2012

Looking at the generated code, here is what is going on:

  • When the {#people} section is defined, the i18n param object is pushed on the context stack.
  • Then since the section object #people is an array, an iteration is done and for each cycle of the iteration, a new context containing just people[i] is pushed.
  • {i18n.firstname} is seen as a pathed reference and the compiler generates code getPath(false, ["i18n", "firstname"]). Since getPath only looks in the current context for i18n, it won't find it since the context is the i-th element of people during the iteration. The desired "i18n" element is up one context level and not accessible to a getPath reference.

Unlike a full blown compiler, I don't think there is any knowledge the i18n is the name of a parameter to this section. All that happens is the section method stashes params on the context stack where a simple {name} reference will find them. Quite separately, the next line is compiled as a getPath with no special knowledge that i18n is a parameter.

This might be made to work if the params context was merged with elem[i] context but since it is a level up, the getPath won't work with the current dust semantics.

Contributor

vybs commented Aug 8, 2012

@jbcpollak to answer the group question

Is there a dustjs mailing list / Google Group / etc for discussion?

So far we have been using github and it has worked, Some questions get asked on stackoverflow

@rragan ane myself discussed the same issue in the gut hub : #47

But I can see your point that it was not was not easy to discover

we can certainly create a google group as well

Contributor

rragan commented Aug 8, 2012

I added material on this case to the new Dust Tutorial at: https://github.com/linkedin/dustjs/wiki/Dust-Tutorial#wiki-Sections_with_parameters

Contributor

vybs commented Aug 8, 2012

We need to promote this page more:)

Contributor

vybs commented Aug 8, 2012

https://www.linkedin.com/groups/dustjs-client-templates-4296431?trk=myg_ugrp_ovr

@iamleppert will create a google groups too, we will use that as the primary mailing list from now on.

@vybs - A Google Group would be great, I'll look for it

@rragan - I see this in the wiki (That tutorial is really much better than the original docs btw):

While you can specify a non-leaf object as a parameter, e.g.

{#A.B foo=A }
    {foo.name}
{/A.B}

you cannot do anything useful with it since the {foo.name} reference is going to look for foo in the current context but that context is the element of the current iteration of the section #A.B (in this case just the name: "Bob", value). Therefore, "foo" won't be found. The foo parameter is on the context stack but one level higher than the current element iteration so unreachable by a path reference.

But this doesn't make sense to me... isn't the idea of a parameter to put the value into the CURRENT context? Otherwise, what is the point, why not just use A? I would expect that:

{#A.B foo=A }

would import 'foo' into {#A.B} context, so {foo.name} should work. I don't think that breaks the original intention of the language and it seems to me what was intended by the documentation.

Oddly enough, I have found this:

{#A.B foo=A }
    {foo} : '{foo.name}'
{/A.B}

will output:

[Object object] : ''

This tells me foo is bound to something, but I haven't figured out how to debug dust yet, so I can't track down what its bound too.

Contributor

rragan commented Aug 8, 2012

In "normal" language design, passing an object would bind the object to the parameter and allow accessing fields within the object. I'm no expert on the dust compiler but I don't think it has any context of what it is compiling, e.g. it compiles each line indendent of it's context.

It sees the section and generates code to call the section method. The section method pushes all the params to the context stack. Then it generates a new context stack level during the iteration over elements that only holds the current element.

Now it compiles the {foo.name} reference in the section body. All xxx.yyy dotted paths are always compiled to a getPath passing ['xxx', 'yyy'] and the getPath method only searches down from the current context. As a result all those lovely params are not visible to any dotted notation reference. There is no concept of "foo" being bound to anything in particular, it is simply treated a a string to pass to getPath.

Your observation about foo being associated with an object is correct. The {foo} reference generates a get call which searches up the context stack and find the parameter "foo" that was pushed to the stack by the section method. However, there is no way to refer into the object from dust. It would be easy to write a dust helper like {@Val path="foo.name"/} that could find foo by looking up the stack and reference down to the value. I'd prefer changing the language reference rules to allow path references to find values up the stack but that's just me.

Contributor

vybs commented Aug 8, 2012

@rragan, may be I am rating the same I did a while back, if we were to change it, then we should use another syntax to walk up from a sub context,

the . according to original dust means it only walks down.

Contributor

rragan commented Aug 8, 2012

A new reference syntax would presumable have the compiler generate a new method call like getPathRef which would otherwise be similar to the the current getPath but with different rules. I'll muse about a syntax more but tossing some ideas out:

{[a.b.c]} - Pro: retains dots, con - uses up a bracket pair and more chars to type
{~a.b.c} - Pro: less to type for common case, con: uses up a prefix character limiting language extension options
{/a.b.c} - Like previous but maybe connotes something about path con: - leading / might be used for abs reference from top of data
{a\b\c}- Pro: path-like but visually different from something like a/b/c, con- might run afoul of escaping in some environment

Contributor

rragan commented Aug 8, 2012

Here's one more: {{a.b.c}}

Granted it looks like mustache/handlebars but it doesn't take way any other special characters and is fast to type since you double-tap the keys.

One downside is it would be a natural notation for indirect reference, e.g. if {a} evaluates to "name", the {{a}} would be equivalent to {name}. This might be powerful but it feels a bit scary for the presentation logic world.

dust is extremely terse, which is probably a barrier to entry for some people, but since it is terse, I think a single character notation would make sense for this extension, rather than wrapping.

How about ',' '+', or '!' ?

{,foo.path} - Its a bit silly, but a comma is like a ., so it might be intuitive to think of "." and "down" and "," as up.

Alternatively, '..', but instead of needing '../../../', '..'' would just indicate, "somewhere up there", and "." would be "just this context":

{..foo.path} <- search up the context stack for foo, then find path
{.foo.path} <- use my local foo

The advantage there is the visual similarity. It is a bit confusing if '.' means 'this context only' and {} means 'search up'.

I'm still not clear on the utility of the '.', but that is a separate discussion I guess.

Contributor

vybs commented Aug 9, 2012

my 2 cents,

IMO, it was easy to understand dust once we understood get and getPath.

get maps to {}, so it walks up

. maps to getPath, it walks down

I am not sure what is the % iof use cases where we need to walk up wit ../ ? j

Hence {#.} has a meaning of it been current context

|| I will try the second option. that is of course a lot more verbose though than accessing via the path.

if there are more than one or say 10 params to acces, how is the following verbose. IMO it seemed clean

Forgive me if this comment is repeating things I've said before, I just want to make sure I'm being clear....

@vybs : I think something is missing from your comment, but what I would like to be able to do is pass a list of objects into a block and have the iterated-HTML output. But inside that block, I need to repeat a series of internationalized strings, and for that I need to pass in a "constant" object as well. For example, give this:

{i18n : { firstname : "First Name", lastname: "Last Name", phoneNumber: "Phone Number" },
people : [
{"name" : "Moe", "lname" : "Foo", "phone" : "1234"},
{"name" : "Curly", "lname" : "Bar", "phone" : "0001"}
]}

I need to output:

<div>
    <b>First Name:</b> Moe<br>
    <b>Last Name:</b> Foo<br>
    <b>Phone Number:</b> 1234<br>
</div>
<div>
    <b>First Name:</b> Curly<br>
    <b>Last Name:</b> Bar<br>
    <b>Phone Number:</b> 0001<br>
</div>

The most concise way to do this is something like this:

{#people}
    <div>
        <b>{..i18n.firstname}</b> {name}<br>
        <b>{..i18n.lastname}</b> {lname}<br>
        <b>{..i18n.phoneNumber}</b> {phone}<br>
    </div>
{/people}

I really don't want to have to do:

{#people firstname=i18n.firstname lastname=i18n.lastname phoneNumber=i18n.phoneNumber}
 ...
 etc
Contributor

vybs commented Aug 9, 2012

I agree that passing params is ugly~~

      {#people}
          <div>
            <b>{#i18n} {firstname} {/i18n}</b> {name}<br>
            <b>{#i18n} {lastname}{/i18n}</b> {lname}<br>
        </div>
      {/people}

Does the above sound too verbose?

Another suggestion, is this a reason why i18n cannot be a child of people, is this some JSON structure you do not control and have to rely on some external apis?

if the i18n block is a child os people then it is even cleaner

     {#people}
        <div>
          {#i18n} 
                <b>{firstname} </b> {name}<br> // this will walk and should find the name in the people, as long as there is no name in i18n
                <b>{lastname}</b> {lname}<br>
            {/i18n}
         </div>
    {/people}

Perhaps I am misunderstanding how dust works, but since {#people} means to me "switch into the people array and iterate over each item in the array".

So when I see:

{#people}
    {#i18n}
        {firstname}
    {/i18n}
        {name}
{/people}

I assume it will do this:

foreach person on people {
    for each i in i18n {
        print(i.firstname);
    }
    print(person.name
}

If the {#i18n} doesn't mean "for each i in i18n", I think it would be inconsistent with {#people}.

Here is how I wanted to use it, using the require.js i18n plugin (this is a simplified version of my code):

require(["i18n!nls/strings"], function(strings) {
    data = {
        "i18n" : strings,
        phoneInfos : [ {
            "phoneNumber" : "1234123456",
            "phoneType" : "MOBILE",
            "carrier" : "",
            "primary" : true,
        }, {
            "phoneNumber" : "1234123457",
            "phoneType" : "HOME",
            "carrier" : "",
        } ]
    };

    dust.render("phoneInfoEditForm", data, function(err, out) {
         $("#phoneInfoFormDiv").html(out);
    });
}

The point is that the constant strings come from a file "nls/strings.js". This file is a set of constants, maintained by our internationalization team. The phoneInfos array would be loaded by ajax from our backend. The point is to separate the text strings from the data.

Also keep in mind the 'strings.js' file will have other strings too, for addresses, errors, whatever. The Javascript shouldn't know anything about the actual strings, that is something only the template should need to be aware of, so it doesn't make sense to me to have to modify each phoneInfo map adding the i18n map to each one.

in reality, the code would be something like this:

phoneInfos = loadPhoneInfoFromAjax();
data = {
    "i18n" : strings,
    "phoneInfos" : phoneInfos
}

I hope that makes sense.

Keep in mind in my production attempt I also had lists of carriers and phone types (home, mobile, etc) and I needed to refer to path fields about each carrier and phoneType object for each phoneInfo, so it is more like this:

{#phoneInfos}
    {i18n.carrierLabel}<br>
    <select id="phoneInfo[{@idx}{.}{/idx}].phoneType" name="phoneInfo[{@idx}{.}{/idx}].phoneType">
        <option value>{select}</option>
        {#phoneTypes}
            <option value="{code}">{name}</option> 
        {/phoneTypes}
    </select>
{/phoneInfos}
Contributor

jimmyhchan commented Aug 9, 2012

I had added a write up about where Dust searches when a value is not defined on the wiki.

In my previous comment I wrote:

If the {#i18n} doesn't mean "for each i in i18n", I think it would be inconsistent with {#people}.

I realize now this is probably accurate, but will work, because there is only one item in the i18n "array" (since its not an array.)

However, this syntax strikes me as too verbose for this simple (and I would think common) operation:

{#i18n} {firstname} {/i18n}

I think the following already has meaning in dust, but it would be ideal I think:

{#i18n.firstname}

@jimmyhchan : Thanks for the writeup, it looks helpful, I'll have to read it more carefully. I think showing example source data would help make understanding the templates make more sense. :)

Contributor

rragan commented Aug 9, 2012

The {#xxx} section conflates two language features: iteration and setting a new context. For the case where the section name only has one element (e.g. not an array), you only get the context change effect.

I think it will be quite common to have multiple peer level chunks of data in the model because each piece would be added by different service/db calls. Those calls are not aware of each other and should not be. The pain arises in the dust view processing when the presentation design requires displaying parts from each of these chunks in a single message. As long as the chunks have only a single top level element the {#xxx} hack to reset context suffices.

Worst case would be something like the following where you need values from both structures. This case may look a bit contrived because I can't get the syntax to work in the current dust version (separate issue thread coming).

The essence of it is that I'm iterating over users and so that is my context. Within the iteration I want to access an element from a paired array using the index of the current iteration. I can't use {#bizId} as it would start a new iteration and I would lose track of my index but if I don't use {#bizId} I cant get the context pointed properly for my path reference to work.

users: [ {name:"Mary"}, {name:"Ted"}],
bizId: [ {"biz: 123"}, {"biz":"345"}]

{#users} {name} - {bizId[$idx].biz} {/users}

Granted this is a bit of an odd example since if the two structures are so tightly coupled, odds are they could be emitted in a different layout to solve the referencing problem.

Looking a bit deeper though in a simple example like:

{#names}{title} {name} {others.jr}{~n}{/names} which does not work because {others.jr} requires reaching up and pathing down

versus

{#names}{title} {name} {#others}{jr}{/others}{~n}{/names}

we see 393 bytes of generated JS code vs 484 bytes of code plus we have to execute more code to create the section and then reference the value versus just a call to reference the value -- more code and longer execution time.

Contributor

vybs commented Aug 10, 2012

i {#users} {name} - {bizId[$idx].biz} {/users}

i think we tend to forget that core concept of logic less means, the server does th data munging in the way the templates need it.

If we are to add so much logic into templates, then may be using these templating languages is not the correct choice.

so i agree: Granted this is a bit of an odd example since if the two structures are so tightly coupled, odds are they could be emitted in a different layout to solve the referencing problem.

we see 393 bytes of generated JS code vs 484 bytes of code
Secondly, we gzip the code, does this really make a difference?

third,

I am not sure there is significant difference in time in js execution of one versus the other code

@vybs: I don't know about @rragan's proposal since I haven't run into that situation yet, but please don't let that distract us from the original point! :)

One of the few negatives mentioned about dust in the LinkedIn blog post about selecting a templating engine was it didn't have built in i18n support... I think adding this ability would make implementing i18n completely trivial and concise. Alternatively, how about a blog post titled "i18n with dust".

Contributor

vybs commented Aug 10, 2012

i dont think I am deviating from the topic, nor am I against it :)

Second i18n and dust is another whole new topic.

This topic is not related to i18n. i18n and dust is a different topic. We solve with a inline helper.

Dust alone cannot solve i18n, and sometimes i118n is more than just replacing a string in different languages. In our case we have complicated rules and i18n helper solves the problem.

it is basically saying I have 2 separate siblings in the JSON, that have data to be combined and rendered together.

Contributor

rragan commented Aug 10, 2012

vybs, touche... You are right. Get a JSON structure from your server you can work with. Not all needs to be solved in the client. Execution time is probably not an issue. Less sure about overall payload size for compiled JS as minimizing page weight is always a concern

@vybs , I apologize, I didn't mean to be confrontational. My point was just that searching up the context for a pathed variable is probably more common than the other use case @rragan brought up.

For example this would be a more common usecase :

{#users}{..i18n.firstname}{name}{/users}

than this:

{#users} {name} - {bizId[$idx].biz} {/users}

I am not so concerned about performance and size of the generated code as the understandability and compactness of the template:

I think something like this:

{#users}{..i18n.firstname}{name}{/users}

is more understandable and shorter than this:

{#users}{#i18n}{firstname}{/i18n}{name}{/users}

especially when you consider the whole #i18n block would need to be repeated for firstname, lastname, title, address, whatever...

I realize in my examples I've been using a naive i18n solution, and real i18n is much more detailed, but it is meant to be an example.

If you can share, I would love to hear more details about the i18n helper you use.

Contributor

rragan commented Aug 10, 2012

I'd be happy to say a bit about our I18N approach. Probably another thread would be better, although it's not really an issue either. In a few words we send needed content strings to the client and render them there with helper tags. The helpers are done in such a way that content strings can be accessed from anywhere regardless of current dust context.

Contributor

rragan commented Aug 10, 2012

Just got done throwing together a helper-based experiment. New helper looks like:

{@Val of="i18n.firstname"/}

and is read as "value of path"

Semantics are:

  • If "of" path discoverable in current context, return that (e.g same results as {i18n.firstname}
  • If not found in current context, look for "i18n" upward. If found, match down for the rest of path. If found, return that.
  • Return undefined if not matched in current context or first higher context with matching first path element.
Contributor

vybs commented Aug 11, 2012

@rragan i18n is another thread, and with helper, it is easy to access any js object on the object and resolve the value inline.

Much harder issues is having access to the translations and not making another AJAX call every time to retrieve the i18n data ( does not scale for large, high speed sites such as paypal or LI )

Contributor

vybs commented Aug 11, 2012

@rragan , sure helper seems like a much cleaner approach than to modify the grammar.

is there a Pull request ?

The helper is more verbose than a modified grammar, which I think would still be useful, but it is better than shifting contexts, so I'll take it. :) I still haven't learned how to write helpers, but I'll take a look.

@rragan , let us know when the helper is available somewhere. :)

As for i18n, definitely another thread, I have questions about that too. Is the google groups mailing list created yet? :)

Contributor

vybs commented Aug 13, 2012

@jbcpollak

helpers are super easy once you know them

Dust helpers are extensions to the core Dust for adding macro like functionality for logic/formatting/i18n etc
simple example of a helper that existed as part of the core Dust is the the @sep. @sep used in conjunction with a #list prints a comma separated list of the elements in the list

Every helper has access to 4 objects in the Dust core javascript library

chunk --> writer object
context --> json object
bodies --> template body content
params --> tag inline params
See below for how Dust helpers are written, they have access to the context object and have full control of how the helper tag processes the context and renders the body of the tag.

sep: function(chunk, context, bodies, params) {
if (context.stack.index === context.stack.of - 1) {
return chunk;
}
return bodies.block(chunk, context);
},
How to use the @sep helper in a template?

{#people}
{@sep}{name},{/sep}
{/people}
See the wiki link we have on github.

Most of think stackoverflow has more traction and we want to use it along with git. ( We worry that a google group might just be

#1 duplicate
#2 become dormant.

Is there anything in GG that SO does not provide?

GG (or some other mailing list) seems like a better place for general chat and announcements, discussioni of the evolution of dust. SO seems like a good resource for "How do I" questions.

For example, SO doesn't seem like a good place to announce new helpers, companion libraries, or discuss the evolution of the formal syntax or internals.

Contributor

vybs commented Aug 13, 2012

@jbcpollak
Sure, good points, agree

I am really try hard to keep all things in one place... so bear with me,

we can have a linked #dustjs twitter group for annoucements.

Any major release is announced on the

http://linkedin.github.com/dustjs/

alright, its your project. :) I just like GG for the archives. I'll look for #dustjs

Contributor

vybs commented Aug 13, 2012

@jbcpollak its not my project:) dont get me wrong. Creating a google group is not a issue. We shall roll it, and see how actiive it will be.

Contributor

vybs commented Aug 22, 2012

close this issue and another one for adding the helper to walk up the tree if required.

May be a better name for the helper to be explicit about what it does

vybs closed this Aug 22, 2012

@rragan , just wondering, did you @Val helper mentioned above ever get integrated into dust or the dust-helpers project?

IE:

{@Val of="i18n.firstname"/}

If not, can you share it with me? Thanks

Contributor

rragan commented May 7, 2013

It was not integrated. There is a some subsequent work done to make native dust behave the way you want for nested paths but that PR is not final. Then a helper would not be needed.

I do have an outstanding PR for a later version of val named access. The code for it is here: rragan/dustjs-helpers@50deade

Contributor

seriousManual commented May 14, 2013

this helper is not of use if someone wants to use this value as a parameter for a helper.
any thoughts on this?

Contributor

rragan commented May 14, 2013

Yes, helpers cannot be used as parameters to other helpers which is a significant limitation to composition. A separate issue is looking at just making nested path references work which would obviate the need for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment