SKIP doesn't behave as expected

[mathrick]

Trivial example:

>>> glom({'foo': 42}, ('foo', lambda x: SKIP))
42

SKIP, in a tuple, essentially results in whatever spec produced it not being there, rather than the entire compound spec (ie. the surrounding tuple) coming to a halt. That is not super important in the example above, but I constantly run into it in my glom-based parser / pattern matcher, where I have a potential match that needs further processing:

glom(data, (lambda x: x if x.get('class') == 'foo' else SKIP, 'val', str))

I'd expect SKIP to essentially abort the entire "branch" of a computation (which is what a tuple of spec is, really), but instead it happily lets through whatever was the target before, leading to a lot unexpected results.

[mahmoud]

Hi Maciej! I've struggled a bit with SKIP in tuples myself, in fact. SKIP's meant to be used for optional processing steps in the tuple. For aborting the processing, you'd want to use STOP. Give that a whirl and let me know what you think. :)

[mathrick]

@mahmoud: STOP was in fact the first thing I tried when I ran into it :). But it doesn't really abort either, just prevents further steps from taking place. For example:

glom({'data': {'class': 'foo', 'val': 42}}, 
     ('data', lambda x: x if x.get('class') == 'bar' else STOP))
>>> {'class': 'foo', 'val': 42}

[mahmoud]

Ah, so the desired behavior is that the target would get dropped if a SKIP (or maybe a STOP) is returned from the tuple? Actually, if you wouldn't mind including the desired output for the example(s) above that'd be great!

[mahmoud]

@mathrick just a gentle ping on the example; I'm still thinkin about this one!

[mathrick]

Ah, sorry, I got busy with another part of my project and I forgot to reply here. The output is a bit of a tricky area, since in the example above, you really want "nothing" (ie. no values at all, not even None), which is not possible. I think returning a SKIP here would be acceptable, since it's a sentinel value already, and the client code will just need to check for that. The advantage of doing so would be that it'd compose nicely inside specs, so SKIP and STOP always mean "abort my chain of computation" (the difference being that STOP also aborts any sibling chains). Determining the scope of the abort would then be the job of individual specs. For example, in a fictional webapp which needs to display a user's name:

display_name = glom(user,
                    (Check(has_completed_profile, default=STOP),
                     Coalesce(
                         (Check(has_new_profile, default=SKIP), 'profile.display_name'),
                         (Check(has_legacy_profile, default=SKIP), 'user_profile.name')),
                     format_for_display))

if display_name == SKIP:
    raise redirect(url_for(complete_user_profile))

Here Check serves as a guard condition, ensuring that the rest of the access can only happen if the check passes. The first Check will not let the Coalesce execute at all if it fails, thanks to the STOP. While the above is a made-up example, it's actually pretty close in spirit to what my use case is, whilst being a bit easier to understand without needing an introduction to my homegrown pattern matching syntax :)

If SKIP as the return value doesn't sit right with you, then perhaps a new sentinel NOTHING would be better?

[mathrick]

@mahmoud: just a quick ping, is this clear enough, or do you need more info?

[mahmoud]

Hmm, yeah I think more info would be helpful. The code blocks above detail some of the data and some prospective specs, so really the only thing remaining would be the expected output. Would love to see that.

By the by, we're working on a Match spec, somewhat positioned as a successor to Check in certain cases, over in #94. Always good to take more use cases into account there.