Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JEP-15 Third-Party Functions #5

Closed
wants to merge 3 commits into from

Conversation

springcomp
Copy link
Contributor

Third-Party Functions

JEP 15
Author Nolan Wood, Maxime Labelle
Status draft
Created 28-Feb-2022

Abstract

This JEP proposes a new distinction between reserved function names
and _third-party function names. This allows the spec to evolve by
introducing new functions without the fear of clashing with existing
functions from popular third-party implementations.

Motivation

Virtually all the current language implementations of JMESPath support
registering additional "custom" functions that add features to JMESPath.

There is currently no restrictions as to what those function names may be
except that is must match the unquoted-string production.

This proposal specifically acknowledges that third-party functions may
be used to extend JMESPath and mandates tha all third-party function
names should start with a leading underscore character.

Functions whose names starts with a letter – known or future – are
effectively reserved exclusively for use by JMESPath.

Specification

This JEP proposes adding the following text to the specification in the
Function Expressions section:

Function Expressions

Functions allow users to easily transform and filter data in JMEPath expressions.
Built-in functions are defined by JMESPath. However, a compliant implementation
MAY support registration of new functions to extend JMESPath.

Functions whose name starts with an alphabetic character (i.e A-Za-z) are reserved
for JMESPath.
Conversely, third-party function names SHOULD start with a leading underscore character
(i.e _ ).

Registration of conflicting function names MAY raise an error.
However, if no error is raised, built-in functions MUST take precedence over
third-party functions.
The resolution of conflicting function names from competing third-party libraries
is undefined.

Rationale

This JEP does not introduce any grammar change.

Although there has been discussions about having a different semantic constructs
for both reserved functions and third-party functions, it has been considered
as an implementation detail. Besides, this would unnecessarily make it more
difficult for library authors to maintain two different logics for evaluating
functions.

@springcomp springcomp added documentation Improvements or additions to documentation, including specifying behaviour JEP-15 labels Feb 28, 2022
@springcomp springcomp changed the title Third-Party Functions JEP-15 Third-Party Functions Feb 28, 2022
@innovate-invent
Copy link
Collaborator

Gitter conversation discussing merit of this JEP and selection of proposed prefix: https://gitter.im/jmespath/chat?at=6219441ad1b64840db3cfb1c

@innovate-invent
Copy link
Collaborator

Thanks so much for writing this up. I have some notes about the final wording.

Conversely, third-party function names SHOULD start with a leading underscore character (i.e _ ).

I think SHOULD needs to be changed to MUST for this to be anything more than a recommendation.

Registration of conflicting function names MAY raise an error. However, if no error is raised, built-in functions MUST take precedence over third-party functions.

This is an implementation detail that doesn't really need to be in language spec. It may be desirable for registration of conflicting functions to silently overwrite the conflict.

The resolution of conflicting function names from competing third-party libraries is undefined.

This is also an implementation detail.

@springcomp
Copy link
Contributor Author

springcomp commented Feb 28, 2022

In theory 3rd party names MUST start with underscore.
However, I feel this renders all existing implementations non-compliant. That's why I changed it to SHOULD to try and preserve backwards compatibility.

From then on, it followed that there must be some priority for built-in functions. Even for future ones. Another way of saying it is ; even though this change would not render your custom implementation non-compliant, you have no guarantee if ever a built-in function appears.

What do you think?

@springcomp
Copy link
Contributor Author

The resolution of conflicting function names from competing third-party libraries is undefined.

This is also an implementation detail.

Agreed. But wanted to leave no room for ambiguity.
Happy to remove if needed.

@springcomp
Copy link
Contributor Author

springcomp commented Feb 28, 2022

Registration of conflicting function names MAY raise an error. However, if no error is raised, built-in functions MUST take precedence over third-party functions.

This is an implementation detail that doesn't really need to be in language spec. It may be desirable for registration of conflicting functions to silently overwrite the conflict.

I think we are saying the same thing. I'm trying to say that it can be silently ignored. But there should no circumstances for a built-in function to be overwritten by a custom implementation. Or are you specifically saying that the spec should not forbid that explicitly?

@innovate-invent
Copy link
Collaborator

In theory 3rd party names MUST start with underscore. However, I feel this renders all existing implementations non-compliant. That's why I changed it to SHOULD to try and preserve backwards compatibility.

My idea was to render all existing implementations non-complaint, with the follow up that most of the functions they currently provide would be added to the spec bringing them back into compliance.

From then on, it followed that there must be some priority for built-in functions. Even for future ones. Another way of saying it is ; even though this change would not render your custom implementation non-compliant, you have no guarantee if ever a built-in function appears.

I think that would be detrimental to backwards compatibility.

@innovate-invent
Copy link
Collaborator

I think we are saying the same thing. I'm trying to say that it can be silently ignored. But there should no circumstances for a built-in function to be overwritten by a custom implementation. Or are you specifically saying that the spec should not forbid that explicitly?

I think that the language spec shouldn't speculate on implementation at all, otherwise it could cause implementer infighting on the various ways of interpreting the speculation. It is better to be ambiguous now and clarify later than to say something now and have to change it later.

@springcomp
Copy link
Contributor Author

My idea was to render all existing implementations non-compliant, with the follow up that most of the functions they currently provide would be added to the spec bringing them back into compliance.

I see.

Maybe we need to coordinate and list which functions we are talking about here.
I assume those would be:

  • items, from_items (previously known as to_object) and zip functions.
  • GorillaStack-style split string manipulation function, but maybe other expected functions as well ?
  • Particularly the lower and upper functions from jmespath.py.

Do you see others?

@jordemort
Copy link

jordemort commented Mar 5, 2022

I'm currently working on an app that is going to embed JMESPath with a bunch of custom functions so this is relevant to my interests. I was wondering how I could avoid my custom functions getting clobbered by a future version of the spec, which is how I ended up here :)

Would you consider x_ or X_ as a prefix instead of just _? IMO, functions named like x_foo, x_bar etc would be more aesthetic and more readable in complex expressions than just _foo and _bar.

I think what would be really ideal is to have some sort of namespace for functions, so I could have something like myapp:foo and myapp:bar but I don't have a strong enough understanding of the grammar to know if something like that would be possible without breaking a bunch of things. Namespaces would make it more obvious that a function is specific to a particular app, though.

@springcomp
Copy link
Contributor Author

@jordemort I'm secretly a fan of namespaces 😏 but we're not sure it's the best way to achieve this. Besides, we plan to include functions in the specification at a much faster pace than previously.

Please head up to the discussions area to be involved.
Also, you might want to read the chat we had on Glitter on 25 February
(Cannot seem to be able to link to the actual conversation from my mobile device)

@springcomp springcomp mentioned this pull request Mar 14, 2022
@springcomp springcomp marked this pull request as draft May 16, 2022 20:32
@springcomp
Copy link
Contributor Author

Closed in favor of corresponding discussion.

@springcomp springcomp closed this Aug 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation, including specifying behaviour
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants