MarkLogic predeclared namespaces mismatch

[charlessimpson]

It looks like there are some mismatches between the XML namespaces that the plugin thinks MarkLogic predeclares and which ones MarkLogic documents and currently implements in 7, 8, and 9.

The only change from MarkLogic 5 and 6 to 7, 8, and 9 is the addition of semantic built-in functions.

I'm not sure where the list of namespaces in https://github.com/ligasgr/intellij-xquery/blob/master/src/main/java/org/intellij/xquery/reference/namespace/marklogic/MarkLogicPredeclaredNamespaces.java came from, but it doesn't match.

As a related issue, MarkLogic uses XQuery 1.0's predeclared namespaces, not 3.0, when a MarkLogic-specific version isn't used.

[rhdunn]

The predefined namespaces list on that page is incomplete. For example, the 1.0-ml section does not list json as a predefined namespace, yet:

 xquery version "1.0-ml";
 <json:a></json:a>

runs without errors, showing the correct json namespace in the xmlns declarations. This is because any built-in function has its namespace declared in the 1.0-ml syntax by default. For the 1.0 syntax, you need to explicitly declare (not import) that namespace to make those built-in functions available.

MarkLogic also provides some functions in modules that need to be explicitly imported to use (e.g. the security functions). It even mixes the two (e.g. for json, it provides json:array and others as built-ins, while several functions such as json:transform-to-json require /MarkLogic/json/json.xqy to be imported as the json namespace).

MarkLogic 7.0 adds the rdf, sc and sem namespaces; 8.0 adds the geo and temporal namespaces; and 9.0 adds the tde namespace to the default available namespaces. For MarkLogic 6.0, the json and xsi namespaces are not listed in the documentation, but are available by default. (Note also that xsi is available by default for the "1.0" syntax, as it is listed as a bound prefix in the XQuery specification). I don't know how this compares for MarkLogic 5.0, as MarkLogic don't make the installers available for versions they have dropped support for,

For exsl, that is a set of built-in functions that are only available in the MarkLogic XSLT implementation, not in XQuery, so shouldn't be predefined.

The other namespaces are for importable modules provided by MarkLogic. It looks like this plugin is handling those as built-in namespaces/functions, instead of resolving the import statement to the MarkLogic modules directory.

NOTE: XQuery 3.0 defines the same set of predeclared namespaces (xml, xs, xsi, fn and local) as XQuery 1.0. That is, the 3.0 specification does not state that the math namespace is predefined (which is the only additional namespace available for XQuery 3.0). XQuery 3.1 adds the math namespace, and also adds the map and array namespaces for the new map and array features added in XQuery 3.1. The Saxon implementation added map support in 9.4, so likely added the map namespace then as well.

[ligasgr]

The list https://github.com/ligasgr/intellij-xquery/blob/master/src/main/java/org/intellij/xquery/reference/namespace/marklogic/MarkLogicPredeclaredNamespaces.java was compiled by me by trying to run random sample of functions from those namespaces without declaring them as I've noticed that the documentation of MarkLogic is not complete in this regard.
If there are cases where something is not actually available in MarkLogic but listed on the page - please let me know. Otherwise I will close the issue.

With regards to importable modules provided by MarkLogic available on that list...
As there is neither a facility in the plugin to connect to remote instance of MarkLogic nor a way to specify local instance of MarkLogic just for the specific purpose of referencing functions from standard installation of MarkLogic I made those functions listed so that the help that you get after Ctrl-P which lists arguments of functions has knowledge of standard arguments of functions available in MarkLogic.
If you see any easier way of achieving this level of code assistance without relying on live instance of MarkLogic - please let me know, I'll be happy to improve this piece of logic.

[willsthompson]

@ligasgr I'm not sure there is a better way than periodically updating the plugin to support the latest MarkLogic-specific conventions. To echo @rhdunn's comment, the json namespace is predeclared in MarkLogic, but not recognized as such by the plugin. Also, all of the json-prefixed functions listed here are available as built-ins without declaring the namespace or importing a module:

https://docs.marklogic.com/8.0/xdmp/json

[rhdunn]

One issue with including the functions that are not built-in in the built-in infrastructure, is that code like:

declare namespace basic = "http://marklogic.com/xdmp/json/basic";
xdmp:quote(json:transform-to-json(<basic:json type="array"/>))

would be valid according to the plugin, whereas it needs an import. The valid form of the code is:

import module namespace json = "http://marklogic.com/xdmp/json" at "/MarkLogic/json/json.xqy";
declare namespace basic = "http://marklogic.com/xdmp/json/basic";
xdmp:quote(json:transform-to-json(<basic:json type="array"/>))

[charlessimpson]

In this particular case, I think I would prefer the plugin to continue to require users to import the namespace.

In the longer term, maybe the plugin could maintain a lookup of predeclared namespaces and functions, rather than just namespaces?

[willsthompson]

@charlessimpson I think that's the only sensible solution. Functions from an external module shouldn't resolve unless the module is explicitly imported.