Handle IPython line magics and shell assignments

[krassowski]

When line magics are detected in the notebook (lines starting with '%') a non-intrusive message should be shown to the user, like that:

It seems that your notebook contains IPython magics (special commands starting with %); Should the LSP linters ignore them?

with options "Ignore" and "Let the server lint lines starting with '%'". This way the few languages in which this could be a false positive would not be affected. Also, we could have a blacklist of languages for which the message would not appear in the first place.

[krassowski]

The current workaround of replacing line magics with "pass" (for Python) and skipping cell magics works but has a drawback: the imported cell/line magics are not included in the virtual document, thus there are false-positive linter messages like "X is imported but unused".

A more robust workaround would add a function call for the magic, reflecting what IPython would do (after initial parsing to determine the magic to be used) - thus the magic would be used in the code.

Ideally, we would allow the user to define custom regular expressions, for example parsing R-magic:

%%R -i input_variable -o output_variable
some R code

into

# noqa
output_variable = R(input_variable)

while this would not work if executed, this is not the goal here - what we want is that the linter registers the that

1. R magic was used (though R magic, in particular, does not require importing but is rather loaded via another magic)
2. a new variable (output_variable) of unknown type was registered in the namespace
3. the input_variable was used.

An initial set of rules (sensible defaults), per language, could be defined in the settings, allowing the user to disable any or all of them and add their own regular expression based rules.

Notes:

• In an alternative reality, we could ask linters to respect all the fancy features introduced by interactive kernels, but there will always be a gap between the introduction of new magic to a specific kernel and all the possible linters picking up on that:
  • for example, LSP for Python depends on over a dozen different specialized linters, many of which may consider the ipython feature requests to be too specific for their scope.
  • on the other hand, the LSP for R has much, much slower rate of development, possibly because most of the casual R users, while excelling in statistical programming are not programmers familiar with advanced general programming aspects.
• there are some transpiling solutions (IPython → plain python file), which we could use as a reference, to avoid re-inventing a wheel

Update: actually, using noqa is not needed as we can simply ignore the result of inspection for these lines. For simplicity, the regular expressions should return a single line as otherwise, the mapping between the virtual document and notebook gets more tricky.

[krassowski]

Current implementation demonstrated and explained in examples/Magics_and_rpy2.ipynb.