Add pre-processer into Processing.R

[gaocegege]

ref https://forum.processing.org/two/discussion/comment/90750/#Comment_90750

As for those conflict function names such as line, we could pre-pass all line calls in R mode to processing.line in Java and r$line to standard line function call in R, because I think R mode users call processing built-in functions more often. And if they need to use same name functions in R standard library, r$something works.

Then processing.r has the same behavior as in Java mode, follow the Law of Least Astonishment to more user-friendly, IMO :)

implement the pre-processor to solve the problem

[gaocegege]

See http://stackoverflow.com/questions/2439782/overload-with-different-return-type-in-java

And there is one thing else should be noticed: The implementation of built-in alpha function is tricky, I don't know why I can't define two function:

public double alpha(int rgb) {
}
public float alpha(int rgb) {
}

in one class, javac tells me that they are one function, and it is an override function. It seems that same function name, same parameter lists and different return types is treated as override, not overloading.

Pre-processor should solve this problem.

[jeremydouglass]

Have you looked at the Processing default (Java mode) preprocessor?

https://github.com/processing/processing/tree/master/java/src/processing/mode/java/preproc

I'm not familiar with it but I believe it is based on ANTLR -- quite complex and heavy-duty, but it might give you some ideas for reusing parts or for just making a very lightweight R preprocessing. Once you have studied it if you have questions you could consult with @JakubValtar and @benfry

[gaocegege]

IMO, the goal of pre-processor is to reorganize the code and transfer the syntax sugars to regular syntax. Then we need a tokenizer and a parser to achieve this.

I am not sure if renjin could do that. I will have a look before the implementation.

BTW, processing.py's pre-processor is another way. It parses the AST from the code as the standard python and does some operations in the AST to achieve the same goal.

[jeremydouglass]

Nice example from processing.py's pre-processor. Is there is a similar AST library approach available for Java? That might work. If I understand you, the single biggest pre-processor issue right now is identifying a list of implemented functions functionName and adding processing$functionName. After that, the logic for moving contents of setup() and settings() in P3 (which processing.py demonstrates). After that, color.

If I'm understanding the big problem correctly, renjin parses R for JVM -- because it isn't "translating R into Java" there is no way to simply use these aspects of Processing's Java pre-processor. This would be ideal, but is impossible.

[gaocegege]

The big problem is right, and renjin could not access Java AST. But I think there may be some tools that could access R AST. Code could be re-organized in R before it is sent to renjin to parse.

[JakubValtar]

@gaocegege

And there is one thing else should be noticed: The implementation of built-in alpha function is tricky, I don't know why I can't define two function:

public double alpha(int rgb) {
}
public float alpha(int rgb) {
}

in one class, javac tells me that they are one function, and it is an override function. It seems that same function name, same parameter lists and different return types is treated as override, not overloading.

Different return type is not enough to make two functions different. Having two different return types introduces ambiguity into the type system. As for Java, you should leave only float function there.

The question is what are you trying to achieve here? Processing uses only float in public API. float (32 bits) gets automatically promoted to double (64 bits) when needed, because it's widening conversion and no data is lost. It works like this:

float foo() {
  return 1.23f;
}

void setup() {
  float f = foo();
  double d = foo();
}

The other way around is narrowing and you have to tell the compiler that you are aware that some data will be lost:

double bar() {
  return 1.23;
}

void setup() {
  double d = bar();
  float f = (float) bar(); // cast to float, otherwise you get a compiler error:
}                          // Type mismatch, double does not match with float

I will write you a general overview of our Java preprocessor later this week, as I'm busy with school these days.

[gaocegege]

Oh, It is my fault. Widening conversion works for alpha. Pre-processor in Processing.R is to remove the prefix.

ref https://forum.processing.org/two/discussion/comment/90747/#Comment_90747
and https://forum.processing.org/two/discussion/comment/90749/#Comment_90749. To slove the problem, we decided to import pre-processor.

And thank you for the overview. It is not so urgent 😄 We will start to write pre-processing logic after a few weeks.

[jeremydouglass]

@gaocegege -- Regarding pre-processing to remove the processing$ prefix -- above you mentioned this example of how pre-processing would work:

• line() ---> processing$line()
• r$line ---> line()

Regarding the second half of pre-processing -- is line actually an R built-in? Are any of the built-in functions from the Processing(Java) API that are being worked on in #16 "Support more built-in functions" going to conflict with R built-in functions? I took a quick look at R builtins()...

http://www.r-fiddle.org/#/fiddle?id=k5Ytlpdb

...and I didn't see any. The closest I saw was Processing's Table class and R's table function, which isn't a conflict, just a potential source of confusion.

It still might be a good idea to simply pre-process r$anything() -> anything() so that there is a built-in workaround in case there is a conflict. But the main purpose of pre-processing would be the convenience of dropping processing$ from all those Processing(Java) API references.

[gaocegege]

ref https://stat.ethz.ch/R-manual/R-devel/library/stats/html/line.html

line is in stats package in r-core. I have tried to define the function directly, and it doesn't works.

[jeremydouglass]

Got it. I should have checked the whole core, not just built-ins! Line is a good example / test case of namespace collision, then.

…

On Fri, May 19, 2017 at 17:30 Ce Gao ***@***.***> wrote: ref https://stat.ethz.ch/R-manual/R-devel/library/stats/html/line.html line is in stats package in r-core. I have tried to define the function directly, and it doesn't works. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAwvaon6sVPi8q8fVyWZCQJ2X_b11ZrAks5r7jQogaJpZM4MU_xS> .

[gaocegege]

Yeah, r-core and r-dev are the main dependencies for R 😄

[gaocegege]

Yet another potential solution

There is one function attach in R, which will attach the namespace into R global namespace. But the functions such as line will fail to attach since there is line already.

[gaocegege]

Attach couldn't work since processing now is a variable, is a external pointer, which is not allowed in renjin to attach.

ref https://github.com/bedatadriven/renjin/blob/e4349cc2c66a1d076b8ab5d2e3db8a1f0796a77b/core/src/main/java/org/renjin/primitives/Environments.java#L375

But I have a new idea: write a renjin extension(package) to package processing and run library(processing) before the real sketch is run. But it will be blocked by #17

[gaocegege]

AST way like processing.py

Tool: https://github.com/graalvm/fastr or https://github.com/bedatadriven/renjin

Just a reminder to me.

[gaocegege]

RParserTest in renjin is helpful to understand the RParser in renjin, which may be used to pre-parse the code.

[gaocegege]

P3D <- "processing.opengl.PGraphics3D"

box <- function(...) {
    processing$box(...)
}

P3D <- "processing.opengl.PGraphics3D"

settings <- function() {
    processing$size(100, 100, P3D)
}

setup <- function() {
    processing$noLoop()
}

draw <- function() {
    processing$translate(58, 48, 0)
    processing$rotateY(0.5)
    processing$noFill()
    box(40, 20, 50)
}

I think I find a better way than AST or regex: define functions in an R script and evaluate it before the sketch, the evaluate the sketch, just like processing.py's preprocessor does.

[gaocegege]

Thes functions are not defined in PApplet, need to deal with them manually.

Wont fix:

• append
• arrayCopy
• concat
• binary
• boolean
• char
• byte
• nf
• nfc
• nfp
• nfs
• match
• matchAll
• float
• int
• printArray
• extend
• join
• shorten
• sort
• splice
• split
• splitTokens
• trim
• str
• subset
• reverse

TODO

• abs
• asin
• sin
• atan
• tan
• atan2
• cos
• exp
• floor
• log
• max
• min
• round
• sqrt
• ceil
• constrain
• day
• hour
• degrees
• hex
• launch
• mag
• lerp
• print
• dist
• month
• minute
• norm
• pow
• second
• radians
• println
• sq
• unbinary
• unhex
• year
• map

[jeremydouglass]

Great list!

I believe that some of these functions (like sin cos atan2) also have R equivalent built-ins.

So one question is whether to map those function manually or whether to leave them accessing their default R built-ins.

We should eventually choose a policy that makes sense -- either mapping everything to Processing, or keeping certain categories of things "native" to R.

[gaocegege]

Yes, I think if these functions have same behavior as in Processing, there if no need to put them in a new category 🤔 If they have different behaviors, tag them and show in different color or something else.

[gaocegege]

Close via #134 #95