Registration, autogenerated symbols and 'no documentation' false positive

[eddelbuettel]

What can we do when .registration=TRUE and we now generate globally-visible identifiers that have to be R and C++ compatible?

Under the default exporting regexp of exportPattern("^[[:alpha:]]+") these are visible, so R nags.

We cannot prefix with dot or underscore because (I think) in either case one language whines. But if we do nothing, R whines. Shall we (auot-)create an Rd (or roxygen) stub?

[jjallaire]

I don't fully get the scenario so hard to say. What's a concrete example of a function that triggers this error?

I personally don't love the exportPattern("^[[:alpha:]]+") as there are typically more non-exported than exported functions in a package and this forces users into naming functions with dot prefixes. But I realize it's common convention so something we need to deal with.

We could always generate a single catch-all Rd file sort of like roxygen does for re-exports. e.g. https://github.com/rstudio/keras/blob/master/man/reexports.Rd

[eddelbuettel]

I don't fully get the scenario so hard to say. What's a concrete example of a function that triggers this error?

It hit me during the class. If we use the package generators (either via Rcpp*.package.skeleton() or the equivalent generators in RStudio) then we get a wild-carded exportPattern(...). Personally I am fine editing it but for newbies it sucks.

Agreed on the re-exports file. I know how to write those in Rd, I failed to get it done with roxygenize() when I tried earlier today on the train. I guess the .Rd route is the way to go then.

[jjallaire]

It seems like .registration = TRUE and exportPattern("^[[:alpha:]]+") together is an anti-pattern, as it will result in spamming the search path with a bunch of intermediate generated functions with awkward names (i.e. prefaced with pkgname_).

I wonder if we should move away from generating packages with exportPattern("^[[:alpha:]]+") entirely. I agree that it's trickier for beginners to get that they need to add exports to NAMESPACE, but the alternative that they never learn about NAMESPACE and have to use dot prefacing for all private functions seems just as bad.

[eddelbuettel]

I hear you (and I wholeheartedly agree about the anti-pattern).

But the pattern is what R does by default. I am a little hesitant to move away from it.

I think ultimately the best may be to have two regexps: all matching 'x' yet excluding to cover our per-package pattern. Defensible idea?

But until then and for maybe some time auto-generate help stubs?

[jjallaire]

But my guess is that R doesn't generate both .registration = TRUE and the export pattern by default? If they do then it's advocating that all native routines should be part of the public API (especially because you can't name a native routine with a dot prefix). I can't believe they would do this intentionally. So the options are: - We could not generate the export pattern in our skeleton functions - Detect the perverse combo and generate an Rd file so the tools don't complain - Introduce a new prefix (e.g. rcpp_export_pkgname_) and update the generated export pattern to exclude it I think option 3 is just another variation of option 2 (papering over something we think is a bad practice). So if it's not going to cause undue support burden I think we are better off taking a position against export patterns in a world where all native symbols automatically match the pattern.

…

On Sat, Jul 1, 2017 at 1:10 PM Dirk Eddelbuettel ***@***.***> wrote: I hear you (and I wholeheartedly agree about the anti-pattern). But the pattern is what R does by default. I am a little hesitant to move away from it. I think ultimately the best may be to have two regexps: all matching 'x' yet excluding to cover our per-package pattern. Defensible idea? But until then and for maybe some time auto-generate help stubs? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#723 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGXx1oQqxPYKDObpcyPx_6DohAGW15Rks5sJn1xgaJpZM4OKUKf> .

[jjallaire]

I just noticed that these exported objects are not even functions:

class(registration::registration_rcpp_hello_world)

[1] "CallRoutine" "NativeSymbolInfo" I wonder if search done by exportPattern shouldn't be automatically discarding these? (i.e. a change to R could resolve this as well). J.J.

…

On Sat, Jul 1, 2017 at 3:57 PM, JJ Allaire ***@***.***> wrote: But my guess is that R doesn't generate both .registration = TRUE and the export pattern by default? If they do then it's advocating that all native routines should be part of the public API (especially because you can't name a native routine with a dot prefix). I can't believe they would do this intentionally. So the options are: - We could not generate the export pattern in our skeleton functions - Detect the perverse combo and generate an Rd file so the tools don't complain - Introduce a new prefix (e.g. rcpp_export_pkgname_) and update the generated export pattern to exclude it I think option 3 is just another variation of option 2 (papering over something we think is a bad practice). So if it's not going to cause undue support burden I think we are better off taking a position against export patterns in a world where all native symbols automatically match the pattern. On Sat, Jul 1, 2017 at 1:10 PM Dirk Eddelbuettel ***@***.***> wrote: > I hear you (and I wholeheartedly agree about the anti-pattern). > > But the pattern is what R does by default. I am a little hesitant to move > away from it. > > I think ultimately the best may be to have two regexps: all matching 'x' > yet excluding to cover our per-package pattern. Defensible idea? > > But until then and for maybe some time auto-generate help stubs? > > — > You are receiving this because you commented. > > > Reply to this email directly, view it on GitHub > <#723 (comment)>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AAGXx1oQqxPYKDObpcyPx_6DohAGW15Rks5sJn1xgaJpZM4OKUKf> > . >

[eddelbuettel]

Nice work. I like this a lot. And possibly a generic issue not just for Rcpp but everything with the same (default) regexp and .registration=TRUE ?

Do you have a moment to send a mail to r-devel? Else I can do it once I settled in a little here.

[eddelbuettel]

Ok, I think I am loosing my marbles. Look for example at this src/init.c file in RcppAnnoy. We use global exports, we have no extra aliases, and we get no warnings from R CMD check --as-cran because the Rcpp Modules functions are exported as _rcpp_modules_*. I think I tried this 'by hand' with another package. Can we not prefix the generated symbols with _rcpp* as well?

[jjallaire]

If that works with no complaints from R CMD check then that is definitely the right solution!

…

On Sun, Jul 2, 2017 at 8:25 AM, Dirk Eddelbuettel ***@***.***> wrote: Ok, I think I am loosing my marbles. Look for example at this src/init.c file in RcppAnnoy <https://github.com/eddelbuettel/rcppannoy/blob/master/src/init.c>. We use global exports, we have no extra aliases, and we get *no* warnings from R CMD check --as-cran because the Rcpp Modules functions are exported as _rcpp_modules_*. I think I tried this 'by hand' with another package. Can we not prefix the generated symbols with _rcpp* as well? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#723 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGXxy6oK50wP69SFbfHL3Mo3FFcRYILks5sJ4w8gaJpZM4OKUKf> .

[jjallaire]

I'll take a look at implementing this today early next week. I also think a note to R-devel on this might be in order as I doubt they intend for all of those objects to actually become part of the public API of the package.

…

On Sun, Jul 2, 2017 at 8:35 AM, JJ Allaire ***@***.***> wrote: If that works with no complaints from R CMD check then that is definitely the right solution! On Sun, Jul 2, 2017 at 8:25 AM, Dirk Eddelbuettel < ***@***.***> wrote: > Ok, I think I am loosing my marbles. Look for example at this src/init.c > file in RcppAnnoy > <https://github.com/eddelbuettel/rcppannoy/blob/master/src/init.c>. We > use global exports, we have no extra aliases, and we get *no* warnings > from R CMD check --as-cran because the Rcpp Modules functions are > exported as _rcpp_modules_*. I think I tried this 'by hand' with another > package. Can we not prefix the generated symbols with _rcpp* as well? > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#723 (comment)>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AAGXxy6oK50wP69SFbfHL3Mo3FFcRYILks5sJ4w8gaJpZM4OKUKf> > . >

[eddelbuettel]

Sorry for the line noise. I had meant to test just this (ie using underscores) 'by hand' last week but I must have messed something up. I can confirm that this use in RcppAnnoy leads to zero complaints from R 3.4.0 and R-devel (as of a few days ago).

Getting this fix into the July release would be great. We should have time.

[jjallaire]

Just discovered that R doesn't like functions with _ prefixes (so the e.g. .Call(_mypkg_myfunc) that is generated within RcppExports.R causes an error)) C++ doesn't like . and R doesn't like _. I guess we could do `_pkgname_myfunc` but that is a pretty baroque workaround.

…

On Sun, Jul 2, 2017 at 8:45 AM, Dirk Eddelbuettel ***@***.***> wrote: Sorry for the line noise. I had meant to test just this (ie using underscores) 'by hand' last week but I must have messed something up. I can confirm that this use in RcppAnnoy leads to zero complaints from R 3.4.0 and R-devel (as of a few days ago). Getting this fix into the July release would be great. We should have time. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#723 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGXx6Ik_PzQbLaXey8Sop6YMXBe1X17ks5sJ5D_gaJpZM4OKUKf> .

[eddelbuettel]

Right. That must be what I ran into -- R complains about _. The backtick-underscore solution is the best I can think of.

What other options do we have? Complain to r-devel?

[jjallaire]

I put in a PR with the back-tick solution. Noted in the PR is that we are now generating C++ code with reserved symbol names (_ prefix). Perhaps we can do the underscore for now and then if/when R 3.5 stops exportPattern from also exporting native symbols then we can remove the underscore.

…

On Sun, Jul 2, 2017 at 10:54 AM, Dirk Eddelbuettel ***@***.*** > wrote: Right. That must be what I ran into -- R complains about _. The backtick-underscore solution is the best I can think of. What other options do we have? Complain to r-devel? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#723 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGXx-dSia_EkG_yRIm0lqsjbzbdgoJOks5sJ680gaJpZM4OKUKf> .