Support for namespace and package environments

[wch]

This is a first attempt at loading the package into a namespace. It works, but the code seriously needs to be cleaned up.

• The namespace is something like <namespace:ggplot2>
  • All the objects in the package is loaded into it, including exported and non-exported items.
  • It is a child environment of R_GlobalEnv
  • I added a name attribute, but "normal" package namespaces don't seem to have this.

There is also a package environment.

• The package environment is something like <package:ggplot2>
  • All of the objects from the namespace are copied to the package environment. Ideally, this should only copy over exported objects.
  • The package environment is a parent of R_GlobalEnv. This is what makes it possible to call functions in the package: first R searches in the global env, then when it doesn't find anything, it searches parent environments.
  • The parent environment of the objects is still the namespace, even when they're found in the package environment. This what makes it possible for a function to use an internal function which is present in the namespace but not in the package environment.

This is really rough and there will probably be problems unloading namespaces.

Here's the first 8 entries in the environment stack when loading ggplot2 with library() and debugging qplot():

<environment: 0x10358dd18> 
<environment: namespace:ggplot2> 
<environment: 0x103162bf8> 
<environment: namespace:base> 
<environment: R_GlobalEnv> 
<environment: package:ggplot2> 
<environment: package:stats> 
<environment: package:graphics> 

And when loading via the modified load_all():

<environment: 0x10cf86540> 
<environment: namespace:ggplot2> 
<environment: 0x105efb9c8> 
<environment: namespace:base> 
<environment: R_GlobalEnv> 
<environment: package:ggplot2> 
<environment: package:MASS> 
<environment: package:proto> 

It looks pretty much the same, although the order seems to be different for the dependency packages (those below ggplot2). I think when loading with library(), those are only visible from the namespace; they're not attached and visible from the global environment.

[hadley]

I wonder if list2env(as.list(src), dest) is a better idiom for this?

[hadley]

Also be careful about objects starting with .

[wch]

OK, got the '.' objects. The list2env isn't appropriate here, since the environment is already created by the makeNamespace function.

[hadley]

I guess we'll need some tests for this too :(

[wch]

Writing tests for this will be a challenge. What do you think about making a another package inside of devtools, specifically for running tests on? Maybe it could go in /inst. It could also be used for testing many other parts of devtools.

[wch]

The imports also need to be properly loaded. They should go in the parent environment of <namespace:ggplot2>.

For example, if you run this code when loading via library(), the

# Print parent frames, and return last one printed
# e: the environment to start in
# n: max number of levels envs to print
# p: print all the environments? If FALSE, just return last frame
printenvs <- function(e = parent.frame(), n = 100, p = TRUE) {
  if (p)  cat(str(e, give.attr=F))
  i <- 1
  while(i < n) {
    if (identical(e, emptyenv()))  break

    e <- parent.env(e)
    if (p)  cat(str(e, give.attr=F))
    i <- i+1
  }
  cat("\n")
  return(e)
}


library(ggplot2)
debug(qplot)
qplot()

# Now at browser prompt
e <- printenvs(n=3)
# <environment: 0x104153e28> 
# <environment: namespace:ggplot2> 
# <environment: 0x103162bf8> 

e
# <environment: 0x103162bf8>
# attr(,"name")
# [1] "imports:ggplot2"

ls(e)
# Shows all the objects from ggplot2 imports, like grid, scales, plyr

The imported packages are loaded into their appropriate namespaces, but the do not have the associated <package:plyr>, which is what makes the functions available in the global env.
Still at the debug prompt:

true
# function (...) 
# TRUE
# <environment: namespace:plyr>

 debug(true)
 true()

# At debug prompt for true()
# Show all parent envs
printenvs(n=5)
# <environment: 0x10360a8a8> 
# <environment: namespace:plyr> 
# <environment: 0x102f37438> 
# <environment: namespace:base> 
# <environment: R_GlobalEnv> 
# <environment: package:ggplot2> 
# <environment: package:stats> 
# <environment: package:graphics> 
# <environment: package:grDevices> 
# <environment: package:utils> 
# <environment: package:datasets> 
# <environment: package:methods> 
# <environment: 0x10093a2e8> 
# <environment: base> 
# <environment: R_EmptyEnv> 

So the imported packages are loaded into their namespace, and then their exported objects are copied over to the "imports:ggplot2" environment, which is a parent of <namespace:ggplot2>.

Also see:
http://digitheadslabnotebook.blogspot.com/2011/06/environments-in-r.html
http://obeautifulcode.com/R/How-R-Searches-And-Finds-Stuff/

[wch]

I copied this function directly from the inside of base::loadNamespace() in R 2.15.1. I don't know if it's correct for other versions of R.

[hadley]

It probably won't be - but I think that's ok.

[wch]

It now imports objects in to the imports environment (instead of loading the dependency packages with require(), which attaches the packages).

Now I'm getting a strange warning in load_code() when I run load_all('ggplot2'). (This is with options(showWarnCalls=T, showNCalls=500)):

Warning message:
In getPackageName(where) :
  Created a package name, ‘2012-08-06 13:09:56’, when none found
Calls: load_all -> load_code -> sys.source -> eval -> eval -> setRefClass -> setClass -> makeClassRepresentation -> getPackageName

This warning happens in the lapply in load_code():

  tryCatch(
    lapply(paths, sys.source, envir = env, chdir = TRUE, 
      keep.source = TRUE), 
    error = function(e) {
      clear_cache()
      stop(e)
    }

When I replace the lapply with a loop, the warning still happens:

  for (path in paths) {
    sys.source(path, envir = env, chdir = TRUE, keep.source = TRUE)
  }

But if I use a loop that uses numerical indexing, the warning doesn't happen!

  for (i in length(paths)) {
    sys.source(paths[i], envir = env, chdir = TRUE, keep.source = TRUE)
  }

[hadley]

A "dependent" package?

[wch]

Doesn't "x is dependent" mean that x depends on something else, as in "John is dependent on nicotine".

"x is a dependency of y" means that y depends on x. It's slightly awkward, but you could say, "nicotine is a dependency of John's".

[hadley]

Are suggests still loaded in here? If so, they probably shouldn't be.

[wch]

fixed.