to.dfs permission denied problem #125

Closed
WinstonWang opened this Issue Aug 20, 2012 · 4 comments

Comments

Projects
None yet
3 participants

To whom it may concern

I'm using the following example to test RHadoop,
This example is generate a list of values and count their occurences.

groups = rbinom(100, n = 500, prob = 0.5)
tapply(groups, groups, length)

require('rmr')
groups = rbinom(100, n = 500, prob = 0.5)
groups = to.dfs(groups)
result = mapreduce(
input = groups,
map = function(k,v) keyval(v, 1),
reduce = function(k,vv) keyval(k, length(vv)))

print(result())

print(from.dfs(result, to.data.frame=T))

But the following is what I got:

groups = rbinom(100, n = 500, prob = 0.5)
tapply(groups, groups, length)
33 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59
1 1 2 2 1 6 5 7 14 27 17 28 27 17 38 37 38 40 34 41 35 24 16 15 8 8
60 61 62 64
3 1 4 3
require('rmr')
Loading required package: rmr
Loading required package: Rcpp
Loading required package: RJSONIO
Loading required package: itertools
Loading required package: iterators
Loading required package: digest
groups = rbinom(100, n = 500, prob = 0.5)
groups = to.dfs(groups)
sh: 1: /home/ecrdc7/hadoop: Permission denied

Any help is appreciate

Collaborator

piccolbo commented Aug 20, 2012

rmr needs access to the hadoop executable. That's why you need to set the HADOOP_CMD variable. I assume you set that variable to /home/ecrdc7/hadoop. Now what is the point of having the path to a command if you can't execute it? I am making educated guesses here. To know for sure you should check the contents of the HADOOP_CMD variable and do a ls -l on /home/ecrdc7/hadoop

Collaborator

piccolbo commented Oct 30, 2012

I assume the problem was solved, so that we don't have too many dormant open cases. If the assumption is wrong, please reopen it with follow up info.

@piccolbo piccolbo closed this Oct 30, 2012

B1ANCHi commented May 29, 2013

I am facing exctly the same problem as described by @WinstonWang :

library(rmr2)
Loading required package: Rcpp
Loading required package: RJSONIO
Loading required package: digest
Loading required package: functional
Loading required package: stringr
Loading required package: plyr
Loading required package: reshape2
small.ints = to.dfs(1:1000)
sh: 1: /usr/local/hadoop: Permission denied
Warning message:
In to.dfs(1:1000) : Converting to.dfs argument to keyval with a NULL key

HADOOP_CMD is set to /usr/local/hadoop. The directory should be accessible, I even used chmod -R 777...

R is selfcompiled v3.0.1 and stored in /usr/local/R.

Are there any good ideas or can anyone give advice what I can try next?

Collaborator

piccolbo commented May 29, 2013

Please open a separate issue in the rmr issue tracker. We switched to independent repos for each package some time ago. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment