Error: could not find function "from.dfs" #131

Closed
pietheinstrengholt opened this Issue Sep 14, 2012 · 2 comments

Projects

None yet

2 participants

@pietheinstrengholt

I've been trying to use R, rmr and hadoop. Hadoop is configured fine. I'm able to use HDFS and MapReduce for other cases. For the R testcase I'm using the deptdelay from:

https://github.com/jseidman/hadoop-R

When I try to do so R quits with the error messages: Error: could not find function "from.dfs"

./deptdelay-rmr12.R
Loading required package: methods
Loading required package: rJava

HADOOP_CMD=/usr/local/hadoop/bin/hadoop

Be sure to run hdfs.init()
Loading required package: Rcpp
Loading required package: RJSONIO
Loading required package: itertools
Loading required package: iterators
Loading required package: digest
Error: could not find function "from.dfs"
Execution halted
hduser@node1:~/TextMining$

Do you have any idea what might be wrong?

@piccolbo
Collaborator

Hi,
first I have to warn you that Jonathan has not committed to keeping those
examples up to date forever, so you may be better off running one of the
many tests distributed with the source code of the package. Second, I am
really puzzled by this. It looks like it is loading all the dependencies
for rmr and rmr itself successfully but then something detaches the package
right away because the function from.dfs exists and is exported. I would
try to simplify things by not loading any other packages such as rhdfs,
which rmr does not depend on, but this is just a general heuristic, I don't
suspect any specific interactions. I would try and ssh into a few random
nodes and do a library(rmr); from.dfs()
It should fail like this

from.dfs()
Error in is.character(input) : 'input' is missing
Not like this
dfdif()
Error: could not find function "dfdif"
If that works then we have to focus on envir differences between R as run
by hadoop and R as run by a user. If it doesn't I don't know what to think.

Antonio
On Fri, Sep 14, 2012 at 3:53 AM, Piethein Strengholt <
notifications@github.com> wrote:

I've been trying to use R, rmr and hadoop. Hadoop is configured fine. I'm
able to use HDFS and MapReduce for other cases. For the R testcase I'm
using the deptdelay from:

https://github.com/jseidman/hadoop-R

When I try to do so R quits with the error messages: Error: could not find
function "from.dfs"

./deptdelay-rmr12.R
Loading required package: methods
Loading required package: rJava

HADOOP_CMD=/usr/local/hadoop/bin/hadoop

Be sure to run hdfs.init()
Loading required package: Rcpp
Loading required package: RJSONIO
Loading required package: itertools
Loading required package: iterators
Loading required package: digest
Error: could not find function "from.dfs"
Execution halted
hduser@node1:~/TextMining$

Do you have any idea what might be wrong?


Reply to this email directly or view it on GitHubhttps://github.com/RevolutionAnalytics/RHadoop/issues/131.

@piccolbo
Collaborator

This has gone silent, I am going to close it provisionally, feel free to reopen as needed

@piccolbo piccolbo closed this Oct 25, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment