Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error in construct.networks.from.list for openssl function networks #3

Closed
bockthom opened this issue Feb 21, 2017 · 3 comments
Closed
Assignees
Labels

Comments

@bockthom
Copy link
Collaborator

After generating openssl function networks (threemonth) for more than 20 hours (using 2 cores), an error appeared:

Error in sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE)) : 
  long vectors not supported yet: fork.c:376
Calls: <Anonymous> ... construct.dependency.network.from.list -> mclapply -> lapply -> FUN -> sendMaster
Error: All inputs to rbind.fill must be data.frames
In addition: Warning message:
In mclapply(list, function(set) { :
  scheduled core 1 encountered error in user code, all values of the job will be affected
Execution halted

Unfortunately, the error message is not that concrete since more than one core was used.

@bockthom
Copy link
Collaborator Author

To be more concrete, the following function of CodefaceProjectData was initially called (which in turn calls the function stated in the initial comment):
get.author.network(relation=AUTHOR.RELATION, directed=TRUE, simple.network=FALSE)

@bockthom
Copy link
Collaborator Author

The described error seems to be a problem with parallelization in mclapply: It also appeared in the busybox function threemonth networks.
Since I have already run that configuration before parallelization and without parallelization no errors have appeared, I guess that the parallelization itself is the problem here...

Perhaps parallelization within already parallelized code may be the problem. If that is not the case, I am not sure where the problem can arise from...

@clhunsen
Copy link
Collaborator

As of commit 930af63, the problematic parallelization is removed and the resulting issue is resolved. The underlying problem of parallelization is to be solved in issue #4.

clhunsen added a commit that referenced this issue Feb 27, 2017
The argument 'filter.base.artifact' passed to the method
'get.author.network.cochange' has not been passed on to the method
'get.artifact2author'. As the argument was set to FALSE instead, this
resulted in not removing the base artifact from the commit list. By
passing on the argument, the argument gets respected now and the method
does not lead to unexpected results.

A long-term consequence of this error was the introduction of too many
edges in the function 'construct.dependency.network.from.list', which is
the explanation for the run-time behavior reported in issue #3.
Basically, the plain re-introduction of parallelization (issue #4)
should be possible now.

Signed-off-by: Claus Hunsen <hunsen@fim.uni-passau.de>
clhunsen added a commit that referenced this issue Feb 27, 2017
This reverts commit 930af63. See also
issues #3 and #4 as well as commits
efcd616 and
91cb0aa.

In combination with commit efcd616,
this resolves #4.

For the following code snippet, the network-construction time gets
reduced from 198 seconds to 11 seconds.
> CF.SELECTION.PROCESS = "threemonth"
> CASESTUDY = "qemu"
> ARTIFACT = "file"
> AUTHOR.RELATION = "cochange"
> ARTIFACT.RELATION = "cochange"
> conf = CodefaceConf$new(CF.DATA, CF.SELECTION.PROCESS, CASESTUDY, ARTIFACT)
> project = CodefaceProjectData$new(conf)
> project$get.author.network(relation = AUTHOR.RELATION, directed = TRUE, simple.network = FALSE)

Signed-off-by: Claus Hunsen <hunsen@fim.uni-passau.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants