Skip to content

Commit

Permalink
Something happened to the compat doc that shouldn't have, putting bac…
Browse files Browse the repository at this point in the history
…k in
  • Loading branch information
piccolbo committed Aug 29, 2012
1 parent b807c3a commit 8bd1e7c
Show file tree
Hide file tree
Showing 4 changed files with 185 additions and 6 deletions.
14 changes: 14 additions & 0 deletions rmr/docs/compatibility.Rmd
@@ -0,0 +1,14 @@
# Compatibility testing for rmr 1.3.x
Please contribute with additional reports. To claim compatibility you need to run `R CMD check path-to-rmr` successfully.
As with any new release, testing on additional platforms is under way. If you build your own Hadoop, see [Which Hadoop for rmr](https://github.com/RevolutionAnalytics/RHadoop/wiki/Which-Hadoop-for-rmr).

<table>
<thead>
<tr><th>rmr</th><th>Hadoop</th><th>R</th><th>OS</th><th>Compatibility</th><th>Reporter</th></tr>
</thead>
<tbody>
<tr><td>1.3.1</td><td>mr1-cdh4.0.0</td><td>Revolution R Enterprise 6.0</td><td>64-bit CentOS 5.6</td><td>only x86_64 and mr1</td><td>Revolution Analytics</td></tr>
<tr><td>1.3.1</td><td>CDH3u4</td><td>Revolution R Enterprise 6.0</td><td>64-bit CentOS 5.6</td><td>only x86_64</td><td>Revolution Analytics</td></tr>
<tr><td>1.3.1</td><td>Apache Hadoop 1.0.2</td><td>Revolution R Enterprise 6.0</td><td>64-bit CentOS 5.6</td><td>only x86_64</td><td>Revolution Analytics</td></tr>
</tbody>
</table>
165 changes: 165 additions & 0 deletions rmr/docs/compatibility.html
@@ -0,0 +1,165 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<!-- saved from url=(0014)about:internet -->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">

<title>Compatibility testing for rmr 1.3.x (Current stable)</title>

<base target="_blank"/>

<style type="text/css">
body, td {
font-family: sans-serif;
background-color: white;
font-size: 12px;
margin: 8px;
}

tt, code, pre {
font-family: 'DejaVu Sans Mono', 'Droid Sans Mono', 'Lucida Console', Consolas, Monaco, monospace;
}

h1 {
font-size:2.2em;
}

h2 {
font-size:1.8em;
}

h3 {
font-size:1.4em;
}

h4 {
font-size:1.0em;
}

h5 {
font-size:0.9em;
}

h6 {
font-size:0.8em;
}

a:visited {
color: rgb(50%, 0%, 50%);
}

pre {
margin-top: 0;
max-width: 95%;
border: 1px solid #ccc;
}

pre code {
display: block; padding: 0.5em;
}

code.r {
background-color: #F8F8F8;
}

table, td, th {
border: none;
}

blockquote {
color:#666666;
margin:0;
padding-left: 1em;
border-left: 0.5em #EEE solid;
}

hr {
height: 0px;
border-bottom: none;
border-top-width: thin;
border-top-style: dotted;
border-top-color: #999999;
}

@media print {
* {
background: transparent !important;
color: black !important;
filter:none !important;
-ms-filter: none !important;
}

body {
font-size:12pt;
max-width:100%;
}

a, a:visited {
text-decoration: underline;
}

hr {
visibility: hidden;
page-break-before: always;
}

pre, blockquote {
padding-right: 1em;
page-break-inside: avoid;
}

tr, img {
page-break-inside: avoid;
}

img {
max-width: 100% !important;
}

@page :left {
margin: 15mm 20mm 15mm 10mm;
}

@page :right {
margin: 15mm 10mm 15mm 20mm;
}

p, h2, h3 {
orphans: 3; widows: 3;
}

h2, h3 {
page-break-after: avoid;
}
}

</style>





</head>

<body>
<h1>Compatibility testing for rmr 1.3.x (Current stable)</h1>

<p>Please contribute with additional reports. To claim compatibility you need to run <code>R CMD check path-to-rmr</code> successfully.<br/>
As with any new release, testing on additional platforms is under way. If you build your own Hadoop, see <a href="https://github.com/RevolutionAnalytics/RHadoop/wiki/Which-Hadoop-for-rmr">Which Hadoop for rmr</a>.</p>

<table>
<thead>
<tr><th>rmr</th><th>Hadoop</th><th>R</th><th>OS</th><th>Compatibility</th><th>Reporter</th></tr>
</thead>
<tbody>
<tr><td>1.3.1</td><td>mr1-cdh4.0.0</td><td>Revolution R Enterprise 6.0</td><td>64-bit CentOS 5.6</td><td>only x86_64 and mr1</td><td>Revolution Analytics</td></tr>
<tr><td>1.3.1</td><td>CDH3u4</td><td>Revolution R Enterprise 6.0</td><td>64-bit CentOS 5.6</td><td>only x86_64</td><td>Revolution Analytics</td></tr>
<tr><td>1.3.1</td><td>Apache Hadoop 1.0.2</td><td>Revolution R Enterprise 6.0</td><td>64-bit CentOS 5.6</td><td>only x86_64</td><td>Revolution Analytics</td></tr>
</tbody>
</table>

</body>

</html>

Binary file added rmr/docs/compatibility.pdf
Binary file not shown.
12 changes: 6 additions & 6 deletions rmr/docs/tutorial.Rmd
@@ -1,9 +1,9 @@

`r read_chunk('../tests/basic-examples.R')`
`r read_chunk('../tests/wordcount.R')`
`r read_chunk('../tests/logistic-regression.R')`
`r read_chunk('../tests/linear-least-squares.R')`
`r read_chunk('../tests/kmeans.R')`
`r read_chunk('../pkg/tests/basic-examples.R')`
`r read_chunk('../pkg/tests/wordcount.R')`
`r read_chunk('../pkg/tests/logistic-regression.R')`
`r read_chunk('../pkg/tests/linear-least-squares.R')`
`r read_chunk('../pkg/tests/kmeans.R')`
`r opts_chunk$set(echo=TRUE, eval=FALSE, cache=FALSE, tidy=FALSE)`

# Mapreduce in R
Expand Down Expand Up @@ -34,7 +34,7 @@ function, which we are not using here, is a regular R function with a few constr
1. It returns a key value pair as returned by the helper function `keyval`, which takes any two R objects as arguments; you can also return a list of such objects, or `NULL`.

In this example, we are not using the key at all, only the value, but we still need both to support the general mapreduce case.
The return value is an object, and you can pass it as input to other jobs or read it into memory (watch out, not good for big data) with `from.dfs`. `from.dfs` is complementary `to.dfs`. It returns a list of key-value pairs, which is the most general data type that mapreduce can handle. If you prefer data frames to lists, you can instruct `from.dfs` to perform a conversion to data frames, which will cover many many important use cases but is not as general as a list of pairs (structured vs. unstructured case). `from.dfs` is useful in defining map reduce algorithms whenever a mapreduce job produces something of reasonable size, like a summary, that can fit in memory and needs to be inspected to decide on the next steps, or to visualize it.
The return value is an object, and you can pass it as input to other jobs or read it into memory (watch out, not good for big data) with `from.dfs`. `from.dfs` is complementary `to.dfs`. It returns a list of key-value pairs, which is the most general data type that mapreduce can handle. `from.dfs` is useful in defining map reduce algorithms whenever a mapreduce job produces something of reasonable size, like a summary, that can fit in memory and needs to be inspected to decide on the next steps, or to visualize it.

## My second mapreduce job

Expand Down

0 comments on commit 8bd1e7c

Please sign in to comment.