Skip to content

Commit

Permalink
Fixd for chaining doc
Browse files Browse the repository at this point in the history
  • Loading branch information
Aklakan committed Oct 8, 2021
1 parent 60581c8 commit 6b02736
Showing 1 changed file with 3 additions and 6 deletions.
9 changes: 3 additions & 6 deletions docs/chaining/chaining.md
Expand Up @@ -8,7 +8,8 @@ While it is easy to create individual (static) functions that transform one rdd
chaining such functions directly is cumbersome to use:

```java
# Hard to read because transformations appear in reverse order of the application
# Hard to read because transformations appear in reverse order of their application
JavaRDD<I> input = ...;
JavaRDD<O> output = fn3.apply(fn2.apply(fn1.apply(input)));
```

Expand All @@ -29,7 +30,6 @@ would allow more naturally placing the input before the transform:
JavaRDD<Resource> output = input.compose(compositeTransform); # NOT POSSIBLE
```


Spark itself unfortunately does not intrisically provide this mechanism, but SANSA provides a little framework
aimed to easy the creation of such transformation chains.

Expand Down Expand Up @@ -59,7 +59,7 @@ JavaRDD<Resource> output = compositeTransform.apply(input);
An further advantage of such transformation functions is that serialization can be tackled inside of the
transformer:

```
```java
public static JavaRddFunction<String, String> myTransform(ParamA inputArgA, ParamB inputArgB) {
return rdd -> {
# Make the arguments serializable; either using lambda serialization or broadcasts
Expand All @@ -73,9 +73,6 @@ public static JavaRddFunction<String, String> myTransform(ParamA inputArgA, Para
});
}
}
```


Expand Down

0 comments on commit 6b02736

Please sign in to comment.