Update main page and Beam Overview page #93
Conversation
hadarhg
commented
Nov 22, 2016
hadarhg
commented
- Move /src/index.md content to /src/get-started/beam-overview.md
- Incorporate some of the content in v2 main page (/src/_layouts/v2home.html) into /src/index.md
|
Refer to this link for build results (access rights to CI server needed): |
|
R: @davorbonaci |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id b3ada9c with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
| # Apache Beam Overview | ||
|
|
||
| Apache Beam is an open source, unified programming model that you can use to create a data processing **pipeline**. You start by building a program that defines the pipeline using one of the open source Beam SDKs. The pipeline is then executed by one of Beam's supported **distributed processing back-ends**, which include [Apache Flink](http://flink.apache.org), [Apache Spark](http://spark.apache.org), and [Google Cloud Dataflow](https://cloud.google.com/dataflow). | ||
| Apache Beam is an open source, unified programming model that you can use to create a data processing **pipeline**. You start by building a program that defines the pipeline using one of the open source Beam SDKs. The pipeline is then executed by one of Beam's supported **distributed processing back-ends**, which include [Apache Apex](http://apex.apache.org), [Apache Flink](http://flink.apache.org), [Apache Spark](http://spark.apache.org), and [Google Cloud Dataflow](https://cloud.google.com/dataflow). | ||
|
|
||
| Beam is particularly useful for [Embarrassingly Parallel](http://en.wikipedia.org/wiki/Embarassingly_parallel) data processing tasks, in which the problem can be decomposed into many smaller bundles of data that can be processed independently and in parallel. You can also use Beam for Extract, Transform, and Load (ETL) tasks and pure data integration. These tasks are useful for moving data between different storage media and data sources, transforming data into a more desirable format, or loading data onto a new system. | ||
|
|
There was a problem hiding this comment.
I think there are more changes here, e.g., Apex to the following table, etc.
| @@ -63,3 +64,16 @@ Beam currently supports Runners that work with the following distributed process | |||
| </table> | |||
There was a problem hiding this comment.
order: apex, flink, spark, dataflow.
|
|
||
| Interested in working with Apache Beam? Great! Here's how to get started: | ||
|
|
||
| * If you are interested in using Beam for your data processing tasks, use the [Get Started]({{ site.baseurl }}/get-started/) section for an overview, quickstart, and examples. Then dive into the [Documentation]({{ site.baseurl }}/documentation/) section to get up to speed on the details. |
There was a problem hiding this comment.
This link is to itself, no? We should remove.
| * If you're interested in contributing to the Beam SDKs or Runners, start with the [Contribute]({{ site.baseurl }}/contribute/) section. | ||
|
|
||
|
|
||
| Apache Beam is an effort undergoing incubation at [The Apache Software Foundation (ASF)](http://www.apache.org) sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. |
| The Beam Pipeline Runners translate the data processing pipeline you define with your Beam program into the API compatible with the distributed processing back-end of your choice. When you run your Beam program, you'll need to specify the appropriate runner for the back-end where you want to execute your pipeline. | ||
|
|
||
| Beam currently supports Runners that work with the following distributed processing back-ends: | ||
| Apache Beam is a unified programming model that allows you to implement batch and streaming data processing jobs that can run on any execution engine. |
There was a problem hiding this comment.
multiple spaces between than and can
| <td>In Development</td> | ||
| </tr> | ||
| </table> | ||
| You start by building a program that defines a pipeline using one of the open source Beam SDKs. The pipeline is then executed by one of Beam’s supported distributed processing back-ends, which include Apache Flink, Apache Spark, and Google Cloud Dataflow. |
|
|
||
| **Note:** You can always execute your pipeline locally for testing and debugging purposes. | ||
| ## Contribute |
There was a problem hiding this comment.
- 3 points: unified, portable, extensible
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id afb12c6 with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
|
||
| * If you are interested in using Beam for your data processing tasks, get started by following the [Quickstart]({{ site.baseurl }}/get-started/quickstart) and the [WordCount Examples Walkthrough]({{ site.baseurl }}/get-started/wordcount-example). Then dive into the [Documentation]({{ site.baseurl }}/documentation/) section to get up to speed on the details. | ||
| * If you're interested in contributing to the Beam SDKs or Runners, start with the [Contribute]({{ site.baseurl }}/contribute/) section. | ||
|
|
There was a problem hiding this comment.
the following two sections can be cut -- this should be on the landing page, no need right here
|
|
||
| **Note:** You can always execute your pipeline locally for testing and debugging purposes. | ||
| ## Contribute |
There was a problem hiding this comment.
swap Get Started and Contribute?
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id 1b0cc2b with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id 60bc0f0 with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id ac752ed with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
LGTM. Merging. |
