Skip to content

[BEAM-2039] Number programming guide chapters#320

Merged
asfgit merged 2 commits intoapache:asf-sitefrom
melap:linecleanup
Sep 22, 2017
Merged

[BEAM-2039] Number programming guide chapters#320
asfgit merged 2 commits intoapache:asf-sitefrom
melap:linecleanup

Conversation

@melap
Copy link

@melap melap commented Sep 15, 2017 •
edited
Loading

Includes the following consistency/formatting changes:

  • Number chapters of programming guide (Re: BEAM-2039). I couldn't figure out how to make the table of contents use 1. 1.1. 1.1.1 etc. for each page, so I have it pulling from the section titles but it unfortunately still has the bullets. any ideas on how to do this better are welcome.
  • Make capitalization of section titles consistent
  • Format text to be more consistent, reasonable line length (no more 500+ character lines)
  • Use the consistent automatic anchor link naming based on section title instead of making our own for arbitrary sections. the auto-generated toc did not work with our hard coded anchors either.

@asfgit
Copy link

asfgit commented Sep 15, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/beam_PreCommit_Website_Stage/701/

Jenkins built the site at commit id 9c2e36b with Jekyll and staged it here. Happy reviewing.

Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again.

@asfgit
Copy link

asfgit commented Sep 15, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/beam_PreCommit_Website_Stage/702/

Jenkins built the site at commit id 7d68149 with Jekyll and staged it here. Happy reviewing.

Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again.

@melap
Copy link
Author

melap commented Sep 18, 2017

R: @kennknowles
CC: @xumingmin

tgroh
tgroh approved these changes Sep 19, 2017
View reviewed changes
Copy link
Member

@tgroh tgroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meaningful content changes are deferrable to future changes.

src/documentation/programming-guide.md Outdated
@@ -1710,21 +2501,27 @@ If our trigger is set to `.accumulatingFiredPanes`, the trigger emits the follow
```


##### Discarding Mode
##### 8.4.1.2. Discarding mode:
Copy link
Member

@tgroh tgroh Sep 19, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aaaaaaaaaaaaaaaaaaaaaa

src/documentation/programming-guide.md Outdated

##### 4.2.4.6. Combining values in a key-grouped collection
Copy link
Member

@tgroh tgroh Sep 19, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'in a keyed PCollection'?

Copy link
Author

@melap melap Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

src/documentation/programming-guide.md Outdated
* [Pre-written Beam transforms for Java]({{ site.baseurl }}/documentation/sdks/javadoc/{{ site.release_latest }}/index.html?org/apache/beam/sdk/transforms/package-summary.html)
* [Pre-written Beam transforms for Python]({{ site.baseurl }}/documentation/sdks/pydoc/{{ site.release_latest }}/apache_beam.transforms.html)

### An example of a composite transform
#### 4.6.1. Composite transform example
Copy link
Member

@tgroh tgroh Sep 19, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is too robotic and the previous is too verbose; maybe

'An example [composite ]transform'

Copy link
Author

@melap melap Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

src/documentation/programming-guide.md Outdated

When you create a pipeline, you often need to read data from some external source, such as a file in external data sink or a database. Likewise, you may want your pipeline to output its result data to a similar external data sink. Beam provides read and write transforms for a [number of common data storage types]({{site.baseurl }}/documentation/io/built-in/). If you want your pipeline to read from or write to a data storage format that isn't supported by the built-in transforms, you can [implement your own read and write transforms]({{site.baseurl }}/documentation/io/io-toc/).
When you create a new `PTransform`, be sure to read the [PTransform Style
Guide]({{ site.baseurl }}/contribute/ptransform-style-guide/). The guide
Copy link
Member

@tgroh tgroh Sep 19, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So demanding!

Maybe "The PTransform style guide contains additional information not included here such as...", "It is a useful starting point when writing new composite PTransforms"?

Copy link
Author

@melap melap Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modified

src/documentation/programming-guide.md Outdated

### Windowing functions
### 7.2. Beam windowing functions
Copy link
Member

@tgroh tgroh Sep 19, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Provided/Built-in window functions"?

Copy link
Author

@melap melap Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

src/documentation/programming-guide.md Outdated

#### Single global window
#### 7.2.4. Single global window
Copy link
Member

@tgroh tgroh Sep 19, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"The single global window"?

Copy link
Author

@melap melap Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

src/documentation/programming-guide.md
default trigger generally requires the entire data set to be available before
processing, which is not possible with continuously updating data. To perform
aggregations on an unbounded `PCollection` that uses global windowing, you
should specify a non-default trigger for that `PCollection`.
Copy link
Member

@tgroh tgroh Sep 19, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should add (at some later point) a section on a custom window fn?

Copy link
Author

@melap melap Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opened JIRA

src/documentation/programming-guide.md

### <a name="setwindowingfunction"></a>Setting your PCollection's windowing function
### 7.3. Setting your PCollection's windowing function
Copy link
Member

@tgroh tgroh Sep 19, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we condense the "How to construct a window fn" to the above section, and have this be just how to apply a WindowFn to a PCollection?

(Feel free to JIRA and do later)

Copy link
Author

@melap melap Sep 22, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opened JIRA

src/documentation/programming-guide.md Outdated
```py
# The Beam SDK for Python does not support triggers.
```

##### <a name="default-trigger"></a>**Default Trigger**
#### 8.1.1. Default trigger
Copy link
Member

@tgroh tgroh Sep 19, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default trigger

Copy link
Author

@melap melap Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

melap
melap commented Sep 22, 2017
View reviewed changes
Copy link
Author

@melap melap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated from feedback and resolved merge conflicts

src/documentation/programming-guide.md Outdated

##### 4.2.4.6. Combining values in a key-grouped collection
Copy link
Author

@melap melap Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

src/documentation/programming-guide.md Outdated
* [Pre-written Beam transforms for Java]({{ site.baseurl }}/documentation/sdks/javadoc/{{ site.release_latest }}/index.html?org/apache/beam/sdk/transforms/package-summary.html)
* [Pre-written Beam transforms for Python]({{ site.baseurl }}/documentation/sdks/pydoc/{{ site.release_latest }}/apache_beam.transforms.html)

### An example of a composite transform
#### 4.6.1. Composite transform example
Copy link
Author

@melap melap Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

src/documentation/programming-guide.md Outdated

When you create a pipeline, you often need to read data from some external source, such as a file in external data sink or a database. Likewise, you may want your pipeline to output its result data to a similar external data sink. Beam provides read and write transforms for a [number of common data storage types]({{site.baseurl }}/documentation/io/built-in/). If you want your pipeline to read from or write to a data storage format that isn't supported by the built-in transforms, you can [implement your own read and write transforms]({{site.baseurl }}/documentation/io/io-toc/).
When you create a new `PTransform`, be sure to read the [PTransform Style
Guide]({{ site.baseurl }}/contribute/ptransform-style-guide/). The guide
Copy link
Author

@melap melap Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modified

src/documentation/programming-guide.md Outdated

### Windowing functions
### 7.2. Beam windowing functions
Copy link
Author

@melap melap Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

src/documentation/programming-guide.md Outdated

#### Single global window
#### 7.2.4. Single global window
Copy link
Author

@melap melap Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

src/documentation/programming-guide.md
default trigger generally requires the entire data set to be available before
processing, which is not possible with continuously updating data. To perform
aggregations on an unbounded `PCollection` that uses global windowing, you
should specify a non-default trigger for that `PCollection`.
Copy link
Author

@melap melap Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opened JIRA

src/documentation/programming-guide.md Outdated
```py
# The Beam SDK for Python does not support triggers.
```

##### <a name="default-trigger"></a>**Default Trigger**
#### 8.1.1. Default trigger
Copy link
Author

@melap melap Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

src/documentation/programming-guide.md

### <a name="setwindowingfunction"></a>Setting your PCollection's windowing function
### 7.3. Setting your PCollection's windowing function
Copy link
Author

@melap melap Sep 22, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opened JIRA

@asfgit
Copy link

asfgit commented Sep 22, 2017

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/beam_PreCommit_Website_Stage/714/

Jenkins built the site at commit id c5d8da6 with Jekyll and staged it here. Happy reviewing.

Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again.

@tgroh
Copy link
Member

tgroh commented Sep 22, 2017

LGTM

@tgroh
Copy link
Member

tgroh commented Sep 22, 2017

@asfgit merge

asfgit pushed a commit that referenced this pull request Sep 22, 2017
This closes #320
0eb4ff4
@asfgit asfgit merged commit c5d8da6 into apache:asf-site Sep 22, 2017
@melap melap deleted the linecleanup branch September 25, 2017 21:21
robertwb pushed a commit to robertwb/incubator-beam that referenced this pull request Jun 5, 2018
This closes apache/beam-site#320
3c46531
robertwb pushed a commit to robertwb/incubator-beam that referenced this pull request Jun 5, 2018
This closes apache/beam-site#320
8b28b0f
melap pushed a commit to apache/beam that referenced this pull request Jun 20, 2018
This closes apache/beam-site#320
7393ec1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tgroh tgroh tgroh approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@melap @asfgit @tgroh