[BEAM-8287] Python 3 docs updates for 2.16.0 #9700

soyrice · 2019-10-01T16:01:29Z

Update docs for Python 3 GA in 2.16.0 SDK version

R: @tvalentyn
CC: @rosetn, @marciw, @aaltay

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Choose reviewer(s) and mention them in a comment (R: @username).
Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

Post-Commit Tests Status (on master branch)

Lang	SDK	Apex	Dataflow	Gearpump	Samza	Spark
Go		---	---	---	---
Java
Python		---		---	---
XLang	---	---	---	---	---	---

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website
Non-portable
Portable	---		---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

rosetn · 2019-10-01T18:54:59Z

Any changes needed for the Quickstart? https://beam.apache.org/get-started/quickstart-py/#check-your-python-version

soyrice · 2019-10-01T19:27:13Z

Any changes needed for the Quickstart? https://beam.apache.org/get-started/quickstart-py/#check-your-python-version

None needed - checked with Valentyn. The snippets are version-agnostic and the opening line still works, but will need to change once 2.7 is deprecated. I'll add a note about the upcoming deprecation like in the programming guide

tvalentyn · 2019-10-01T20:32:08Z

website/src/documentation/programming-guide.md

@@ -39,6 +39,9 @@ how to implement Beam concepts in your pipelines.
  </ul>
 </nav>

+{:.language-py}
+New versions of the Python SDK will only support Python 3.5 or higher. Currently, the Python SDK still supports Python 2.7.x. We recommend using the latest Python 3 version.


How about the following wording:

New releases of the Python SDK will soon require Python 3.5 or higher. We recommend that new users start using Beam with Python 3.

Or:

Currently, the Python SDK still supports Python 2.7.x, however new releases of the Python SDK will soon require Python 3.5 or higher. We recommend that new users start using Beam with Python 3.

Is there a link to a site/document/mailing list discussion we can link to related to sunsetting plan?

Is there a particular release I can mention here? Or is "soon" as specific as we can get right now?

Do you recommend this only for new users? It seems that all users would benefit from switching to Python 3

I think we can't say more specific than soon at this point.

sure, we can drop "new" - and recommend the switch to all users.

soyrice · 2019-10-01T20:43:24Z

Staging links:
http://apache-beam-website-pull-requests.storage.googleapis.com/9700/get-started/quickstart-py/index.html
http://apache-beam-website-pull-requests.storage.googleapis.com/9700/documentation/programming-guide/index.html
http://apache-beam-website-pull-requests.storage.googleapis.com/9700/roadmap/python-sdk/index.html

tvalentyn · 2019-10-01T22:28:43Z

website/src/roadmap/python-sdk.md

@@ -22,7 +22,7 @@ limitations under the License.

 ## Python 3 Support

-Apache Beam first offered Python 3.5 support with the 2.11.0 SDK release and added Python 3.6, Python 3.7 support with the 2.14.0 version. However, we continue to polish some [rough edges](https://issues.apache.org/jira/browse/BEAM-1251?focusedCommentId=16890504&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-1689050) and strengthen Beam's Python 3 offering:
+Apache Beam supports Python 3.5 or higher with the 2.16.0 SDK release. Python 3.5 beta support was first offered with the 2.11.0 SDK release; beta support for Python 3.6, Python 3.7 was added with the 2.14.0 version. We continue to polish some [rough edges](https://issues.apache.org/jira/browse/BEAM-1251?focusedCommentId=16890504&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-1689050) and strengthen Beam's Python 3 offering:


Beam does not have notions of Alpha/Beta. I suggest the following wording:

Apache Beam offers Python 3.5, 3.6, 3.7 support since Beam 2.14.0. We continue to polish remaining rough edges to improve the experience of Python 3 users and prepare for eventual sunsetting of Beam Python 2 offering:

I think as of this release we are confident enough to say:

As of Apache Beam 2.16.0, Beam supports Python 3.5 and higher. <subnote here about it has been supported for a while and for a few minor remaining issues see the BEAM-1215>

Add note about Python version support Resolve comments

aaltay · 2019-10-02T00:52:52Z

website/src/documentation/programming-guide.md

@@ -39,6 +39,9 @@ how to implement Beam concepts in your pipelines.
  </ul>
 </nav>

+{:.language-py}
+New versions of the Python SDK will only support Python 3.5 or higher. Currently, the Python SDK still supports Python 2.7.x. We recommend using the latest Python 3 version.


Is there a link to a site/document/mailing list discussion we can link to related to sunsetting plan?

aaltay · 2019-10-02T00:53:36Z

website/src/get-started/quickstart-py.md

@@ -27,6 +27,8 @@ If you're interested in contributing to the Apache Beam Python codebase, see the
 * TOC
 {:toc}

+New versions of the Python SDK will only support Python 3.5 or higher. Currently, the Python SDK still supports Python 2.7.x. We recommend using the latest Python 3 version.


Should we add a deprecation warning here. And probably to the download page and to the 2.16 blog.

/cc @markflyhigh

Is there a special format for deprecation warnings?

aaltay · 2019-10-02T00:56:03Z

website/src/roadmap/python-sdk.md

@@ -22,7 +22,7 @@ limitations under the License.

 ## Python 3 Support

-Apache Beam first offered Python 3.5 support with the 2.11.0 SDK release and added Python 3.6, Python 3.7 support with the 2.14.0 version. However, we continue to polish some [rough edges](https://issues.apache.org/jira/browse/BEAM-1251?focusedCommentId=16890504&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-1689050) and strengthen Beam's Python 3 offering:
+Apache Beam supports Python 3.5 or higher with the 2.16.0 SDK release. Python 3.5 beta support was first offered with the 2.11.0 SDK release; beta support for Python 3.6, Python 3.7 was added with the 2.14.0 version. We continue to polish some [rough edges](https://issues.apache.org/jira/browse/BEAM-1251?focusedCommentId=16890504&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-1689050) and strengthen Beam's Python 3 offering:


I think as of this release we are confident enough to say:

As of Apache Beam 2.16.0, Beam supports Python 3.5 and higher. <subnote here about it has been supported for a while and for a few minor remaining issues see the BEAM-1215>

website/src/roadmap/python-sdk.md

tvalentyn · 2019-10-02T18:31:37Z

We don't have a sunsetting doc yet, but Beam SDK itself comes with a warning that Python 2 will sunset in future releases. We can link https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E that has a discussion.

aaltay · 2019-10-03T01:03:21Z

We don't have a sunsetting doc yet, but Beam SDK itself comes with a warning that Python 2 will sunset in future releases. We can link https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E that has a discussion.

It might be good to document this plan somewhere and point users to it. A jira issue might be good, and that way users can add comments if they want to.

tvalentyn · 2019-10-09T19:31:44Z

I created https://issues.apache.org/jira/browse/BEAM-8371 to track Python 2 sunsetting. We can reference this jira in this PR.

tvalentyn · 2019-10-09T19:33:13Z

website/src/documentation/programming-guide.md

@@ -39,6 +39,9 @@ how to implement Beam concepts in your pipelines.
  </ul>
 </nav>

+{:.language-py}
+The Python SDK supports Python 2.7, 3.5, or higher; soon, new Python SDK releases won't support Python 2.7. For best results, use Beam with Python 3.


We could say:
...new Python SDK releases won't support Python 2.7 (BEAM-8371).

tvalentyn · 2019-10-09T19:33:39Z

website/src/roadmap/python-sdk.md

@@ -22,7 +22,7 @@ limitations under the License.

 ## Python 3 Support

-Apache Beam first offered Python 3.5 support with the 2.11.0 SDK release and added Python 3.6, Python 3.7 support with the 2.14.0 version. However, we continue to polish some [rough edges](https://issues.apache.org/jira/browse/BEAM-1251?focusedCommentId=16890504&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-1689050) and strengthen Beam's Python 3 offering:
+Apache Beam 2.14.0 and higher support Python 3.5, 3.6, and 3.7. We continue to [improve](https://issues.apache.org/jira/browse/BEAM-1251?focusedCommentId=16890504&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-1689050) the experience for Python 3 users and phase out Python 2 support:


We could say:
...and phase out Python 2 support (BEAM-8371).

tvalentyn

Thanks, @soyrice !

aaltay · 2019-10-09T23:13:51Z

website/src/documentation/programming-guide.md

@@ -39,6 +39,9 @@ how to implement Beam concepts in your pipelines.
  </ul>
 </nav>

+{:.language-py}
+The Python SDK supports Python 2.7, 3.5, or higher; soon, new Python SDK releases won't support Python 2.7 ([BEAM-8371](https://issues.apache.org/jira/browse/BEAM-8371)). For best results, use Beam with Python 3.


Should we spell out 3.5, or higher here similarly as 3.5, 3.6 and 3.7?

SGTM, would not hurt. We should have a conversation in Beam about definition of support for minor version. Direct runner might just work on 3.8, dataflow wouldn't. There are no 3.8 tests. Do we want to stop enthusiastic users from trying 3.8? Probably not. Do we claim to support it? Probably not.

aaltay · 2019-10-09T23:14:27Z

website/src/documentation/programming-guide.md

@@ -39,6 +39,9 @@ how to implement Beam concepts in your pipelines.
  </ul>
 </nav>

+{:.language-py}
+The Python SDK supports Python 2.7, 3.5, or higher; soon, new Python SDK releases won't support Python 2.7 ([BEAM-8371](https://issues.apache.org/jira/browse/BEAM-8371)). For best results, use Beam with Python 3.


How about, instead of new Python SDK releases won't support -> starting from 2020 newer Python SDKreleases won't support ?

Based on latest round of discussions in the mailing list, we decided that we will do it somewhere in 2020, but not necessarily in the beginning. We will revisit this conversation. For now I think we can say:

New Python SDK releases will stop supporting Python 2.7 in 2020 ([BEAM-8371]).

aaltay · 2019-10-10T00:17:11Z

Suggestions from the last round of comments LGTM.

tvalentyn · 2019-10-10T15:40:08Z

website/src/get-started/quickstart-py.md

@@ -27,6 +27,8 @@ If you're interested in contributing to the Apache Beam Python codebase, see the
 * TOC
 {:toc}

+The Python SDK supports Python 2.7, 3.5, or higher; soon, new Python SDK releases won't support Python 2.7 ([BEAM-8371](https://issues.apache.org/jira/browse/BEAM-8371)). For best results, use Beam with Python 3.


Let's make quickstart wording consistent with programming guide. Thanks.

Good catch - done.

soyrice force-pushed the python3_ga_docs_updates branch 2 times, most recently from ec0541c to 7ae4978 Compare October 1, 2019 18:34

tvalentyn reviewed Oct 1, 2019

View reviewed changes

Document Python 3 GA

9137cad

Add note about Python version support Resolve comments

soyrice force-pushed the python3_ga_docs_updates branch from b5b3e15 to 9137cad Compare October 2, 2019 16:40

soyrice marked this pull request as ready for review October 2, 2019 17:33

aaltay reviewed Oct 2, 2019

View reviewed changes

tvalentyn changed the title ~~[BEAM-8287] Python3 GA docs updates~~ [BEAM-8287] Python 3 docs updates for 2.16.0 Oct 2, 2019

tvalentyn reviewed Oct 9, 2019

View reviewed changes

Cyrus Maden and others added 3 commits October 9, 2019 15:47

Link to Python 2 deprecation tracking issue

9fc0dcc

Link to Python 2 deprecation tracking issue

1b77c05

Link to Python 2 deprecation tracking issue

75bac98

tvalentyn approved these changes Oct 9, 2019

View reviewed changes

aaltay approved these changes Oct 9, 2019

View reviewed changes

Note Python 2 deprecation in programming guide

a1ac340

tvalentyn reviewed Oct 10, 2019

View reviewed changes

Update quickstart to match

29350f6

tvalentyn merged commit 71a4690 into apache:master Oct 10, 2019

robinyqiu pushed a commit to robinyqiu/beam that referenced this pull request Oct 14, 2019

[BEAM-8287] Python 3 docs updates for 2.16.0 (apache#9700)

3be76a5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BEAM-8287] Python 3 docs updates for 2.16.0 #9700

[BEAM-8287] Python 3 docs updates for 2.16.0 #9700

soyrice commented Oct 1, 2019 •

edited

Loading

rosetn commented Oct 1, 2019

soyrice commented Oct 1, 2019

tvalentyn Oct 1, 2019

aaltay Oct 2, 2019

soyrice Oct 2, 2019

soyrice Oct 2, 2019

tvalentyn Oct 2, 2019

soyrice commented Oct 1, 2019

tvalentyn Oct 1, 2019

aaltay Oct 2, 2019

soyrice Oct 2, 2019

aaltay Oct 2, 2019

aaltay Oct 2, 2019

soyrice Oct 3, 2019

aaltay Oct 2, 2019

tvalentyn commented Oct 2, 2019 •

edited

Loading

aaltay commented Oct 3, 2019

tvalentyn commented Oct 9, 2019

tvalentyn Oct 9, 2019

tvalentyn Oct 9, 2019

tvalentyn left a comment

aaltay Oct 9, 2019

tvalentyn Oct 10, 2019

soyrice Oct 10, 2019

aaltay Oct 9, 2019

tvalentyn Oct 10, 2019

soyrice Oct 10, 2019

aaltay commented Oct 10, 2019

tvalentyn Oct 10, 2019

soyrice Oct 10, 2019

[BEAM-8287] Python 3 docs updates for 2.16.0 #9700

[BEAM-8287] Python 3 docs updates for 2.16.0 #9700

Conversation

soyrice commented Oct 1, 2019 • edited Loading

Post-Commit Tests Status (on master branch)

Pre-Commit Tests Status (on master branch)

rosetn commented Oct 1, 2019

soyrice commented Oct 1, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

soyrice commented Oct 1, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tvalentyn commented Oct 2, 2019 • edited Loading

aaltay commented Oct 3, 2019

tvalentyn commented Oct 9, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tvalentyn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaltay commented Oct 10, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

soyrice commented Oct 1, 2019 •

edited

Loading

tvalentyn commented Oct 2, 2019 •

edited

Loading