Initial commit (#1)

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,12 @@
+**Please** add a meaningful description for your change here
+
+------------------------
+
+Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
+
+ - [ ] I agree that my contributions are licensed with both [Apache ASL2](../../LICENSE-APACHE) and [MIT](../../LICENSE-MIT).
+ - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
+ - [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
+ - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
+
+See the [`CONTRIBUTING.md`](../../CONTRIBUTING.md) guide for more information.

.github/workflows/test.yaml

@@ -0,0 +1,30 @@
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+name: Test
+
+on: [push]
+
+jobs:
+  tests:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.7", "3.8", "3.9"]
+    steps:
+    - uses: actions/checkout@v3
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v3
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install local package
+      run: |
+        pip install -U pip
+        pip install -e .
+    - name: Run tests
+      run: python -m unittest -v

.gitignore

@@ -0,0 +1,40 @@
+# IDEs
+.vscode/
+.idea/
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# Virtual environments.
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/

CONTRIBUTING.md

@@ -0,0 +1,69 @@
+# Contributing
+
+🎉🎊 Thanks for taking the time to contribute! 🎉🎊
+
+There are many ways to contribute, here are some.
+
+## Filing an issue
+
+If there's any issue you encounter or anything that needs to be fixed, feel free to [create a GitHub issue](https://github.com/apache/beam-starter-python/issues).
+
+## Contributing to this starter project
+
+If this is your first time contributing to a GitHub repo,
+we recommmend going through the
+[GitHub quickstart](https://docs.github.com/en/get-started/quickstart/hello-world).
+
+It's a good idea to discuss your plans with the Beam community through the dev@beam.apache.org mailing list before doing any changes.
+
+Here's a small overview of the process.
+
+1. [Fork the repo](https://docs.github.com/en/get-started/quickstart/fork-a-repo).
+
+1. Clone the repo.
+
+    ```sh
+    export GITHUB_USERNAME="my-github-username"
+
+    git clone git@github.com:$GITHUB_USERNAME/beam-starter-python.git
+    ```
+
+1. Set the [upstream remote branch](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/configuring-a-remote-for-a-fork).
+
+    ```sh
+    cd beam-starter-python
+    git remote add upstream git@github.com:apache/beam-starter-python.git
+    ```
+
+1. Create and change to a new branch.
+
+    ```sh
+    git checkout -B my-branch-name
+    ```
+
+1. Modify the code! 😱
+
+1. Run the tests. For steps on how to run them see the [`README.md`](README.md).
+
+1. Commit and push your changes to your branch in `origin`.
+
+    ```sh
+    git commit -m "one line description of your changes"
+    git push
+    ```
+
+1. [Create a Pull Request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request).
+
+1. Add reviewers, and [address review comments](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests).
+
+1. Once it's approved, we can merge the Pull Request.
+
+For more information about proposing changes to a GitHub repository, see the
+[Propose changes](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-branches)
+page in the GitHub docs.
+
+## Contributing to Apache Beam
+
+For information on how to contribute to
+[Apache Beam](https://github.com/apache/beam), see the
+[Contribution guide](https://beam.apache.org/contribute/).

LICENSE

@@ -1,3 +1,38 @@
+This software is distributed under the terms of both the MIT license and the
+Apache License (Version 2.0).
+
+
+MIT license
+
+Copyright 2022 Google LLC
+
+Permission is hereby granted, free of charge, to any
+person obtaining a copy of this software and associated
+documentation files (the "Software"), to deal in the
+Software without restriction, including without
+limitation the rights to use, copy, modify, merge,
+publish, distribute, sublicense, and/or sell copies of
+the Software, and to permit persons to whom the Software
+is furnished to do so, subject to the following
+conditions:
+
+The above copyright notice and this permission notice
+shall be included in all copies or substantial portions
+of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
+ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
+TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
+PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
+IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+DEALINGS IN THE SOFTWARE.
+
+
+Apache 2 license
+
                                  Apache License
                            Version 2.0, January 2004
                         http://www.apache.org/licenses/
@@ -198,4 +233,4 @@
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
-   limitations under the License.
+   limitations under the License.

MANIFEST.in

@@ -0,0 +1 @@
+include requirements.txt

NOTICE

@@ -0,0 +1,5 @@
+Apache Beam
+Copyright [2022-] The Apache Software Foundation
+
+This product includes software developed at
+The Apache Software Foundation (http://www.apache.org/).

README.md

@@ -0,0 +1,78 @@
+# Apache Beam starter for Python
+
+If you want to clone this repository to start your own project,
+you can choose the license you prefer and feel free to delete anything related to the license you are dropping.
+
+## Before you begin
+
+Make sure you have a [Python 3](https://www.python.org/) development environment ready.
+If you don't, you can download and install it from the
+[Python downloads page](https://www.python.org/downloads/).
+
+We recommend using a virtual environment to isolate your project's dependencies.
+
+```sh
+# Create a new Python virtual environment.
+python -m venv env
+
+# Activate the virtual environment.
+source env/bin/activate
+```
+
+While activated, your `python` and `pip` commands will point to the virtual environment,
+so any changes or install dependencies are self-contained.
+
+As a one time setup, let's install the project's dependencies from the [`requirements.txt`](requirements.txt) file.
+
+```py
+# It's always a good idea to update pip before installing dependencies.
+pip install -U pip
+
+# Install the project as a local package, this installs all the dependencies as well.
+pip install -e .
+```
+
+> ℹ️ Once you are done, you can run the `deactivate` command to go back to your global Python installation.
+
+### Running the pipeline
+
+Running your pipeline in Python is as easy as running the script file directly.
+
+```sh
+# You can run the script file directly.
+python main.py
+
+# To run passing command line arguments.
+python main.py --input-text="🎉"
+
+# To run the tests.
+python -m unittest -v
+```
+
+## GitHub Actions automated testing
+
+This project already comes with automated testing via [GitHub Actions](https://github.com/features/actions).
+
+To configure it, look at the [`.github/workflows/test.yaml`](.github/workflows/test.yaml) file.
+
+## Using other runners
+
+To keep this template small, it only includes the [Direct Runner](https://beam.apache.org/documentation/runners/direct/).
+
+For a comparison of what each runner currently supports, look at the [Beam Capability Matrix](https://beam.apache.org/documentation/runners/capability-matrix/).
+
+To add a new runner, visit the runner's page for instructions on how to include it.
+
+## Contributing
+
+Thank you for your interest in contributing!
+All contributions are welcome! 🎉🎊
+
+Please refer to the [`CONTRIBUTING.md`](CONTRIBUTING.md) file for more information.
+
+# License
+
+This software is distributed under the terms of both the MIT license and the
+Apache License (Version 2.0).
+
+See [LICENSE](LICENSE) for details.

main.py

@@ -0,0 +1,32 @@
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+from apache_beam.options.pipeline_options import PipelineOptions
+
+from my_app import app
+
+
+if __name__ == "__main__":
+    import argparse
+    import logging
+
+    logging.getLogger().setLevel(logging.INFO)
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "--input-text",
+        default="Default input text",
+        help="Input text to print.",
+    )
+    args, beam_args = parser.parse_known_args()
+
+    beam_options = PipelineOptions(save_main_session=True, setup_file="./setup.py")
+    app.run(
+        input_text=args.input_text,
+        beam_options=beam_options,
+    )

my_app/__init__.py

@@ -0,0 +1,9 @@
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+from .app import *

my_app/app.py

@@ -0,0 +1,27 @@
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+from typing import Callable, Optional
+import apache_beam as beam
+from apache_beam.options.pipeline_options import PipelineOptions
+
+
+def run(
+    input_text: str,
+    beam_options: Optional[PipelineOptions] = None,
+    test: Callable[[beam.PCollection], None] = lambda _: None,
+) -> None:
+    with beam.Pipeline(options=beam_options) as pipeline:
+        elements = (
+            pipeline
+            | "Create elements" >> beam.Create(["Hello", "World!", input_text])
+            | "Print elements" >> beam.Map(print)
+        )
+
+        # Used for testing only.
+        test(elements)

requirements.txt

@@ -0,0 +1 @@
+apache-beam==2.39.0

setup.py

@@ -0,0 +1,22 @@
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+from setuptools import setup, find_packages
+
+with open("requirements.txt") as f:
+    requirements = f.readlines()
+
+setup(
+    name="My app",
+    version="1.0",
+    description="Python Apache Beam pipeline.",
+    author="My name",
+    author_email="my@email.com",
+    packages=find_packages(),
+    install_requires=requirements,
+)

test/__init__.py

@@ -0,0 +1,7 @@
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.