[Beam-3208] Add Distinct PTransform to mirror Java SDK

[ttanay]

The Java SDK introduced a Distinct PTransform to replace the RemoveDuplicates PTransform.
But, the Python SDK still had the RemoveDuplicates PTransform.
A new Distinct PTransform was added which does the same thing as RemoveDuplicates. RemoveDuplicates is now an alias to the Distinct PTransform and is deprecated.

JIRA Issue: https://issues.apache.org/jira/browse/BEAM-3208

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• If this contribution is large, please file an Apache Individual Contributor License Agreement. (N/A)

Post-Commit Tests Status (on master branch)

Lang	SDK	Apex	Dataflow	Flink	Gearpump	Samza	Spark
Go		---	---	---	---	---	---
Java
Python		---			---	---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

[ttanay]

Run Portable_Python PreCommit

[ttanay]

Run RAT PreCommit

[ttanay]

Run Portable_Python PreCommit

[aaltay]

@ttanay is this ready for review? (There is still WIP in the title.)

[ttanay]

Thanks for checking this out @aaltay.
I realized there is a better way to do the same.
Hence, switched to WIP.
Need to make that change. Will let you know post that. :)

[ttanay]

Run Python_pytest PreCommit

[ttanay]

Python_pytest is failing with message:

Task 'pytest' not found in root project 'beam'. Some candidates are: 'goTest', 'test'.

Did something change?

[pabloem]

Hi Tanay. The pytest thing showed up only today. I don't know where it came from, but it should be safe to ignore for now.

[aaltay]

@udim might have an idea about the pytest.

[udim]

Sorry, that is a test job of mine. It shouldn't be running automatically. 🤷‍♂️

[ttanay]

Hi Tanay. The pytest thing showed up only today. I don't know where it came from, but it should be safe to ignore for now.

Cool. 🚀

@aaltay @pabloem This is ready for review now.

[pabloem]

Hello Tanay. Happy to work on this PR together.

I like how you used the appropriate Beam deprecation marker for the old transform. That's great.

I believe that this new utility that you're adding is complex, and will be easy to misuse. Furthermore, the transform is quite simple. Would you consider using a different method to do this? (e.g. subclassing distinct, or having removeduplicates expand into distinct?) - we'd love to add the least complexity possible.

If you're in the ASF slack, feel free to ping me directly to chat.

[ttanay]

Hey @pabloem
I made the changes.
Thank you for helping out!

[pabloem]

Hi @ttanay - this is great. Would you please squash the commits so we can merge? - This LGTM

[ttanay]

Hi @ttanay - this is great. Would you please squash the commits so we can merge? - This LGTM

@pabloem Should I change the deprecation version? It is currently "2.11" here:
https://github.com/apache/beam/pull/7918/files#diff-50f2e48308c52f12d0c907b571032195R206

[pabloem]

Since 2.11 is released already, yeah, I'd think you want to set it to 2.12

[ttanay]

Since 2.11 is released already, yeah, I'd think you want to set it to 2.12

Cool. I'll make the change.

[ttanay]

@pabloem Squashed commits, changed deprecation version to "2.12":
https://github.com/apache/beam/pull/7918/files#diff-50f2e48308c52f12d0c907b571032195R206
PTAL

[pabloem]

Thanks Tanay!