[BEAM-13793] Adding external transform registry for schema-based payl…

[ihji]

…oad to Python expansion service

Please add a meaningful description for your change here

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

ValidatesRunner compliance status (on master branch)

Lang	ULR	Dataflow	Flink	Samza	Spark	Twister2
Go	---					---
Java
Python	---					---
XLang						---

Examples testing status on various runners

Lang	ULR	Dataflow	Flink	Samza	Spark	Twister2
Go	---	---	---	---	---	---	---
Java	---		---	---	---	---	---
Python	---	---	---	---	---	---	---
XLang	---	---	---	---	---	---	---

Post-Commit SDK/Transform Integration Tests Status (on master branch)

Go	Java	Python

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website	Whitespace	Typescript
Non-portable
Portable	---			---	---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[codecov]

Codecov Report

Merging #16800 (c887d46) into master (9833b7b) will decrease coverage by 0.00%.
The diff coverage is 88.37%.

@@            Coverage Diff             @@
##           master   #16800      +/-   ##
==========================================
- Coverage   73.82%   73.82%   -0.01%     
==========================================
  Files         667      668       +1     
  Lines       87392    87350      -42     
==========================================
- Hits        64521    64489      -32     
+ Misses      21768    21758      -10     
  Partials     1103     1103              

Flag	Coverage Δ
python	83.65% <88.37%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...runners/portability/external_transform_registry.py	87.80% <87.80%> (ø)
sdks/python/apache_beam/portability/common_urns.py	100.00% <100.00%> (ø)
...ache_beam/runners/portability/expansion_service.py	93.18% <100.00%> (ø)
sdks/python/apache_beam/io/localfilesystem.py	91.47% <0.00%> (-0.78%)	⬇️
...pache_beam/runners/interactive/interactive_beam.py	80.62% <0.00%> (-0.78%)	⬇️
...runners/interactive/display/pcoll_visualization.py	85.85% <0.00%> (-0.51%)	⬇️
...dks/python/apache_beam/options/pipeline_options.py	94.85% <0.00%> (-0.43%)	⬇️
...hon/apache_beam/runners/worker/bundle_processor.py	93.51% <0.00%> (-0.13%)	⬇️
...he_beam/portability/api/external_transforms_pb2.py	100.00% <0.00%> (ø)
...am/portability/api/external_transforms_pb2_urns.py	100.00% <0.00%> (ø)
... and 5 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9833b7b...c887d46. Read the comment docs.

[ihji]

R: @robertwb @chamikaramj

[robertwb]

I haven't looked at the code itself, but as mentioned on the doc, I'm not sure how much value we can get from a transform registry if the transforms in question need to be explicitly imported in order to be registered.

[aaltay]

What is the next step for this PR?

[robertwb]

I think we can drop this PR now, right?

[ihji]

Yes, we don't need this.