[Bug]: Importing Python module apache_beam.dataframe.convert raises AttributeError

[jzxu]

What happened?

In Python, with Apache beam 2.48.0 and Pandas >=2.0.0 installed, running the following import statement raises an exception:

>>> import apache_beam.dataframe.convert
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jzxu/env/lib/python3.11/site-packages/apache_beam/dataframe/convert.py", line 33, in <module>
    from apache_beam.dataframe import transforms
  File "/home/jzxu/env/lib/python3.11/site-packages/apache_beam/dataframe/transforms.py", line 33, in <module>
    from apache_beam.dataframe import frames  # pylint: disable=unused-import
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jzxu/env/lib/python3.11/site-packages/apache_beam/dataframe/frames.py", line 1231, in <module>
    class DeferredSeries(DeferredDataFrameOrSeries):
  File "/home/jzxu/env/lib/python3.11/site-packages/apache_beam/dataframe/frames.py", line 1338, in DeferredSeries
    @frame_base.populate_defaults(pd.Series)
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jzxu/env/lib/python3.11/site-packages/apache_beam/dataframe/frame_base.py", line 600, in wrap
    base_argspec = getfullargspec(unwrap(getattr(base_type, func.__name__)))
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: type object 'Series' has no attribute 'append'. Did you mean: '_append'?

This is because the function Series.append has been removed in Pandas 2.0.0:

https://pandas.pydata.org/docs/dev/whatsnew/v2.0.0.html

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[tvalentyn]

Thanks, this is a duplicate of #27221. Left a comment there.

[tvalentyn]

closing as a duplicate, but it is an issue that should be fixed, let's continue on #27221.