Raise error if dataframe is empty

[pankajastro]

Description

closes: #810

What is the current behavior?

In case of an empty dataframe the data frame operator throws an SQL syntax error.

What is the new behavior?

If the dataframe is empty then raise an error

Does this introduce a breaking change?

No

Checklist

• Created tests which fail without the change (if possible)
• Extended the README / documentation, if necessary

[codecov]

Codecov Report

Base: 94.13% // Head: 94.80% // Increases project coverage by +0.66% 🎉

Coverage data is based on head (2fac372) compared to base (a99f949).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1238      +/-   ##
==========================================
+ Coverage   94.13%   94.80%   +0.66%     
==========================================
  Files          17       69      +52     
  Lines         597     3235    +2638     
  Branches       67      378     +311     
==========================================
+ Hits          562     3067    +2505     
- Misses         22      104      +82     
- Partials       13       64      +51     

Impacted Files	Coverage Δ
python-sdk/src/astro/databases/aws/redshift.py	93.93% <100.00%> (ø)
python-sdk/src/astro/databases/base.py	96.09% <100.00%> (ø)
python-sdk/src/astro/databases/google/bigquery.py	94.56% <100.00%> (ø)
python-sdk/src/astro/databases/postgres.py	96.55% <100.00%> (ø)
python-sdk/src/astro/databases/snowflake.py	95.69% <100.00%> (ø)
python-sdk/src/astro/exceptions.py	100.00% <0.00%> (ø)
python-sdk/src/astro/sql/operators/drop.py	100.00% <0.00%> (ø)
python-sdk/src/astro/table.py	100.00% <0.00%> (ø)
... and 47 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[pankajkoti]

Do we need to add this check for postgres, redshift and bigquery too? Since we're overriding the load_pandas_dataframe_to_table method in those implementations too?

[pankajkoti]

In my opinion, we should not be creating tables as part of load_pandas_dataframe_to_table if an empty dataframe is passed, as to me, it sounds like a side effect.
If the user really wants to create a table by passing an empty dataframe, we could add a separate method for that. But load_pandas_dataframe_to_table would mean load a dataframe and if that is empty, in my opinion, we should alert instead of silently just creating the table.

[sunank200]

I feel that we should have a separate method which should check if dataframe is empty in the base class. That method can be overridden for Postgres and Bigquery.

[sunank200]

In my opinion, we should not be creating tables as part of load_pandas_dataframe_to_table if an empty dataframe is passed, as to me, it sounds like a side effect. If the user really wants to create a table by passing an empty dataframe, we could add a separate method for that. But load_pandas_dataframe_to_table would mean load a dataframe and if that is empty, in my opinion, we should alert instead of silently just creating the table.

+1 on this. I feel we should not do multiple things as part of same method.

[sunank200]

Shouldn't _assert_not_empty_df(df) be overridden for PostGres and Bigquery like you had mentioned?