pandas_gbq.to_gbq always replace the destination table.

[rmoralesd]

I'm trying to run a Cloud Function for inserting data in a BigQuery table. The table is already created, it is monthly partitioned by a datetime column and It was like 4000 rows. The cloud function should insert a new row, but instead it drops the table and I end up with a table with just the new row.

• Cloud Function
• 1st gen
• Python version: 3.10

My requeriments.txt file

google-cloud-bigquery
pandas
pandas_gbq
db-dtypes

Steps to reproduce

1. Create a dataset in BigQuery
2. Create a table
3. Fill the table with some data
4. Execute the cloud function in order to insert a new row.

Expected result: new row inserted in the table, keeping the old rows.
Actual result: existing rows are deleted, the table only have one row

Code example

# example
df = pd.DataFrame({
        "ticketid":'777777777',
        "ticket_num":12345,
        "Placa":"ABC123",
        "Entrada":"Prueba",
        "FechaHoraIngreso":pd.Timestamp(year=2022,month=1,day=31,hour=20)        
    },index=[0])   

    pandas_gbq.to_gbq(df,'dataset_name.table_name','project_id',if_exists='append')

Stack trace

# example
There are no errors during the execution. 

Thanks!

[meredithslota]

Can you confirm what versions of each of those libraries you're using? The "if_exists" behavior hasn't changed in ~5 years, if I'm looking in the right place (https://github.com/googleapis/python-bigquery-pandas/blame/7978a917aa4326b7118ecd20258330ac5882057d/pandas_gbq/gbq.py#L1034) and we have tests running to validate this, but I'm wondering if there's something we're missing re: Cloud Functions.

[rmoralesd]

Hello @meredithslota . I don't really remember how I solved this, I believe I had some kind of misunderstanding about the parameters of the functions. Thanks anyway for the answer.

[meredithslota]

Ok, thanks! I'm going to close this out for now, but please do file a new issue if this crops up again. Thanks!