DynamicAllocation is ignored when I use the connector.

[jgustave]

If I don't use the bigquery connector, then my project will DynamicAllocate to make use of the cluster automatically.
Once I add the call to the bq connector, the job is no longer dynamic and uses the specified spark.executor.cores, etc. I verified that the configs in the environment are identical and have dynamicAllocation enabled.

[Gaurangi94]

Hello,

Could you answer the following questions about your setup? It will help us resolve the issue faster.

1. How do you enable dynamicallocation on your cluster? Have you added this setting in spark-defaults.conf or do you enable it for each job?
2. What parameters are you using in running the job WITH and WITHOUT sparkbqconnector?

Thanks!