Set expiry time for spark created temp table

[karimEssawi]

Hi,

I've noticed that when I have a job reading from bigquery, a temp table prefixed with _sbc_ is created. I gather that this temp table is used to stage query results and used in the job later on. The problem is that we have multiple jobs running on a schedular - e.g. every 5 mins - so they generate quite a lot of temp tables and this clutters up bigquery itself. The temp tables do have an expiry time of 24 hours but I was wondering if there is an option to control the expiry time to 15 mins for example?

[davidrabinowitz]

Fixed in version 0.19.0