Skip to content

Commit

Permalink
Update index.js
Browse files Browse the repository at this point in the history
  • Loading branch information
mservidio committed Nov 5, 2019
1 parent 463014e commit d5c2473
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion ingestion/function/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ async function transform(config) {
const transformQuery = await fromStorage(config.bucket,
`${processPrefix}/${config.destinationTable}.${transformFileName}`) || defaultTransformQuery;
const dataset = bigqueryClient.dataset(config.dataset);
// const exists = tableExists(config.dataset, config.destinationTable);
// const exists = await tableExists(config.dataset, config.destinationTable);

This comment has been minimized.

Copy link
@salsferrazza

salsferrazza Nov 5, 2019

Contributor

So If the table exists, it will still append based on write disposition hint specified in schema.json. But it will reset the column descriptions and modes. If table doesn't exist, it will create since there's CREATE_IF_NEEDED specified. If an existing publisher updates to this version of the code and runs an ingest, they will lose any metadata they applied to a destination table out of band, correct?

// if (!exists) {
// console.log(`creating table ${config.destinationTable} with ${config.metadata.fields}`);
// await dataset.createTable(config.destinationTable, { schema: config.metadata.fields });
Expand Down

0 comments on commit d5c2473

Please sign in to comment.