-
Notifications
You must be signed in to change notification settings - Fork 101
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dynamic table name assign support #19
Comments
I think assigning a table name by record value has difficulties about validation (field existence, type, contained characters). |
I know that exploring a data/field driven approach to tables would help many use cases tremendously. It is worth exploring and is not quite the same as what is provided by plugin forest. For instance, in a multi-tenant system data may be ingested with a value indicating the customer. Moving that data into tables specifically for that customer is one way to reduce data usage in BigQuery. BigQuery also supports creating tables from a schema table. A table that uses the schema of the named table, but is named by the passed in value. This is definitely worth exploring I'm just not yet knowledgeable enough with this plugin to start implementing it. |
Thank you for comments. |
I saw the plugin could support the table name with the time format a few days ago.
But sometimes the time format table name maybe not enough.
Is it possible let plugin read table name by the fields?
For example :
table accesslog_%Y_%m_%{name}
"name" is one of the fileds in fluentd.
The text was updated successfully, but these errors were encountered: