Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: example in readme #2

Open
wants to merge 125 commits into
base: postgres
Choose a base branch
from
Open

fix: example in readme #2

wants to merge 125 commits into from

Conversation

jsbrain
Copy link

@jsbrain jsbrain commented Feb 25, 2022

Example not working as IF EXISTS arguments are at wrong position.

@svenklemm @phemmer
initial import of postgres output plugin
a1bd367
@svenklemm @phemmer
make address configurable
0afa350
@svenklemm @phemmer
add helper functions for create table and insert
bcca5c8
@svenklemm @phemmer
add tests
a13ac44
@svenklemm @phemmer
fix sql placeholder
1a6b1c4
@svenklemm @phemmer
let pgx handle time conversion, remove old code
5ea8868
@svenklemm @phemmer
adjust test cases to datatype changes
8aa03cf
@svenklemm @phemmer
remove debug prints
749e757
@svenklemm @phemmer
check if table exists before creating
e157258
@svenklemm @phemmer
allow skipping tags
05df41b
@svenklemm @phemmer
refactoring
4c35bff
@svenklemm @phemmer
implement TagsAsForeignkeys
07dcf4e
@svenklemm @phemmer
fix tests
8348b7e
@svenklemm @phemmer
add SampleConfig
feee01c
@svenklemm @phemmer
register driver
7b0315b
@svenklemm @phemmer
update README
cd4363f
@svenklemm @phemmer
prepare for create table template
8d5ad62
@svenklemm @phemmer
quote identifier
a1e6b9a
@svenklemm @phemmer
refactor generateInsert
b87f066
@svenklemm @phemmer
use timestamp for time column to allow pg10 partitioning
22d8a1e
@svenklemm @phemmer
remove nondeterministic tests
212125f
@svenklemm @phemmer
use template for create table query generation
c3874cf
@svenklemm @phemmer
make TableTemplate configurable
d5f27b4
@svenklemm @phemmer
add quoteLiteral helper function
8ee4803
@svenklemm @phemmer
add tests for quoting
7e3330c
@svenklemm @phemmer
add TABLELITERAL to template variables
ee6fd6b
@svenklemm @phemmer
fix template in doc
236ddb9
@svenklemm @phemmer
dont add primary key
09f311a
@svenklemm @phemmer
allow using jsonb for fields and tags and make it default
9dc0fed
@svenklemm @phemmer
document jsonb settings
8fb25ae 
remove IgnoredTags since tagexclude achieves the same
phemmer and others added 28 commits August 13, 2021 11:08
@phemmer
outputs/postgresql: address PR comments
93aec6f
@phemmer
outputs/postgresql: Refactor initialization to use telegraf convention
ff644a9
@phemmer
Merge remote-tracking branch 'origin/master' into postgres
fa5f937 
* origin/master: (183 commits)
  fix: CrateDB replace dots in tag keys with underscores (influxdata#9566)
  feat: Pull metrics from multiple AWS CloudWatch namespaces (influxdata#9386)
  fix: improve Clickhouse corner cases for empty recordset in aggregation queries, fix dictionaries behavior (influxdata#9401)
  fix(opcua): clean client on disconnect so that connect works cleanly (influxdata#9583)
  fix: Refactor ec2 init for config-api (influxdata#9576)
  fix: sort logs by timestamp before writing to Loki (influxdata#9571)
  fix: muting tests for udp_listener (influxdata#9578)
  fix: Do not return on disconnect to avoid breaking reconnect (influxdata#9524)
  fix: Fixing k8s nodes and pods parsing error (influxdata#9581)
  feat: OpenTelemetry output plugin (influxdata#9228)
  feat: Support AWS Web Identity Provider (influxdata#9411)
  fix: upgraded sensu/go to v2.9.0 (influxdata#9577)
  fix: Normalize unix socket path (influxdata#9554)
  docs: fix aws ec2 readme inconsistency (influxdata#9567)
  feat: Modbus Rtu over tcp enhancement (influxdata#9570)
  docs: information on new conventional commit format (influxdata#9573)
  docs: Add logo (influxdata#9574)
  docs: Adding links to net_irtt and dht_sensor external plugins (influxdata#9569)
  Upgrade hashicorp/consul/api to 1.9.1 (influxdata#9565)
  Update vmware/govmomi to v0.26.0 (influxdata#9552)
  Do not skip good quality nodes after a bad quality node is encountered (influxdata#9550)
  fix test so it hits a fake service (influxdata#9564)
  Update changelog
  Fix procstat plugin README to match sample config (influxdata#9553)
  Fix metrics reported as written but not actually written  (influxdata#9526)
  Prevent segfault in persistent volume claims (influxdata#9549)
  Update procstat to support cgroup globs & include systemd unit children (Copy of influxdata#7890) (influxdata#9488)
  Fix attempt to connect to an empty list of servers. (influxdata#9503)
  Fix handling bool in sql input plugin (influxdata#9540)
  Suricata alerts (influxdata#9322)
  Linter fixes for plugins/inputs/[fg]* (influxdata#9387)
  For Prometheus Input add ability to query Consul Service catalog (influxdata#5464)
  Support Landing page on Prometheus landing page (influxdata#8641)
  [Docs] Clarify tagging behavior (influxdata#9461)
  Change the timeout from all queries to per query (influxdata#9471)
  Attach the pod labels to the `kubernetes_pod_volume` & `kubernetes_pod_network` metrics. (influxdata#9438)
  feat(http_listener_v2): allows multiple paths and add path_tag (influxdata#9529)
  Bug Fix Snmp empty metric name (influxdata#9519)
  Worktable workfile stats (influxdata#8587)
  Update Go to v1.16.6 (influxdata#9542)
  ...
@phemmer
outputs/postgresql: address linter complaints
04d2764
@phemmer
outputs/postgresql: update licenses
a1316ae
@phemmer
outputs/postgresql: remove maybeTempError
55f071b
@phemmer
outputs/postgresql: configure integration testing
ed55f2f
@phemmer
outputs.postgresql: change default log level to warn
50c8b39
@phemmer
outputs.postgresql: fix error handling on sequential writes
3f946a2 
permanent errors on sub-batches were resulting in retries
@phemmer
outputs.postgresql: don't try to detect mismatched data types
48315bb 
the admin may deliberately re-type a column because because they know a value will never exceed a more optimized type. The error response is the same in both cases (permanent with sub-batch drop), so let the DB handle it instead.
In the future, we may want to detect values that won't fit inside a column type on a per-row basis (in TableSource), but this means more code, and right now handling the admin's broken database scheme isn't worth it.
@phemmer
outputs.postgresql: add support for pguint
1936779
@phemmer
outputs.postgresql: rewrite schema updates to address concurrency issues
0038e57 
Total rewrite of TableManager.EnsureStructure. This addresses concurrency issues with multiple telegraf processes attempting to create tables at the same time. The code is longer, but also simpler.
@phemmer
outputs.postgresql: update README & add test to keep updated
1bcc5c2
@phemmer
outputs.postgresql: fix missing .allColumns template values
94f0b74 
In the event that the local table cache was missing columns, but the DB had them, when performing a table update the missing columns wouldn't be passed to the template.
@phemmer
outputs.postgresql: use timestamp without time zone
40ef1f9 
major performance hit in timescaledb for some queries when timestamp with time zone is used:
timescale/timescaledb#118 (comment)
@phemmer
outputs.postgresql: set pg connection application name
2d84174
@phemmer
outputs.postgresql: fixes for cache
a00e292 
Addresses 2 problems with the tag cache:
1. The same tag ID in use by multiple tables would collide, causing one not to be inserted.
2. The cache sizing was grossly underestimated.

Also added some debug logging for cache statistics.
@phemmer
outputs.postgresql: add "connection" parameter to sample config
10c0810
@phemmer
outputs.postgresql: better handling of DDL errors
96a1b41 
When a DDL statement (adding columns) errors, instead of failing the whole write operation, treat permanent errors as if DDL statements were disabled. Meaning on fields, just drop the field, and on tags, drop the metric.
Temporary errors still fail so that retry won't result in data loss.
@phemmer
outputs.postgresql: fix tests error logging & handling of %w
04cfa33
@phemmer
outputs.postgresql: fix column removal
402b4ec 
fixes incorrect column re-indexing when a column was removed from TableSource
@phemmer
Merge remote-tracking branch 'origin/master' into postgres
02ee794 
* origin/master: (176 commits)
  fix: Linter fixes for plugins/inputs/[h-j]* (influxdata#9986)
  fix(inputs/kube_inventory): don't skip resources with zero s/ns timestamps (influxdata#9978)
  fix: update gjson to v1.10.2 (influxdata#9998)
  fix: procstat tags were not getting generated correctly (influxdata#9973)
  chore: create bug report form (influxdata#9976)
  fix: Allow for non x86 macs in Go install script (influxdata#9982)
  test: add sqlserver plugin integration tests (influxdata#9943)
  feat: plugins/common/tls/config.go: Filter client certificates by DNS names (influxdata#9910)
  feat: add option to skip table creation in azure data explorer output (influxdata#9942)
  docs: update nightlies links (influxdata#9989)
  fix: add s390x to nightlies (influxdata#9990)
  feat: Add more details to processors.ifname logmessages (influxdata#9984)
  docs: Create SECURITY.md (influxdata#9951)
  fix: set NIGHTLY=1 for correctly named nightly artifacts (influxdata#9987)
  feat: Kafka Add metadata full to config (influxdata#9833)
  chore: Update to AWS SDK v2 (influxdata#9647)
  chore: lint ignore fmt.Printf unhandled error (influxdata#9967)
  fix: starlark pop operation for non-existing keys (influxdata#9954)
  feat: update etc/telegraf.conf and etc/telegraf_windows.conf (influxdata#9876)
  fix: Check return code of zfs command for FreeBSD. (influxdata#9956)
  chore: update go to 1.17.2 (influxdata#9873)
  fix: Graylog plugin TLS support and message format (influxdata#9862)
  docs: update README with info on package repos (influxdata#9964)
  feat: Modbus connection settings (serial) (influxdata#9256)
  fix: segfault in ingress, persistentvolumeclaim, statefulset in kube_inventory (influxdata#9585)
  fix: add normalization of tags for ethtool input plugin (influxdata#9901)
  chore: remove empty build.py (influxdata#9958)
  fix: internet_speed input plugin not collecting/reporting latency (influxdata#9957)
  chore: reference db2 external plugin (influxdata#9952)
  chore: update readme go version from 1.14 to 1.17 (influxdata#9944)
  fix: decode Prometheus scrape path from Kuberentes labels (influxdata#9662)
  docs: fix broken link (influxdata#9812)
  fix: Correct conversion of int with specific bit size (influxdata#9933)
  fix: update golanci-lint to v1.42.1 (influxdata#9932)
  feat: Azure Event Hubs output plugin (influxdata#9346)
  feat: more fields for papertrail event webhook (influxdata#9940)
  fix: solve compatibility issue for mongodb inputs when using 5.x relicaset (influxdata#9892)
  docs: Add symlink to command documentation (influxdata#9926)
  docs: update contributing.md (influxdata#9914)
  chore: reference oracle external plugin (influxdata#9934)
  ...
@phemmer
outputs.postgresql: fix tags_as_foreign_keys typo in README
0665544
@phemmer
Merge remote-tracking branch 'origin/master' into postgres
ddb5575 
* origin/master: (133 commits)
  chore: restart service if it is already running and upgraded via RPM (influxdata#9970)
  feat: update etc/telegraf.conf and etc/telegraf_windows.conf (influxdata#10237)
  fix: Handle duplicate registration of protocol-buffer files gracefully. (influxdata#10188)
  fix(http_listener_v2): fix panic on close (influxdata#10132)
  feat: add Vault input plugin (influxdata#10198)
  feat: support aws managed service for prometheus (influxdata#10202)
  fix: Make telegraf compile on Windows with golang 1.16.2 (influxdata#10246)
  Update changelog
  feat: Modbus add per-request tags (influxdata#10231)
  fix: Implement NaN and inf handling for elasticsearch output (influxdata#10196)
  feat: add nomad input plugin (influxdata#10106)
  fix: Print loaded plugins and deprecations for once and test (influxdata#10205)
  fix: eliminate MIB dependency for ifname processor (influxdata#10214)
  feat: Optimize locking for SNMP MIBs loading. (influxdata#10206)
  feat: Add SMART plugin concurrency configuration option, nvme-cli v1.14+ support and lint fixes. (influxdata#10150)
  feat: update configs (influxdata#10236)
  fix(inputs/kube_inventory): set TLS server name config properly (influxdata#9975)
  fix: Sudden close of Telegraf caused by OPC UA input plugin (influxdata#10230)
  fix: bump github.com/eclipse/paho.mqtt.golang from 1.3.0 to 1.3.5 (influxdata#9913)
  fix: json_v2 parser timestamp setting (influxdata#10221)
  fix: ensure graylog spec fields not prefixed with '_' (influxdata#10209)
  docs: remove duplicate links in CONTRIBUTING.md (influxdata#10218)
  fix: pool detection and metrics gathering for ZFS >= 2.1.x (influxdata#10099)
  fix: parallelism fix for ifname processor (influxdata#10007)
  chore: Forbids "log" package only for aggregators, inputs, outputs, parsers and processors (influxdata#10191)
  docs: address documentation gap when running telegraf in k8s (influxdata#10215)
  feat: update etc/telegraf.conf and etc/telegraf_windows.conf (influxdata#10211)
  fix: mqtt topic extracting no longer requires all three fields (influxdata#10208)
  fix: windows service - graceful shutdown of telegraf (influxdata#9616)
  feat: update etc/telegraf.conf and etc/telegraf_windows.conf (influxdata#10201)
  feat: Modbus support multiple slaves (gateway feature) (influxdata#9279)
  fix: Revert unintented corruption of the Makefile from influxdata#10200. (influxdata#10203)
  chore: remove triggering update-config bot in CI (influxdata#10195)
  Update changelog
  feat: Implement deprecation infrastructure (influxdata#10200)
  fix: extra lock on init for safety (influxdata#10199)
  fix: resolve influxdata#10027 (influxdata#10112)
  fix: register bigquery to output plugins influxdata#10177 (influxdata#10178)
  fix: sysstat use unique temp file vs hard-coded (influxdata#10165)
  refactor: snmp to use gosmi (influxdata#9518)
  ...
@phemmer
outputs/postgresql: fix go test detection for pguint extension
0e511d6
@phemmer
Merge remote-tracking branch 'origin/master' into postgres
e87744d 
* origin/master: (211 commits)
  feat: update configs (influxdata#10676)
  feat[elastic output]: add elastic pipeline flags (influxdata#10505)
  Update changelog
  fix: ensure folders do not get loaded more than once (influxdata#10551)
  docs: update VMWare doc links (influxdata#10663)
  fix: prometheusremotewrite wrong timestamp unit (influxdata#10547)
  feat: update configs (influxdata#10662)
  fix: add graylog toml tags (influxdata#10660)
  feat: add socks5 proxy support for kafka output plugin (influxdata#8192)
  docs: override reported OpenSearch version (influxdata#10586)
  feat: update etc/telegraf.conf and etc/telegraf_windows.conf (influxdata#10659)
  fix: bump all go.opentelemetry.io dependencies (influxdata#10647)
  feat: collection offset implementation (influxdata#10545)
  chore: update go to 1.17.7 (influxdata#10658)
  fix: check for nil client before closing in amqp (influxdata#10635)
  fix: timestamp change during execution of json_v2 parser. (influxdata#10657)
  fix: bump github.com/signalfx/golib/v3 from 3.3.38 to 3.3.43 (influxdata#10652)
  fix: bump github.com/aliyun/alibaba-cloud-sdk-go (influxdata#10653)
  fix: incorrect handling of json_v2 timestamp_path (influxdata#10618)
  feat: gather additional stats from memcached (influxdata#10641)
  feat: update etc/telegraf.conf and etc/telegraf_windows.conf (influxdata#10649)
  fix: Revert deprecation of http_listener_v2 (influxdata#10648)
  fix: bump github.com/denisenkom/go-mssqldb from 0.10.0 to 0.12.0 (influxdata#10503)
  fix: bump github.com/gopcua/opcua from 0.2.3 to 0.3.1 (influxdata#10626)
  fix: use current time as ecs timestamp (influxdata#10636)
  fix: bump github.com/nats-io/nats-server/v2 from 2.6.5 to 2.7.2 (influxdata#10638)
  chore: add -race flag to go tests (influxdata#10629)
  feat: update etc/telegraf.conf and etc/telegraf_windows.conf (influxdata#10631)
  fix: license doc outdated causing CI failure (influxdata#10630)
  fix: bump k8s.io/client-go from 0.22.2 to 0.23.3 (influxdata#10589)
  feat: Implemented support for reading raw values, added tests and doc (influxdata#6501)
  fix: Improve parser tests by using go-cmp/cmp (influxdata#10497)
  feat(mongodb): add FsTotalSize and FsUsedSize informations (influxdata#10625)
  docs: update quay docs for auth (influxdata#10612)
  chore: allow downgrade of go version in windows script (influxdata#10614)
  chore: update CI go to 1.17.6 (influxdata#10611)
  feat: update etc/telegraf.conf and etc/telegraf_windows.conf (influxdata#10600)
  fix(inputs.opcua): add more data to error log (influxdata#10465)
  fix: bump github.com/aws/aws-sdk-go-v2/service/kinesis from 1.6.0 to 1.13.0 (influxdata#10601)
  refactor: use early return pattern (influxdata#10591)
  ...
@phemmer
outputs/postgresql: update README for linter
e27281f
@jsbrain
fix: example in readme
efaecae
@phemmer phemmer force-pushed the postgres branch 2 times, most recently from a981c96 to bfa2e66 Compare June 3, 2022 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
5 participants
@jsbrain @svenklemm @saaros @rikonen @phemmer