Main merge 2 #1426

mattdurham · 2022-02-25T15:17:24Z

Merge main from yesterday

* Add node_exporter to depcheck * update weaveworks/common dependency * map current release flags and changed defaults * documentation * revert accidental checkin * print out flags when node_exporter test fails to assist debugging * oops, i introduced some flags from master by mistake

* [dev.multiple-integrations] Enable present integrations by default, deprecate enabled field (#1062) * integrations: default to enabled by default * document deprecation of enabled * pkg/integrations: support *_configs field for integrations (#1130) Creates the basic code to unmarshal integrations from a YAML field called <integration name>_configs, which is a slice of that integration. Note that this is NOT wired up to the integrations manager yet, and trying to run the agent with more than one integration of the same type will likely cause problems. * [dev.multiple-integrations] Prototype new integrations subsystem (#1142) * wip: prototype new integrations subsystem * implement Controller with basic logic for Integration and UpdateIntegration * Implement HTTPIntegration for Controller * decouple controller and subsystem * don't have controller implement integration slightly less smelly now * multiplexer integration * rely on boilerplate for multiplexing for now generics would be nice here * remove multiplex_integration.go Also a little code smelly. Instead of having integrations that run other integrations, I'm going to fall back to having only one controller. * introduce Subsystem, unexport Controller start wiring up things to Subsystem * introduce v2 agent integration to use for testing * start wiring metrics integrations * rename Options to Globals call a spade a spade * add subsystem options to globals * remove dead code * metricsutils: calculate self-scraping based on globals * complete HTTP target API * working example with agent integration * appease the linter * don't return an error when context to cancel an integration is closed * once again i am asking the linter to forgive my typos * fix bug where labels from individual targets were getting dropped at the API endpoint * pkg/config: fix broken test * finish unit tests for integrations v2 controller * metricsutil/metricshandler_integration: make job name unique Before this change, the job name would have collided when using multiple instances of the same integration. * ensure that global subsystem labels are injected into targets * integrations/v2: Infer target hostname from SD API host (#1175) * [dev.multiple-integrations] integrations/v2: allow shimming between v1 and v2 integrations. (#1179) * integrations/v2: allow shimming between v1 and v2 integrations. Shimming is done by changing how the integration registration works; a new RegisterDynamic was added that allows for creating Configs at runtime. Here be dragons; this should be removed whenever we no longer have a need for it. * fix lint * pkg/integrations/v2: use "RegisterLegacy" instead of a generic mechanism * fine, I won't add the deprecation notice if it will make the linter sad * pkg/integrations: re-align (#1181) This commit reverts 69ba2dd in favor of allowing the new subsystem to handle multiple instances of integrations. This commit also removes the wal_truncate_frequency field from integrations as it is the only field from old integrations that does not have a current counterpart. * [dev.multiple-integrations] Hide integrations/v2 behind a feature flag (#1185) * feature flag wip * dynamically switch between integrations v1 and v2 default to v1. * pkg/integrations/versionselector to file in pkg/config * pkg/config: fix defaults for Integrations * pkg/config: use more generic way to unmarshal differently based on flag * add missing godoc comment * more comments * switch to deferred unmarshaling * remove unused Config field * simplify completeUnmarshal * do not perform lazy deferred unmarshaling * enable cadvisor by default * switch to using real feature flag * fix postgres_exporter * Merge main into dev.multiple-integrations (#1184) * Fix typo (#1141) * Traces: Improved pod association in PromSD processor (#1137) * Improve k8s pod association * Add tests * Changelog * typo * Add prom_sd_pod_association * Extend tests for pod associations * Docs for pod association config * Lint fixes * Move to unreleased * Add instrumentation recommendations * Remove uncessary constants * Improve tests * remote config with http(s) provider (#1143) * sample remote config code with http provider * use t.TempDir() in unit test * no need to clean up after T.TempDir() * use NewClientFromConfig and make caller responsible for calling SetDirectory * handle nil HTTPClientConfig * remove blank identifier assignment * pass basic auth command line flags for remote config * address pr nits * add expiremental flag * set loader inline * update changelog * add remote config section in docs * pr comment updates * announce patch releases for cve-2021-41090 (#1152) * Merge patch release to main (#1153) * Add secret type to sensitive values * Break out config tests to their own implementation. Also remove username has a sensitive value. * Update changelog * Fix failing test * Scrub secrets when marshaling instance configs * update for v0.21 * Updated changes from the merge. * Remove changelog * Scrub out receivers has ***receivers_scrubber***:null * obscure etcd/consul credentials * Update pkg/traces/config_test.go Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Update pkg/config/config.go * go fmt * Change to using custom object and return <secret> * Fix bad merge * [v0.21.2] toggle config endpoint (#19) * disable /-/config endpoint by default * disable scraping api get endpoint as well * fix new test * add test and rename flag Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Update version to v0.21.2 * Update defaults.go * fix /-/config endpoint * also fix non-pointer config bug * temporarily disable linting for release * fix lint errors Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> * Fix POSTGRES_EXPORTER_DATA_SOURCE_NAME usage for postgres_exporter (#1162) * Fix POSTGRES_EXPORTER_DATA_SOURCE_NAME usage for postgres_exporter A recent change broke the usage of POSTGRES_EXPORTER_DATA_SOURCE_NAME for the postgres_exporter. As the incorrect variable was checked in the if clause, it always raises an error. * changelog: keep feature -> enhancement -> bugfix order * postgres_exporter: add regression test Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Fix syntax error in Jsonnet logs helper method (#1174) Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * cAdvisor Integration (#1081) * Add cadvisor module * Begin creating common config for cadvisor * Don't export internal state * Finish config options for cadvisor * Set config options, and implement cAdvisor collectors * Linting * Buildflags for cadvisor only in linux * I R LEArN Build Tags * Don't zero value the zero value * Offload sketchy global var manipulation to the integrations Run func * Remove unused collectors * Lint * Create generic stub integration and use it for cadvisor * Lint * Final refactor of cAdvisor config for unsupported platforms. Pared down stub integrations. * Lint * Docs for cadvisor config * Update changelog * Update pkg/integrations/stub_integration.go Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Reorder changelog * Instance key clarity * Inclusive naming * Finish name changes Keep default disable metric list in sync with upstream Idiomatic golang * Hardcode disabled metrics for cadvisor Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Remove log-level flag from systemd unit file (#1177) * Upgrade to OTel v0.40.0 (#1176) * Upgrade to OTel v0.40.0 * Changelog * Add factories check * go mod tidy * config/features: create package to standardize experimental features (#1170) * config/features: create package to standardize experiemental features This commit introduces a new package, pkg/config/features, which allows defining a set of features and validating whether flags associated with those features are allowed to be set. Closes #1163 * update documentation (also s/enabled-features/enable-features) * Fix typo * Update pkg/config/features/features.go Co-authored-by: Robert Lankford <rlankfo@gmail.com> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * enable cadvisor by default * switch to using real feature flag * fix postgres_exporter Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: f11r <fiete.gruenter@rwth-aachen.de> Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * Revert "Merge main into dev.multiple-integrations (#1184)" (#1189) This reverts commit ad76ec5. * [dev.multiple-integrations] Revert breaking changes to existing integrations (#1191) * revert breaking changes to integrations v1 This commit reverts #1062 in favor of making breaking changes directly in integrations-next instead. The part of #1181 to remove `wal_truncate_frequency` has also been reverted. As part of this change, the enabled field is removed from the v2 common metrics configs, and v2 integrations can no longer be disabled. v2 integrations can only be disabled by removing them from the YAML. * integrations/v2: remove stale reference to ErrDisabled (fix typo too) * integrations/v2: bring in common config decoupling * [dev.multiple-integrations] Introduce autoscraper (#1195) * pkg/integrations/v2: introduce self-scraping * linting * [dev.multiple-integrations] Multiple instances of integrations (#1196) * multiple instances of integrations opt in relevant v1 integrations into supporting multiple instances * shims should check for instance key override * Document integrations-next (#1197) * document integrations-next * remove json tags since they make markdown unhappy * changelog * s/Run/RunIntegration * remove stale comment about integrations.controller purpose * create dedicated run method for instanceScraper * s/expoter/exporter/g * Document why an autoscrape.Scraper manages a set of per-instance scrapers * spell out prerequisite instead of pre-req * use go.uber.org/atomic to make the code a little easier to follow * remove started callback for running integration * use smaller interface for autoscrape Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: f11r <fiete.gruenter@rwth-aachen.de> Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com>

…tly unmarshaled (#1235)

* remove jsonnet vendor This adds all vendor folders into .gitignore and removes cached vendor files from the repository. Closes #1221 * Update scripts and instructions for jsonnet vendor removal * `make example-dashboards` will now also run `jb install` * k3d environment instructions now include `jb install` * smoke-test.bash will now run `jb install` prior to `tk apply`