Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the map type inaccessible to users #2976

Merged
merged 10 commits into from Mar 1, 2023

Conversation

dominiklohmann
Copy link
Member

@dominiklohmann dominiklohmann commented Feb 28, 2023

We want to remove the map type, but doing that is a larger operation. To get the user-facing breaking change out earlier, i.e., in time for 3.0, we're just making it inaccessible first and are then removing it later on.

Additionally, this PR removes the only place in VAST that ever used the map type for table slices written to disk: The vast.metrics event, whose metadata was never queryable.

Now, behold the beauty of actually queryable metrics from the self-sink:

$ vast -qq export json '#type == "vast.metrics.passive-store.init.runtime"
  | select ts,value
  | summarize min(value), max(value) by ts resolution 10s'
{"ts": "2023-02-28T17:21:50.000000", "min(value)": 0.218875, "max(value)": 107.280125}
{"ts": "2023-02-28T17:20:00.000000", "min(value)": 0.549292, "max(value)": 0.549292}

To enable the self-sink for testing, set the following:

# <prefix>/etc/vast/vast.yaml
vast:
  enable-metrics: true
  metrics:
    self-sink:
      enable: true

Tasks

Edit tasklist title
Beta Give feedback Tasklist Tasks, more options

Delete tasklist

Delete tasklist block?
Are you sure? All relationships in this tasklist will be removed.
  1. Make the map type inaccessible
    Options
  2. Avoid using the map type for metrics
    Options
  3. Update the VAST v3.0 blog post draft
    Options
  4. Update the documentation @mavam will do that in Document schema and type definition rules #2967 for me
    Options
  5. Add changelog entry
    Options

We want to remove the map type, but doing that is a larger operation. To
get the user-facing breaking change out earlier, i.e., in time for 3.0,
we're just making it inaccessible first and are then removing it later
on.
This was the only place in VAST that ever used the map type for table
slices written to disk: The `vast.metrics` event, whose metadata was
never queryable.

Now, behold the beauty of actually queryable metrics from the self-sink:

```
$ vast -qq export json '#type == "vast.metrics.passive-store.init.runtime"
  | select ts,value
  | summarize min(value), max(value) by ts resolution 10s'
{"ts": "2023-02-28T17:21:50.000000", "min(value)": 0.218875, "max(value)": 107.280125}
{"ts": "2023-02-28T17:20:00.000000", "min(value)": 0.549292, "max(value)": 0.549292}
```

To enable the self-sink for testing, set the following:

```yaml
# <prefix>/etc/vast/vast.yaml
vast:
  enable-metrics: true
  metrics:
    self-sink:
      enable: true
```
@mavam
Copy link
Member

mavam commented Feb 28, 2023

Loving the "self sink" example!

@dominiklohmann dominiklohmann marked this pull request as ready for review March 1, 2023 08:19
Copy link
Member

@mavam mavam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

web/docs/setup/monitor.md Outdated Show resolved Hide resolved
web/docs/setup/monitor.md Outdated Show resolved Hide resolved
web/docs/setup/monitor.md Outdated Show resolved Hide resolved
web/docs/setup/monitor.md Outdated Show resolved Hide resolved
libvast/src/system/accountant.cpp Outdated Show resolved Hide resolved
web/blog/vast-v3.0/index.md Outdated Show resolved Hide resolved
web/docs/setup/monitor.md Show resolved Hide resolved
web/docs/setup/monitor.md Show resolved Hide resolved
Co-authored-by: Matthias Vallentin <matthias@vallentin.net>
@dominiklohmann dominiklohmann merged commit e0cbccf into master Mar 1, 2023
38 checks passed
@dominiklohmann dominiklohmann deleted the topic/make-map-type-inaccessible branch March 1, 2023 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants