Skip to content

Make the map type inaccessible to users #2976

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Mar 1, 2023

Conversation

dominiklohmann
Copy link
Member

@dominiklohmann dominiklohmann commented Feb 28, 2023

We want to remove the map type, but doing that is a larger operation. To get the user-facing breaking change out earlier, i.e., in time for 3.0, we're just making it inaccessible first and are then removing it later on.

Additionally, this PR removes the only place in VAST that ever used the map type for table slices written to disk: The vast.metrics event, whose metadata was never queryable.

Now, behold the beauty of actually queryable metrics from the self-sink:

$ vast -qq export json '#type == "vast.metrics.passive-store.init.runtime"
  | select ts,value
  | summarize min(value), max(value) by ts resolution 10s'
{"ts": "2023-02-28T17:21:50.000000", "min(value)": 0.218875, "max(value)": 107.280125}
{"ts": "2023-02-28T17:20:00.000000", "min(value)": 0.549292, "max(value)": 0.549292}

To enable the self-sink for testing, set the following:

# <prefix>/etc/vast/vast.yaml
vast:
  enable-metrics: true
  metrics:
    self-sink:
      enable: true
### Tasks
- [x] Make the map type inaccessible
- [x] Avoid using the map type for metrics
- [x] Update the VAST v3.0 blog post draft
- [x] ~Update the documentation~ @mavam will do that in tenzir/vast#2967 for me
- [x] Add changelog entry

We want to remove the map type, but doing that is a larger operation. To
get the user-facing breaking change out earlier, i.e., in time for 3.0,
we're just making it inaccessible first and are then removing it later
on.
This was the only place in VAST that ever used the map type for table
slices written to disk: The `vast.metrics` event, whose metadata was
never queryable.

Now, behold the beauty of actually queryable metrics from the self-sink:

```
$ vast -qq export json '#type == "vast.metrics.passive-store.init.runtime"
  | select ts,value
  | summarize min(value), max(value) by ts resolution 10s'
{"ts": "2023-02-28T17:21:50.000000", "min(value)": 0.218875, "max(value)": 107.280125}
{"ts": "2023-02-28T17:20:00.000000", "min(value)": 0.549292, "max(value)": 0.549292}
```

To enable the self-sink for testing, set the following:

```yaml
# <prefix>/etc/vast/vast.yaml
vast:
  enable-metrics: true
  metrics:
    self-sink:
      enable: true
```
@mavam
Copy link
Member

mavam commented Feb 28, 2023

Loving the "self sink" example!

@dominiklohmann dominiklohmann requested a review from mavam March 1, 2023 08:19
@dominiklohmann dominiklohmann marked this pull request as ready for review March 1, 2023 08:19
Copy link
Member

@mavam mavam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Co-authored-by: Matthias Vallentin <matthias@vallentin.net>
@dominiklohmann dominiklohmann enabled auto-merge March 1, 2023 09:55
@dominiklohmann dominiklohmann merged commit e0cbccf into master Mar 1, 2023
@dominiklohmann dominiklohmann deleted the topic/make-map-type-inaccessible branch March 1, 2023 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants