Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to Docusaurus v3 #3800

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion web/_extensions/codeblock-details/codeblock-details.lua
@@ -1,6 +1,6 @@
function CodeBlock(code_block)
if code_block.attributes["code-fold"] == "true" then
local open = "<details><summary>Code</summary>\n"
local open = "<details>\n<summary>Code</summary>\n"
local close = "\n</details>"
local open_block = pandoc.RawBlock("html", open)
local close_block = pandoc.RawBlock("html", close)
Expand Down
9 changes: 6 additions & 3 deletions web/blog/a-git-retrospective/index.md
Expand Up @@ -33,7 +33,8 @@ For the statistics, we’ll switch to R. In all subsequent figures, a single poi
corresponds to a merge commit. The reduced opacity alleviates the effects of
overplotting.

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
library(dplyr)
Expand Down Expand Up @@ -102,7 +103,8 @@ the small team set sails. Throughput increased as core contributors joined the
team. Fast-forward to 2020 when we started doing public releases. The figure
below shows how this process matured.

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
library(ggrepel)
Expand Down Expand Up @@ -140,7 +142,8 @@ vs. non-breaking changes.
Let’s zoom in on all releases since v1.0. At this time, we had a solid
engineering and release process in place.

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
library(tidyr)
Expand Down
3 changes: 2 additions & 1 deletion web/blog/from-slack-to-discord/index.md
Expand Up @@ -25,7 +25,8 @@ that abandons Slack. [Numerous][meilisearch] [open-source][appwrite]
![Slack-to-Discord](/img/blog/slack-to-discord.excalidraw.svg)

:::info Discord Invite Link
You can join our Discord community chat via <https://discord.tenzir.com>.
You can join our Discord community chat via
[discord.tenzir.com](https://discord.tenzir.com).
:::

Here are the top four reasons why we are switching:
Expand Down
2 changes: 1 addition & 1 deletion web/blog/migrating-from-vast-to-tenzir/index.md
Expand Up @@ -30,7 +30,7 @@ In addition to that, the following things have changed.

- The repository moved from `tenzir/vast` to `tenzir/tenzir`.
- Our Discord server is now the *Tenzir Community*. Join us at
<https://discord.tenzir.com>!
[discord.tenzir.com](https://discord.tenzir.com)!
- The documentation moved from [vast.io](https://vast.io) to
[docs.tenzir.com](https://docs.tenzir.com).

Expand Down
45 changes: 30 additions & 15 deletions web/blog/parquet-and-feather-writing-security-telemetry/index.md
Expand Up @@ -162,7 +162,8 @@ Every row corresponds to a single store file where we varied some of these
parameters. We used [hyperfine](https://github.com/sharkdp/hyperfine) as
benchmark driver tool, configured with 8 runs. Let’s take a look at the data.

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
library(dplyr)
Expand Down Expand Up @@ -269,7 +270,8 @@ internally VAST stores the full-qualified type as schema name.

How many events do we have per schema?

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
schemas <- normalized |>
Expand Down Expand Up @@ -299,7 +301,8 @@ Between 1 and 100M events, we almost see everything.

What’s the typical event size?

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
schemas |>
Expand Down Expand Up @@ -413,7 +416,8 @@ use two other types of “stores” for the analysis to facilitate comparison:
Let’s kick of the analysis by getting a better understanding at the size
distribution.

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
unified |>
Expand Down Expand Up @@ -451,7 +455,8 @@ The y-axis of above plot is log-scaled, which makes it hard for relative
comparison. Let’s focus on the medians (the bars in the box) only and bring the
y-axis to a linear scale:

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
medians <- unified |>
Expand Down Expand Up @@ -504,7 +509,8 @@ a compression ratio of **\~0.9**.
The above analysis covered averages across schemas. If we juxtapose Parquet and
Feather per schema, we see the difference between the two formats more clearly:

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
library(ggrepel)
Expand Down Expand Up @@ -573,7 +579,8 @@ compressors like Zstd shine.
Zooming in to the bottom left area with average event size of less than 100B,
and removing the log scaling, we see the following:

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
parquet_vs_feather_size |>
Expand All @@ -599,7 +606,8 @@ If we take pick a single point, e.g., `zeek.conn` with
we can confirm that the relative performance matches the results of our analysis
above:

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
unified |>
Expand All @@ -622,7 +630,8 @@ import Svg7 from './index_files/figure-gfm/plot-zeek-suricata-1.svg';
Finally, we look at the fraction of space Parquet takes compared to Feather on a
per schema basis, restricted to schemas with more than 10k events:

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
library(tibble)
Expand Down Expand Up @@ -670,7 +679,8 @@ than Parquet. Is that the case when enabling compression for both?
To avoid distortion of small events, we also restrict the analysis to schemas
with more than 100k events.

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
unified |>
Expand Down Expand Up @@ -714,7 +724,8 @@ Parquet compression is slower. Please don’t hesitate to reach out, e.g., via o

Let’s compare Parquet and Feather by compression level, per schema:

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
parquet_vs_feather_duration <- unified |>
Expand Down Expand Up @@ -776,7 +787,8 @@ Finally, we combine the size and speed analysis into a single benchmark. Our
goal is to find an optimal parameterization, i.e., one that strictly dominates
others. To this end, we now plot size against speed:

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
cleaned <- unified |>
Expand Down Expand Up @@ -848,7 +860,8 @@ cloud is unwieldy for relative comparison. To better represent the variance of
schemas for a given configuration, we can strip the “inner” points and only look
at their convex hull:

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
# Native convex hull does the job, no need to leverage ggforce.
Expand Down Expand Up @@ -879,7 +892,8 @@ is “good” in that it offers more predictable behavior. The high amount of
overlap makes it still difficult to perform clearer comparisons. If we facet by
store type, it becomes easier to compare the areas:

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
cleaned |>
Expand Down Expand Up @@ -909,7 +923,8 @@ Arranging the facets above row-wise makes it easier to compare the y-axis, i.e.,
speed, where lower polygons are better. Arranging them column-wise makes it easier
to compare the x-axis, i.e., size, where the left-most polygons are better:

<details><summary>Code</summary>
<details>
<summary>Code</summary>

``` r
cleaned |>
Expand Down
13 changes: 7 additions & 6 deletions web/blog/public-roadmap-and-open-rfcs/index.md
Expand Up @@ -58,14 +58,15 @@ existing environment. Our GitHub presence includes code, documentation, website
content, and third-party integrations. We also promote use of GitHub Discussions
to engage with us. This makes GitHub the focal point to engage with the content.
Therefore, we decided to encode the roadmap as GitHub issues; for clarity in a
dedicated repository at <https://github.com/tenzir/public-roadmap>.
dedicated repository at
[tenzir/public-roadmap](https://github.com/tenzir/public-roadmap).

We decided against dual-purposing the issue tracker of our main repository
<https://github.com/tenzir/vast> because it would add roadmap items as many
open, long-running issues that scatter the attention and potentially confuse the
community. That said, the primary value of the issue tracker is the layer on top
of issues: [GitHub Projects][github-projects], which allows for organizing
issues across multiple dimensions in a visually appealing way.
[tenzir/vast](https://github.com/tenzir/vast) because it would add roadmap items
as many open, long-running issues that scatter the attention and potentially
confuse the community. That said, the primary value of the issue tracker is the
layer on top of issues: [GitHub Projects][github-projects], which allows for
organizing issues across multiple dimensions in a visually appealing way.

[github-projects]: https://docs.github.com/en/issues/planning-and-tracking-with-projects

Expand Down
7 changes: 2 additions & 5 deletions web/docs/contribute/documentation.md
Expand Up @@ -20,10 +20,6 @@ Then build and serve the site via:
```bash
yarn start
```

Browse to <http://localhost:3000/> to view the site. Docusaurus should spawn
your default browser automatically upon invoking `yarn start`.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this no longer needed?

## Write content

Docusaurus uses an [enhanced flavor of
Expand All @@ -50,7 +46,8 @@ PNG or SVG images.

Our editing workflow looks as follows:

1. Open <https://excalidraw.com> and click *Upload* in the top left
1. Open [excalidraw.com](https://excalidraw.com) and click *Upload* in the top
left
2. Select the SVG you would like to edit
3. Make your edits in Excalidraw
4. Uncheck the box "Background" to ensure a transparent background.
Expand Down
4 changes: 3 additions & 1 deletion web/docs/get-started.md
Expand Up @@ -12,7 +12,9 @@ platform:
3. **Platform**: Manages nodes and global user data. Nodes connect to the
platform and you can manage them through [app.tenzir.com][app].

![Tenzir Moving Parts](moving-parts.excalidraw.svg)
import MovingParts from './moving-parts.excalidraw.svg'

<MovingParts className="excalidraw-svg" alt="Tenzir Moving Parts" />

[app]: https://app.tenzir.com

Expand Down
2 changes: 1 addition & 1 deletion web/docs/integrations/splunk.md
Expand Up @@ -113,7 +113,7 @@ volumes:

After you spun up the containers, configure Splunk as follows:

1. Go to <http://localhost:8000> and login with `admin`:`tenzir123`
`admin`:`tenzir123`
2. Navigate to *Add data* → *Monitor* → *HTTP Event Collector*
3. Configure the event collector:
- Name: Tenzir
Expand Down