-
Notifications
You must be signed in to change notification settings - Fork 205
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add documentation for Streams API #1576
Conversation
docs/sources/next/javascript-api/k6-experimental/streams/readablestream/_index.md
Outdated
Show resolved
Hide resolved
c5f0471
to
845c2ea
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left a few comments but they're mostly small edits and changes to make it consistent with other parts of the docs. :) There is one empty example in the "releaseLock.md" file.
Overall it looks great, thanks for writing these docs! 🙇
docs/sources/next/javascript-api/k6-experimental/streams/_index.md
Outdated
Show resolved
Hide resolved
docs/sources/next/javascript-api/k6-experimental/streams/_index.md
Outdated
Show resolved
Hide resolved
}, | ||
|
||
// The pull function is called repeatedly to get data, while the | ||
// internal high water mark is not reached. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is "high water mark" referring to the end of the file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The high water mark is a streams-specific concept. It's representing the amount of chunks a stream can pull/contain before blocking.
For instance a stream with a high water mark of five, will holds at most 5 chunks in its internal queue. So if the client is really slow to pull data from the stream, we can control the total memory and CPU usage that the streams consume for instance.
docs/sources/next/javascript-api/k6-experimental/streams/_index.md
Outdated
Show resolved
Hide resolved
docs/sources/next/javascript-api/k6-experimental/streams/_index.md
Outdated
Show resolved
Hide resolved
...urces/next/javascript-api/k6-experimental/streams/readablestreamdefaultcontroller/enqueue.md
Outdated
Show resolved
Hide resolved
...urces/next/javascript-api/k6-experimental/streams/readablestreamdefaultcontroller/enqueue.md
Outdated
Show resolved
Hide resolved
...sources/next/javascript-api/k6-experimental/streams/readablestreamdefaultcontroller/error.md
Outdated
Show resolved
Hide resolved
...sources/next/javascript-api/k6-experimental/streams/readablestreamdefaultcontroller/error.md
Outdated
Show resolved
Hide resolved
docs/sources/next/javascript-api/k6-experimental/streams/readablestream/_index.md
Outdated
Show resolved
Hide resolved
Co-authored-by: Heitor Tashiro Sergent <heitortsergent@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was a bit late with the review, so LGTM, in general, and my comments could be applied optionally in other PRs.
One thing that I feel is important to mention is Streams API from another page, like https://grafana.com/docs/k6/latest/testing-guides/running-large-tests/ where we name options about improved memory consumption.
|
||
```javascript | ||
import { CountQueuingStrategy, ReadableStream } from 'k6/experimental/streams'; | ||
import { setTimeout } from 'k6/timers'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
import { setTimeout } from 'k6/timers'; |
|
||
Streaming involves breaking a resource that one wants to process or consume into smaller chunks, which can be processed or consumed incrementally. This is particularly useful when dealing with large files or data sources, as it allows you to work with the data in smaller, more manageable pieces. | ||
|
||
With the Streams API support in k6, you can start processing raw data with Javascript bit by bit, as soon as it's available, without needing to generate a full in-memory representation of the data. This helps to reduce memory usage and improve performance, especially when working with large data sets. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With the Streams API support in k6, you can start processing raw data with Javascript bit by bit, as soon as it's available, without needing to generate a full in-memory representation of the data. This helps to reduce memory usage and improve performance, especially when working with large data sets. | |
With the Streams API support in k6, you can start processing raw data with JavaScript bit by bit, as soon as it's available, without needing to generate a full in-memory representation of the data. This helps to reduce memory usage and improve performance, especially when working with large data sets. |
|
||
```javascript | ||
import { ReadableStream } from 'k6/experimental/streams'; | ||
import { setTimeout } from 'k6/timers'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
import { setTimeout } from 'k6/timers'; |
|
||
```javascript | ||
import { ReadableStream } from 'k6/experimental/streams'; | ||
import { setTimeout } from 'k6/timers'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
import { setTimeout } from 'k6/timers'; |
|
||
```javascript | ||
import { ReadableStream } from 'k6/experimental/streams'; | ||
import { setTimeout } from 'k6/timers'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
import { setTimeout } from 'k6/timers'; |
|
||
```javascript | ||
import { ReadableStream } from 'k6/experimental/streams'; | ||
import { setTimeout } from 'k6/timers'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
import { setTimeout } from 'k6/timers'; |
|
||
```javascript | ||
import { ReadableStream } from 'k6/experimental/streams'; | ||
import { setTimeout } from 'k6/timers'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
import { setTimeout } from 'k6/timers'; |
|
||
```javascript | ||
import { ReadableStream } from 'k6/experimental/streams'; | ||
import { setTimeout } from 'k6/timers'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
import { setTimeout } from 'k6/timers'; |
|
||
## Returns | ||
|
||
A promise which fullfils or rejects with a value depending on the state of the stream: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A promise which fullfils or rejects with a value depending on the state of the stream: | |
A promise which fulfills or rejects with a value depending on the state of the stream: |
I'll address your comments in a separate PR 🙏🏻 🙇🏻 |
What?
This Pull Request adds documentation for the new
k6/experimental/streams
module.Checklist
npm start
command locally and verified that the changes look good.docs/sources/next
folder of the documentation.