Add support for streaming logs for multiple modules at the same time #1033
Conversation
|
Is it possible to get all of the API changes together in one place so that they can be reviewed? |
|
If the |
|
Yes, we can call it pattern. Should we call it the same in the json as well? I was wondering if that might cause more confusion.. A "pattern" like "edgeAgent" or "edgeHub" works. So in the simplest case, users can simply use the module Id (which is why I called it Id). For advanced users, the id can act as a pattern as well. |
| => this.clientWebSocket.CloseAsync(closeStatus, statusDescription, cancellationToken); | ||
| public async Task CloseAsync(WebSocketCloseStatus closeStatus, string statusDescription, CancellationToken cancellationToken) | ||
| { | ||
| using (await this.clientWebSocketLock.LockAsync(cancellationToken)) |
There was a problem hiding this comment.
Is this cancellation token always set to something non-none?
There was a problem hiding this comment.
Yes - for the streaming case, we have a real cancellation token, which is cancelled if the client closes, etc.
| @@ -60,7 +64,8 @@ Task ProcessLogsFrame(ArraySegment<byte> bytes) | |||
|
|
|||
| async Task<LogsStreamRequest> ReadLogsStreamingRequest(IClientWebSocket clientWebSocket, CancellationToken cancellationToken) | |||
| { | |||
| var buf = new byte[1024]; | |||
| // Max size of the request can be 8k | |||
| var buf = new byte[MaxLogRequestSizeBytes]; | |||
There was a problem hiding this comment.
Does this truncate if longer than the max?
There was a problem hiding this comment.
Yes, we only read up to this size. Ideally should never exceed the max size - the requests currently are < 1k, so 8k seemed enough. If we feel, I can increase to 16k?
The EdgeAgent support log streams with the following schema -
{
"schemaVersion": "1.0",
"items": {
[
{
"id": "edge",
"filter": ""
},
{
"id": "temp",
"filter": ""
}
],
"encoding": "gzip",
"contentType": "text"
}
}
The id is a regex. So "edge" would match both edgeHub and edgeAgent.
Logs from all the modules are streamed over the requesting stream.