{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":341674352,"defaultBranch":"master","name":"cockroach","ownerLogin":"abarganier","currentUserCanPush":false,"isFork":true,"isEmpty":false,"createdAt":"2021-02-23T20:00:29.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/8194877?v=4","public":true,"private":false,"isOrgOwned":false},"refInfo":{"name":"","listCacheKey":"v0:1718031449.0","currentOid":""},"activityList":{"items":[{"before":"001010e5b342ad76eb0578876b1c0bbe5f8403f3","after":"85dbe3f646b5000251a90f284aea39c1988485d8","ref":"refs/heads/buffsink-tech-note","pushedAt":"2024-06-11T18:17:05.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/docs: add tech-note for `bufferedSink` in `pkg/util/log`\n\nThis commit adds a tech note explaining the core concepts behind the\n`bufferedSink`, which is the primary async log buffering mechanism used\nwithin `pkg/util/log`.\n\nRelease note: none","shortMessageHtmlLink":"pkg/docs: add tech-note for <code>bufferedSink</code> in <code>pkg/util/log</code>"}},{"before":"533372d85fedee4444b16c2b77d1cd93de5a1697","after":"3a3acc9860e7cd96286639fa1e99450c51c53dbd","ref":"refs/heads/flush-test-fix","pushedAt":"2024-06-10T15:21:29.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/obs/logstream: increase timeout used in TestBlockedFlush\n\nAddresses: https://github.com/cockroachdb/cockroach/issues/125290\n\nTestBlockedFlush only provided 100 milliseconds of tolerance for a\ngoroutine to be created, make a call to\n`(*asyncProcessorRouter).Process()`, and then signal on a channel.\nUnder nightly stress, the test was timing out because the goroutine\nfailed to signal on the channel within 100 milliseconds, which certainly\nfeels possible if the test is running under nightly stress.\n\nThis patch simply updates the test with a longer timeout (10ms -> 10s)\nto make the timeout less likely when run under stress.\n\nRelease note: none","shortMessageHtmlLink":"pkg/obs/logstream: increase timeout used in TestBlockedFlush"}},{"before":null,"after":"533372d85fedee4444b16c2b77d1cd93de5a1697","ref":"refs/heads/flush-test-fix","pushedAt":"2024-06-10T14:57:29.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/obs/logstream: increase timeout used in TestBlockedFlush\n\nAddresses: https://github.com/cockroachdb/cockroach/issues/125290\n\nTestBlockedFlush only provided 100 milliseconds of tolerance for a\ngoroutine to be created, make a call to\n`(*asyncProcessorRouter).Process()`, and then signal on a channel.\nUnder nightly stress, the test was timing out because the goroutine\nfailed to signal on the channel within 100 milliseconds, which certainly\nfeels possible if the test is running under nightly stress.\n\nThis patch simply updates the test with a longer timeout (10ms -> 10s)\nto make the timeout less likely when run under stress.\n\nRelease note: none","shortMessageHtmlLink":"pkg/obs/logstream: increase timeout used in TestBlockedFlush"}},{"before":null,"after":"94a210ab6ead184ab2c1b4a5e56bacc1f712a77d","ref":"refs/heads/backport24.1.0-rc-124219","pushedAt":"2024-06-07T15:48:56.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"log: fix race in TestLogEntryPropagation\n\nFixes: https://github.com/cockroachdb/cockroach/issues/122791\n\nTestLogEntryPropagation was unsafely reassigning `OrigStderr` in the log\npackage which was tripping the race detector.\n\nThis patch makes use of the existing `hijackStderr` function which\nallows us to safely reassign stderr.\n\nRelease note: none","shortMessageHtmlLink":"log: fix race in TestLogEntryPropagation"}},{"before":"9e0e2080b31d4d7cf02de3ec1095701bfa4f9d10","after":"56faec2d873c52c139dd6e2084aa8c2ed9fe21d7","ref":"refs/heads/skip-logstream-test","pushedAt":"2024-06-07T15:36:38.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/obs/logstream: skip TestBlockedFlush\n\nRelates to: https://github.com/cockroachdb/cockroach/issues/125290\n\nThere seems to be some kind of deadlock happening in the test related to\nthe channels used to coordinate the various goroutines involved.\n\nWe've been unable to reproduce this locally, so it will require some\ndeeper investigation. In the meantime, we'll skip the test to unblock\nmaster while we investigate.\n\nRelease note: none","shortMessageHtmlLink":"pkg/obs/logstream: skip TestBlockedFlush"}},{"before":null,"after":"9e0e2080b31d4d7cf02de3ec1095701bfa4f9d10","ref":"refs/heads/skip-logstream-test","pushedAt":"2024-06-07T15:35:25.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/obs/logstream: skip TestBlockedFlush\n\nRelates to: https://github.com/cockroachdb/cockroach/issues/125290\n\nThere seems to be some kind of deadlock happening in the test related to\nthe channels used to coordinate the various goroutines involved.\n\nWe've been unable to reproduce this locally, so it will require some\ndeeper investigation. In the meantime, we'll skip the test to unblock\nmaster while we investigate.\n\nRelease note: none","shortMessageHtmlLink":"pkg/obs/logstream: skip TestBlockedFlush"}},{"before":null,"after":"63a936eafcec631206c7e16983e64471bc28b912","ref":"refs/heads/hotrange-23.2","pushedAt":"2024-06-06T20:55:54.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/server: remove flaky assertion from TestHotRangesResponse (23.2)\n\nFixes: https://github.com/cockroachdb/cockroach/issues/125222\n\nThe assertions removed in this patch are known to be flaky. The root of\nthe problem was addressed in\nhttps://github.com/cockroachdb/cockroach/pull/119723, but due to\nthe fact that the changes affected KV QPS calculations in a critical\nway, we decided not to backport.\n\nThe result is that older release branches continue to experience flakes\nfor this test, which has become quite noisy.\n\nOur solution is to remove the flaky assertions while leaving the\nremainder of the test.\n\nRelease note: none","shortMessageHtmlLink":"pkg/server: remove flaky assertion from TestHotRangesResponse (23.2)"}},{"before":null,"after":"c173c153a30acb73df9e0d5c092bcf366c0571c7","ref":"refs/heads/hotrange-23.1","pushedAt":"2024-06-06T20:52:02.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/server: remove flaky assertion from TestHotRangesResponse (23.1)\n\nFixes: https://github.com/cockroachdb/cockroach/issues/125222\n\nThe assertions removed in this patch are known to be flaky. The root of\nthe problem was addressed in\nhttps://github.com/cockroachdb/cockroach/pull/119723, but due to\nthe fact that the changes affected KV QPS calculations in a critical\nway, we decided not to backport.\n\nThe result is that older release branches continue to experience flakes\nfor this test, which has become quite noisy.\n\nOur solution is to remove the flaky assertions while leaving the\nremainder of the test.\n\nRelease note: none","shortMessageHtmlLink":"pkg/server: remove flaky assertion from TestHotRangesResponse (23.1)"}},{"before":null,"after":"001010e5b342ad76eb0578876b1c0bbe5f8403f3","ref":"refs/heads/buffsink-tech-note","pushedAt":"2024-06-05T21:20:02.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/docs: add tech-note for `bufferedSink` in `pkg/util/log`\n\nThis commit adds a tech note explaining the core concepts behind the\n`bufferedSink`, which is the primary async log buffering mechanism used\nwithin `pkg/util/log`.\n\nRelease note: none","shortMessageHtmlLink":"pkg/docs: add tech-note for <code>bufferedSink</code> in <code>pkg/util/log</code>"}},{"before":"0f55f80c4c9701f150121639ac936628cad30039","after":"39ba92b5e79517ded7bf904fa2997e48724f6191","ref":"refs/heads/structured-log-frmwk","pushedAt":"2024-06-05T20:09:45.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/obs/logstream: introduce mechanism to consume structured logs internally\n\nThis patch expands upon the new structured logging facilities, adding a\nmechanism to consume emitted structured logs internally.\n\nThis is done primarily via the new pkg/obs/logstream package, which\nhandles buffering, routing, & processing of events logged via\nlog.Structured.\n\nIt can be used in conjunction with the eventagg package, and the\nKVProcessor interface, to provide users of the eventagg package a way\nto consume streams of events flushed from their aggregations. This\nenables engineers to use the aggregated data flushed from their\naggregations to build features internal to CRDB. Features that are\npowered by the same data that could be consumed externally via the\nSTRUCTURED_EVENTS log channel.\n\nThe provided log config can be updated to make use of this new channel.\nFor example:\n```\nsinks:\n  file-groups:\n    structured-events:\n      channels: [STRUCTURED_EVENTS]\n```\n\nThe changes aim to complete the eventagg pipeline/ecosystem, which now\nallows engineers to use common facilities to define aggregations, log\nthe aggregated results, and consume the logged events internally as\ninput data.\n\nFinally, it completes the toy StmtStats example by defining a processor\nfor the aggregated events that are logged.\n\nRelease note: none","shortMessageHtmlLink":"pkg/obs/logstream: introduce mechanism to consume structured logs int…"}},{"before":"236f2f83d3dd19a4e971bb18936b5b4a14d22890","after":"0f55f80c4c9701f150121639ac936628cad30039","ref":"refs/heads/structured-log-frmwk","pushedAt":"2024-06-04T21:03:58.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/obs/logstream: introduce mechanism to consume structured logs internally\n\nThis patch expands upon the new structured logging facilities, adding a\nmechanism to consume emitted structured logs internally.\n\nThis is done primarily via the new pkg/obs/logstream package, which\nhandles buffering, routing, & processing of events logged via\nlog.Structured.\n\nIt can be used in conjunction with the eventagg package, and the\nKVProcessor interface, to provide users of the eventagg package a way\nto consume streams of events flushed from their aggregations. This\nenables engineers to use the aggregated data flushed from their\naggregations to build features internal to CRDB. Features that are\npowered by the same data that could be consumed externally via the\nSTRUCTURED_EVENTS log channel.\n\nThe provided log config can be updated to make use of this new channel.\nFor example:\n```\nsinks:\n  file-groups:\n    structured-events:\n      channels: [STRUCTURED_EVENTS]\n```\n\nThe changes aim to complete the eventagg pipeline/ecosystem, which now\nallows engineers to use common facilities to define aggregations, log\nthe aggregated results, and consume the logged events internally as\ninput data.\n\nFinally, it completes the toy StmtStats example by defining a processor\nfor the aggregated events that are logged.\n\nRelease note: none","shortMessageHtmlLink":"pkg/obs/logstream: introduce mechanism to consume structured logs int…"}},{"before":"9b61fa7da1e83ab54dfcaf8cee50ffc24253047f","after":"dfde8af2186d93c390437bb281a926f66c05066c","ref":"refs/heads/jobs-hist-poc","pushedAt":"2024-06-03T16:01:03.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"kyle-a-wong","name":"Kyle Wong","path":"/kyle-a-wong","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37189875?s=80&v=4"},"commit":{"message":"WIP: Add structured log processor","shortMessageHtmlLink":"WIP: Add structured log processor"}},{"before":"b4256ba0b1580c1a695d0161fbf569acc62c6c35","after":"236f2f83d3dd19a4e971bb18936b5b4a14d22890","ref":"refs/heads/structured-log-frmwk","pushedAt":"2024-05-31T16:32:51.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"kyle-a-wong","name":"Kyle Wong","path":"/kyle-a-wong","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37189875?s=80&v=4"},"commit":{"message":"pkg/obs/logstream: introduce mechanism to consume structured logs internally\n\nThis patch expands upon the new structured logging facilities, adding a\nmechanism to consume emitted structured logs internally.\n\nThis is done primarily via the new pkg/obs/logstream package, which\nhandles buffering, routing, & processing of events logged via\nlog.Structured.\n\nIt can be used in conjunction with the eventagg package, and the\nKVProcessor interface, to provide users of the eventagg package a way\nto consume streams of events flushed from their aggregations. This\nenables engineers to use the aggregated data flushed from their\naggregations to build features internal to CRDB. Features that are\npowered by the same data that could be consumed externally via the\nSTRUCTURED_EVENTS log channel.\n\nThe provided log config can be updated to make use of this new channel.\nFor example:\n```\nsinks:\n  file-groups:\n    structured-events:\n      channels: [STRUCTURED_EVENTS]\n```\n\nThe changes aim to complete the eventagg pipeline/ecosystem, which now\nallows engineers to use common facilities to define aggregations, log\nthe aggregated results, and consume the logged events internally as\ninput data.\n\nFinally, it completes the toy StmtStats example by defining a processor\nfor the aggregated events that are logged.\n\nRelease note: none","shortMessageHtmlLink":"pkg/obs/logstream: introduce mechanism to consume structured logs int…"}},{"before":"be28df82e0b229ecd2b53543857d517d5851062f","after":"9b61fa7da1e83ab54dfcaf8cee50ffc24253047f","ref":"refs/heads/jobs-hist-poc","pushedAt":"2024-05-22T21:31:03.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"kyle-a-wong","name":"Kyle Wong","path":"/kyle-a-wong","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37189875?s=80&v=4"},"commit":{"message":"jobs: job state change processor wip","shortMessageHtmlLink":"jobs: job state change processor wip"}},{"before":"8078238686f2a399787721909401abd1f9f0b282","after":"37cd3b5d4f45b465e446628d01b5adb36dd0a8d0","ref":"refs/heads/stderr-concurrent","pushedAt":"2024-05-21T18:39:49.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"log: fix race in TestLogEntryPropagation\n\nFixes: https://github.com/cockroachdb/cockroach/issues/122791\n\nTestLogEntryPropagation was unsafely reassigning `OrigStderr` in the log\npackage which was tripping the race detector.\n\nThis patch makes use of the existing `hijackStderr` function which\nallows us to safely reassign stderr.\n\nRelease note: none","shortMessageHtmlLink":"log: fix race in TestLogEntryPropagation"}},{"before":"0b7b18a57ee9ad563fdb2e48c32649968af0b115","after":"8078238686f2a399787721909401abd1f9f0b282","ref":"refs/heads/stderr-concurrent","pushedAt":"2024-05-21T16:56:07.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"log: fix race in TestLogEntryPropagation\n\nFixes: https://github.com/cockroachdb/cockroach/issues/122791\n\nTestLogEntryPropagation was unsafely reassigning `OrigStderr` in the log\npackage which was tripping the race detector.\n\nThis patch makes use of the existing `hijackStderr` function which\nallows us to safely reassign stderr.\n\nRelease note: none","shortMessageHtmlLink":"log: fix race in TestLogEntryPropagation"}},{"before":"1188486f430d152a05daaa0a82f2c5f138bb37f2","after":"0b7b18a57ee9ad563fdb2e48c32649968af0b115","ref":"refs/heads/stderr-concurrent","pushedAt":"2024-05-21T16:55:02.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"log: fix race in TestLogEntryPropagation\n\nFixes: https://github.com/cockroachdb/cockroach/issues/122791\n\nTestLogEntryPropagation was unsafely reassigning `OrigStderr` in the log\npackage which was tripping the race detector.\n\nThis patch makes use of the existing `redirectStderr` function which\nallows us to safely reassign stderr.\n\nRelease note: none","shortMessageHtmlLink":"log: fix race in TestLogEntryPropagation"}},{"before":"cafcf37e8ae1891b88832982c571fb9dd76ce70b","after":"1188486f430d152a05daaa0a82f2c5f138bb37f2","ref":"refs/heads/stderr-concurrent","pushedAt":"2024-05-21T16:54:16.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"log: fix race in TestLogEntryPropagation\n\nTestLogEntryPropagation was unsafely reassigning `OrigStderr` in the log\npackage which was tripping the race detector.\n\nThis patch makes use of the existing `redirectStderr` function which\nallows us to safely reassign stderr.\n\nRelease note: none","shortMessageHtmlLink":"log: fix race in TestLogEntryPropagation"}},{"before":"8e510b5f2c6eefdd766843d5e516f6d65bcd0334","after":"cafcf37e8ae1891b88832982c571fb9dd76ce70b","ref":"refs/heads/stderr-concurrent","pushedAt":"2024-05-21T14:50:05.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/util/log: make use of OrigStderr thread safe\n\nFixes: https://github.com/cockroachdb/cockroach/issues/122791\n\nThe log package exports a variable called `OrigStderr` which is meant\nto represent the original stderr `os.File` for the process at startup.\n\nVarious operations in the log package may \"hijack\" the stderr, for\nexample, to redirect the process's stderr logs to a file instead.\nThe package maintains a reference to the original stderr in order to\nmaintain the ability to still make use of it under certain\ncircumstances while still redirecting stderr to a file.\n\nA race condition was appearing in tests that was reassigning this\nvariable while it was in use by the process.\n\nThis patch fixes the race condition by wrapping the os.File in a mutex,\nand then implementing the `io.Writer` interface on the wrapper type.\nSwapping out the underlying file can now be an atomic operation, and we\nalso expose an API to perform atomic operations on the underlying file.\nA RWMutex is used to reduce contention.\n\nRelease note: none","shortMessageHtmlLink":"pkg/util/log: make use of OrigStderr thread safe"}},{"before":"2a312932b3297c8b180f44c6d991f37830851cc8","after":"b4256ba0b1580c1a695d0161fbf569acc62c6c35","ref":"refs/heads/structured-log-frmwk","pushedAt":"2024-05-16T18:50:28.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/obs/logstream: introduce mechanism to consume structured logs internally\n\nThis patch expands upon the new structured logging facilities, adding a\nmechanism to consume emitted structured logs internally.\n\nThis is done primarily via the new pkg/obs/logstream package, which\nhandles buffering, routing, & processing of events logged via\nlog.Structured.\n\nIt can be used in conjunction with the eventagg package, and the\nKVProcessor interface, to provide users of the eventagg package a way\nto consume streams of events flushed from their aggregations. This\nenables engineers to use the aggregated data flushed from their\naggregations to build features internal to CRDB. Features that are\npowered by the same data that could be consumed externally via the\nSTRUCTURED_EVENTS log channel.\n\nThe provided log config can be updated to make use of this new channel.\nFor example:\n```\nsinks:\n  file-groups:\n    structured-events:\n      channels: [STRUCTURED_EVENTS]\n```\n\nThe changes aim to complete the eventagg pipeline/ecosystem, which now\nallows engineers to use common facilities to define aggregations, log\nthe aggregated results, and consume the logged events internally as\ninput data.\n\nFinally, it completes the toy StmtStats example by defining a processor\nfor the aggregated events that are logged.\n\nRelease note: none","shortMessageHtmlLink":"pkg/obs/logstream: introduce mechanism to consume structured logs int…"}},{"before":"87c882719621a23b48dc449d38eb89a2b3347f58","after":"e87a48929f525a9f74bfc04439a594dd5bbe157c","ref":"refs/heads/antenna-poc-sql-stats","pushedAt":"2024-05-16T17:59:29.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"eventagg: add FlushTrigger mechanism to MapReduceAggregator\n\nThis patch introduces the FlushTrigger interface, which can be used by\nthe MapReduceAggregator to determine when it's time to flush the current\naggregation.\n\nAlong with the interface, an initial implementation is provided called\n`WindowedFlush`. `WindowedFlush` aligns event aggregations to truncated\ntime intervals given a user-provided time window.\n\nFor example, if a window of 5 minutes was given, the `WindowedFlush`\nwould enforce the following window boundaries:\n\n- [12:00:00, 12:05:00)\n- [12:05:00, 12:10:00)\n- [12:10:00, 12:15:00)\n- etc.\n\nThis is a first pass implementation of the flush mechanism used in\nthe eventagg package. As needs evolve, the interface and/or\nimplementation is subject to change. For the purposed of prototyping\nthough, this meets our needs.\n\nRelease note: none","shortMessageHtmlLink":"eventagg: add FlushTrigger mechanism to MapReduceAggregator"}},{"before":null,"after":"be28df82e0b229ecd2b53543857d517d5851062f","ref":"refs/heads/jobs-hist-poc","pushedAt":"2024-05-15T19:35:54.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"jobs: structured log wip","shortMessageHtmlLink":"jobs: structured log wip"}},{"before":"86d7c0148ea720dd528107f832aa4f14e86d98c7","after":"8e510b5f2c6eefdd766843d5e516f6d65bcd0334","ref":"refs/heads/stderr-concurrent","pushedAt":"2024-05-15T17:30:02.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/util/log: make use of OrigStderr thread safe\n\nFixes: https://github.com/cockroachdb/cockroach/issues/122791\n\nThe log package exports a variable called `OrigStderr` which is meant\nto represent the original stderr `os.File` for the process at startup.\n\nVarious operations in the log package may \"hijack\" the stderr, for\nexample, to redirect the process's stderr logs to a file instead.\nThe package maintains a reference to the original stderr in order to\nmaintain the ability to still make use of it under certain\ncircumstances while still redirecting stderr to a file.\n\nA race condition was appearing in tests that was reassigning this\nvariable while it was in use by the process.\n\nThis patch fixes the race condition by wrapping the os.File in a mutex,\nand then implementing the `io.Writer` interface on the wrapper type.\nSwapping out the underlying file can now be an atomic operation, and we\nalso expose an API to perform atomic operations on the underlying file.\nA RWMutex is used to reduce contention.\n\nRelease note: none","shortMessageHtmlLink":"pkg/util/log: make use of OrigStderr thread safe"}},{"before":"424e6100e5f7e7c1004b82685fc59ccd1d9efdc9","after":"86d7c0148ea720dd528107f832aa4f14e86d98c7","ref":"refs/heads/stderr-concurrent","pushedAt":"2024-05-15T17:28:05.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/util/log: make use of OrigStderr thread safe\n\nFixes: https://github.com/cockroachdb/cockroach/issues/122791\n\nThe log package exports a variable called `OrigStderr` which is meant\nto represent the original stderr `os.File` for the process at startup.\n\nVarious operations in the log package may \"hijack\" the stderr, for\nexample, to redirect the process's stderr logs to a file instead.\nThe package maintains a reference to the original stderr in order to\nmaintain the ability to still make use of it under certain\ncircumstances while still redirecting stderr to a file.\n\nA race condition was appearing in tests that was reassigning this\nvariable while it was in use by the process.\n\nThis patch fixes the race condition by wrapping the os.File in a mutex,\nand then implementing the `io.Writer` interface on the wrapper type.\nSwapping out the underlying file can now be an atomic operation, and we\nalso expose an API to perform atomic operations on the underlying file.\nA RWMutex is used to reduce contention.\n\nRelease note: none","shortMessageHtmlLink":"pkg/util/log: make use of OrigStderr thread safe"}},{"before":null,"after":"424e6100e5f7e7c1004b82685fc59ccd1d9efdc9","ref":"refs/heads/stderr-concurrent","pushedAt":"2024-05-15T15:15:01.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/util/log: make use of OrigStderr thread safe\n\nFixes: https://github.com/cockroachdb/cockroach/issues/122791\n\nThe log package exports a variable called `OrigStderr` which is meant\nto represent the original stderr `os.File` for the process at startup.\n\nVarious operations in the log package may \"hijack\" the stderr, for\nexample, to redirect the process's stderr logs to a file instead.\nThe package maintains a reference to the original stderr in order to\nmaintain the ability to still make use of it under certain\ncircumstances while still redirecting stderr to a file.\n\nA race condition was appearing in tests that was reassigning this\nvariable while it was in use by the process.\n\nThis patch fixes the race condition by wrapping the os.File in a mutex,\nand then implementing the `io.Writer` interface on the wrapper type.\nSwapping out the underlying file can now be an atomic operation, and we\nalso expose an API to perform atomic operations on the underlying file.\nA RWMutex is used to reduce contention.\n\nRelease note: none","shortMessageHtmlLink":"pkg/util/log: make use of OrigStderr thread safe"}},{"before":"6a92c01149a3c78f2f87ea5d293bfa132bb8fd64","after":"87c882719621a23b48dc449d38eb89a2b3347f58","ref":"refs/heads/antenna-poc-sql-stats","pushedAt":"2024-05-13T18:53:15.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"eventagg: add FlushTrigger mechanism to MapReduceAggregator\n\nThis patch introduces the FlushTrigger interface, which can be used by\nthe MapReduceAggregator to determine when it's time to flush the current\naggregation.\n\nAlong with the interface, an initial implementation is provided called\n`WindowedFlush`. `WindowedFlush` aligns event aggregations to truncated\ntime intervals given a user-provided time window.\n\nFor example, if a window of 5 minutes was given, the `WindowedFlush`\nwould enforce the following window boundaries:\n\n- [12:00:00, 12:05:00)\n- [12:05:00, 12:10:00)\n- [12:10:00, 12:15:00)\n- etc.\n\nThis is a first pass implementation of the flush mechanism used in\nthe eventagg package. As needs evolve, the interface and/or\nimplementation is subject to change. For the purposed of prototyping\nthough, this meets our needs.\n\nRelease note: none","shortMessageHtmlLink":"eventagg: add FlushTrigger mechanism to MapReduceAggregator"}},{"before":"92480128b3cf2fc753cd185850ba9ba02babeef0","after":"c52aa2b5e028af41424742037e8b7ae04dd63ab0","ref":"refs/heads/backport23.2-120522","pushedAt":"2024-05-13T18:47:06.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/sql/sqlstats: counter metric for flushed fingerprint counts\n\nAddresses: https://github.com/cockroachdb/cockroach/issues/119779\n\nThe count of unique fingeprints flushed to `system.statement_statistics`\nand `system.transaction_statistics` is the core component that\ndetermines data cardinality within the SQL stats subsystem. Today, we\ndon't have good metrics around this source of cardinality. As we aim to\nreduce cardinality by improving our fingerprinting algorithms, creating\na metric to count the number of unique statement and transaction\nfingerprints included in each flush of the in-memory SQL stats will be a\nhelpful measurement to benchmark cardinality reduction.\n\nThis patch adds a new metric to track the # of unique fingerprints (stmt\nand txn) included in each flush.\n\nRelease note (ops change): A new counter metric,\n`sql.stats.flush.fingerprint.count`, has been introduced. The metric\ntracks the number of unique statement and transaction fingerprints\nincluded in the SQL Stats flush.","shortMessageHtmlLink":"pkg/sql/sqlstats: counter metric for flushed fingerprint counts"}},{"before":"de3b8526bfbda4b29fedd6a696868c12c5055a72","after":"92480128b3cf2fc753cd185850ba9ba02babeef0","ref":"refs/heads/backport23.2-120522","pushedAt":"2024-05-13T18:13:25.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"pkg/sql/sqlstats: counter metric for flushed fingerprint counts\n\nAddresses: https://github.com/cockroachdb/cockroach/issues/119779\n\nThe count of unique fingeprints flushed to `system.statement_statistics`\nand `system.transaction_statistics` is the core component that\ndetermines data cardinality within the SQL stats subsystem. Today, we\ndon't have good metrics around this source of cardinality. As we aim to\nreduce cardinality by improving our fingerprinting algorithms, creating\na metric to count the number of unique statement and transaction\nfingerprints included in each flush of the in-memory SQL stats will be a\nhelpful measurement to benchmark cardinality reduction.\n\nThis patch adds a new metric to track the # of unique fingerprints (stmt\nand txn) included in each flush.\n\nRelease note (ops change): A new counter metric,\n`sql.stats.flush.fingerprint.count`, has been introduced. The metric\ntracks the number of unique statement and transaction fingerprints\nincluded in the SQL Stats flush.","shortMessageHtmlLink":"pkg/sql/sqlstats: counter metric for flushed fingerprint counts"}},{"before":"96a1e2e25e2894b61aeb37ac987b47625daf2db0","after":"6a92c01149a3c78f2f87ea5d293bfa132bb8fd64","ref":"refs/heads/antenna-poc-sql-stats","pushedAt":"2024-05-13T17:54:26.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"eventagg: add FlushTrigger mechanism to MapReduceAggregator\n\nThis patch introduces the FlushTrigger interface, which can be used by\nthe MapReduceAggregator to determine when it's time to flush the current\naggregation.\n\nAlong with the interface, an initial implementation is provided called\n`WindowedFlush`. `WindowedFlush` aligns event aggregations to truncated\ntime intervals given a user-provided time window.\n\nFor example, if a window of 5 minutes was given, the `WindowedFlush`\nwould enforce the following window boundaries:\n\n- [12:00:00, 12:05:00)\n- [12:05:00, 12:10:00)\n- [12:10:00, 12:15:00)\n- etc.\n\nThis is a first pass implementation of the flush mechanism used in\nthe eventagg package. As needs evolve, the interface and/or\nimplementation is subject to change. For the purposed of prototyping\nthough, this meets our needs.\n\nRelease note: none","shortMessageHtmlLink":"eventagg: add FlushTrigger mechanism to MapReduceAggregator"}},{"before":null,"after":"2a312932b3297c8b180f44c6d991f37830851cc8","ref":"refs/heads/structured-log-frmwk","pushedAt":"2024-05-13T17:54:09.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"abarganier","name":"Alex Barganier","path":"/abarganier","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8194877?s=80&v=4"},"commit":{"message":"eventagg: introduce mechanism to consume structured logs internally\n\nThis patch expands upon the new structured logging facilities, adding a\nmechanism to consume emitted structured logs internally.\n\nThis is done primarily via the new pkg/obs/logstream package, which\nhandles buffering, routing, & processing of events logged via\nlog.Structured.\n\nIt can be used in conjunction with the eventagg package, and the\nKVProcessor interface, to provide users of the eventagg package a way\nto consume streams of events flushed from their aggregations. This\nenables engineers to use the aggregated data flushed from their\naggregations to build features internal to CRDB. Features that are\npowered by the same data that could be consumed externally via the\nSTRUCTURED_EVENTS log channel.\n\nThe provided log config can be updated to make use of this new channel.\nFor example:\n```\nsinks:\n  file-groups:\n    structured-events:\n      channels: [STRUCTURED_EVENTS]\n```\n\nThe changes aim to complete the eventagg pipeline/ecosystem, which now\nallows engineers to use common facilities to define aggregations, log\nthe aggregated results, and consume the logged events internally as\ninput data.\n\nFinally, it completes the toy StmtStats example by defining a processor\nfor the aggregated events that are logged.\n\nRelease note: none","shortMessageHtmlLink":"eventagg: introduce mechanism to consume structured logs internally"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEYokfpQA","startCursor":null,"endCursor":null}},"title":"Activity · abarganier/cockroach"}