{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":237328841,"defaultBranch":"master","name":"sdb","ownerLogin":"sdimitro","currentUserCanPush":false,"isFork":true,"isEmpty":false,"createdAt":"2020-01-30T23:53:23.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/2281588?v=4","public":true,"private":false,"isOrgOwned":false},"refInfo":{"name":"","listCacheKey":"v0:1692748121.0","currentOid":""},"activityList":{"items":[{"before":"c509f8ce70a6ecff36b665e249184b0b60fa91b3","after":"907884803c81f522ff8581355f38116bb6e33d20","ref":"refs/heads/task_state","pushedAt":"2023-08-23T00:00:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"DLPX-87579 sdb: use drgn helper for task states\n\n= Problem\n\nCurrently having our own custom function for figuring out a task's\nstate has two drawbacks:\n\n1] As we saw in a2bdd57cf33a08f938b2d67c529d331a2ac1cac6 things can\n   get out of date and it is up to us to fix them.\n\n2] There are some weird edge cases that we don't handle as well\n   like the following crash that I have never been able to reproduce\n   locally but it occasionally reproduces in the Github runners\n   of PR https://github.com/delphix/sdb/pull/337:\n\n```\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/internal/repl.py\", line 107, in eval_cmd\n    for obj in invoke([], input_):\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/pipeline.py\", line 152, in invoke\n    yield from execute_pipeline(first_input, pipeline)\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/pipeline.py\", line 84, in execute_pipeline\n    yield from massage_input_and_call(pipeline[-1], this_input)\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/pipeline.py\", line 67, in massage_input_and_call\n    yield from cmd.call(objs)\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/command.py\", line 413, in call\n    yield from self.__invalid_memory_objects_check(\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/command.py\", line 358, in __invalid_memory_objects_check\n    for obj in objs:\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/command.py\", line 625, in _call\n    self.pretty_print(self.caller(objs))\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/commands/linux/stacks.py\", line 407, in pretty_print\n    self.print_stacks(filter(self.match_stack, objs))\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/commands/linux/stacks.py\", line 382, in print_stacks\n    for stack_key, tasks in KernelStacks.aggregate_stacks(objs):\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/commands/linux/stacks.py\", line 375, in aggregate_stacks\n    stack_key = (KernelStacks.task_struct_get_state(task),\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/commands/linux/stacks.py\", line 221, in task_struct_get_state\n    return KernelStacks.TASK_STATES[(state | exit_state) & 0x7f]\nKeyError: 101\n```\n\n= This Patch\n\nUses the drgn helper whose implementation handles more edge cases\nand is more probable to stay up to date with the latest kernels\nwhile keeping backwards compatibility.\n\n= Testing\n\nThe above stack trace that I was able to reproduce consistently\nin that PR no longer shows up with this patch.","shortMessageHtmlLink":"DLPX-87579 sdb: use drgn helper for task states"}},{"before":"b5f4a2d722ce189c8288a154a80609b71269da1d","after":"c509f8ce70a6ecff36b665e249184b0b60fa91b3","ref":"refs/heads/task_state","pushedAt":"2023-08-22T23:57:35.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"DLPX-87579 sdb: use drgn helper for task states\n\n= Problem\n\nCurrently having our own custom function for figuring out a task's\nstate has two drawbacks:\n\n1] As we saw in a2bdd57cf33a08f938b2d67c529d331a2ac1cac6 things can\n   get out of date and it is up to us to fix them.\n\n2] There are some weird edge cases that we don't handle as well\n   like the following crash that I have never been able to reproduce\n   locally but it occasionally reproduces in the Github runners\n   of PR https://github.com/delphix/sdb/pull/337:\n\n```\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/internal/repl.py\", line 107, in eval_cmd\n    for obj in invoke([], input_):\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/pipeline.py\", line 152, in invoke\n    yield from execute_pipeline(first_input, pipeline)\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/pipeline.py\", line 84, in execute_pipeline\n    yield from massage_input_and_call(pipeline[-1], this_input)\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/pipeline.py\", line 67, in massage_input_and_call\n    yield from cmd.call(objs)\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/command.py\", line 413, in call\n    yield from self.__invalid_memory_objects_check(\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/command.py\", line 358, in __invalid_memory_objects_check\n    for obj in objs:\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/command.py\", line 625, in _call\n    self.pretty_print(self.caller(objs))\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/commands/linux/stacks.py\", line 407, in pretty_print\n    self.print_stacks(filter(self.match_stack, objs))\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/commands/linux/stacks.py\", line 382, in print_stacks\n    for stack_key, tasks in KernelStacks.aggregate_stacks(objs):\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/commands/linux/stacks.py\", line 375, in aggregate_stacks\n    stack_key = (KernelStacks.task_struct_get_state(task),\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/commands/linux/stacks.py\", line 221, in task_struct_get_state\n    return KernelStacks.TASK_STATES[(state | exit_state) & 0x7f]\nKeyError: 101\n```\n\n= This Patch\n\nUses the drgn helper whose implementation handles more edge cases\nand is more probable to stay up to date with the latest kernels\nwhile keeping backwards compatibility.\n\n= Testing\n\nThe above stack trace that I was able to reproduce consistently\nin that PR no longer shows up with this patch.","shortMessageHtmlLink":"DLPX-87579 sdb: use drgn helper for task states"}},{"before":null,"after":"b5f4a2d722ce189c8288a154a80609b71269da1d","ref":"refs/heads/task_state","pushedAt":"2023-08-22T23:48:41.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"DLPX-87579 sdb: use drgn helper for task states\n\n= Problem\n\nCurrently having our own custom function for figuring out a task's\nstate has two drawbacks:\n\n1] As we saw in a2bdd57cf33a08f938b2d67c529d331a2ac1cac6 things can\n   get out of date and it is up to us to fix them.\n\n2] There are some weird edge cases that we don't handle as well\n   like the following crash that I have never been able to reproduce\n   locally but it occasionally reproduces in the Github runners\n   of PR https://github.com/delphix/sdb/pull/337:\n\n```\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/internal/repl.py\", line 107, in eval_cmd\n    for obj in invoke([], input_):\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/pipeline.py\", line 152, in invoke\n    yield from execute_pipeline(first_input, pipeline)\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/pipeline.py\", line 84, in execute_pipeline\n    yield from massage_input_and_call(pipeline[-1], this_input)\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/pipeline.py\", line 67, in massage_input_and_call\n    yield from cmd.call(objs)\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/command.py\", line 413, in call\n    yield from self.__invalid_memory_objects_check(\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/command.py\", line 358, in __invalid_memory_objects_check\n    for obj in objs:\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/command.py\", line 625, in _call\n    self.pretty_print(self.caller(objs))\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/commands/linux/stacks.py\", line 407, in pretty_print\n    self.print_stacks(filter(self.match_stack, objs))\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/commands/linux/stacks.py\", line 382, in print_stacks\n    for stack_key, tasks in KernelStacks.aggregate_stacks(objs):\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/commands/linux/stacks.py\", line 375, in aggregate_stacks\n    stack_key = (KernelStacks.task_struct_get_state(task),\n  File \"/usr/local/lib/python3.8/dist-packages/sdb-0.1.0-py3.8.egg/sdb/commands/linux/stacks.py\", line 221, in task_struct_get_state\n    return KernelStacks.TASK_STATES[(state | exit_state) & 0x7f]\nKeyError: 101\n```\n\n= This Patch\n\nUses the drgn helper whose implementation handles more edge cases\nand is more probable to stay up to date with the latest kernels\nwhile keeping backwards compatibility.\n\n= Testing\n\nThe above stack trace that I was able to reproduce consistently\nin that PR no longer shows up with this patch.","shortMessageHtmlLink":"DLPX-87579 sdb: use drgn helper for task states"}},{"before":"bae41463f2c4173c4161c2041c99bf204aeb7309","after":"fbc215c1b98d89d923b5d56572c848cc073920f3","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-22T23:38:15.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early\n\n= Problem\n\nWith our switch to the new v5.15 kernel a subset of SDB commands\nbroke without us realizing until we actually needed them. Our\nregression dumps helps us to ensure we don't introduce regressions\nfor older kernels when developing new features but they can't help\nus in detecting changes in the upstream kernel or ZFS that break\nour commands.\n\n= This Patch\n\nThis patch attempts to provide a rudimentary mechanism for catching\nregression introduced by the upstream Ubuntu kernels by running a\nfew basic SDB commands in a Github action that's run nightly and\nfor every PR.\n\nSpecifically this patch makes it so we have such a test for each\nUbuntu LTS kernel starting from 20.04 (currently the `ubuntu-latest`\nGithub runner tag points to 22.04 so we'd test that twice but in\nthe future that tag will point to 24.04, etc...).\n\nWe also change for all the available Python versions for each\nUbuntu version to further ensure SDB's compatibility with future\nPython versions.\n\n= Misc Notes\n\nIn order to use SDB in the Github runner I had to introduce an\nextra script that downloads the kernel's debug info. See the\n`install-live-kernel-dbg.sh` script for more info.\n\nI also made sure to decouple the apt-install of the python-dev\nfiles to its own shell script too as different Ubuntu versions\nship with different Python versions. See `install-python-dev.sh`\nfor more info.\n\n= Potential Future Items\n\nIn the future we may want to detect whenever our ZFS commands are\nnot getting out of date. `test_live_kernel.sh` has a way of\ndetecting whether the ZFS module is installed and running a few\nZFS commands on the live kernel. The idea is that we can either\nintroduce Github Actions like the upstream openzfs that install\nour kernel module to the runner and run the commands there OR\nwe can create a BlackBox test that clones the repo and runs this\nscript.","shortMessageHtmlLink":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early"}},{"before":"fbc215c1b98d89d923b5d56572c848cc073920f3","after":"bae41463f2c4173c4161c2041c99bf204aeb7309","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-22T23:14:45.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"stack state WIP","shortMessageHtmlLink":"stack state WIP"}},{"before":"52feab9a73debe0780c8b3bdcee9424c93dd2828","after":"fbc215c1b98d89d923b5d56572c848cc073920f3","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-22T22:31:23.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early\n\n= Problem\n\nWith our switch to the new v5.15 kernel a subset of SDB commands\nbroke without us realizing until we actually needed them. Our\nregression dumps helps us to ensure we don't introduce regressions\nfor older kernels when developing new features but they can't help\nus in detecting changes in the upstream kernel or ZFS that break\nour commands.\n\n= This Patch\n\nThis patch attempts to provide a rudimentary mechanism for catching\nregression introduced by the upstream Ubuntu kernels by running a\nfew basic SDB commands in a Github action that's run nightly and\nfor every PR.\n\nSpecifically this patch makes it so we have such a test for each\nUbuntu LTS kernel starting from 20.04 (currently the `ubuntu-latest`\nGithub runner tag points to 22.04 so we'd test that twice but in\nthe future that tag will point to 24.04, etc...).\n\nWe also change for all the available Python versions for each\nUbuntu version to further ensure SDB's compatibility with future\nPython versions.\n\n= Misc Notes\n\nIn order to use SDB in the Github runner I had to introduce an\nextra script that downloads the kernel's debug info. See the\n`install-live-kernel-dbg.sh` script for more info.\n\nI also made sure to decouple the apt-install of the python-dev\nfiles to its own shell script too as different Ubuntu versions\nship with different Python versions. See `install-python-dev.sh`\nfor more info.\n\n= Potential Future Items\n\nIn the future we may want to detect whenever our ZFS commands are\nnot getting out of date. `test_live_kernel.sh` has a way of\ndetecting whether the ZFS module is installed and running a few\nZFS commands on the live kernel. The idea is that we can either\nintroduce Github Actions like the upstream openzfs that install\nour kernel module to the runner and run the commands there OR\nwe can create a BlackBox test that clones the repo and runs this\nscript.","shortMessageHtmlLink":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early"}},{"before":"2ac14fc23db78b8502cb9b6ea1541bc5d7fa030a","after":"52feab9a73debe0780c8b3bdcee9424c93dd2828","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-22T22:15:15.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early\n\n= Problem\n\nWith our switch to the new v5.15 kernel a subset of SDB commands\nbroke without us realizing until we actually needed them. Our\nregression dumps helps us to ensure we don't introduce regressions\nfor older kernels when developing new features but they can't help\nus in detecting changes in the upstream kernel or ZFS that break\nour commands.\n\n= This Patch\n\nThis patch attempts to provide a rudimentary mechanism for catching\nregression introduced by the upstream Ubuntu kernels by running a\nfew basic SDB commands in a Github action that's run nightly and\nfor every PR.\n\nSpecifically this patch makes it so we have such a test for each\nUbuntu LTS kernel starting from 20.04 (currently the `ubuntu-latest`\nGithub runner tag points to 22.04 so we'd test that twice but in\nthe future that tag will point to 24.04, etc...).\n\nWe also change for all the available Python versions for each\nUbuntu version to further ensure SDB's compatibility with future\nPython versions.\n\n= Misc Notes\n\nIn order to use SDB in the Github runner I had to introduce an\nextra script that downloads the kernel's debug info. See the\n`install-live-kernel-dbg.sh` script for more info.\n\nI also made sure to decouple the apt-install of the python-dev\nfiles to its own shell script too as different Ubuntu versions\nship with different Python versions. See `install-python-dev.sh`\nfor more info.\n\n= Potential Future Items\n\nIn the future we may want to detect whenever our ZFS commands are\nnot getting out of date. `test_live_kernel.sh` has a way of\ndetecting whether the ZFS module is installed and running a few\nZFS commands on the live kernel. The idea is that we can either\nintroduce Github Actions like the upstream openzfs that install\nour kernel module to the runner and run the commands there OR\nwe can create a BlackBox test that clones the repo and runs this\nscript.","shortMessageHtmlLink":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early"}},{"before":"aa27170d61b93a40c37c9314f7284426300276dc","after":"2ac14fc23db78b8502cb9b6ea1541bc5d7fa030a","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-22T22:10:22.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early\n\n= Problem\n\nWith our switch to the new v5.15 kernel a subset of SDB commands\nbroke without us realizing until we actually needed them. Our\nregression dumps helps us to ensure we don't introduce regressions\nfor older kernels when developing new features but they can't help\nus in detecting changes in the upstream kernel or ZFS that break\nour commands.\n\n= This Patch\n\nThis patch attempts to provide a rudimentary mechanism for catching\nregression introduced by the upstream Ubuntu kernels by running a\nfew basic SDB commands in a Github action that's run nightly and\nfor every PR.\n\nSpecifically this patch makes it so we have such a test for each\nUbuntu LTS kernel starting from 20.04 (currently the `ubuntu-latest`\nGithub runner tag points to 22.04 so we'd test that twice but in\nthe future that tag will point to 24.04, etc...).\n\nWe also change for all the available Python versions for each\nUbuntu version to further ensure SDB's compatibility with future\nPython versions.\n\n= Misc Notes\n\nIn order to use SDB in the Github runner I had to introduce an\nextra script that downloads the kernel's debug info. See the\n`install-live-kernel-dbg.sh` script for more info.\n\nI also made sure to decouple the apt-install of the python-dev\nfiles to its own shell script too as different Ubuntu versions\nship with different Python versions. See `install-python-dev.sh`\nfor more info.\n\n= Potential Future Items\n\nIn the future we may want to detect whenever our ZFS commands are\nnot getting out of date. `test_live_kernel.sh` has a way of\ndetecting whether the ZFS module is installed and running a few\nZFS commands on the live kernel. The idea is that we can either\nintroduce Github Actions like the upstream openzfs that install\nour kernel module to the runner and run the commands there OR\nwe can create a BlackBox test that clones the repo and runs this\nscript.","shortMessageHtmlLink":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early"}},{"before":"abf539ebaeaac42ce498ccb95550feba98ef0e7b","after":"aa27170d61b93a40c37c9314f7284426300276dc","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-22T21:42:02.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early\n\n= Problem\n\nWith our switch to the new v5.15 kernel a subset of SDB commands\nbroke without us realizing until we actually needed them. Our\nregression dumps helps us to ensure we don't introduce regressions\nfor older kernels when developing new features but they can't help\nus in detecting changes in the upstream kernel or ZFS that break\nour commands.\n\n= This Patch\n\nThis patch attempts to provide a rudimentary mechanism for catching\nregression introduced by the upstream Ubuntu kernels by running a\nfew basic SDB commands in a Github action that's run nightly and\nfor every PR.\n\nSpecifically this patch makes it so we have such a test for each\nUbuntu LTS kernel starting from 20.04 (currently the `ubuntu-latest`\nGithub runner tag points to 22.04 so we'd test that twice but in\nthe future that tag will point to 24.04, etc...).\n\nWe also change for all the available Python versions for each\nUbuntu version to further ensure SDB's compatibility with future\nPython versions.\n\n= Misc Notes\n\nIn order to use SDB in the Github runner I had to introduce an\nextra script that downloads the kernel's debug info. See the\n`install-live-kernel-dbg.sh` script for more info.\n\nI also made sure to decouple the apt-install of the python-dev\nfiles to its own shell script too as different Ubuntu versions\nship with different Python versions. See `install-python-dev.sh`\nfor more info.\n\n= Potential Future Items\n\nIn the future we may want to detect whenever our ZFS commands are\nnot getting out of date. `test_live_kernel.sh` has a way of\ndetecting whether the ZFS module is installed and running a few\nZFS commands on the live kernel. The idea is that we can either\nintroduce Github Actions like the upstream openzfs that install\nour kernel module to the runner and run the commands there OR\nwe can create a BlackBox test that clones the repo and runs this\nscript.","shortMessageHtmlLink":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early"}},{"before":"1f1cb887acd62c843088e844a2261e67424619c1","after":"abf539ebaeaac42ce498ccb95550feba98ef0e7b","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-22T21:30:50.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early\n\n= Problem\n\nWith our switch to the new v5.15 kernel a subset of SDB commands\nbroke without us realizing until we actually needed them. Our\nregression dumps helps us to ensure we don't introduce regressions\nfor older kernels when developing new features but they can't help\nus in detecting changes in the upstream kernel or ZFS that break\nour commands.\n\n= This Patch\n\nThis patch attempts to provide a rudimentary mechanism for catching\nregression introduced by the upstream Ubuntu kernels by running a\nfew basic SDB commands in a Github action that's run nightly and\nfor every PR.\n\nSpecifically this patch makes it so we have such a test for each\nUbuntu LTS kernel starting from 20.04 (currently the `ubuntu-latest`\nGithub runner tag points to 22.04 so we'd test that twice but in\nthe future that tag will point to 24.04, etc...).\n\nWe also change for all the available Python versions for each\nUbuntu version to further ensure SDB's compatibility with future\nPython versions.\n\n= Misc Notes\n\nIn order to use SDB in the Github runner I had to introduce an\nextra script that downloads the kernel's debug info. See the\n`install-live-kernel-dbg.sh` script for more info.\n\nI also made sure to decouple the apt-install of the python-dev\nfiles to its own shell script too as different Ubuntu versions\nship with different Python versions. See `install-python-dev.sh`\nfor more info.\n\n= Potential Future Items\n\nIn the future we may want to detect whenever our ZFS commands are\nnot getting out of date. `test_live_kernel.sh` has a way of\ndetecting whether the ZFS module is installed and running a few\nZFS commands on the live kernel. The idea is that we can either\nintroduce Github Actions like the upstream openzfs that install\nour kernel module to the runner and run the commands there OR\nwe can create a BlackBox test that clones the repo and runs this\nscript.","shortMessageHtmlLink":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early"}},{"before":"3a927368ec566a30335cd47bf5ba3721107e891e","after":"1f1cb887acd62c843088e844a2261e67424619c1","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-22T21:27:15.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early\n\n= Problem\n\nWith our switch to the new v5.15 kernel a subset of SDB commands\nbroke without us realizing until we actually needed them. Our\nregression dumps helps us to ensure we don't introduce regressions\nfor older kernels when developing new features but they can't help\nus in detecting changes in the upstream kernel or ZFS that break\nour commands.\n\n= This Patch\n\nThis patch attempts to provide a rudimentary mechanism for catching\nregression introduced by the upstream Ubuntu kernels by running a\nfew basic SDB commands in a Github action that's run nightly and\nfor every PR.\n\nSpecifically this patch makes it so we have such a test for each\nUbuntu LTS kernel starting from 20.04 (currently the `ubuntu-latest`\nGithub runner tag points to 22.04 so we'd test that twice but in\nthe future that tag will point to 24.04, etc...).\n\nWe also change for all the available Python versions for each\nUbuntu version to further ensure SDB's compatibility with future\nPython versions.\n\n= Misc Notes\n\nIn order to use SDB in the Github runner I had to introduce an\nextra script that downloads the kernel's debug info. See the\n`install-live-kernel-dbg.sh` script for more info.\n\nI also made sure to decouple the apt-install of the python-dev\nfiles to its own shell script too as different Ubuntu versions\nship with different Python versions. See `install-python-dev.sh`\nfor more info.\n\n= Potential Future Items\n\nIn the future we may want to detect whenever our ZFS commands are\nnot getting out of date. `test_live_kernel.sh` has a way of\ndetecting whether the ZFS module is installed and running a few\nZFS commands on the live kernel. The idea is that we can either\nintroduce Github Actions like the upstream openzfs that install\nour kernel module to the runner and run the commands there OR\nwe can create a BlackBox test that clones the repo and runs this\nscript.","shortMessageHtmlLink":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early"}},{"before":"37c294487c9293b50ffb3823cc3b2f8a744eda88","after":"3a927368ec566a30335cd47bf5ba3721107e891e","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-22T20:46:33.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early\n\n= Problem\n\nWith our switch to the new v5.15 kernel a subset of SDB commands\nbroke without us realizing until we actually needed them. Our\nregression dumps helps us to ensure we don't introduce regressions\nfor older kernels when developing new features but they can't help\nus in detecting changes in the upstream kernel or ZFS that break\nour commands.\n\n= This Patch\n\nThis patch attempts to provide a rudimentary mechanism for catching\nregression introduced by the upstream Ubuntu kernels by running a\nfew basic SDB commands in a Github action that's run nightly and\nfor every PR.\n\nSpecifically this patch makes it so we have such a test for each\nUbuntu LTS kernel starting from 20.04 (currently the `ubuntu-latest`\nGithub runner tag points to 22.04 so we'd test that twice but in\nthe future that tag will point to 24.04, etc...).\n\nWe also change for all the available Python versions for each\nUbuntu version to further ensure SDB's compatibility with future\nPython versions.\n\n= Misc Notes\n\nIn order to use SDB in the Github runner I had to introduce an\nextra script that downloads the kernel's debug info. See the\n`install-live-kernel-dbg.sh` script for more info.\n\nI also made sure to decouple the apt-install of the python-dev\nfiles to its own shell script too as different Ubuntu versions\nship with different Python versions. See `install-python-dev.sh`\nfor more info.\n\n= Potential Future Items\n\nIn the future we may want to detect whenever our ZFS commands are\nnot getting out of date. `test_live_kernel.sh` has a way of\ndetecting whether the ZFS module is installed and running a few\nZFS commands on the live kernel. The idea is that we can either\nintroduce Github Actions like the upstream openzfs that install\nour kernel module to the runner and run the commands there OR\nwe can create a BlackBox test that clones the repo and runs this\nscript.","shortMessageHtmlLink":"DLPX-87572 sdb: want live kernel tests to find kernel regressions early"}},{"before":null,"after":"ac662a5d85d843b6da618d1a865733d1311fddd7","ref":"refs/heads/dmesg_update","pushedAt":"2023-08-22T17:04:21.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"DLPX-87560 sdb: dmesg broken for 5.15 kernels\n\nThis patch replaces our hand-written implementation of dmesg\nwhich doesn't work for most vanilla kernel versions after 5.10\nwith the drgn helper which stays up to date more often.\n\nIn order for this to work I had to change the type of our\ndmesg command from a Locator+PrettyPrinter to a standard\ncommand. This is not a big deal as the only reason we would\never walk and filter dmesg would be to filter the kernel logs\nby level. I've added this as an argument to the command.\n\n= Github Issue Tracker Automation\n\nCloses #336","shortMessageHtmlLink":"DLPX-87560 sdb: dmesg broken for 5.15 kernels"}},{"before":"d70ff50f8f3a0fc1872884a2b13f6eb5f532570f","after":"81271ece9c5835ea8c5e347b2cb372717aad7d78","ref":"refs/heads/undo_v","pushedAt":"2023-08-22T00:47:45.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"DLPX-87531 Fix SDB test regression due to drgn API change\n\nThere were a few changes recently in drgn involving data\nstructures used for holding pointers to type information\ndata. These changes introduced subtle differences in the\nAPIs behavior, one of which affected our test suite.\n\nBefore when a type was looked up by name and there was no\nexterned/global type with that name but multiple local\n(source file-specific ones), then one of those local ones\nwas returned consistently. With the recent drgn changes,\nthe behavior is that one of those local types will be\nrandomly picked making one of our regression tests fail\nsometimes while other times it passed.\n\nThe specific test is `ptype 'struct v'` which we expect\nto output:\n```\nstruct v {\n\tuint8_t b[32];\n}\n```\nBut sometimes we get:\n```\nstruct v {\n\tuint8_t b[64];\n}\n```\n\nThe reason is that there are multiple definitions of this\nstruct in the vdev_raidz code of ZFS:\n```\n~/repos/zfs$ git grep -B 5 -A 2 \"struct v \"\n...snip...\nmodule/zfs/vdev_raidz_math_avx2.c-\nmodule/zfs/vdev_raidz_math_avx2.c-extern const uint8_t gf_clmul_mod_lt[4*256][16];\nmodule/zfs/vdev_raidz_math_avx2.c-\nmodule/zfs/vdev_raidz_math_avx2.c-#define       ELEM_SIZE 32\nmodule/zfs/vdev_raidz_math_avx2.c-\nmodule/zfs/vdev_raidz_math_avx2.c:typedef struct v {\nmodule/zfs/vdev_raidz_math_avx2.c-      uint8_t b[ELEM_SIZE] __attribute__((aligned(ELEM_SIZE)));\nmodule/zfs/vdev_raidz_math_avx2.c-} v_t;\n--\nmodule/zfs/vdev_raidz_math_avx512bw.c-\nmodule/zfs/vdev_raidz_math_avx512bw.c-extern const uint8_t gf_clmul_mod_lt[4*256][16];\nmodule/zfs/vdev_raidz_math_avx512bw.c-\nmodule/zfs/vdev_raidz_math_avx512bw.c-#define   ELEM_SIZE 64\nmodule/zfs/vdev_raidz_math_avx512bw.c-\nmodule/zfs/vdev_raidz_math_avx512bw.c:typedef struct v {\nmodule/zfs/vdev_raidz_math_avx512bw.c-  uint8_t b[ELEM_SIZE] __attribute__((aligned(ELEM_SIZE)));\nmodule/zfs/vdev_raidz_math_avx512bw.c-} v_t;\n...snip...\n```\n\nThis patch gets rid of this regression test as it causes\noccassional failures AND it's not really needed as the\nfunctionality that it tests is already covered by other\ntests.\n\nA side-improvement of this patch is also printing in the\nGithub test result output the differences between what\nwe expect the output to be vs what we get from our ref\ncrash dumps.","shortMessageHtmlLink":"DLPX-87531 Fix SDB test regression due to drgn API change"}},{"before":"de328deed3e6094842acda5d93b1d257ab490ccf","after":"d70ff50f8f3a0fc1872884a2b13f6eb5f532570f","ref":"refs/heads/undo_v","pushedAt":"2023-08-22T00:34:29.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"DLPX-87531 Fix SDB test regression due to drgn API change\n\nThere were a few changes recently in drgn involving data\nstructures used for holding pointers to type information\ndata. These changes introduced subtle differences in the\nAPIs behavior, one of which affected our test suite.\n\nBefore when a type was looked up by name and there was no\nexterned/global type with that name but multiple local\n(source file-specific ones), then one of those local ones\nwas returned consistently. With the recent drgn changes,\nthe behavior is that one of those local types will be\nrandomly picked making one of our regression tests fail\nsometimes while other times it passed.\n\nThe specific test is `ptype 'struct v'` which we expect\nto output:\n```\nstruct v {\n\tuint8_t b[32];\n}\n```\nBut sometimes we get:\n```\nstruct v {\n\tuint8_t b[64];\n}\n```\n\nThe reason is that there are multiple definitions of this\nstruct in the vdev_raidz code of ZFS:\n```\n~/repos/zfs$ git grep -B 5 -A 2 \"struct v \"\n...snip...\nmodule/zfs/vdev_raidz_math_avx2.c-\nmodule/zfs/vdev_raidz_math_avx2.c-extern const uint8_t gf_clmul_mod_lt[4*256][16];\nmodule/zfs/vdev_raidz_math_avx2.c-\nmodule/zfs/vdev_raidz_math_avx2.c-#define       ELEM_SIZE 32\nmodule/zfs/vdev_raidz_math_avx2.c-\nmodule/zfs/vdev_raidz_math_avx2.c:typedef struct v {\nmodule/zfs/vdev_raidz_math_avx2.c-      uint8_t b[ELEM_SIZE] __attribute__((aligned(ELEM_SIZE)));\nmodule/zfs/vdev_raidz_math_avx2.c-} v_t;\n--\nmodule/zfs/vdev_raidz_math_avx512bw.c-\nmodule/zfs/vdev_raidz_math_avx512bw.c-extern const uint8_t gf_clmul_mod_lt[4*256][16];\nmodule/zfs/vdev_raidz_math_avx512bw.c-\nmodule/zfs/vdev_raidz_math_avx512bw.c-#define   ELEM_SIZE 64\nmodule/zfs/vdev_raidz_math_avx512bw.c-\nmodule/zfs/vdev_raidz_math_avx512bw.c:typedef struct v {\nmodule/zfs/vdev_raidz_math_avx512bw.c-  uint8_t b[ELEM_SIZE] __attribute__((aligned(ELEM_SIZE)));\nmodule/zfs/vdev_raidz_math_avx512bw.c-} v_t;\n...snip...\n```\n\nThis patch gets rid of this regression test as it causes\noccassional failures AND it's not really needed as the\nfunctionality that it tests is already covered by other\ntests.\n\nA side-improvement of this patch is also printing in the\nGithub test result output the differences between what\nwe expect the output to be vs what we get from our ref\ncrash dumps.","shortMessageHtmlLink":"DLPX-87531 Fix SDB test regression due to drgn API change"}},{"before":"6b911b360d66fcb205f8ee788a3c535e6bac2367","after":"de328deed3e6094842acda5d93b1d257ab490ccf","ref":"refs/heads/undo_v","pushedAt":"2023-08-21T23:58:54.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"debug","shortMessageHtmlLink":"debug"}},{"before":"f0bb74e2eb673be97ef9a122e77ddb494dc3a3a4","after":"6b911b360d66fcb205f8ee788a3c535e6bac2367","ref":"refs/heads/undo_v","pushedAt":"2023-08-21T23:42:39.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"debug","shortMessageHtmlLink":"debug"}},{"before":"ea2a93585668e1e4190f2799f11fae77c247ebb9","after":"f0bb74e2eb673be97ef9a122e77ddb494dc3a3a4","ref":"refs/heads/undo_v","pushedAt":"2023-08-21T23:04:45.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"WIP","shortMessageHtmlLink":"WIP"}},{"before":"c47691761f1b78fb9259e96f13957501b5c79541","after":"37c294487c9293b50ffb3823cc3b2f8a744eda88","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-21T21:34:08.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"WIP","shortMessageHtmlLink":"WIP"}},{"before":"82956902d73de00187424d770e1e91b209253160","after":"c47691761f1b78fb9259e96f13957501b5c79541","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-21T21:23:40.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"WIP","shortMessageHtmlLink":"WIP"}},{"before":"5d04aedf224f7ee9332b01b423ccb0efa0bc152c","after":"82956902d73de00187424d770e1e91b209253160","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-21T21:19:43.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"WIP","shortMessageHtmlLink":"WIP"}},{"before":"509d4d6f4644ba1eac4efba3f72949d8a588cc31","after":"5d04aedf224f7ee9332b01b423ccb0efa0bc152c","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-21T21:15:25.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"WIP","shortMessageHtmlLink":"WIP"}},{"before":"33549fb29410ae9377873e1c7d10eae548f4c484","after":"509d4d6f4644ba1eac4efba3f72949d8a588cc31","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-21T21:01:55.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"WIP","shortMessageHtmlLink":"WIP"}},{"before":"62e726065668a299e7ac5735ab934d96dce37644","after":"33549fb29410ae9377873e1c7d10eae548f4c484","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-21T20:58:37.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"WIP","shortMessageHtmlLink":"WIP"}},{"before":"a35e44970650955c9e4831aff5c5f8a09cc78366","after":"ea2a93585668e1e4190f2799f11fae77c247ebb9","ref":"refs/heads/undo_v","pushedAt":"2023-08-21T16:54:00.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"WIP","shortMessageHtmlLink":"WIP"}},{"before":null,"after":"a35e44970650955c9e4831aff5c5f8a09cc78366","ref":"refs/heads/undo_v","pushedAt":"2023-08-21T15:53:06.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"WIP","shortMessageHtmlLink":"WIP"}},{"before":"1f6a2f8052bfdbf9713f11488880034749611cfe","after":"62e726065668a299e7ac5735ab934d96dce37644","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-17T18:34:40.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"WIP","shortMessageHtmlLink":"WIP"}},{"before":"2460839779d0e211ec74482883e52485fb0c201d","after":"1f6a2f8052bfdbf9713f11488880034749611cfe","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-17T18:30:59.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"WIP","shortMessageHtmlLink":"WIP"}},{"before":"006cad14e9c497d9464f5a53e3a658bf5a60eedd","after":"2460839779d0e211ec74482883e52485fb0c201d","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-17T18:27:56.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"WIP","shortMessageHtmlLink":"WIP"}},{"before":"1c18b24df5b9d52dddeb41038584dd2b4a303249","after":"006cad14e9c497d9464f5a53e3a658bf5a60eedd","ref":"refs/heads/ubuntu_live_test","pushedAt":"2023-08-17T18:19:19.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"sdimitro","name":"Serapheim Dimitropoulos","path":"/sdimitro","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2281588?s=80&v=4"},"commit":{"message":"WIP","shortMessageHtmlLink":"WIP"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAADcFhZxgA","startCursor":null,"endCursor":null}},"title":"Activity · sdimitro/sdb"}